In this research paper,we propose a corpus for the task of detecting religious extremism in social networks and open sources and compare various machine learning algorithms for the binary classification problem using ...In this research paper,we propose a corpus for the task of detecting religious extremism in social networks and open sources and compare various machine learning algorithms for the binary classification problem using a previously created corpus,thereby checking whether it is possible to detect extremist messages in the Kazakh language.To do this,the authors trained models using six classic machine-learning algorithms such as Support Vector Machine,Decision Tree,Random Forest,K Nearest Neighbors,Naive Bayes,and Logistic Regression.To increase the accuracy of detecting extremist texts,we used various characteristics such as Statistical Features,TF-IDF,POS,LIWC,and applied oversampling and undersampling techniques to handle imbalanced data.As a result,we achieved 98%accuracy in detecting religious extremism in Kazakh texts for the collected dataset.Testing the developed machine learningmodels in various databases that are often found in everyday life“Jokes”,“News”,“Toxic content”,“Spam”,“Advertising”has also shown high rates of extremism detection.展开更多
基金This work was supported by the grant“Development of models,algorithms for semantic analysis to identify extremist content in web resources and creation the tool for cyber forensics”funded by the Ministry of Digital Development,Innovations and Aerospace industry of the Republic of Kazakhstan.Grant No.IRN AP06851248.Supervisor of the project is Shynar Mussiraliyeva,email:mussiraliyevash@gmail.com.
文摘In this research paper,we propose a corpus for the task of detecting religious extremism in social networks and open sources and compare various machine learning algorithms for the binary classification problem using a previously created corpus,thereby checking whether it is possible to detect extremist messages in the Kazakh language.To do this,the authors trained models using six classic machine-learning algorithms such as Support Vector Machine,Decision Tree,Random Forest,K Nearest Neighbors,Naive Bayes,and Logistic Regression.To increase the accuracy of detecting extremist texts,we used various characteristics such as Statistical Features,TF-IDF,POS,LIWC,and applied oversampling and undersampling techniques to handle imbalanced data.As a result,we achieved 98%accuracy in detecting religious extremism in Kazakh texts for the collected dataset.Testing the developed machine learningmodels in various databases that are often found in everyday life“Jokes”,“News”,“Toxic content”,“Spam”,“Advertising”has also shown high rates of extremism detection.
