AUT Journal of Modeling and Simulation

AUT Journal of Modeling and Simulation

Efficient Arabic Hate Speech Detection via LLaMA-3: A Prompting and Instruction-Tuning Approach

Document Type : Research Article

Authors
Anas Khudhur Abbass 1
Heshaam Faili 2
1 Alborz Campus, University of Tehran, Tehran, Iran
2 School of Electrical and Computer Engineering, College of Engineering, University of Tehran, Tehran, Iran.
10.22060/miscj.2025.24262.5414
Abstract
Cyberspace generates user-generated content daily, but also gives freedom of expression, potentially spreading hate speech and endangering minorities. So, there’s the issue of identifying hate speech as quickly as possible to stop it from being shared. This is particularly challenging for Arabic given its rich morphology and lack of good quality linguistic resources. In this article, we examine zero-shot and few-shot prompting for detecting Arab hate speech using the LLaMA-3-8B language model while also refining performance via supervised fine-tuning utilizing a custom instruction-based dataset. In the zero-shot approach, the outputs from the model are unstructured textual outputs, so we take the unstructured responses and run them through a lightweight TF-IDF + Logistic Regression classifier to classify the responses in one of the predefined hate speech categories. To obtain better classification, we construct the instruction-based training set by creating tweet embeddings from Arabic-BERT and using K-Means clustering to enforce semantic/topical variety. Next, we use the GPT-4o model to generate representative instructions from each cluster and create an instruction-based fine-tuning data set. We then fine-tune LLaMA-3-8B using QLoRA, which also allows the model to be fine-tuned with a lower memory footprint. The experimental results presented in this paper show that zero-shot and few-shot prompting achieved relatively low F1-scores of 42.2% and 45.0%, respectively, and instruction-tuning fine-tuning achieves the overall performance of an F1-score of 90.1%, which exceeds stronger benchmarks like AraBERT. Our results exemplify the potential impact of instruction tuning and QLoRA-based fine-tuning over prompting-based approaches in low-resource contexts like Arabic.
Keywords
Hate Speech Detection in Arabic
Zero-Shot Prompting
Few-shot Prompting
Supervised Fine-Tuning
LLaMA-3-8B
Subjects
[1] J.Q. Dong, C.-H. Yang, Business value of big data analytics: A systems-theoretic approach and empirical test, Information & Management, 57(1) (2020) 103124.
[2] N.A. Ghani, S. Hamid, I.A.T. Hashem, E. Ahmed, Social media big data analytics: A survey, Computers in Human behavior, 101 (2019) 417-428.
[3] K. Müller, C. Schwarz, Fanning the flames of hate: Social media and hate crime, Journal of the European Economic Association, 19(4) (2021) 2131-2167.
[4] A. Al-Hassan, H. Al-Dossari, Detection of hate speech in social networks: a survey on multilingual corpus, in:  6th international conference on computer science and information technology, ACM, 2019, pp. 10-5121.
[5] N. Badri, F. Kboubi, A. Habacha Chaibi, Abusive and Hate speech Classification in Arabic Text Using Pre-trained Language Models and Data Augmentation, ACM Transactions on Asian and Low-Resource Language Information Processing,  (2024).
[6] N. Badri, F. Kboubi, A.H. Chaibi, Combining fasttext and glove word embedding for offensive and hate speech text detection, Procedia Computer Science, 207 (2022) 769-778.
[7] Z. Boulouard, M. Ouaissa, M. Ouaissa, Machine learning for hate speech detection in arabic social media, in:  Computational Intelligence in Recent Communication Networks, Springer, 2022, pp. 147-162.
[8] N. Albadi, M. Kurdi, S. Mishra, Are they our brothers? analysis and detection of religious hate speech in the arabic twittersphere, in:  2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), IEEE, 2018, pp. 69-76.
[9] K. Darwish, W. Magdy, A. Mourad, Language processing for arabic microblog retrieval, in:  Proceedings of the 21st ACM international conference on Information and knowledge management, 2012, pp. 2427-2430.
[10] Y. Matrane, F. Benabbou, N. Sael, A systematic literature review of Arabic dialect sentiment analysis, Journal of King Saud University-Computer and Information Sciences, 35(6) (2023) 101570.
[11] M. Hedhli, F. Kboubi, Cnn-bilstm model for arabic dialect identification, in:  International Conference on Computational Collective Intelligence, Springer, 2023, pp. 213-225.
[12] J.D.M.-W.C. Kenton, L.K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, in:  Proceedings of naacL-HLT, Minneapolis, Minnesota, 2019, pp. 2.
[13] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, Language models are unsupervised multitask learners, OpenAI blog, 1(8) (2019) 9.
[14] R. Alshaalan, H. Al-Khalifa, Hate speech detection in saudi twittersphere: A deep learning approach, in:  Proceedings of the fifth Arabic natural language processing workshop, 2020, pp. 12-23.
[15] B. Alrashidi, A. Jamal, A. Alkhathlan, Abusive content detection in arabic tweets using multi-task learning and transformer-based models, Applied Sciences, 13(10) (2023) 5825.
[16] M. Ibrahim, CUFE at NADI 2024 shared task: Fine-Tuning Llama-3 To Translate From Arabic Dialects To Modern Standard Arabic, in:  Proceedings of The Second Arabic Natural Language Processing Conference, 2024, pp. 769-773.
[17] M.S. Jahan, M. Oussalah, D.R. Beddia, N. Arhab, A Comprehensive Study on NLP Data Augmentation for Hate Speech Detection: Legacy Methods, BERT, and LLMs, arXiv preprint arXiv:2404.00303,  (2024).
[18] H. ElSahar, S.R. El-Beltagy, Building large arabic multi-domain resources for sentiment analysis, in:  International conference on intelligent text processing and computational linguistics, Springer, 2015, pp. 23-34.
[19] P. Nandwani, R. Verma, A review on sentiment analysis and emotion detection from text, Soc Netw Anal Min, 11(1) (2021) 81.
[20] F.M. Plaza-Del-Arco, M.D. Molina-González, L.A. Ureña-López, M.T. Martín-Valdivia, A multi-task learning approach to hate speech detection leveraging sentiment analysis, IEEE Access, 9 (2021) 112478-112489.
[21] H. Mubarak, K. Darwish, W. Magdy, Abusive language detection on Arabic social media, in:  Proceedings of the first workshop on abusive language online, 2017, pp. 52-56.
[22] I. Aljarah, M. Habib, N. Hijazi, H. Faris, R. Qaddoura, B. Hammo, M. Abushariah, M. Alfawareh, Intelligent detection of hate speech in Arabic social network: A machine learning approach, Journal of Information Science, 47(4) (2021) 483-501.
[23] F.Y.A. Anezi, Arabic Hate Speech Detection Using Deep Recurrent Neural Networks, Applied Sciences, 12(12) (2022) 6010.
[24] H. Mohaouchane, A. Mourhir, N.S. Nikolov, Detecting offensive language on arabic social media using deep learning, in:  2019 sixth international conference on social networks analysis, management and security (SNAMS), IEEE, 2019, pp. 466-471.
[25] K. Salameh, S. Hamza, S. Atiani, Enhancing Arabic Hate Speech Detection: The Role of Word Embedding Techniques with Deep Learning Models, in:  2025 International Conference on New Trends in Computing Sciences (ICTCS), 2025, pp. 334-341.
[26] W. Antoun, F. Baly, H. Hajj, Arabert: Transformer-based model for arabic language understanding, arXiv preprint arXiv:2003.00104,  (2020).
[27] M. Abdul-Mageed, A. Elmadany, E.M.B. Nagoudi, ARBERT & MARBERT: Deep bidirectional transformers for Arabic, arXiv preprint arXiv:2101.01785,  (2020).
[28] A.S. Alammary, BERT models for Arabic text classification: a systematic review, Applied Sciences, 12(11) (2022) 5720.
[29] R. Khezzar, A. Moursi, Z. Al Aghbari, arHateDetector: detection of hate speech from standard and dialectal Arabic Tweets, Discover Internet of Things, 3(1) (2023) 1.
[30] K.E. Daouadi, Y. Boualleg, O. Guehairia, Deep Random Forest and AraBert for Hate Speech Detection from Arabic Tweets, J. Univers. Comput. Sci., 29(11) (2023) 1319-1335.
[31] M. Alwateer, I. Gad, M. Elmarhomy, G. Elmarhomy, H. Hashim, M. Almaliki, E.-S. Atlam, Interpretable Arabic Hate Speech Detection using Large Language Model, in:  2025 2nd International Conference on Advanced Innovations in Smart Cities (ICAISC), IEEE, 2025, pp. 1-8.
[32] P. Welsby, B.M. Cheung, ChatGPT, in, Oxford University Press, 2023, pp. 1047-1048.
[33] M. Das, S.K. Pandey, A. Mukherjee, Evaluating ChatGPT against functionality tests for hate speech detection, in:  Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), 2024, pp. 6370-6380.
[34] R. Pan, J.A. García-Díaz, R. Valencia-García, Comparing Fine-Tuning, Zero and Few-Shot Strategies with Large Language Models in Hate Speech Detection in English, CMES-Computer Modeling in Engineering & Sciences, 140(3) (2024).
[35] AI-Meta, meta-llama/Meta-Llama-3-8B-Instruct,  (2024).
[36] H. Mulki, H. Haddad, C.B. Ali, H. Alshabani, L-hsab: A levantine twitter dataset for hate speech and abusive language, in:  Proceedings of the third workshop on abusive language online, 2019, pp. 111-118.
[37] H.a.H. Mulki, Hatem and Ali, Chedi Bechikh and Alshabani, Halima, L-HSAB dataset,  (2019).
[38] F. Husain, O. Uzuner, Transfer Learning Across Arabic Dialects for Offensive Language Detection, in:  2022 International Conference on Asian Language Processing (IALP), 2022, pp. 196-205.
[39] H. Al-Jarrah, M. Al-Smadi, M. Hammad, F. Shannaq, Using Deep Learning Techniques to Detect Hate and Abusive Language in Arabic Tweets, International Journal of Intelligent Engineering & Systems, 17(5) (2024).
AUT Journal of Modeling and Simulation
Volume 57, Issue 2
December 2025
Pages 127-138