AI-Driven and Large Language Models-Based Translation of Arabic News Texts into English: A Comparative Evaluation
Keywords:
Accuracy, AI-driven Translation, Arabic News, Evaluation, Large Language Models, Performance, ChatGPT-4Abstract
The proliferation of artificial intelligence (AI) has profoundly reshaped machine translation, particularly through the advent of Large Language Models (LLMs). This study provides a systematic comparative evaluation of three prominent AI-driven translation tools (Google Translate, Reverso, Yandex) and three state-of-the-art LLMs (ChatGPT-4, Gemini-1.5-Pro, Bing) for translating Arabic news texts into English. Employing a quantitative research design, a corpus of twenty diverse Arabic news articles from major outlets was compiled. Expert-validated human translations served as benchmarks. Translation outputs were analyzed using a three-tiered framework: (1) classification of errors into lexico-semantic, syntactic, and formatting types; (2) performance assessment via a five-point scoring rubric; and (3) determination of accuracy levels. Results reveal that lexico-semantic errors were the most prevalent (45.22%), followed by formatting (32.27%) and syntactic errors (22.50%). Among all systems, ChatGPT-4 demonstrated superior performance, committing the fewest total errors (19 out of 471) and achieving the highest mean accuracy score (7.68/8.00), with 75% of its outputs rated as "highly accurate." In stark contrast, the AI-driven tool Reverso performed least effectively, recording the highest error count (128) and the lowest mean score (5.94/8.00). The findings establish a clear performance hierarchy, indicating that LLMs, especially ChatGPT-4, significantly outperform traditional AI-driven tools in handling the linguistic and contextual complexities of Arabic news translation. However, persistent error patterns underscore the continued necessity for human post-editing to ensure precision in professional and media-specific translation contexts.
Downloads
References
Abdelaal, N., & Alazzawie, A. (2020). Machine translation: The case of Arabic-English translation of news texts. Theory and Practice in Language Studies, 10(4), 408–418. https://doi.org/xxxxx
Abdulaal, M. A. A.-D. (2022). Tracing machine and human translation errors in some literary texts with some implications for EFL translators. Journal of Language and Linguistic Studies, 18, xx–xx.
Ahmed, M. N. (2024). Ethics of translation and journalism: Truth, accuracy and cultural sensitivity in media communication. Al-Noor Journal for Digital Media Studies, 1(3), 35–45.
Ali, M. A. (2020). Quality and machine translation: An evaluation of online machine translation of English into Arabic texts. Open Journal of Modern Linguistics, 10(5), 524–548.
Al-Maaytah, M., & Almahasees, Z. (n.d.). A linguistic investigation for a case study of ChatGPT and Google Translate in rendering special needs texts from English into Arabic: A synchronic case study. Journal name, volume(issue), pages.
Almaaytah, S. A., & Alzobidy, S. A. (2023). Challenges in rendering Arabic text to English using machine translation: A systematic literature review. IEEE Access, 11, 94772–94779. https://doi.org/xxxxx
Al-Salman, S., & Haider, A. S. (2024). Assessing the accuracy of MT and AI tools in translating humanities or social sciences Arabic research titles into English: Evidence from Google Translate, Gemini, and ChatGPT. International Journal of Data and Network Science, 8(4), 2483–2498.
Benbada, M. L., & Benaouda, N. (2023). Investigation of the role of artificial intelligence in developing machine translation quality: Case study of Reverso Context and Google Translate translations of expressive and descriptive texts (Arabic-English/English-Arabic) (Doctoral dissertation, Faculty of Letters and Languages–Department of English).
Chacha, L., & Mwangi, I. (2024). Challenges of translating conversational implicatures from English to Kiswahili using computer-assisted tools: A case of Google Translate. Mwanga wa Lugha, 9(1), 129–135.
Chandra, R., Chaudhary, A., & Rayavarapu, Y. (2025). An evaluation of LLMs and Google Translate for translation of selected Indian languages via sentiment and semantic analyses. arXiv preprint arXiv:2503.21393.
Chen, L., Wang, W., & Hu, D. (2024). E³: Optimizing language model training for translation via enhancing efficiency and effectiveness. In China National Conference on Chinese Computational Linguistics (pp. 75–90). Singapore: Springer Nature Singapore.
Deng, L. (2016). Deep learning: From speech recognition to language and multimodal processing. APSIPA Transactions on Signal and Information Processing, 5, e1.
Falempin, A., & Ranadireksa, D. (2024). Human vs. machine: The future of translation in an AI-driven world. In Widyatama International Conference on Engineering 2024 (WICOENG 2024) (pp. 177–183). Atlantis Press.
Farghal, M., & Haider, A. S. (2024). Translating classical Arabic verse: Human translation vs. AI large language models (Gemini and ChatGPT). Cogent Social Sciences, 10(1), 2410998.
Jiang, Z., Lv, Q., Zhang, Z., & Lei, L. (2023). Distinguishing translations by human, NMT, and ChatGPT: A linguistic and statistical approach. arXiv preprint arXiv:2312.10750.
Mohsen, M. (2024). Artificial intelligence in academic translation: A comparative study of large language models and Google Translate. Psycholinguistics, 35(2), 134–156.
Shafia, S. S. H. (2021). News and news translation: History and strategies. Turkish Journal of Computer and Mathematics Education, 12(11), 6710–6719.
Sholikhah, N. F. M., & Indah, R. N. (2021). Common lexical errors made by machine translation on cultural text. Edulingua: Jurnal Linguistik Terapan dan Pendidikan Bahasa Inggris, 8(1), 39–50.
Sidiya, A. M., Alzaher, H., Almahdi, R., & Elkafrawy, P. (2024). From analysis to implementation: A comprehensive review for advancing Arabic-English machine translation. In 2024 21st Learning and Technology Conference (L&T) (pp. 109–114). IEEE.
Siu, S. C. (2024). Revolutionising translation with AI: Unravelling neural machine translation and generative pre-trained large language models. In New advances in translation technology: Applications and pedagogy (pp. 29–54). Singapore: Springer Nature Singapore.
Tekgurler, M. (2025). LLMs for translation: Historical, low-resourced languages and contemporary AI models. arXiv preprint arXiv:2503.11898.
Zanaty, D. G. (2024). When translating from Arabic to English and vice versa, is Google Bard a trustworthy tool? مجلة وادي النيل للدراسات والبحوث الإنسانية والاجتماعية والتربوية, 44(44), 205–236.
Zinhom, H. (2024). The challenges of using machine translation in rendering Arabic texts into English: Applied perspective. Journal for Foreign Languages, 16(1), 175–198.
Published
How to Cite
Issue
Section
Copyright (c) 2025 Abdalwahid Noman
This work is licensed under a Creative Commons Attribution 4.0 International License.
