Mathematical Problem Solving in Arabic: Assessing Large Language Models

This paper comprehensively evaluates the efficacy of different large language models (LLMs) in addressing mathematical challenges expressed in natural languages, mainly focusing on low-resource languages like Arabic. The main challenge of this problem is that despite the considerable size and impressive problem-solving capabilities of these models, they still require enhancements to achieve satisfactory performance for mathematical problem-solving. An LLM must be trained or fine-tuned extensively in understanding and solving mathematical problems articulated in natural language contexts. This is further complicated by the scarcity of high-quality, annotated datasets in low-resource languages. To overcome this problem, we introduce a method for generating synthetic data to address the challenge of data scarcity in Arabic. This approach includes using LLMs to create high-quality, synthetic mathematical problem datasets that can be used to augment the training process. We propose the Synthetic Data Augmentation (SDA) framework, an advanced variant of traditional data augmentation techniques explicitly tailored for mathematical problem-solving in low-resource languages. Experimental studies reveal that using synthetic data significantly improves the performance of LLMs in solving mathematical problems in Arabic. This study contributes to the field of natural language processing and provides insights into the potential of synthetic data generation to enhance LLM performance in low-resource languages. © 2024 Elsevier B.V.. All rights reserved.