EVALUATING MULTILINGUAL AND LANGUAGE SPECIFIC TRANSFORMERS FOR ENGLISH- HINDI SEMANTIC ALIGNMENT

Abstract

This research provides a detailed comparative evaluation of three prominent transformer based language models: multilingual BERT (mBERT), XLM-ROBERTa, and the Hindi specific L3Cube-HindBERT. The primary objective was to assess their effectiveness in learning and aligning cross-lingual semantic representations between English and Hindi. The study utilized the Bharat Parallel Corpus Collection (BPCC), a significant resource for Indian languages, to form the basis of this investigation. A synthetic classification task was designed to evaluate the models’ ability to differentiate between genuine English-Hindi translated sentence pairs and randomly mismatched pairs, thereby gauging their capacity to capture semantic equivalence. While the core research paper focused on performance metrics and training dynamics without visualizations, this report will also touch upon the experimental scripts’ capabilities for such visual analysis as part of a broader method ological discussion. The findings indicate that all three models are capable of aligning cross-lingual representations, though their learning trajectories and ultimate performance vary due to architectural and pretraining differences. Notably, mBERT demonstrated the most stable training convergence and achieved the best overall performance on the clas sification task, suggesting advantages from its extensive multilingual pretraining. XLM ROBERTa showed slightly higher validation losses but strong performance, indicative of its robust pretraining regimen. L3Cube-HindBERT, while initially slower to converge, showed benefits of domain adaptation for Hindi. This study underscores the trade-offs between generalized multilingual models and language-specific architectures in the context of cross-lingual tasks.