Initially, NLP models were language-specific, addressing each language in isolation due to distinct linguistic characteristics. However, with the advent of transformer-based architectures, multilingual models have emerged as a more efficient approach. These models demonstrate superior performance, particularly in classification tasks for low-resource languages, by leveraging joint pre-training across multiple languages. A key focus of this article is the challenge of handling low-resource languages within multilingual models. We discuss the issue of data imbalance, where languages with abundant resources overshadow those with less data, impacting overall model performance. To address this, the article examines the use of exponential smoothing in training data sampling. This technique adjusts the probability of language selection, enhancing the representation of low-resource languages while maintaining the quality for high-resource languages. We provide mathematical formulations and practical scenarios illustrating the effectiveness of this approach. The article concludes by underscoring the significance of exponential smoothing in both NLP and multi-locale models, highlighting its role in ensuring balanced training data and improving the performance of multilingual models. This article contributes to the understanding and development of more equitable and efficient multilingual NLP models.