Abstract :

This study examines whether transfer learning can improve machine learning performance under data-scarce forecasting conditions. The empirical application focuses on forecasting foreign direct investment (FDI) inflows using CatBoostRegressor. The source domain is represented by a broad international panel of countries, while the target domain consists of Central Asian economies.

CatBoost is evaluated in two settings: without transfer learning, where the model is trained only on target-region data, and with transfer learning, where the model is first pretrained on the source domain and then fine-tuned on the target domain. The results are compared with simple baseline models, including a lagged-target naive model and a regional mean benchmark.

The findings show that transfer learning substantially improves predictive accuracy. Compared with the target-only CatBoost model, the transfer learning model reduces RMSE from 4.0421 to 3.0426 and MAE from 3.3504 to 2.3369. also improves from − 5.8205 to − 0.0761. These results suggest that transfer learning can help stabilize machine learning forecasts when target-domain observations are limited.

---

Keywords :

CatBoost, Data-Scarce Economies, FDI, Machine learning, Transfer learning

---

References :

1. Pan, S. J., & Yang, Q. (2010). A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22(10), 1345-1359.
2. Shimodaira, H. (2000). Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of Statistical Planning and Inference, 90(2), 227-244.
3. Weiss, K., Khoshgoftaar, T. M., & Wang, D. (2016). A survey of transfer learning. Journal of Big Data, 3(1), 1-40.
4. Thrun, S., & Pratt, L. (1998). Learning to learn: Introduction and overview. In S. Thrun & L. Pratt (Eds.), Learning to Learn (pp. 3-17). Springer.
5. Pan, S. J., & Yang, Q. (2010). A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22(10), 1345-1359.
6. Shimodaira, H. (2000). Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of Statistical Planning and Inference, 90(2), 227-244.
7. Weiss, K., Khoshgoftaar, T. M., & Wang, D. (2016). A survey of transfer learning. Journal of Big Data, 3(1), Article 9.
8. Thrun, S., & Pratt, L. (1998). Learning to learn: Introduction and overview. In S. Thrun & L. Pratt (Eds.), Learning to Learn (pp. 3-17). Springer.
9. Caruana, R. (1997). Multitask learning. Machine Learning, 28, 41-75.
10. Niu, S., Liu, Y., Wang, J., & Song, H. (2020). A decade survey of transfer learning (2010-2020). IEEE Transactions on Artificial Intelligence, 1(2), 151-166.
11. Farahani, A., Voghoei, S., Rasheed, K., & Arabnia, H. R. (2021). A brief review of domain adaptation. In Advances in Data Science and Information Engineering (pp. 877-894). Springer.
12. Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., Xiong, H., & He, Q. (2021). A comprehensive survey on transfer learning. Proceedings of the IEEE, 109(1), 43-76.
13. Hosna, A., Merry, E., Gyalmo, J., Alom, Z., Aung, Z., & Azim, M. A. (2022). Transfer learning: A friendly introduction. Journal of Big Data, 9(1), Article 102.
14. Weber, M., Auch, M. L., Doblander, C., Mandl, P., & Jacobsen, H.-A. (2021). Transfer learning with time series data: A systematic mapping study. IEEE Access, 9, 165409-165428.
15. Solís, M., & Calvo-Valverde, L.-A. (2022). Performance of deep learning models with transfer learning for multiple-step-ahead forecasts in monthly time series. Inteligencia Artificial, 25, 110-125.
16. Solís, M., & Calvo-Valverde, L.-A. (2023). A proposal of transfer learning for monthly macroeconomic time series forecast. Engineering Proceedings, 39, Article 58.
17. Nguyen, H. T., & Nguyen, D. T. (2020). Transfer learning for macroeconomic forecasting. In 2020 7th NAFOSTED Conference on Information and Computer Science (NICS) (pp. 332-337). IEEE.
18. Mlinarič, J., Pregelj, B., Boškoski, P., Petrovčič, J., & Dolanc, G. (2025). A transfer learning approach to machine learning-based end-of-line quality inspection. Advances in Production Engineering & Management, 20(2), 277-290.
19. Laurer, M., van Atteveldt, W., Casas, A., & Welbers, K. (2024). Less annotating, more classifying: Addressing the data scarcity issue of supervised machine learning with deep transfer learning and BERT-NLI. Political Analysis, 32(1), 84-100.
20. Topsakal, O., & Akinci, T. C. (2023). A review of transfer learning: Advantages, strategies and types. In the International Conference on Modern and Advanced Research (pp. 390-394).
21. Alolayan, O. S., Raymond, S. J., Montgomery, J. B., & Williams, J. R. (2021). Towards better shale gas production forecasting using transfer learning. arXiv preprint arXiv:2106.11051.
22. Medeiros, M. C., Vasconcelos, G. F. R., Veiga, Á., & Zilberman, E. (2021). Forecasting inflation in a data-rich environment: The benefits of machine learning methods. Journal of Business & Economic Statistics, 39(1), 98-119.
23. Naghi, A. A., O’Neill, E., & Zaharieva, M. (2024). The benefits of forecasting inflation with machine learning: New evidence. Journal of Applied Econometrics, 39(7), 1321-1331.
24. Islam, M. M., Jannat, A., & Rahman, M. M. (2024). Forecasting foreign direct investment inflow to Bangladesh: Using an autoregressive integrated moving average and a machine learning-based random forest approach. Journal of Risk and Financial Management, 17(10), Article 451.
25. Thrun, S., & Pratt, L. (Eds.). (1998). Learning to Learn. Springer.
26. Torrey, L., & Shavlik, J. (2010). Transfer learning. In E. Soria Olivas, J. D. Martín Guerrero, M. Martínez Sober, J. R. Magdalena Benedito, & A. J. Serrano López (Eds.), Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques (pp. 242–264). IGI Global.
27. Weiss, K., Khoshgoftaar, T. M., & Wang, D. (2016). A survey of transfer learning. Journal of Big Data, 3, Article 9.
28. Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A. V., & Gulin, A. (2018). CatBoost: Unbiased boosting with categorical features. Advances in Neural Information Processing Systems, 31.