Abstract :

The objectives of this research are (1) to analyze the fulfillment of unidimensionality assumptions on Applied Mathematics FSE instruments by the Rasch Model, (2) to identify and analyze the existence of items that have differential item functioning (DIF) among study programs in the polytechnic, and (3) to interpret as well as recommend improvements to instruments based on results from analysis about unidimensionality and DIF. This survey is quantitative research with a cross-sectional design. The sample consists of 206 students coming from three study programs: the Mechanical Engineering, the Refrigeration and Air Conditioning Engineering, and the Utility Engineering Technology, Politeknik Negeri Bali (PNB), selected by the total sampling method. The research instrument comprises five questions constructed according to an outcome-based education (OBE) curriculum where content validity was checked through Aiken’s V method, giving a value equal to 0.91. Data were analyzed using Winsteps 5.9 software based on the Rasch model. Results of the analysis indicated that the test has outstanding reliability and separation indices both on respondents and items. Most items fall within the fitting criteria of the Rasch Model. The unidimensionality test proved that it can consistently measure a single construct in applied mathematics competency. Based on DIF analysis, most items work well across different learning categories, but a few items have large and significant DIFs. These imply that this instrument is suitable to be used as an assessment tool for learning outcomes where several parts still need improvement to enhance fairness in measuring. It raises awareness about conducting comprehensive evaluations on assessment tools within the OBE context in vocational education.

---

Keywords :

Applied Mathematics, Differential Item Functioning, Rasch Model, Unidimensionality, vocational education

---

References :

1. Hake, R. R. Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. J. Phys. 66, 64–74 (1998).
2. Bond, T. G. & Fox, C. M. Applying the Rasch model: Fundamental measurement in the human sciences,. (Routledge, 2015). doi:https://doi.org/10.1111/j.1745-3984.2003.tb01103.x.
3. Boone, W. J., Staver, J. R. & Yale, M. S. Rasch Analysis in the human sciences. (Springer., 2014). doi:10.1007/978-94-007-6857-4.
4. Tennant, A., McKenna, S. P. & Hagell, P. Application of Rasch analysis in the development and application of quality of life instruments. Value Heal. 7, 22–26 (2004).
5. Wright, B. D. & Stone, M. H. Measurement essentials. Measurement (WIDE RANGE, INC, 1999).
6. Linacre, J. M. What do infit and outfit, mean-square and standardized mean? Rasch Meas. Trans. 6, 878 (2012).
7. Tennant, A. & Küçükdeveci, A. A. Application of the Rasch measurement model in rehabilitation research and practice: Early developments, current practice, and future challenges. Rehabil. Sci. 4, 01–17 (2023).
8. Dorans, N. J. & Holland, P. W. DIF Detection and Description: Mantel-Haenszel and Standardization. Differential Item Functioning (N.J: Lawrence Erlbaum Associates., 1993).
9. Linacre, J. M., Boone, W., Green, R., Trevor, G. & Christine, M. A User’s Guide to WINSTEPS® / MINISTEP Rasch-model computer programs. 2–4 (2023).
10. Hope, D., Adamson, K., McManus, I. C., Chis, L. & Elder, A. Using differential item functioning to evaluate potential bias in a high stakes postgraduate knowledge based assessment. BMC Med. Educ. 18, 1–7 (2018).
11. Zumbo, B. D. Psychometric methods for investigating DIF and test bias during test adaptation across languages and cultures. 1–85 (2006).
12. Holland, P. W. & Wainer, H. Differential item functioning. (Routledge., 2012).
13. Creswell, J. W. & Creswell, J. D. Research design qualitative, quantitative, and mixed methods approaches. (SAGE Publications, Inc., 2017).
14. Kesmodel, U. S. Cross-sectional studies – what are they good for? Acta Obstet. Gynecol. Scand. 97, 388–393 (2018).
15. Aiken, L. R. Three Coefficients for Analyzing the Reliability and Validity of Ratings. Psychol. Meas. 45, 131–142 (1985).
16. Wright, B. D. & Masters, G. N. Rating scale analysis item response modeling approach. (Mesa Press, 1982).
17. Linacre, J. M. Reasonable mean-square fit values revisited. Rasch Meas. Trans. 36, 1–6 (2024).
18. Wright, B. D. & Linacre, J. M. Reasonable mean-square fit values. Rasch Meas. Trans. 8, 370 (1994).
19. Linacre, J. M. Optimizing rating scale category effectiveness optimizing rating scale category effectiveness university of Chicago. Appl. Meas. 3, 85–106 (2002).
20. Linacre, J. M. Data Variance Explained by Rasch Measures. Rasch Meas. Trans. 20, 1045 (2006).
21. Supriatna, M., Suryana, D., Anzhali, M. N. & Noor, A. M. Evaluating career self-awareness instruments for victims of violence using the Rasch model. BMC Psychol. 13, 1–13 (2025).
22. Sumintono, B. & Widhiarso, W. Aplikasi Pemodelan Rasch pada Assessment Pendidikan. (Trim Komunikata, 2015).
23. Linacre, J. M. Predicting responses from rasch measures. Appl. Meas. 11, 1–10 (2010).
24. Bond, T. G., Yan, Z. & Heene, M. Applying the Rasch model fundamental measurement in the human sciences. (Routledge, 2020). doi:https://doi.org/10.4324/9780429030499.
25. Hambleton, R. K., Swaminathan, H. & Rogers, H. J. Fundamentals of item response theory. Sosiological Assoc. 21, 289–290 (1992).
26. Biggs, J. & Tang, C. Teaching for quality learning at university. What the student does (4th Edn.). Innovations in Education and Teaching International 50 (SRHE and Open University Press, 2011).
27. Zumbo, B. D. A handbook on the theory and methods of differential item functioning (DIF). Ottawa: National Defense Headquarters (Directorate of Human Resources Research and Evaluation Department of National Defen, 1999).
28. Linacre, J. M. Rasch measurement transactions: Teaching Rasch measurement. 31, 1630–1642 (2017).
29. Zumbo, B. D. & Gelin, M. N. A matter of test bBias in educational policy research: Bringing the context into picture by iInvestigating sociological / community moderated (or mediated) test and item bias. Educ. Res. Policy Stud. 5, 1–23 (2005).
30. Engelhard, G. & Wang, J. Rasch Models for Solving Measurement Problems: Invariant Measurement in the Social Sciences. (SAGE Publications, Inc., 2022). doi:10.4135/9781071878675.