Abstract :

This study validates the Applied Mathematics exam (AME) test on Diploma 3 (D3) Study Program of Refrigeration and Air Conditioning Engineering (RACE), Politeknik Negeri Bali (PNB) to determine the validity, reliability, unidemensional, level of difficulty, and discriminatory power of the test. Validation uses the modern test theory approach of the Rasch model. Data were collected during the online final exam of the even semester of the 2023/2024 academic year. The instrument uses a multiple-choice test form with 5 answer options. The sample involved 73 even semester students of the D3 RACE study program who took applied mathematics courses. The data collected were analyzed using the Rasch Model assisted by the Winsteps application. The results of the analysis show that the AME test has an adequate level of validity, most of the questions meet the fit criteria for the Rasch Model. The level of test reliability is categorized as very good with a person and item reliability value of 0.90. Several questions still show misfits that require improvement. Item difficulty and person ability show a proportional distribution between the level of difficulty of the questions and the students’ abilities. Overall, the test’s discriminatory power is categorized as good, although there is one question that needs to be reviewed further for improvement. The implication is that the use of the Rasch Model in validating online test instruments can help teachers in compiling questions that are more valid, reliable, and in accordance with the level of student ability. The implication is that the application of the Rasch Model in validating test instruments can help lecturers in constructing more valid and reliable tests. The results of this study can be used as an empirical example of the application of the Rasch model theory to produce more valid and reliable measurements. It is recommended that the development of future test tests really needs to pay attention to the balance between the level of difficulty of the items and the abilities of the students, and ensure that the measurements are more valid and reliable, especially in the context of polytechnic education.

---

Keywords :

Applied Mathematics, Polytechnic, Rasch Model, Reliability., Test, Validity.

---

References :

1. Bernard, R. M., Borokhovski, E., Schmid, R. F., Tamim, R. M. & Abrami, P. C. A meta-analysis of blended learning and technology use in higher education: From the general to the applied. Comput. High. Educ. 26, 87–122 (2014).
2. Li, T., Yeung, M., Li, E. & Leung, B. How formative are assessments for learning activities towards summative assessment? J. Teach. Educ. 9, 42–57 (2021).
3. Rawlusyk, P. . Assessment in Higher Education and Student Learning. Instr. Pedagog. 21, 1–34 (2018).
4. Sekyi, E. T. A. Assessment, Student Learning and Classroom Practice: A Review. Educ. Pract. 7, 1–6 (2016).
5. Brookhart, S. M. & Nitko, A. J. Educational Assessment of Students. (Pearson Education, 2011).
6. Crocker, L. M. & Algina, J. Introduction to Classical and Modern Test Theory. (Wadsworth Publishing, 1986).
7. Nunnally, J. C. & Bernstein, I. H. Psychometric Theory. (McGraw-Hill Book Company, 1994).
8. Allen, M. J. & Yen, W. M. Introduction to Measurement Theory. (Waveland Press, 2001).
9. Wright, B. D. & Stone, M. H. Best Test Design. University of Chicago (Mesa Press, 1979). doi:10.2307/jj.3685368.13.
10. Bond, T. G. & Fox, C. M. Applying the Rasch Model : Fundamental Measurement in the Human Sciences Second Edition. (Lawrence Erlbaum Associates, Inc, 2007).
11. Baghaei, P. & Amrahi, N. Validation of a Multiple Choice English Vocabulary Test with the Rasch Model. Lang. Teach. Res. 2, (2011).
12. Anastasi, A. & Urbina, S. Psychological Testing. (PT. Indeks, Gramedia Grup, 2007).
13. Gronlud, N. E. & Linn, R. L. Meansurment and Evaluation in Teaching. (Macmillan Publishing Co., Inc, 1990).
14. Mardapi, D. Pengukuran, penilaian dan evaluasi pendidikan. (: Nuha Medika, 2016).
15. Bond, T. G. & Fox, C. M. Applying the Rasch Model: Fundamental Measurement in the Human Sciences,. (Routledge, 2015).
16. Thompson, B. Exploratory and Confirmatory Factor Analysis. American Psychological Association (American Psychological Association, 2004).
17. Anderson, L. W. & Krathwohl, D. R. A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives: A Revision of Bloom’s Taxonomy of Educational Objectives. (Pearson, 2001).
18. Boone, W. J., Staver, J. R. & Yale, M. S. Rasch Analysis in the Human SciencesNo Title. (Springer., 2014).
19. Boone, W. J. Rasch analysis for instrument development: Why,when,and how? CBE Life Sci. Educ. 15, 1–7 (2016).
20. Linacre, J. M. Data Variance Explained by Rasch Measures. Rasch Meas. Trans. 20, 1045 (2006).
21. Linacre, J. M. A User’s Guide to Winsteps, Ministep Rasch-Model Computer Programs. Winsteps https://www.winsteps.com/winman/copyright.htm (2019).
22. Kubinger, K. D., Rasch, D. & Yanagida, T. A new approach for testing the Rasch model. Res. Eval. An Int. J. Theory Pract. 17, 321–333 (2011).
23. Zanon, C., Hutz, C. S., Yoo, H. & Hambleton, R. K. An application of item response theory to psychological test development. Reflex. e Crit. 29, (2016).
24. Erfan, M., Maulyda, M. A., Hidayati, V. R., Astria, F. P. & Ratu, T. Tes Klasik Dan Model Rasch. J. Educ. Res. Rev. 3, 11–19 (2020).
25. Susdelina, Perdana, S. A. & Febrian. Analisis Kualitas Instrumen Pengukuran Pemahaman Konsep Persamaan Kuadrat Melalui Teori Tes Klasik Dan Rasch Model. Kiprah 6, 41–48 (2018).
26. Bichi, A. A., Talib, R., Atan, N. A., Ibrahim, H. & Yusof, S. M. Validation of a developed university placement test using classical test theory and Rasch measurement approach. J. Adv. Appl. Sci. 6, 22–29 (2019).
27. Boone, W. J. & Noltemeyer, A. Rasch analysis: A primer for school psychology researchers and practitioners. Cogent Educ. 4, (2017).
28. Mahmud, Z. & Porter, A. Using rasch analysis to explore what students learn about probability concepts. Math. Educ. 6, 1–10 (2015).
29. Runnels, J. & Bunkyo, H. Using the Rasch model to validate a multiple choice English achievement test. J. Lang. Stud. 6, 141–153 (2015).
30. Sumintono, B. & Widhiarso, W. Aplikasi Pemodelan Rasch pada Assessment Pendidikan. (Trim Komunikata, 2015).
31. Linacre, J. M. What do Infit and Outfit, Mean-square and Standardized mean? Rasch Meas. Trans. 6, 878 (2012).
32. Linacre, J. M. Optimizing rating scale category effectiveness Optimizing Rating Scale Category Effectiveness University of Chicago. Appl. Meas. 3, 85–106 (2002).
33. Gliem, J. A. & Gliem, R. R. Calculating, Interpreting, and Reporting Cronbach’s Alpha Reliability Coefficient for Likert-Type Scales. in Midwest Research to Practice Conference in Adult, Continuing, and Community Education 83–88 (The Ohio State University, Columbus, OH, 2003).
34. Planinic, M., Boone, W. J., Susac, A. & Ivanjek, L. Rasch analysis in physics education research: Why measurement matters. Rev. Phys. Educ. Res. 15, 20111 (2019).
35. Taylor, R. Interpretation of the Correlation Coefficient: A Basic Review. Dianostic Med. Sonogr. 6, 35–39 (1990).
36. Schober, P. & Schwarte, L. A. Correlation coefficients: Appropriate use and interpretation. Analg. 126, 1763–1768 (2018).
37. Metode Penelitian Kuantitatif, Kualitatif dan R&D. (Alfabeta, 2017).
38. Linacre, J. M. Predicting responses from rasch measures. Appl. Meas. 11, 1–10 (2010).
39. Wright, B. D. & Masters, G. N. RATING SCALE ANALYSIS-n Item Response Modeling Approach. (1982).
40. Coladarci, T. & Cobb, C. D. Fundamentals of Statistical Reasoning in Education. (John Wiley & Sons, Inc., 2013).
41. Wright, B. D. & Stone, M. H. Measurement Essentials. Measurement (WIDE RANGE, INC, 1999).
42. Glas, C. A. W. & Vos, H. J. Adaptive Mastery TestingUsing the Rasch Model and Bayesian Sequential Decision Theory. ERIC https://eric.ed.gov/?id=ED417056 (1998).