Articles

Unidimensionality Analysis and Differential Item Functioning in the Applied Mathematics Final Examination Test of Politeknik Negeri Bali Students Using the Rasch Model

/in /by

The objectives of this research are (1) to analyze the fulfillment of unidimensionality assumptions on Applied Mathematics FSE instruments by the Rasch Model, (2) to identify and analyze the existence of items that have differential item functioning (DIF) among study programs in the polytechnic, and (3) to interpret as well as recommend improvements to instruments based on results from analysis about unidimensionality and DIF. This survey is quantitative research with a cross-sectional design. The sample consists of 206 students coming from three study programs: the Mechanical Engineering, the Refrigeration and Air Conditioning Engineering, and the Utility Engineering Technology, Politeknik Negeri Bali (PNB), selected by the total sampling method. The research instrument comprises five questions constructed according to an outcome-based education (OBE) curriculum where content validity was checked through Aiken’s V method, giving a value equal to 0.91. Data were analyzed using Winsteps 5.9 software based on the Rasch model. Results of the analysis indicated that the test has outstanding reliability and separation indices both on respondents and items. Most items fall within the fitting criteria of the Rasch Model. The unidimensionality test proved that it can consistently measure a single construct in applied mathematics competency. Based on DIF analysis, most items work well across different learning categories, but a few items have large and significant DIFs. These imply that this instrument is suitable to be used as an assessment tool for learning outcomes where several parts still need improvement to enhance fairness in measuring. It raises awareness about conducting comprehensive evaluations on assessment tools within the OBE context in vocational education.

Validation of Applied Mathematics Exam Test Using Rasch Model Approach: Case Study in Diploma 3 Study Program of Refrigeration and Air Conditioning Engineering Politeknik Negeri Bali

/in /by

This study validates the Applied Mathematics exam (AME) test on Diploma 3 (D3) Study Program of Refrigeration and Air Conditioning Engineering (RACE), Politeknik Negeri Bali (PNB) to determine the validity, reliability, unidemensional, level of difficulty, and discriminatory power of the test. Validation uses the modern test theory approach of the Rasch model. Data were collected during the online final exam of the even semester of the 2023/2024 academic year. The instrument uses a multiple-choice test form with 5 answer options. The sample involved 73 even semester students of the D3 RACE study program who took applied mathematics courses. The data collected were analyzed using the Rasch Model assisted by the Winsteps application. The results of the analysis show that the AME test has an adequate level of validity, most of the questions meet the fit criteria for the Rasch Model. The level of test reliability is categorized as very good with a person and item reliability value of 0.90. Several questions still show misfits that require improvement. Item difficulty and person ability show a proportional distribution between the level of difficulty of the questions and the students’ abilities. Overall, the test’s discriminatory power is categorized as good, although there is one question that needs to be reviewed further for improvement. The implication is that the use of the Rasch Model in validating online test instruments can help teachers in compiling questions that are more valid, reliable, and in accordance with the level of student ability. The implication is that the application of the Rasch Model in validating test instruments can help lecturers in constructing more valid and reliable tests. The results of this study can be used as an empirical example of the application of the Rasch model theory to produce more valid and reliable measurements. It is recommended that the development of future test tests really needs to pay attention to the balance between the level of difficulty of the items and the abilities of the students, and ensure that the measurements are more valid and reliable, especially in the context of polytechnic education.