Analysis of Rasch Model for the Validation of Chemistry National Exam Instruments

Ayi Darmana, Ani Sutiani, Haqqi Annazili Nasution, Ismanisa Ismanisa*, Nurhaswinda Nurhaswinda

Abstract


Information about score obtained from a test is often interpreted as an indicator of the student's ability level. This is one of the weaknesses of classical analysis that are unable to provide meaningful and fair information. The acquisition of the same score if it comes from a test item with a different level of difficulty, must show different abilities. Analysis of the Rasch model will overcome this weakness. The purpose of this study was to analyze the quality of the items by validating the national chemistry exam instrument using the Rasch model. The research sample was 212 new students of the Department of Chemistry at the State University of Medan. The data collected was in the form of respondent's answer data to the 2013 chemistry UN questions, which amounted to 40 items multiple choice and uses the documentation method. Data analysis technique used the Rasch Model with Ministep software. The results of the analysis show the quality of the Chemistry National Exam (UN) questions is categorized as very good based on the following aspects: unidimension, item fit test, person map item, difficulty test level, person and item reliability. There is one item found to be gender bias, in which men benefit more than women. The average chemistry ability of respondents is above the average level of difficulty of the test items

Keywords


National exam, Dichotomy, Rasch model, Ministep

Full Text:

PDF

References


Adedoyin, O. & Mokobi, T. 2013. Using IRT psychometric analysis in examining the quality of junior certificate mathematics multiple choice examination test items. International Journal of Asian Social Science, 3(4):992-1011.

Afrassa, T.M. 2005. Monitoring mathematics achievement over time. In Applied Rasch Measurement: A Book of Exemplars, Springer, Dordrecht, pp.61-77.

Ardiyanti, D. 2016. Aplikasi model rasch pada pengembangan skala efikasi diri dalam pengambilan keputusan karir peserta didik. Jurnal Psikologi, 43(3):248–263.

Baker, F.B. & Kim, S.H. 2017. The Basics of Item Response Theory using R. Springer, New York, pp.55-67.

Bond, T.G. & Fox, C.M. 2007. Applying the Rasch Model: Fundamental Measurement in the Human Sciences, 2nd Edition, Lawrence Erlbaum Associates, Publisers, Mahwah, New Jersey, London.

Boone, W.J., Staver, J.R., & Yale, M.S. 2014. Rasch Analysis in the Human Sciences, Springer Science dan Business Media.

Brown, R.L., Obasi, C.N., & Barrett, B. 2016. Rasch analysis of the WURSS-21 dimensional validation and assessment of invariance. Journal of Lung, Pulmonary & Respiratory Research, 3(2):46-53.

Chan, S.W., Ismail, Z., & Sumintono, B. 2014. A rasch model analysis on secondary students’ statistical reasoning ability in descriptive statistics. Procedia-Social and Behavioral Sciences, 129:133-139.

Chernyshenko, O.S., Stark, S., Chan, K.Y., Drasgow, F., & Williams, B. 2001. Fitting item response theory models to two personality inventories: issues and insights. Multivariate Behavioral Research, 36(4):523-562.

Darmana, A., Jasmidi, & Sutiani, A. 2020. Development of the thermochemistry-Hots-tawheed multiple choice instrument. In Journal of Physics Conference Series, 1462(1):1-9.

Field, A. 2009. Discovering Statistics using SPSS, 3rd edition, Sage Publication Ltd, London.

Greiff, S., Fischer, A., Wüstenberg, S., Sonnleitner, P., Brunner, M., & Martin, R. 2013. A multitrait–multimethod study of assessment instruments for complex problem solving. Intelligence, 41(5):579-596.

Guler, N., Uyanik, G.K., & Teker, G.T. 2014. Comparison of classical test theory and item response theory in terms of item parameters. European Journal of Research on Education, 2(1):1-6.

Hambleton R.K. & Swaminathan, H. 1985. Items Response Theory: Principles And Application, Kluwer-Nijjhoff Publish, Boston.

Hayati, S. & Lailatussaadah, L. 2016. Validitas dan reliabilitas instrumen pengetahuan pembelajaran aktif, kreatif dan menyenangkan (PAKEM) menggunakan model Rasch. Jurnal Ilmiah Didaktika: Media Ilmiah Pendidikan dan Pengajaran, 16(2):169-179.

Ibrahim, F.M., Shariff, A.A., & Tahir, R.M. 2015. Using rasch model to analyze the ability of pre-university students in vector. In AIP Conference Proceedings, 1682(1): 030009.

Isnani, I., Utami, W.B., Susongko, P., & Lestiani, H.T. 2019. Estimation of college students’ ability on real analysis course using rasch model. REiD (Research and Evaluation in Education), 5(2):95-102.

Jüttner, M., Boone, W., Park, S., & Neuhaus, B.J. 2013. Development and use of a test instrument to measure biology teachers’ content knowledge (CK) and pedagogical content knowledge (PCK). Educational Assessment, Evaluation and Accountability, 25(1):45-67.

Kaiser, H.F. 1974. An index of factorial simplicity. Phyhcometrika, 39(1):31-36.

Lia, R.M., Rusilowati, A., & Isnaeni, W. 2020. NGSS-Oriented chemistry test instruments: validity and reliability analysis with the rasch model. REiD (Research and Evaluation in Education), 6(1):41-50.

Linacre, J.M. 2016. A User’s Guide to WINSTEPS MINISTEP Rasch-Model Computer Programs, IL: Winsteps.com, Chicago.

Mahmud, Z., Ramli, W.S.W., Sapri, S., & Ahmad, S. 2017. Diagnosis of students’ ability in a statistical course based on rasch probabilistic outcome. In AIP Conference Proceedings, 1836(1):020048.

Osterlind, S.J. 1983. Test Item Bias, CA: Sage Publication Inc, Beverly Hills.

Pratama, D. & Husnayaini, I. 2020. Applying rasch model to measure students reading comprehension. JISAE: Journal of Indonesian Student Assessment and Evaluation, 6(2):203-209.

Reckase, M.D. 1979. Unifactor latent trait models applied to multifactor tests: results and implications. Journal of Educational Statistics, 4(3):207-230.

Rosana, D., Widodo, E., Setianingsih, W., & Setyawarno, D. 2020. Developing assessment instruments of PISA model to measure students’ problem-solving skills and scientific literacy in junior high schools. Jurnal Pendidikan Sains Indonesia, 8(2):292-305.

Sabekti, A.W. & Khoirunnisa, F. 2018. Penggunaan rasch model untuk mengembangkan instrumen pengukuran kemampuan berpikir kritis peserta didik pada topik ikatan kimia. Jurnal Zarah, 6(2):68–75.

Saidi, S.S. & Siew, N.M. 2019. Reliability and validity analysis of statistical reasoning test survey instrument using the rasch measurement model. International Electronic Journal of Mathematics Education, 14(3):535-546.

Samritin, S., Wijaya, R.S., Tarno, T., Suranata, K., Ardi, Z., Ifdil, I., ... & Rangka, I.B. 2019. Matching the student’s ability and their math test using rasch analysis. In Journal of Physics: Conference Series, 1318(1):012059.

Setiawan, B., Panduwangi, M., & Sumintono, B. 2018. A Rasch analysis of the community’s preference for different attributes of islamic banks in Indonesia. International Journal of Social Economics, 45(12):1647-1662.

Sihombing, R.U., Naga, D.S., & Rahayu, W. 2019. A Rasch model measurement analysis on science literacy test of indonesian students: smart way to improve the learning assessment. IJER-Indonesian Journal of Educational Review, 6(1):44-55.

Smits, N., Cuijpers, P., & Van Straten, A. 2011. Applying computerized adaptive testing to the CES-D scale: A simulation study. Psychiatry Research, 188(1):147-155.

Sumintono, B. & Widhiarso, W. 2015. Aplikasi Pemodelan Rasch pada Assessmen Pendidikan, Trim Komunikata, Cimahi.

Susongko, P. 2016. Validation of science achievement test with the rasch model. Jurnal Pendidikan IPA Indonesia, 5(2):268-277.

Wati, M., Mahtari, S., Hartini, S., & Amelia, H. 2019. A Rasch model analysis on Junior High School students' scientific reasoning ability. International Journal of Interactive Mobile Technologies (iJIM), 13(7):141-149.

Wibisono, S. 2019. Aplikasi model rasch untuk validasi instrumen pengukuran fundamentalisme agama bagi responden muslim. JP3I (Jurnal Pengukuran Psikologi dan Pendidikan Indonesia), 3(3):729-750.

Wu, Q., Zhang, Z., Song, Y., Zhang, Y., Zhang, Y., Zhang, F., & Miao, D. 2013. The development of mathematical test based on item response theory. International Journal of Advancements in Computing Technology, 5(10):209–216.

Yasin, S.N.T.M., Yunus, M.F.M., & Ismail, I. 2018. The Use of rasch measurement model for the validity and reliability. Journal of Counseling and Educational Technology, 1(2):22-27.




DOI: https://doi.org/10.24815/jpsi.v9i3.19618

Article Metrics

Abstract view : 0 times
PDF - 0 times

Refbacks

  • There are currently no refbacks.


Copyright (c) 2021 Ismanisa Ismanisa, Ayi Darmana, Ani Sutiani, Haqqi Annazili Nasution, Nurhaswinda Nurhaswinda


Jurnal Pendidikan Sains Indonesia (Indonesian Journal of Science Education)

ISSN 2338-4379  (print) | 2615-840X (online)
Organized by Universitas Syiah Kuala 
Published by Master of Science Education Study Program Graduate School Universitas Syiah Kuala, Banda Aceh, Indonesia
Website : http://jurnal.usk.ac.id/jpsi
Email     : jpsi@usk.ac.id

  

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.