Developing an Instrument to Assess Chemical Literacy for Prospective Chemistry Teachers

Ananta Ardyansyah, Sri Rahayu*, Munzil Munzil, Yahmin Yahmin, Naif Mastoor Alsulami

Abstract


The lack of instruments designed to assess chemical literacy among prospective chemistry teachers highlights the need for robust tools to support science education. This study develops and validates an instrument to measure chemical literacy, addressing this gap and contributing to improved pedagogical practices. In this study, a chemical literacy instrument for general chemistry was developed, drawing on existing frameworks of chemical literacy. This research employs a development-based approach (R&D), encompassing literature review, instrument development, and quality testing. The quality of the instrument was evaluated based on its reliability and validity. The validity was evaluated by expert panel review, difficulty index, discrimination index, and pearson correlation value. In this study, a multiple-choice chemical literacy test instrument was successfully developed using the PISA 2025 scientific literacy framework, resulting in 23 valid items. The PISA framework was chosen for its perceived dynamism compared to other frameworks. The reliability of the developed instrument reached 0.818, indicating that the questions are highly reliable. Future improvements to the instrument are necessary, with a focus on ensuring that all items fully meet the established validity criteria

Keywords


assessment; chemical literacy; general chemistry; PISA; test

Full Text:

PDF

References


Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. Educational and Psychological Measurement, 40(4), 955–959. https://doi.org/10.1177/001316448004000419

Arjoon, J. A., Xu, X., & Lewis, J. E. (2013). Understanding the state of the art for measurement in chemistry education research: Examining the psychometric evidence. Journal of Chemical Education, 90(5), 536–545. https://doi.org/10.1021/ed3002013

Barbera, J., Harshman, J., & Komperda, R. (2023). The Chemistry Instrument Review and Assessment Library (CHIRAL): A New Resource for the Chemistry Education Community. Journal of Chemical Education, 100(4), 1455–1459. https://doi.org/10.1021/acs.jchemed.2c00762

Bond, D. (1989). In pursuit of chemical literacy: A place for chemical reactions. Journal of Chemical Education, 66(2), 157. https://doi.org/10.1021/ed066p157

Bybee, R., McCrae, B., & Laurie, R. (2009). PISA 2006: An assessment of scientific literacy. Journal of Research in Science Teaching, 46(8), 865–883. https://doi.org/10.1002/tea.20333

Crocker, L., & Algina, J. (2008). Introduction to classical and modern test theory- Procedures for Estimating Reliability. In Harcourt Brace Jovanovich College.

DeBoer, G. E. (2000). Scientific literacy: Another look at its historical and contemporary meanings and its relationship to science education reform. Journal of Research in Science Teaching, 37(6), 582–601. https://doi.org/https://doi.org/10.1002/1098-2736(200008)37:6<582::AID-TEA5>3.0.CO;2-L

Dewi, C. A., Khery, Y., & Erna, M. (2019). An ethnoscience study in chemistry learning to develop scientific literacy. Jurnal Pendidikan IPA Indonesia, 8(2), 279–287. https://doi.org/10.15294/jpii.v8i2.19261

Djaen, N., Rahayu, S., Yahmin, Y., & Muntholib, M. (2021). Chemical Literacy of First Year Students on Carbon Chemistry. J-PEK (Jurnal Pembelajaran Kimia), 6(1), 41–62. https://doi.org/10.17977/um026v6i12021p041

Dzin, N. H. M., & Lay, Y. F. (2021). Assessing the Validity and Reliability of Science Multiple Choice Test Using Rasch Dichotomous Measurement Model. Journal of Baltic Science Education, 20(6), 927–941. https://doi.org/10.33225/JBSE/21.20.927

Fadly, D., Rahayu, S., Dasna, I. W., & Yahmin, Y. (2022). The Effectiveness of a SOIE Strategy Using Socio-scientific Issues on Students Chemical Literacy. International Journal of Instruction, 15(1), 237–258. https://doi.org/10.29333/iji.2022.15114a

Haetami, A., Arwien, R. T., Marhadi, M. A., Rudi, L., & Arsyad, M. (2023). Increasing Literacy Ability in Chemistry on A Socio-Scientific Basis. Jurnal Penelitian Pendidikan IPA, 9(6), 154–161. https://doi.org/10.29303/jppipa.v9i6.3908

Hazen, R. M., & Trefil, J. S. (1991). Achieving chemical literacy. Journal of Chemical Education, 68(5), 392. https://doi.org/10.1021/ed068p392

Hernani, Saefulloh, & Mudzakir, A. (2017). The first year pre-service teachers’ chemical literacy in individual learning case using the fuel cell technology based-chemical enrichment book. AIP Conference Proceedings, 1868. https://doi.org/10.1063/1.4995106

Holbrook, J., & Rannikmae, M. (2009). The Meaning of Scientific Literacy. International Journal of Envirnomental and Science Education, 4(3), 275–288.

Istyadji, M., & Sauqina. (2023). Conception of scientific literacy in the development of scientific literacy assessment tools: a systematic theoretical review. Journal of Turkish Science Education, 20(2), 281–308. https://doi.org/10.36681/tused.2023.016

Khan, G.-A. N., Ishrat, N., & Khan, A. Q. (2015). Using Item analysis on essay types questions given in summative examination of medical college students: Facility value, Discrimination index. International Journal of Research in Medical Sciences, 3(1), 1. https://doi.org/10.5455/2320-6012.ijrms20150131

Kline, P. (2015). A Handbook of Test Construction. Routledge. https://doi.org/9781138905122

Lagowski, J. J. (1991). Scientific literacy revisited. Journal of Chemical Education, 68(9), 713. https://doi.org/10.1021/ed068p713

Laugksch, R. C. (2000). Scientific literacy: A conceptual overview. Science Education, 84(1), 71–94. https://doi.org/https://doi.org/10.1002/(SICI)1098-237X(200001)84:1<71::AID-SCE6>3.0.CO;2-C

Lazenby, K., Tenney, K., Marcroft, T. A., & Komperda, R. (2023). Practices in instrument use and development in chemistry education research and practice 2010-2021. Chemistry Education Research and Practice, 24(3), 882–895. https://doi.org/10.1039/d2rp00275b

Marfuatun, M., Nahadi, N., Yuliani, G., & Hernani, H. (2024). The Framework and Types of Chemical Literacy Tests: A Systematic Review. Jurnal Penelitian Pendidikan IPA, 10(6), 269–276. https://doi.org/10.29303/jppipa.v10i6.7641

Mellyzar, M., Lukman, I. R., Alvina, S., Pasaribu, A. I., & Fadli, M. R. (2022). Chemical Literacy of High School Students: Analysis of Cognitive Abilities on Colloid Material. Jurnal Penelitian Pendidikan IPA, 8(6), 3128–3133. https://doi.org/10.29303/jppipa.v8i6.2377

Missen, R. W., & Smith, W. R. (1989). A question of basic chemical literacy? Journal of Chemical Education, 66(3), 217–218. https://doi.org/10.1021/ed066p217

Mozeika, D., & Bilbokaite, R. (2011). Teaching and Learning Method for Enhancing 15-16 Years Old Students`Students`Knowledge as One Of Scientific Literacy Aspect in Chemistry: Results Based on Research and Approbation. Educational Research Association The International Journal of Educational Researchers, 2010(1), 1–16.

Muntholib, Ibnu, S., Rahayu, S., Fajaroh, F., Kusairi, S., & Kuswandi, B. (2020). Chemical literacy: Performance of first year chemistry students on chemical kinetics. Indonesian Journal of Chemistry, 20(2), 468–482. https://doi.org/10.22146/ijc.43651

Norris, S. P., & Phillips, L. M. (2003). How literacy in its fundamental sense is central to scientific literacy. Science Education, 87(2), 224–240. https://doi.org/https://doi.org/10.1002/sce.10066

OECD. (2023). PISA 2025 Science Framework. May 2023, 1–93.

Panwala, T., Rajdev, S., Mulla, S., Pérez Rodríguez, N. M., Galloway, R. L., Blau, D. M., Traxler, R., Bhatnagar, J., Zaki, S. R., Rivera, A., Torres, J. V., Noyd, D., Santiago-Albizu, X. E., García, B. R., Tomashek, K. M., Bower, W. A., Sharp, T. M., Technologique, E., Mwachui, M. A., … Levett, P. N. (2017). Kualitas Pengembangan Profesi Guru di SMK Muhammadiyah 1 Wates. In PLoS Neglected Tropical Diseases (Vol. 9, Issue 5). https://doi.org/10.1016/j.tmaid.2020.101607%0Ahttps://doi.org/10.1016/j.ijsu.2020.02.034%0Ahttps://onlinelibrary.wiley.com/doi/abs/10.1111/cjag.12228%0Ahttps://doi.org/10.1016/j.ssci.2020.104773%0Ahttps://doi.org/10.1016/j.jinf.2020.04.011%0Ahttps://doi.o

Peterson, R. A. (1994). A Meta-analysis of Cronbach’s Coefficient Alpha. Journal of Consumer Research, 21(2), 381–391. https://doi.org/10.1086/209405

Prasemmi, S., Rahayu, S., Fajaroh, F., & Almuntasheri, S. (2021). Chemical Literacy Skill of High School Students on the Solution Chemistry. JTK (Jurnal Tadris Kimiya), 6(1), 1–15. https://doi.org/10.15575/jtk.v6i1.10392

Pratama, F. I., & Rohaeti, E. (2023). Students’ Chemical Literacy Ability on Hydrocarbon Material: A Case of Toxic Compounds in Fried Food. Jurnal Penelitian Pendidikan IPA, 9(9), 6795–6802. https://doi.org/10.29303/jppipa.v9i9.4554

Primadianningsih, C., Sumarni, W., & Sudarmin, S. (2023). Systematic Literature Review: Analysis of Ethno-STEM and Student’s Chemistry Literacy Profile in 21st Century. Jurnal Penelitian Pendidikan IPA, 9(2), 650–659. https://doi.org/10.29303/jppipa.v9i2.2559

Roberts, D. A. (2007). Scientific Literacy/Science Literacy. In S. K. Abell & N. G. Lederman (Eds.), Handbook of Research on Science Education (1st ed., Issue 25, pp. 729–781). Routledge.

Sadler, T. D., & Zeidler, D. L. (2009). Scientific literacy, PISA, and socioscientific discourse: Assessment for progressive aims of science education. Journal of Research in Science Teaching, 46(8), 909–921. https://doi.org/10.1002/tea.20327

Schmitt, N. (1996). Uses and abuses of coefficient alpha. Psychological Assessment, 8(4), 350–353. https://doi.org/10.1037/1040-3590.8.4.350

Seprianto, Jofrishal, & Mauliza. (2017). Preservice Chemistry Teachers ’ Chemical Literacy Based on Pisa and Timss Results for International and Indonesian Students. Jurnal Pengajaran MIPA, 22(2), 151–157. https://doi.org/10.18269/jpmipa.v22i2.8647

Shwartz, Y., Ben-Zvi, R., & Hofstein, A. (2006). Chemical literacy: What does this mean to scientists and school teachers? Journal of Chemical Education, 83(10), 1557–1561. https://doi.org/10.1021/ed083p1557

Stašević, F., Miletić, N., Nikolić, J. Đ., & Gutman, I. (2023). Do Serbian high school students possess knowledge of basic chemical facts related to real life as a prerequisite for chemical literacy? Journal of the Serbian Chemical Society, 88(3), 343–354. https://doi.org/10.2298/JSC211126083S

Sulistina, O., Tiara, F. A., & Habiddin, H. (2021). Chemical literacy skills on competencies and attitude aspects of senior high school students. AIP Conference Proceedings, 2330(March). https://doi.org/10.1063/5.0043343

Sumarni, W., Sudarmin, Wiyanto, Rusilowati, A., & Susilaningsih, E. (2017). Chemical literacy of teaching candidates studying the integrated food chemistry ethnosciences course. Journal of Turkish Science Education, 14(3), 60–72. https://doi.org/10.12973/tused.10204a

Suwahyu, F. A., & Rahayu, S. (2023). Development and Utilization of Instrument Using PISA Framework to Improve Chemistry Literacy Ability: A Systematic Review. AIP Conference Proceedings, 2569(January). https://doi.org/10.1063/5.0113478

Taber, K. S. (2018). The Use of Cronbach’s Alpha When Developing and Reporting Research Instruments in Science Education. Research in Science Education, 48(6), 1273–1296. https://doi.org/10.1007/s11165-016-9602-2

Thummathong, R., & Thathong, K. (2018). Chemical literacy levels of engineering students in Northeastern Thailand. Kasetsart Journal of Social Sciences, 39(3), 478–487. https://doi.org/10.1016/j.kjss.2018.06.009

Tosun, C. (2024). Analysis of the Last 40 Years of Science Education Research via Bibliometric Methods. Science and Education, 33(2), 451–480. https://doi.org/10.1007/s11191-022-00400-9

Wattanakasiwich, P., Taleab, P., Sharma, M. D., & Johnston, I. D. (2013). Development and implementation of a conceptual survey in thermodynamics. International Journal of Innovation in Science and Mathematics Education, 21(1), 29–53.




DOI: https://doi.org/10.24815/jpsi.v13i1.42095

Article Metrics

Abstract view : 0 times
PDF - 0 times

Refbacks

  • There are currently no refbacks.


Copyright (c) 2025 Ananta Ardyansyah, Sri Rahayu, Munzil Munzil, Yahmin Yahmin


Jurnal Pendidikan Sains Indonesia (Indonesian Journal of Science Education)

ISSN 2338-4379  (print) | 2615-840X (online)
Organized by Universitas Syiah Kuala 
Published by Master of Science Education Study Program Graduate School Universitas Syiah Kuala, Banda Aceh, Indonesia
Website : http://jurnal.usk.ac.id/jpsi
Email     : jpsi@usk.ac.id

  

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.