Development of Linear Motion Assessment Using Evidence-Centered Design

Hepi Kharisda Hulu, Judyanto Sirait*, Firdaus Firdaus, Nuri Balta

Abstract


Linear motion is one of the basic concepts in physics, so an assessment that can measure students' abilities in this material is necessary. The assessments created by teachers tend to only evaluate students' abilities in formula calculations without measuring their understanding in depth. This research aims to develop assessments on the topic of linear motion using the evidence-centered design (ECD) framework. ECD is a framework that focuses on evidence of students' abilities. ECD consists of three components: claim, evidence, and task. The process of developing this assessment uses the research and development method by following the ADDIE stages: analysis, design, development, implementation, and evaluation. Data were collected through interviews with physics teachers, analyzing teachers’ documents, validating the assessment, and conducting task trials. The trials were conducted in two stages: one-to-one trials with one student and small-scale trials with 10 students. The analysis techniques included calculating validity using the Aiken Index, descriptive analysis of student responses in the one-to-one trial, and small-scale trials. The competencies measured in the assessment are based on scientific practice, including using and developing representations, analyzing and interpreting data, explaining with evidence, and evaluating information. Each measured competency in the assessment consists of two essay tasks. The validation results show that the developed assessment has high validity with an average Aiken score of 0.97. Teachers can use this assessment as a reference through stages to develop assessments that evaluate the achievement of learning objectives


Keywords


R&D, Linear Motion, Assessment, Evidence-Centered Design, Scientific Practice

Full Text:

PDF

References


ACER; UNESCO Institute for Statistics. 2017. Principles of Good Practice in Learning Assessment. ACER, Melbourne.

Aiken, L. R. 1985. Three coefficients for analyzing the reliability and validity of ratings. Educational and Psychological Measurement, 45(1):131-142. https://doi.org/10.1177/0013164485451012

Apriyani, N. Ariani, T. & Arini, W. 2020. Pengembangan Modul Fisika Berbasis Discovery Learning pada Materi Fluida Statis Siswa Kelas XI SMA Negeri 1 Lubuklinggau Tahun Pelajaran 2019/2020. Silampari Jurnal Pendidikan Ilmu Fisika, 2(1): 41-54. https://doi.org/10.31540/sjpif.v2i1

Arends, R. I. 2012. Learning to Teach (9th Edition). McGraw-Hill Education, New York

Arifin, Z. 2016. Evaluasi Pembelajaran: Prinsip, Teknik, dan Prosedur. PT Remaja Rosdakarya, Bandung

Arikunto, S. 2012. Dasar-dasar Evaluasi Pendidikan. Bumi Aksara, Jakarta

Bashooir, K., & Supahar. 2018. Validitas dan Reliabilitas Instrumen Asesmen Kinerja Literasi Sains Pelajaran Fisika Berbasis STEM. Jurnal Penelitian dan Evaluasi Pendidikan, 22(2): 219-230. http://dx.doi.org/10.21831/pep.v22i2.19590

Bechard, E., Clark, A., Romine, R. S., Karvonen, M., Neal, K., Erickson, K. 2019. Use of Evidence-Centered Design to Develop Learning Maps-Based Assessments. International Journal of Testing, 19(2): 188-205. http://dx.doi.org/10.1080/15305058.2018.1543310

Bhakti, Y. B., Arthur, R., & Supriyati, Y. 2023. Development of an assessment instrument for critical thinking skills in physics: a systematic literature review. Journal of Physics: Conference Series, 2596(1): 012067. http://dx.doi.org/10.1088/1742-6596/2596/1/012067

Branch, RM. 2009. ‘Instructional Design: The ADDIE Approach’. Springer, United States of America.

Brookhart, S. M. 2010. How to Assess Higher-Order Thinking Skills in Your Classroom. VA: ASCD, Alexandria.

Brookhart, S. M. 2013. How to Create and Use Rubrics for Formative Assessment and Grading. ASCD, Alexandria.

Crawford, B. A., & Capps, D. K. 2018. Teacher Cognition of Engaging Children in Scientific Practices. In Y. Dori, Z. Mevarech, & D. R. Baker (Eds.), Cognition, metacognition, and culture in STEM education: Learning, teaching and assessment (Vol. 24, Ser. Cognition, Metacognition, and Culture in STEM Education. Innovations in Science Education and Technology). Springer International Publishing AG. 24. https://doi.org/10.1007/978-3-319-66659-4_2

Duschl, R. A. & Bybee, R. W. 2014. Planning and carrying out investigations: An entry to learning and to teacher professional development around NGSS science and engineering practices. International Journal of STEM Education, 1(12): 1-9. https://doi.org/10.1186/s40594-014-0012-6

Ellianawati, E., Mufiatunnikmah, S., Setyaningsih, N., & Subali, B. 2020. Asesmen Multi Representasi Berbasis Keterampilan Abad Ke-21 pada Materi Gerak Lurus. Physics Education Research Journal, 2(1): 19-34. https://doi.org/10.21580/perj.2020.2.1.5038

Giancoli, D. C. 2001. Fisika (5th ed.). Erlangga, Jakarta.

Harris, C. J., Krajcik, S. C., Pellegrino, J. W., & DeBarger, A. H. 2019. Designing Knowledge-In-Use Assessments to Promote Deeper Learning. Educational Measurement: Issues and Practice, 38(2): 53-67. https://doi.org/10.1111/emip.12253

Henderson, R. 2024. Redesigning Legacy Conceptual Inventories: Using Evidence-Centered Design to Develop Valid, Equitable, and Flexible Items for Conceptual Understanding. PERC 2024.

Jufriadi, A., Sutopo., Kusairi, S., & Sunaryono. 2023. Assessment of kinematic concepts comprehension: A systematic review. International Journal of Evaluation and Research in Education, 12(3):1449-1459. http://doi.org/10.11591/ijere.v12i3.24546

Kemendikbud. 2017. Panduan Penilaian oleh Pendidik dan Satuan Pendidikan pada Sekolah Dasar dan Sekolah Menengah Pertama. Direktorat Pembinaan Sekolah Dasar, Jakarta.

Kemendikburistek, 2022. Pembelajaran dan Asesmen: Pendidikan Anak Usia Dini, Pendidikan Dasar, dan Menengah. Badan Standar, Kurikulum, dan Asesmen Pendidikan, Jakarta

Klein, P., Muller, A., & Kuhn, J. 2017. Assessment of representational competence in kinematics. Physical Review Physics Education Research, 13(1): 010132, https://doi.org/10.1103/PhysRevPhysEducRes.13.010132

Lager, A., & Lavonen, J. 2023. Engaging Students in Scientific Practices in a Remote Setting. Education Sciences: 13(5):431. https://doi.org/10.3390/educsci13050431

Laverty, J. T., Underwood, S. M., Matz, R. L., Posey, L. A., Carmel, J. H., Caballero, M. D., Fata-Hartley, C. L., Ebert-May, D., Jardeleza, S. E., & Cooper, M. M. 2016. Characterizing College Science Assessments: The Three-Dimensional Learning Assessment Protocol. PloS one, 11(9), e0162333. https://doi.org/10.1371/journal.pone.0162333

Mislevy, R. J. & Haertel, G.D. 2007. Implications of Evidence-Centered Design for Educational Testing. Educational Measurement: Issues and Practice, 25(4): 6-20. https://doi.org/10.1111/j.1745-3992.2006.00075.x

Mislevy, R. J., Behrens, J. T., Dicerbo, K. E. & Levy, R. 2012. Design and Discovery in Educational Assessment: Evidence-Centered Design, Psychometrics, and Educational Data Mining. Journal of Educational Data Mining, 4(1), 11–48, DOI: https://doi.org/10.5281/zenodo.3554641

Mislevy, R. J., Steinberg, L. S., & Almond, R. G. 2003. Focus Article: On the Structure of Educational Assessments. Measurement: Interdisciplinary Research and Perspectives, 1(1), 3–62. https://doi.org/10.1207/S15366359MEA0101_02

Müftüoğlu, C. T., & Sahin, M. 2024. Framework of Assessment Design Based on Evidence-Centered Design for Assessment Analytics. In: Sahin, M., Ifenthaler, D. (eds) Assessment Analytics in Education. Advances in Analytics for Learning and Teaching. Springer, Cham. https://doi.org/10.1007/978-3-031-56365-2_8

Newton, S., Alemdar, M., Rutstein, D., Edwards, D., Helms, M., Hernandez, D., & Usselman, M. 2021. Utilizing Evidence-Centered Design to Develop Assessments: A High School Introductory Computer Science Course. Frontiers in Education, 6(June), 1–19.

Pellegrino, J. W., DiBello, L. V., & Goldman, S. R. 2016. A Framework for Conceptualizing and Evaluating the Validity of Instructionally Relevant Assessments. Educational Phychologist, 51(1): 59-81. http://dx.doi.org/10.1080/00461520.2016.1145550

Pellegrino, J. W., Wilson, M. R., Koenig, J. A., & Beatty, A. S. 2014. Developing assessments for the Next Generation Science Standards. National Academies Press, USA.

Poutot, G., & Blandin, B. 2015. Exploration of Students’ Misconceptions in Mechanics using the FCI. American Journal of Educational Research. 3(2):116-120. http://dx.doi.org/10.12691/education-3-2-2

Rahmadani, U., Sundari, PD, Hidayati, H., & Dewi, WS. 2023. Tinjauan Sistematis Kesalahpahaman dalam Kinematika: Identifikasi, Penyebab, dan Remediasi. Jurnal Penelitian Pendidikan IPA , 9 (12): 1274–1283. https://doi.org/10.29303/jppipa.v9i12.4956

Santrock, J. W. 2011. Educational Psychology. McGraw-Hill Education, Amerika Serikat.

Saribas, D., & Ceyhan, G. D. 2015. Learning to Teach Scientific Prcatice: Pedagogical Decisions and Reflections During a Course for Pre-Service Science Teacher. International Journal of STEM Education, (2)7: 1-13. https://doi.org/10.1186/s40594-015-0023-y

Sirait, J. 2024. Merancang Asesmen Fisika Menggunakan Evidence-Centered Design (ECD) dan Three-Dimensional Learning Protocol (3D-LAP). Jurnal Pendidikan Informatika dan Sains, 13(2):165-175. https://doi.org/10.31571/saintek.v13i2.8228

Sirait, J. & Oktavianty, E. 2021. Pengembangan dan Validasi Angket Motivasi Belajar Fisika (AMBF): Studi Pilot. Jurnal Ilmiah Pendidikan Fisika. 5(3):305-316. https://doi.org/10.20527/jipf.v5i3.3829

Sirait, J., Firdaus, Hidayatullah, M.M.S., & Habellia, R.C. 2023. Development and validation of force test to assess physics education students’ representational competence. Jurnal Pendidikan Sains Indonesia (Indonesian Journal of Science Education), 11(1):306-317. https://dx.doi.org/10.24815/jpsi.v11i2.28178

Sirait, J., Ricky, P., & Hamdani. 2023. Development and Validation of the Force Diagrams Representation Test. JPPPF (Jurnal Penelitian dan Pengembangan Pendidikan Fisika). 9(2):183-196. https://doi.org/10.21009/1.09201

Steinkuehler, C. 2011. Learning Science Through Computer Games and Simulations. National Academies Press, USA.

Stoen, S. M., McDaniel, M. A., Frey. R. F., Hynes, K. M., & Cahill, M. J. 2020. Force Concept Inventory: More than just conceptual understanding. Physical Review Physics Education Research, 16(1): 010105. https://dOi.org/10.1103/PhysRevPhysEducRes.16.010105

Sudijono, A. 2016. Pengantar Evaluasi Pendidikan. Rajawali Pers, Jakarta.

Sugiyono. 2017. Metodologi Penelitian Kuantitatif, Kualitatif, dan R&D. Alfabeta, Bandung

Weisdiyanti, N., & Juliani, R. 2023. Development of Critical Thinking Skills-Based Physics Test Instruments for High School. Proceedings of the 4th International Conference on Science and Technology Applications, ICoSTA 2022, 2022 November, Medan, North Sumatera Province, Indonesia.

Widodo, K. Handoyo, E., & Masyhar, A. 2021. Performance of Educational Assessments: Integrated Assessment as an Assessment Innovation during the Covid-19 Pandemic. Turkish Journal of Computer and Mathematics Education, 12(6): 2708-2718. https://doi.org/10.17762/turcomat.v12i6.5777

Windschitl, M., Thompson, J., & Braaten, M. 2018. Ambitious Science Teaching. Harvard Education Press, London

Yuliantaningrum, L. & Sunarti, T. 2020. Pengembangan Instrumen Soal HOTS untuk Mengukur Keterampilan Berpikir Kritis, Berpikir Kreatif, dan Pemecahan Masalah Materi Gerak Lurus pada Peserta Didik SMA. IPF: Inovasi Pendidikan Fisika, 9(2):75-82. http://dx.doi.org/10.30872/diglosia.v5i2.414




DOI: https://doi.org/10.24815/jipi.v9i1.42940

Article Metrics

Abstract view : 18 times
PDF - 14 times

Refbacks

  • There are currently no refbacks.


Copyright (c) 2025 Jurnal IPA & Pembelajaran IPA



JIPI (Jurnal IPA dan Pembelajaran IPA)

ISSN 2614-0500  (print) | 2620-553X (online)
Organized by Universitas Syiah Kuala 
Published by Master of Science Education Study Program Graduate School Universitas Syiah Kuala, Banda Aceh, Indonesia
Website : http://jurnal.usk.ac.id/jipi
Email     : jipi@usk.ac.id

Creative Commons License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.