•  
  •  
 

Jurnal Penelitian dan Evaluasi Pendidikan

Keywords

cut score; standard setting; generalizability theory

Document Type

Article

Abstract

Penelitian ini bertujuan untuk mengetahui metode yang lebih akurat dalam mengestimasi kesalahan pengukuran standard setting pada metode Ebel, Bookmark, dan Contrasting group. Data penelitian ini merupakan dokumen Dinas Pendidikan dan Kebudayaan Kabupaten Sumbawa berupa respon peserta Ujian Nasional Matematika Paket P0C5520 tahun ajaran 2015/2016 yang berjumlah 352 siswa. Guru juga dilibatkan dalam penelitian sebagai panelis dalam Focus Group Discussion (FGD). Data yang terkumpul kemudian dianalisis melaui tiga tahap.. Tahap pertama yaitu persiapan, kegiatan pada tahap ini mencakup penyiapan data, penggolongan SMP, dan penentuan karakteristik butir. Tahap kedua yaitu FGD dilakukan dalam dua putaran. Tahap ketiga yaitu mengestimasi kesalahan pengukuran dengan menggunakan pendekatan Generalizability Theory dengan bantuan program eduG. Hasil penelitian menunjukkan bahwa metode Contrasting group memiliki estimasi kesalahan pengukuran paling kecil dibandingkan metode Ebel dan Bookmark, oleh karena itu, metode Contrasting grouplebih akurat dibandingkan dengan dua metode lainnya.

Kata kunci: cut score, standard setting, generalizability theory

MEASUREMENT ERROR ESTIMATION OF STANDARD SETTING IN MATHEMATICS COMPETENCY ASSESSMENT FOR JUNIOR HIGH SCHOOL IN SUMBAWA REGENCY

Abstract

This research aims to find the most accurate methods in estimating measurement error of standard setting among Ebel, Bookmark, and Contrasting group methods. The data used in this study were 352 students' responses on Mathematics National Exam Package P0C5520 in the academic year of 2015/2016. The document was collected from the Department of Education and Culture in Sumbawa Regency. Teachers were also involved in this research as panelists in the Focus Group Discussion (FGD). The data collected were then analyzed through three stages. The first stage was preparation stage, including the activities of data preparation, school classification, and item characteristics analysis. The second stage was two-round FGD. The third stage was estimating the measurement error using Generalizability Theory approach assisted by eduG program. The research result shows that Contrasting Group method produces the smallest measurement error estimation compared to Ebel and Bookmark methods, therefore, Contrasting group method is considered as the most accurate method. Keywords: cut score, standard setting, generalizability theory

First Page

24

Last Page

34

Issue

1

Volume

22

Digital Object Identifier (DOI)

10.21831/pep.v22i1.16492

References

Alsmadi, A. A. (2007). A comparative studi of two standard-setting technique. Social Behavior and Personality, 38(4), 479-486.

Berk, R. A. (1986). A Consumer's guide to setting performance standards on criterion-referenced tests. Review of Educational Research, 56(1), 137-172. https://doi.org/10.3102/00346543056001137

Cizek, G. J. (1996). An NCME instructional module on: setting passing scores. Educational Measurement: Issues and Practice, 15(2), 20-31. https://doi.org/10.1111/j.1745-3992.1996.tb00809.x

Cizek, G. J., & Bunch, M. B. (2007). Standard setting: a guide to establishing and evaluating performance standards for tests. California: Sage Publication, Inc.

Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. New York: CBS College Publishing.

Glass, G. V. (1978). Standards and criteria. Journal of Educational Measurement, 15(4), 237-261. https://doi.org/10.1111/j.1745-3984.1978.tb00072.x

Karantonis, A., & Sireci, S. G. (2006). The bookmark standard-setting method: a literature review. Educational Measurement: Issues and Practice, 25(1), 4-12. https://doi.org/10.1111/j.1745-3992.2006.00047.x

Lee, G., & Lewis, D. M. (2008). A Generalizability theory approach to standard error estimates for bookmark standard settings. Educational and Psychological Measurement, 68(4), 603-620. https://doi.org/10.1177/0013164407312603

Livingstone, S. A., & Zieky, M. J. (1982). Passing scores: a manual for setting standards of performance on educational and occupational tests. Princeton, New Jersey: Educational Testing Service.

MacCann, R. G., & Stanley, G. (2006). The use of rasch modeling to improve standard setting. Practical Assessment, Research & Evaluation, 11(2), 1 - 17.

Menteri Pendidikan dan Kebudayaan Republik Indonesia. Peraturan Menteri Pendidikan dan Kebudayaan Republik Indonesia Nomor 5 Tahun 2015 tentang Kriteria Kelulusan Peserta Didik, Penyelenggaraan Ujian Nasional, dan Penyelenggaraan Ujian Sekolah/Madrasah/Pendidikan Kesetaraan Pada Smp/Mts atau yang Sederajat d (2015).

Nichols, P., Twing, J., Mueller, C. D., & O'Malley, K. (2010). Standard-setting methods as measurement processes. Educational Measurement: Issues and Practice, 29(1), 14-24. https://doi.org/10.1111/j.1745-3992.2009.00166.x

Nudell, H. (2008). Making the cut score, that is establishing a pass/fail score is a highly technical process. ICSC Certified Professionals Newsletter.

Prijowuntato, S. W., Mardapi, D., & Budiyono, B. (2015). Perbandingan estimasi kesalahan pengukuran standard setting dalam penilaian kompetensi akuntansi SMK. Jurnal Penelitian Dan Evaluasi Pendidikan, 19(2). Retrieved from https://journal.uny.ac.id/index.php/jpep/article/view/5578

Retnawati, H. (2014). Teori respon butir dan penerapannya. Yogyakarta: Nuha Medika.

Saunders, J. C., Ryan, J. P., & Huynh, H. (1980). A comparison of two ways of setting passing scores based on the nedelsky procedure. Publication Series in Mastery Testing. South Carolina: University of South Carolina.

Yin, P., & Sconing, J. (2008). Estimating standard errors of cut scores for item rating and mapmark procedure: a generalizability theory approach. Educational and Psychological Measurement, 68(1), 182-197.

Zieky, M. J., Perie, M., & Livingston, S. A. (2008). Cutscores: a manual for setting standards of performance on educational and occupational tests. Princeton, New Jersey: Educational Testing Service.

Download

Share

COinS