REID (Research and Evaluation in Education)
Keywords
item selection strategy, item exposure, step-size, adaptive testing
Document Type
Article
Abstract
One of the ability estimation methods that is widely applied to the Computerized Adaptive Testing (CAT) algorithm is the maximum likelihood estimation (MLE). However, the maximum likelihood method has the disadvantage of being unable to find a solution to the ability estimation of test-takers when the test takers' scores do not have a pattern. If there are test takers who get either score of 0 or perfect score, then the abilities of test-takers are usually estimated using the step-size model. However, the step-size model often results in item exposure where certain items will appear more often than other items. This surely threatens the security of the test because items that often appear will be easier to recognize. This study tries to provide an alternative strategy by modifying the step-size model and randomizing the calculation results of the information function obtained. Based on the results of the study, it is found that alternative strategies for item selection can make more varied items appear to improve the security of tests on the CAT.
Page Range
32-40
Issue
1
Volume
6
Digital Object Identifier (DOI)
10.21831/reid.v6i1.30508
Source
https://journal.uny.ac.id/index.php/reid/article/view/30508
Recommended Citation
Suhardi, I. (2020). Alternative item selection strategies for improving test security in computerized adaptive testing of the algorithm. REID (Research and Evaluation in Education), 6(1), 32-40. https://doi.org/10.21831/reid.v6i1.30508
References
Baker, F. B. (1992). Item response theory: Parameter estimation techniques. New York, NY: Marcel Dekker.
Birnbaum, A. (1968). Some latent trait models and their uses in inferring an examinee's ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental rest scores (pp. 397-479). Reading, MA: Addison-Wesley.
Dodd, B. G. (1990). The effect of item selection procedure and stepsize on computerized adaptive attitude measurement using the rating scale model. Applied Psychological Measurement, 14(4), 355-366. https://doi.org/10.1177/014662169001400403
Eignor, D. R., Stocking, M. L., Way, W. D., & Steffen, M. (1993). Case studies in computer adaptive test design through simulation. https://doi.org/10.1002/j.2333-8504.1993.tb01567.x
Grist, S. (1989). Computerized adaptive tests. In ERIC Digest No. 107. Retrieved from https://files.eric.ed.gov/fulltext/ED315425.pdf
Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston, MA: Kluwer Nijhoff.
Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage Publications.
Haryanto, H. (2013). Pengembangan computerized adaptive testing (CAT) dengan algoritma logika Fuzzy. Jurnal Penelitian Dan Evaluasi Pendidikan, 15(1), 47-70. https://doi.org/10.21831/pep.v15i1.1087
Higgins, P. (2009). Candidate measured ability and use of time. Retrieved from https://www.rasch.org/mra/mra-10-09.htm
Lord, Frederic M. (1977). A broad-range tailored test of verbal ability. Applied Psychological Measurement, 1(1), 95-100. https://doi.org/10.1177/014662167700100115
Martinez, L. (2009). Time usage and candidate performance. Retrieved from http://www.rasch.org/mra/mra-06-09.htm
McBride, J. R., & Martin, J. T. (1983). Reliability and validity of adaptive ability tests in a military setting. In D. J. Weiss (Ed.), New horizons in testing: Latent trait test theory and computerized adaptive testing (pp. 224-236). New York, NY: Academic Press.
Mills, C. N. (1999). Development and introduction of a computer adaptive Graduate Record Examinations General Test. In F. Drasgow & J. B. Olson-Buchanan (Eds.), Innovations in computerized assessment (pp. 117-135). Mahwah, NJ: Lawrence Erlbaum Associates.
Pressman, R. S. (2001). Software engineering: A practitioner's approach (5th ed.). New York, NY: McGraw-Hill Higher Education.
Rudner, L. M. (1998). An on-line, interactive, computer adaptive testing tutorial. Retrieved from http://edres.org/scripts/cat
Santoso, A. (2010). Pengembangan computerized adaptive testing untuk mengukur hasil belajar mahasiswa Universitas Terbuka. Jurnal Penelitian Dan Evaluasi Pendidikan, 14(1), 62-83. https://doi.org/10.21831/pep.v14i1.1976
Thissen, D. (1990). Reliability and measurement precision. In H. Wainer, N. J. Dorans, R. Flaugher, B. F. Green, R. J. Mislevy, L. Steinberg, & D. Thissen (Eds.), Computerized adaptive testing: A primer (2nd ed., pp. 161-186). Hillsdale, NJ: Erlbaum.
Vispoel, W. P. (1999). Creating computerized adaptive tests of music aptitude: Problems, solutions, and future directions. In F. Drasgow & J. B. Olson-Buchanan (Eds.), Innovations in computerized assessment (pp. 151-176). Mahwah, NJ: Lawrence Erlbaum Associates.
Winarno, W. (2013). Pengembangan computerized adaptive testing (CAT) menggunakan metode pohon segitiga keputusan. Jurnal Penelitian Dan Evaluasi Pendidikan, 16(2), 574-592. https://doi.org/10.21831/pep.v16i2.1132