The Appropriate Sample Size for Test Length on Estimating Item Parameters in Item Response Theory

Abstract

An important drawback in item response theory (IRT) is that the sample size must be sufficiently large to accurately estimate item parameters. Therefore, to save budget, time, and manpower, it is necessary to determine the minimum sample size suitable for different test lengths. Thus, the purpose of this research was to compare the results of different test lengths and sample sizes on estimating item parameters in IRT. The data used in this study were secondary data from The National Institute of Educational Testing Service (Public Organization). These data consisted of results from individual exams in the O-NET test, grade nine, which included only multiple-choice exams with binary item scores. Three test lengths were considered: Mathematics with 20 items, Thai language with 40 items, and English language with 50 items. A sample size of 5,000 students was selected to represent the true item parameter values for each test. Six conditional sample sizes (200, 300, 400, 500, 700, and 1,000 students) were obtained through systematic random sampling. The data were analyzed using R and SPSS programs. To compare the test parameter estimates with the actual parameters, a 3-parameter model was employed, with the criterion that the correlation coefficient (r) should be ≥ .70 and the root mean square difference (RMSD) ≤ 0.33 across all parameters (a, b, c). The results showed that for a 20-item test length, a sample size greater than 1,000 is recommended. A minimum sample size of 700 is appropriate for a 40-item test length, and for a 50-item test length with a first component variance of less than 10%, a sample size greater than 1,000 is recommended.