01483nas a2200157 4500008003900000245004600039210004600085300001200131490000700143520104600150100001901196700002601215700001201241700001901253856005301272 2016 d00aOptimal Reassembly of Shadow Tests in CAT0 aOptimal Reassembly of Shadow Tests in CAT a469-4850 v403 aEven in the age of abundant and fast computing resources, concurrency requirements for large-scale online testing programs still put an uninterrupted delivery of computer-adaptive tests at risk. In this study, to increase the concurrency for operational programs that use the shadow-test approach to adaptive testing, we explored various strategies aiming for reducing the number of reassembled shadow tests without compromising the measurement quality. Strategies requiring fixed intervals between reassemblies, a certain minimal change in the interim ability estimate since the last assembly before triggering a reassembly, and a hybrid of the two strategies yielded substantial reductions in the number of reassemblies without degradation in the measurement accuracy. The strategies effectively prevented unnecessary reassemblies due to adapting to the noise in the early test stages. They also highlighted the practicality of the shadow-test approach by minimizing the computational load involved in its use of mixed-integer programming.1 aChoi, Seung, W1 aMoellering, Karin, T.1 aLi, Jie1 aLinden, Wim, J uhttp://apm.sagepub.com/content/40/7/469.abstract01559nas a2200169 4500008003900000022001400039245007700053210006900130300001300199490000700212520105800219100002101277700001401298700001901312700001701331856004101348 2015 d a1745-398400aAssessing Individual-Level Impact of Interruptions During Online Testing0 aAssessing IndividualLevel Impact of Interruptions During Online  a80–1050 v523 aWith an increase in the number of online tests, the number of interruptions during testing due to unexpected technical issues seems to be on the rise. For example, interruptions occurred during several recent state tests. When interruptions occur, it is important to determine the extent of their impact on the examinees' scores. Researchers such as Hill and Sinharay et al. examined the impact of interruptions at an aggregate level. However, there is a lack of research on the assessment of impact of interruptions at an individual level. We attempt to fill that void. We suggest four methodological approaches, primarily based on statistical hypothesis testing, linear regression, and item response theory, which can provide evidence on the individual-level impact of interruptions. We perform a realistic simulation study to compare the Type I error rate and power of the suggested approaches. We then apply the approaches to data from the 2013 Indiana Statewide Testing for Educational Progress-Plus (ISTEP+) test that experienced interruptions.1 aSinharay, Sandip1 aWan, Ping1 aChoi, Seung, W1 aKim, Dong-In uhttp://dx.doi.org/10.1111/jedm.1206401617nas a2200193 4500008003900000022001400039245007400053210006900127300001400196490000700210520105700217100002101274700001401295700001901309700001701328700001801345700001901363856004101382 2014 d a1745-398400aDetermining the Overall Impact of Interruptions During Online Testing0 aDetermining the Overall Impact of Interruptions During Online Te a419–4400 v513 a

With an increase in the number of online tests, interruptions during testing due to unexpected technical issues seem unavoidable. For example, interruptions occurred during several recent state tests. When interruptions occur, it is important to determine the extent of their impact on the examinees’ scores. There is a lack of research on this topic due to the novelty of the problem. This article is an attempt to fill that void. Several methods, primarily based on propensity score matching, linear regression, and item response theory, were suggested to determine the overall impact of the interruptions on the examinees’ scores. A realistic simulation study shows that the suggested methods have satisfactory Type I error rate and power. Then the methods were applied to data from the Indiana Statewide Testing for Educational Progress-Plus (ISTEP+) test that experienced interruptions in 2013. The results indicate that the interruptions did not have a significant overall impact on the student scores for the ISTEP+ test.

1 aSinharay, Sandip1 aWan, Ping1 aWhitaker, Mike1 aKim, Dong-In1 aZhang, Litong1 aChoi, Seung, W uhttp://dx.doi.org/10.1111/jedm.1205201203nas a2200145 4500008003900000245005800039210005600097300001000153490000700163520077300170100001900943700002200962700002100984856005201005 2011 d00aA New Stopping Rule for Computerized Adaptive Testing0 aNew Stopping Rule for Computerized Adaptive Testing a37-530 v713 a

The goal of the current study was to introduce a new stopping rule for computerized adaptive testing (CAT). The predicted standard error reduction (PSER) stopping rule uses the predictive posterior variance to determine the reduction in standard error that would result from the administration of additional items. The performance of the PSER was compared with that of the minimum standard error stopping rule and a modified version of the minimum information stopping rule in a series of simulated adaptive tests, drawn from a number of item pools. Results indicate that the PSER makes efficient use of CAT item pools, administering fewer items when predictive gains in information are small and increasing measurement precision when information is abundant.

1 aChoi, Seung, W1 aGrady, Matthew, W1 aDodd, Barbara, G uhttp://epm.sagepub.com/content/71/1/37.abstract01148nas a2200133 4500008003900000245006700039210006700106300001200173490000700185520072700192100001900919700002300938856005300961 2009 d00aComparison of CAT Item Selection Criteria for Polytomous Items0 aComparison of CAT Item Selection Criteria for Polytomous Items a419-4400 v333 a

Item selection is a core component in computerized adaptive testing (CAT). Several studies have evaluated new and classical selection methods; however, the few that have applied such methods to the use of polytomous items have reported conflicting results. To clarify these discrepancies and further investigate selection method properties, six different selection methods are compared systematically. The results showed no clear benefit from more sophisticated selection criteria and showed one method previously believed to be superior—the maximum expected posterior weighted information (MEPWI)—to be mathematically equivalent to a simpler method, the maximum posterior weighted information (MPWI).

1 aChoi, Seung, W1 aSwartz, Richard, J uhttp://apm.sagepub.com/content/33/6/419.abstract