01294nas a2200133   4500008004100000245005700041210005700098300000900155490000800164520084700172653003401019100002301053856008401076       2008                        eng d00aSome new developments in adaptive testing technology0 aSome new developments in adaptive testing technology  a3-110 v2163 a<p>In an ironic twist of history, modern psychological testing has returned to an adaptive format quite common when testing was not yet standardized. Important stimuli to the renewed interest in adaptive testing have been the development of item-response theory in psychometrics, which models the responses on test items using separate parameters for the items and test takers, and the use of computers in test administration, which enables us to estimate the parameter for a test taker and select the items in real time. This article reviews a selection from the latest developments in the technology of adaptive testing, such as constrained adaptive item selection, adaptive testing using rule-based item generation, multidimensional adaptive testing, adaptive use of test batteries, and the use of response times in adaptive testing.</p>
10acomputerized adaptive testing1 avan der Linden, WJ  uhttp://mail.iacat.org/content/some-new-developments-adaptive-testing-technology01964nas a2200265   4500008004100000020002200041245008200063210006900145260002600214300001000240490000700250520112900257653001501386653003401401653001401435653001701449653002401466653001501490653002201505653001501527100002301542700001301565700001801578856010201596       2006                        eng d  a1076-9986 (Print)00aAssembling a computerized adaptive testing item pool as a set of linear tests0 aAssembling a computerized adaptive testing item pool as a set of  bSage Publications: US  a81-990 v313 aTest-item writing efforts typically results in item pools with an undesirable correlational structure between the content attributes of the items and their statistical information. If such pools are used in computerized adaptive testing (CAT), the algorithm may be forced to select items with less than optimal information, that violate the content constraints, and/or have unfavorable exposure rates. Although at first sight somewhat counterintuitive, it is shown that if the CAT pool is assembled as a set of linear test forms, undesirable correlations can be broken down effectively. It is proposed to assemble such pools using a mixed integer programming model with constraints that guarantee that each test meets all content specifications and an objective function that requires them to have maximal information at a well-chosen set of ability values. An empirical example with a previous master pool from the Law School Admission Test (LSAT) yielded a CAT with nearly uniform bias and mean-squared error functions for the ability estimator and item-exposure rates that satisfied the target for all items in the pool. 10aAlgorithms10acomputerized adaptive testing10aitem pool10alinear tests10amathematical models10astatistics10aTest Construction10aTest Items1 avan der Linden, WJ1 aAriel, A1 aVeldkamp, B P  uhttp://mail.iacat.org/content/assembling-computerized-adaptive-testing-item-pool-set-linear-tests01585nas a2200205   4500008004100000020002200041245005000063210005000113260002600163300001200189490000700201520094200208653003401150653002801184653001901212653002001231653003301251100002301284856007201307       2006                        eng d  a0146-6216 (Print)00aEquating scores from adaptive to linear tests0 aEquating scores from adaptive to linear tests  bSage Publications: US  a493-5080 v303 aTwo local methods for observed-score equating are applied to the problem of equating an adaptive test to a linear test. In an empirical study, the methods were evaluated against a method based on the test characteristic function (TCF) of the linear test and traditional equipercentile equating applied to the ability estimates on the adaptive test for a population of test takers. The two local methods were generally best. Surprisingly, the TCF method performed slightly worse than the equipercentile method. Both methods showed strong bias and uniformly large inaccuracy, but the TCF method suffered from extra error due to the lower asymptote of the test characteristic function. It is argued that the worse performances of the two methods are a consequence of the fact that they use a single equating transformation for an entire population of test takers and therefore have to compromise between the individual score distributions. 10acomputerized adaptive testing10aequipercentile equating10alocal equating10ascore reporting10atest characteristic function1 avan der Linden, WJ  uhttp://mail.iacat.org/content/equating-scores-adaptive-linear-tests02153nas a2200229   4500008004100000020002200041245008700063210006900150260004100219300001200260490000700272520135500279653002101634653001501655653002401670653002601694653002501720653002101745653003101766100002301797856010301820       2005                        eng d  a0022-0655 (Print)00aA comparison of item-selection methods for adaptive tests with content constraints0 acomparison of itemselection methods for adaptive tests with cont  bBlackwell Publishing: United Kingdom  a283-3020 v423 aIn test assembly, a fundamental difference exists between algorithms that select a test sequentially or simultaneously. Sequential assembly allows us to optimize an objective function at the examinee's ability estimate, such as the test information function in computerized adaptive testing. But it leads to the non-trivial problem of how to realize a set of content constraints on the test—a problem more naturally solved by a simultaneous item-selection method. Three main item-selection methods in adaptive testing offer solutions to this dilemma. The spiraling method moves item selection across categories of items in the pool proportionally to the numbers needed from them. Item selection by the weighted-deviations method (WDM) and the shadow test approach (STA) is based on projections of the future consequences of selecting an item. These two methods differ in that the former calculates a projection of a weighted sum of the attributes of the eventual test and the latter a projection of the test itself. The pros and cons of these methods are analyzed. An empirical comparison between the WDM and STA was conducted for an adaptive version of the Law School Admission Test (LSAT), which showed equally good item-exposure rates but violations of some of the constraints and larger bias and inaccuracy of the ability estimator for the WDM.10aAdaptive Testing10aAlgorithms10acontent constraints10aitem selection method10ashadow test approach10aspiraling method10aweighted deviations method1 avan der Linden, WJ  uhttp://mail.iacat.org/content/comparison-item-selection-methods-adaptive-tests-content-constraints01613nas a2200217   4500008004100000020002200041245008200063210006900145260004300214300001200257490000700269520084600276653003401122653002601156653003501182653001601217653001701233100002301250700001801273856010401291       2004                        eng d  a1076-9986 (Print)00aConstraining item exposure in computerized adaptive testing with shadow tests0 aConstraining item exposure in computerized adaptive testing with  bAmerican Educational Research Assn: US  a273-2910 v293 aItem-exposure control in computerized adaptive testing is implemented by imposing item-ineligibility constraints on the assembly process of the shadow tests. The method resembles Sympson and Hetter’s (1985) method of item-exposure control in that the decisions to impose the constraints are probabilistic. The method does not, however, require time-consuming simulation studies to set values for control parameters before the operational use of the test. Instead, it can set the probabilities of item ineligibility adaptively during the test using the actual item-exposure rates. An empirical study using an item pool from the Law School Admission Test showed that application of the method yielded perfect control of the item-exposure rates and had negligible impact on the bias and mean-squared error functions of the ability estimator. 10acomputerized adaptive testing10aitem exposure control10aitem ineligibility constraints10aProbability10ashadow tests1 avan der Linden, WJ1 aVeldkamp, B P  uhttp://mail.iacat.org/content/constraining-item-exposure-computerized-adaptive-testing-shadow-tests01908nas a2200217   4500008004100000020002200041245007000063210006900133260004100202300001200243490000700255520117100262653003201433653003301465653001801498653002401516100001301540700001801553700002301571856009601594       2004                        eng d  a0022-0655 (Print)00aConstructing rotating item pools for constrained adaptive testing0 aConstructing rotating item pools for constrained adaptive testin  bBlackwell Publishing: United Kingdom  a345-3590 v413 aPreventing items in adaptive testing from being over- or underexposed is one of the main problems in computerized adaptive testing. Though the problem of overexposed items can be solved using a probabilistic item-exposure control method, such methods are unable to deal with the problem of underexposed items. Using a system of rotating item pools, on the other hand, is a method that potentially solves both problems. In this method, a master pool is divided into (possibly overlapping) smaller item pools, which are required to have similar distributions of content and statistical attributes. These pools are rotated among the testing sites to realize desirable exposure rates for the items. A test assembly model, motivated by Gulliksen's matched random subtests method, was explored to help solve the problem of dividing a master pool into a set of smaller pools. Different methods to solve the model are proposed. An item pool from the Law School Admission Test was used to evaluate the performances of computerized adaptive tests from systems of rotating item pools constructed using these methods. (PsycINFO Database Record (c) 2007 APA, all rights reserved)10acomputerized adaptive tests10aconstrained adaptive testing10aitem exposure10arotating item pools1 aAriel, A1 aVeldkamp, B P1 avan der Linden, WJ  uhttp://mail.iacat.org/content/constructing-rotating-item-pools-constrained-adaptive-testing01592nas a2200145   4500008004100000245005200041210005200093300001200145490000700157520113200164653003401296100001601330700002301346856007701369       2003                        eng d00aComputerized adaptive testing with item cloning0 aComputerized adaptive testing with item cloning  a247-2610 v273 a(from the journal abstract) To increase the number of items available for adaptive testing and reduce the cost of item writing, the use of techniques of item cloning has been proposed. An important consequence of item cloning is possible variability between the item parameters. To deal with this variability, a multilevel item response (IRT) model is presented which allows for differences between the distributions of item parameters of families of item clones. A marginal maximum likelihood and a Bayesian procedure for estimating the hyperparameters are presented. In addition, an item-selection procedure for computerized adaptive testing with item cloning is presented which has the following two stages: First, a family of item clones is selected to be optimal at the estimate of the person parameter. Second, an item is randomly selected from the family for administration. Results from simulation studies based on an item pool from the Law School Admission Test (LSAT) illustrate the accuracy of these item pool calibration and adaptive testing procedures. (PsycINFO Database Record (c) 2003 APA, all rights reserved).10acomputerized adaptive testing1 aGlas, C A W1 avan der Linden, WJ  uhttp://mail.iacat.org/content/computerized-adaptive-testing-item-cloning01437nas a2200205   4500008004100000245008800041210007000129300001200199490000700211520067700218653002100895653003000916653002400946653002500970653002600995653005201021100001901073700002301092856011601115       2003                        eng d00aOptimal stratification of item pools in α-stratified computerized adaptive testing0 aOptimal stratification of item pools in αstratified computerized  a262-2740 v273 aA method based on 0-1 linear programming (LP) is presented to stratify an item pool optimally for use in α-stratified adaptive testing. Because the 0-1 LP model belongs to the subclass of models with a network flow structure, efficient solutions are possible. The method is applied to a previous item pool from the computerized adaptive testing (CAT) version of the Graduate Record Exams (GRE) Quantitative Test. The results indicate that the new method performs well in practical situations. It improves item exposure control, reduces the mean squared error in the θ estimates, and increases test reliability. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10aAdaptive Testing10aComputer Assisted Testing10aItem Content (Test)10aItem Response Theory10aMathematical Modeling10aTest Construction computerized adaptive testing1 aChang, Hua-Hua1 avan der Linden, WJ  uhttp://mail.iacat.org/content/optimal-stratification-item-pools-%CE%B1-stratified-computerized-adaptive-testing01521nas a2200157   4500008004100000245009500041210006900136300001200205490000700217520090100224653002101125653003001146653004501176100002301221856011901244       2003                        eng d00aSome alternatives to Sympson-Hetter item-exposure control in computerized adaptive testing0 aSome alternatives to SympsonHetter itemexposure control in compu  a249-2650 v283 aTheHetter and Sympson (1997; 1985) method is a method of probabilistic item-exposure control in computerized adaptive testing. Setting its control parameters to admissible values requires an iterative process of computer simulations that has been found to be time consuming, particularly if the parameters have to be set conditional on a realistic set of values for the examinees’ ability parameter. Formal properties of the method are identified that help us explain why this iterative process can be slow and does not guarantee admissibility. In addition, some alternatives to the SH method are introduced. The behavior of these alternatives was estimated for an adaptive test from an item pool from the Law School Admission Test (LSAT). Two of the alternatives showed attractive behavior and converged smoothly to admissibility for all items in a relatively small number of iteration steps. 10aAdaptive Testing10aComputer Assisted Testing10aTest Items computerized adaptive testing1 avan der Linden, WJ  uhttp://mail.iacat.org/content/some-alternatives-sympson-hetter-item-exposure-control-computerized-adaptive-testing01517nas a2200229   4500008004100000245008700041210006900128300001200197490000700209520075300216653002100969653001300990653003001003653003401033653001101067653001501078653001501093653001801108100002301126700002701149856011101176       2003                        eng d00aUsing response times to detect aberrant responses in computerized adaptive testing0 aUsing response times to detect aberrant responses in computerize  a251-2650 v683 aA lognormal model for response times is used to check response times for aberrances in examinee behavior on computerized adaptive tests. Both classical procedures and Bayesian posterior predictive checks are presented. For a fixed examinee, responses and response times are independent; checks based on response times offer thus information independent of the results of checks on response patterns. Empirical examples of the use of classical and Bayesian checks for detecting two different types of aberrances in response times are presented. The detection rates for the Bayesian checks outperformed those for the classical checks, but at the cost of higher false-alarm rates. A guideline for the choice between the two types of checks is offered.10aAdaptive Testing10aBehavior10aComputer Assisted Testing10acomputerized adaptive testing10aModels10aperson Fit10aPrediction10aReaction Time1 avan der Linden, WJ1 aKrimpen-Stoop, E M L A  uhttp://mail.iacat.org/content/using-response-times-detect-aberrant-responses-computerized-adaptive-testing01694nas a2200277   4500008004100000020001000041245006500051210006400116260009700180300001100277520074500288653002101033653002201054653002501076653002801101653002501129653001601154653001801170653005501188653001501243653001201258100001801270700002301288700001301311856009201324       2002                        eng d  a02-0900aMathematical-programming approaches to test item pool design0 aMathematicalprogramming approaches to test item pool design  aTwente, The NetherlandsbUniversity of Twente, Faculty of Educational Science and Technology  a93-1083 a(From the chapter) This paper presents an approach to item pool design that has the potential to improve on the quality of current item pools in educational and psychological testing and hence to increase both measurement precision and validity. The approach consists of the application of mathematical programming techniques to calculate optimal blueprints for item pools. These blueprints can be used to guide the item-writing process. Three different types of design problems are discussed, namely for item pools for linear tests, item pools computerized adaptive testing (CAT), and systems of rotating item pools for CAT. The paper concludes with an empirical example of the problem of designing a system of rotating item pools for CAT.10aAdaptive Testing10aComputer Assisted10aComputer Programming10aEducational Measurement10aItem Response Theory10aMathematics10aPsychometrics10aStatistical Rotation computerized adaptive testing10aTest Items10aTesting1 aVeldkamp, B P1 avan der Linden, WJ1 aAriel, A  uhttp://mail.iacat.org/content/mathematical-programming-approaches-test-item-pool-design01261nas a2200145   4500008004100000245006500041210006500106300001000171490000700181520076500188653003400953100002300987700001601010856008901026       2000                        eng d00aCapitalization on item calibration error in adaptive testing0 aCapitalization on item calibration error in adaptive testing  a35-530 v133 a(from the journal abstract) In adaptive testing, item selection is sequentially optimized during the test. Because the optimization takes place over a pool of items calibrated with estimation error, capitalization on chance is likely to occur. How serious the consequences of this phenomenon are depends not only on the distribution of the estimation errors in the pool or the conditional ratio of the test length to the pool size given ability, but may also depend on the structure of the item selection criterion used. A simulation study demonstrated a dramatic impact of capitalization on estimation errors on ability estimation. Four different strategies to minimize the likelihood of capitalization on error in computerized adaptive testing are discussed.10acomputerized adaptive testing1 avan der Linden, WJ1 aGlas, C A W  uhttp://mail.iacat.org/content/capitalization-item-calibration-error-adaptive-testing01091nas a2200133   4500008004100000245007800041210006900119300001200188490000700200520059200207653003400799100002300833856010100856       1999                        eng d00aMultidimensional adaptive testing with a minimum error-variance criterion0 aMultidimensional adaptive testing with a minimum errorvariance c  a398-4120 v243 aAdaptive testing under a multidimensional logistic response model is addressed. An algorithm is proposed that minimizes the (asymptotic) variance of the maximum-likelihood estimator of a linear combination of abilities of interest. The criterion results in a closed-form expression that is easy to evaluate. In addition, it is shown how the algorithm can be modified if the interest is in a test with a "simple ability structure". The statistical properties of the adaptive ML estimator are demonstrated for a two-dimensional item pool with several linear combinations of the abilities. 10acomputerized adaptive testing1 avan der Linden, WJ  uhttp://mail.iacat.org/content/multidimensional-adaptive-testing-minimum-error-variance-criterion01191nas a2200157   4500008004100000245010900041210006900150300001200219490000700231520058300238653003400821100002300855700001600878700001800894856012100912       1999                        eng d00aUsing response-time constraints to control for differential speededness in computerized adaptive testing0 aUsing responsetime constraints to control for differential speed  a195-2100 v233 aAn item-selection algorithm is proposed for neutralizing the differential effects of time limits on computerized adaptive test scores. The method is based on a statistical model for distributions of examinees’ response times on items in a bank that is updated each time an item is administered. Predictions from the model are used as constraints in a 0-1 linear programming model for constrained adaptive testing that maximizes the accuracy of the trait estimator. The method is demonstrated empirically using an item bank from the Armed Services Vocational Aptitude Battery. 10acomputerized adaptive testing1 avan der Linden, WJ1 aScrams, D J1 aSchnipke, D L  uhttp://mail.iacat.org/content/using-response-time-constraints-control-differential-speededness-computerized-adaptive01440nas a2200145   4500008004100000245005300041210005100094300001200145490000700157520098100164653003401145100002301179700001501202856007701217       1998                        eng d00aA model for optimal constrained adaptive testing0 amodel for optimal constrained adaptive testing  a259-2700 v223 aA model for constrained computerized adaptive testing is proposed in which the information in the test at the trait level (0) estimate is maximized subject to a number of possible constraints on the content of the test. At each item-selection step, a full test is assembled to have maximum information at the current 0 estimate, fixing the items already administered. Then the item with maximum in-formation is selected. All test assembly is optimal because a linear programming (LP) model is used that automatically updates to allow for the attributes of the items already administered and the new value of the 0 estimator. The LP model also guarantees that each adaptive test always meets the entire set of constraints. A simulation study using a bank of 753 items from the Law School Admission Test showed that the 0 estimator for adaptive tests of realistic lengths did not suffer any loss of efficiency from the presence of 433 constraints on the item selection process. 10acomputerized adaptive testing1 avan der Linden, WJ1 aReese, L M  uhttp://mail.iacat.org/content/model-optimal-constrained-adaptive-testing