01395nas a2200181 4500008004500000022001400045245008700059210006900146490000700215520080200222653003401024653002501058653001901083653001901102653001601121100002101137856005501158 2024 Engldsh  a2165-659200aThe Influence of Computerized Adaptive Testing on Psychometric Theory and Practice0 aInfluence of Computerized Adaptive Testing on Psychometric Theor0 v113 a

The major premise of this article is that part of the stimulus for the evolution of psychometric theory since the 1950s was the introduction of the concept of computerized adaptive testing (CAT) or its earlier non-CAT variations. The conceptual underpinnings of CAT that had the most influence on psychometric theory was the shift of emphasis from the test (or test score) as the focus of analysis to the test item (or item score). The change in focus allowed a change in the way that test results are conceived of as measurements. It also resolved the conflict among a number of ideas that were present in the early work on psychometric theory. Some of the conflicting ideas are summarized below to show how work on the development of CAT resolved some of those conflicts.

 

10acomputerized adaptive testing10aItem Response Theory10aparadigm shift10ascaling theory10atest design1 aReckase, Mark, D uhttps://jcatpub.net/index.php/jcat/issue/view/34/900542nas a2200169 4500008004500000245006600045210006600111300001000177490000700187653002100194653000800215653000800223653003000231653001300261100002000274856007800294 2023 Engldsh 00aExpanding the Meaning of Adaptive Testing to Enhance Validity0 aExpanding the Meaning of Adaptive Testing to Enhance Validity a22-310 v1010aAdaptive Testing10aCAT10aCBT10atest-taking disengagement10avalidity1 aWise, Steven, L uhttp://mail.iacat.org/expanding-meaning-adaptive-testing-enhance-validity00733nas a2200193 4500008004500000245009100045210006900136300001000205490000700215653003500222653003400257653002900291653002600320100001900346700002700365700002400392700002100416856010200437 2023 Engldsh 00aHow Do Trait Change Patterns Affect the Performance of Adaptive Measurement of Change?0 aHow Do Trait Change Patterns Affect the Performance of Adaptive  a32-580 v1010aadaptive measurement of change10acomputerized adaptive testing10alongitudinal measurement10atrait change patterns1 aTai, Ming, Him1 aCooperman, Allison, W.1 aDeWeese, Joseph, N.1 aWeiss, David, J. uhttp://mail.iacat.org/how-do-trait-change-patterns-affect-performance-adaptive-measurement-change00480nas a2200133 4500008003900000245007300039210006700112490000600179653003400185653001300219653003500232100002600267856005300293 2022 d00aThe (non)Impact of Misfitting Items in Computerized Adaptive Testing0 anonImpact of Misfitting Items in Computerized Adaptive Testing0 v910acomputerized adaptive testing10aitem fit10athree-parameter logistic model1 aDeMars, Christine, E. uhttps://jcatpub.net/index.php/jcat/issue/view/2601514nas a2200145 4500008004100000245003400041210003300075260005500108520107900163653002101242653001601263653001701279100002201296856005001318 2017 eng d00aGrow a Tiger out of Your CAT 0 aGrow a Tiger out of Your CAT aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

The main focus in the community of test developers and researchers is on improving adaptive test procedures and methodologies. Yet, the transition from research projects to larger-scale operational CATs is facing its own challenges. Usually, these operational CATs find their origin in government tenders. “Scalability”, “Interoperability” and “Transparency” are three keywords often found in these documents. Scalability is concerned with parallel system architectures which are based upon stateless selection algorithms. Design capacities often range from 10,000 to well over 100,000 concurrent students. Interoperability is implemented in standards like QTI, standards that were not designed with adaptive testing in mind. Transparency is being realized by open source software: the adaptive test should not be a black box. These three requirements often complicate the development of an adaptive test, or sometimes even conflict.

Session Video

10ainteroparability10aScalability10atransparency1 aVerschoor, Angela uhttp://mail.iacat.org/grow-tiger-out-your-cat03464nas a2200145 4500008004100000245004500041210004500086260005500131520301500186653000803201653001803209653001603227100001403243856006103257 2017 eng d00aItem Response Time on Task Effect in CAT0 aItem Response Time on Task Effect in CAT aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

Introduction. In addition to reduced test length and increased measurement efficiency, computerized adaptive testing (CAT) can provide new insights into the cognitive process of task completion that cannot be mined via conventional tests. Response time is a primary characteristic of the task completion procedure. It has the potential to inform us about underlying processes. In this study, the relationship between response time and response accuracy will be investigated.

Hypothesis. The present study argues that the relationship between response time on task and response accuracy, which may be positive, negative, or curvilinear, will depend on cognitive nature of task items, holding ability of the subjects and difficulty of the items constant. The interpretations regarding the associations are not uniform either.

Research question. Is there a homogeneous effect of response time on test outcome across Graduate

Proposed explanations. If the accuracy of cognitive test responses decreases with response time, then it is an indication that the underlying cognitive process is a degrading process such as knowledge retrieval. More accessible knowledge can be retrieved faster than less accessible knowledge. It is inherent to knowledge retrieval that the success rate declines with elapsing response time. For instance, in reading tasks, the time on task effect is negative and the more negative, the easier a task is. However, if the accuracy of cognitive test responses increases with response time, then the process is of an upgrading nature, with an increasing success rate as a function of response time. For example, problem-solving takes time, and fast responses are less likely to be well-founded responses. It is of course also possible that the relationship is curvilinear, as when an increasing success rate is followed by a decreasing success rate or vice versa.

Methodology. The data are from computer-based GRE quantitative and verbal tests and will be analyzed with generalized linear mixed models (GLMM) framework after controlling the effect of ability and item difficulty as possible confounding factors. A linear model means a linear combination of predictors determining the probability of person p for answering item i correctly. The models are equivalent with advanced IRT models that go beyond the regular modeling of test responses in terms of one or more latent variables and item parameters. The lme4 package for R will be utilized to conduct the statistical calculation.

Implications. The right amount of testing time in CAT is important—too much is wasteful and costly, too little impacts score validity. The study is expected to provide new perception on the relationship between response time and response accuracy, which in turn, contribute to a better understanding of time effects and relevant cognitive process in CA.

Session Video

10aCAT10aResponse time10aTask effect1 aShi, Yang uhttp://mail.iacat.org/item-response-time-task-effect-cat00516nas a2200193 4500008004500000022001400045245004500059210004200104300000900146490000600155653001300161653001500174653001300189653001200202653001100214653001200225100002100237856006400258 2015 Engldsh  a2165-659200aImplementing a CAT: The AMC Experience 0 aImplementing a CAT The AMC Experience a1-120 v310aadaptive10aAssessment10acomputer10amedical10aonline10aTesting1 aBarnard, John, J uhttp://www.iacat.org/jcat/index.php/jcat/article/view/52/2500689nas a2200193 4500008004500000022001400045245012100059210006900180300001000249490000600259653003100265653002300296653002200319653003200341653002500373653001800398100001500416856006400431 2014 Engldsh  a2165-659200aDetecting Item Preknowledge in Computerized Adaptive Testing Using Information Theory and Combinatorial Optimization0 aDetecting Item Preknowledge in Computerized Adaptive Testing Usi a37-580 v210acombinatorial optimization10ahypothesis testing10aitem preknowledge10aKullback-Leibler divergence10asimulated annealing.10atest security1 aBelov, D I uhttp://www.iacat.org/jcat/index.php/jcat/article/view/36/1800483nas a2200145 4500008004100000245006100041210005900102653000800161653001600169653001300185100001900198700001600217700002200233856008200255 2011 eng d00aA Heuristic Of CAT Item Selection Procedure For Testlets0 aHeuristic Of CAT Item Selection Procedure For Testlets10aCAT10ashadow test10atestlets1 aChien, Yuehmei1 aShin, David1 aWay, Walter Denny uhttp://mail.iacat.org/content/heuristic-cat-item-selection-procedure-testlets00782nas a2200205 4500008004100000245003400041210003200075260001200107520027300119653000800392653000800400653002300408653001000431653001200441653000800453100002200461700001900483700001600502856005800518 2011 eng d00aA Test Assembly Model for MST0 aTest Assembly Model for MST c10/20113 a

This study is just a short exploration in the matter of optimization of a MST. It is extremely hard or maybe impossible to chart influence of item pool and test specifications on optimization process. Simulations are very helpful in finding an acceptable MST.

10aCAT10amst10amultistage testing10aRasch10arouting10atif1 aVerschoor, Angela1 aRadtke, Ingrid1 aEggen, Theo uhttp://mail.iacat.org/content/test-assembly-model-mst01393nas a2200145 4500008004100000245007100041210006900112260001200181520089200193653000801085653001801093653001701111100002601128856009301154 2011 eng d00aWalking the Tightrope: Using Better Content Control to Improve CAT0 aWalking the Tightrope Using Better Content Control to Improve CA c10/20113 a

All testing involves a balance between measurement precision and content considerations. CAT item-selection algorithms have evolved to accommodate content considerations. Reviews CAT evolution including: Original/”Pure” adaptive exams, Constrained CAT, Weighted-deviations method, Shadow-Test Approach, Testlets instead of fully adapted tests, Administration of one item may preclude the administration of other item(s), and item relationships.

Research Questions

 

10aCAT10aCAT evolution10atest content1 aGialluca, Kathleen, A uhttp://mail.iacat.org/content/walking-tightrope-using-better-content-control-improve-cat02752nas a2200433 4500008004100000020004600041245012800087210006900215250001500284300001200299490000700311520139300318653003401711653001501745653001001760653000901770653002201779653002501801653001101826653001101837653000901848653001601857653001501873653003801888653001901926653003101945653002801976653004802004653002202052100002002074700001202094700001402106700001602120700001402136700001702150700001502167700001502182856012102197 2009 eng d a1878-5921 (Electronic)0895-4356 (Linking)00aAn evaluation of patient-reported outcomes found computerized adaptive testing was efficient in assessing stress perception0 aevaluation of patientreported outcomes found computerized adapti a2008/07/22 a278-2870 v623 aOBJECTIVES: This study aimed to develop and evaluate a first computerized adaptive test (CAT) for the measurement of stress perception (Stress-CAT), in terms of the two dimensions: exposure to stress and stress reaction. STUDY DESIGN AND SETTING: Item response theory modeling was performed using a two-parameter model (Generalized Partial Credit Model). The evaluation of the Stress-CAT comprised a simulation study and real clinical application. A total of 1,092 psychosomatic patients (N1) were studied. Two hundred simulees (N2) were generated for a simulated response data set. Then the Stress-CAT was given to n=116 inpatients, (N3) together with established stress questionnaires as validity criteria. RESULTS: The final banks included n=38 stress exposure items and n=31 stress reaction items. In the first simulation study, CAT scores could be estimated with a high measurement precision (SE<0.32; rho>0.90) using 7.0+/-2.3 (M+/-SD) stress reaction items and 11.6+/-1.7 stress exposure items. The second simulation study reanalyzed real patients data (N1) and showed an average use of items of 5.6+/-2.1 for the dimension stress reaction and 10.0+/-4.9 for the dimension stress exposure. Convergent validity showed significantly high correlations. CONCLUSIONS: The Stress-CAT is short and precise, potentially lowering the response burden of patients in clinical decision making.10a*Diagnosis, Computer-Assisted10aAdolescent10aAdult10aAged10aAged, 80 and over10aConfidence Intervals10aFemale10aHumans10aMale10aMiddle Aged10aPerception10aQuality of Health Care/*standards10aQuestionnaires10aReproducibility of Results10aSickness Impact Profile10aStress, Psychological/*diagnosis/psychology10aTreatment Outcome1 aKocalevent, R D1 aRose, M1 aBecker, J1 aWalter, O B1 aFliege, H1 aBjorner, J B1 aKleiber, D1 aKlapp, B F uhttp://mail.iacat.org/content/evaluation-patient-reported-outcomes-found-computerized-adaptive-testing-was-efficient02598nas a2200337 4500008004100000020004600041245012800087210006900215250001500284300000700299490000600306520149200312653003201804653002301836653002501859653003401884653001101918653001101929653000901940653002601949653003101975653002702006653001102033653001802044100001502062700001202077700001402089700001802103700001202121856012702133 2009 eng d a1477-7525 (Electronic)1477-7525 (Linking)00aReduction in patient burdens with graphical computerized adaptive testing on the ADL scale: tool development and simulation0 aReduction in patient burdens with graphical computerized adaptiv a2009/05/07 a390 v73 aBACKGROUND: The aim of this study was to verify the effectiveness and efficacy of saving time and reducing burden for patients, nurses, and even occupational therapists through computer adaptive testing (CAT). METHODS: Based on an item bank of the Barthel Index (BI) and the Frenchay Activities Index (FAI) for assessing comprehensive activities of daily living (ADL) function in stroke patients, we developed a visual basic application (VBA)-Excel CAT module, and (1) investigated whether the averaged test length via CAT is shorter than that of the traditional all-item-answered non-adaptive testing (NAT) approach through simulation, (2) illustrated the CAT multimedia on a tablet PC showing data collection and response errors of ADL clinical functional measures in stroke patients, and (3) demonstrated the quality control of endorsing scale with fit statistics to detect responding errors, which will be further immediately reconfirmed by technicians once patient ends the CAT assessment. RESULTS: The results show that endorsed items could be shorter on CAT (M = 13.42) than on NAT (M = 23) at 41.64% efficiency in test length. However, averaged ability estimations reveal insignificant differences between CAT and NAT. CONCLUSION: This study found that mobile nursing services, placed at the bedsides of patients could, through the programmed VBA-Excel CAT module, reduce the burden to patients and save time, more so than the traditional NAT paper-and-pencil testing appraisals.10a*Activities of Daily Living10a*Computer Graphics10a*Computer Simulation10a*Diagnosis, Computer-Assisted10aFemale10aHumans10aMale10aPoint-of-Care Systems10aReproducibility of Results10aStroke/*rehabilitation10aTaiwan10aUnited States1 aChien, T W1 aWu, H M1 aWang, W-C1 aCastillo, R V1 aChou, W uhttp://mail.iacat.org/content/reduction-patient-burdens-graphical-computerized-adaptive-testing-adl-scale-tool-development03158nas a2200493 4500008004100000020002200041245008900063210006900152250001500221260000800236300001000244490000700254520169600261653003401957653002001991653001502011653001002026653000902036653002602045653003202071653003102103653001102134653001102145653000902156653003202165653001602197653002902213653004402242653002902286653003102315653003102346653001702377100001702394700001402411700001602425700001302441700001702454700002202471700001702493700001402510700001402524700001702538856010902555 2008 eng d a1075-2730 (Print)00aUsing computerized adaptive testing to reduce the burden of mental health assessment0 aUsing computerized adaptive testing to reduce the burden of ment a2008/04/02 cApr a361-80 v593 aOBJECTIVE: This study investigated the combination of item response theory and computerized adaptive testing (CAT) for psychiatric measurement as a means of reducing the burden of research and clinical assessments. METHODS: Data were from 800 participants in outpatient treatment for a mood or anxiety disorder; they completed 616 items of the 626-item Mood and Anxiety Spectrum Scales (MASS) at two times. The first administration was used to design and evaluate a CAT version of the MASS by using post hoc simulation. The second confirmed the functioning of CAT in live testing. RESULTS: Tests of competing models based on item response theory supported the scale's bifactor structure, consisting of a primary dimension and four group factors (mood, panic-agoraphobia, obsessive-compulsive, and social phobia). Both simulated and live CAT showed a 95% average reduction (585 items) in items administered (24 and 30 items, respectively) compared with administration of the full MASS. The correlation between scores on the full MASS and the CAT version was .93. For the mood disorder subscale, differences in scores between two groups of depressed patients--one with bipolar disorder and one without--on the full scale and on the CAT showed effect sizes of .63 (p<.003) and 1.19 (p<.001) standard deviation units, respectively, indicating better discriminant validity for CAT. CONCLUSIONS: Instead of using small fixed-length tests, clinicians can create item banks with a large item pool, and a small set of the items most relevant for a given individual can be administered with no loss of information, yielding a dramatic reduction in administration time and patient and clinician burden.10a*Diagnosis, Computer-Assisted10a*Questionnaires10aAdolescent10aAdult10aAged10aAgoraphobia/diagnosis10aAnxiety Disorders/diagnosis10aBipolar Disorder/diagnosis10aFemale10aHumans10aMale10aMental Disorders/*diagnosis10aMiddle Aged10aMood Disorders/diagnosis10aObsessive-Compulsive Disorder/diagnosis10aPanic Disorder/diagnosis10aPhobic Disorders/diagnosis10aReproducibility of Results10aTime Factors1 aGibbons, R D1 aWeiss, DJ1 aKupfer, D J1 aFrank, E1 aFagiolini, A1 aGrochocinski, V J1 aBhaumik, D K1 aStover, A1 aBock, R D1 aImmekus, J C uhttp://mail.iacat.org/content/using-computerized-adaptive-testing-reduce-burden-mental-health-assessment01949nas a2200301 4500008004500000020001400045245012900059210006900188300001200257490000700269520093500276653002501211653002101236653002501257653003001282653003001312653001001342653001501352653002601367653002501393653002401418653001501442653001501457100001301472700001701485700002101502856012401523 2007 Engldsh  a0146-621600aComputerized adaptive testing for polytomous motivation items: Administration mode effects and a comparison with short forms0 aComputerized adaptive testing for polytomous motivation items Ad a412-4290 v313 aIn a randomized experiment (n=515), a computerized and a computerized adaptive test (CAT) are compared. The item pool consists of 24 polytomous motivation items. Although items are carefully selected, calibration data show that Samejima's graded response model did not fit the data optimally. A simulation study is done to assess possible consequences of model misfit. CAT efficiency was studied by a systematic comparison of the CAT with two types of conventional fixed length short forms, which are created to be good CAT competitors. Results showed no essential administration mode effects. Efficiency analyses show that CAT outperformed the short forms in almost all aspects when results are aggregated along the latent trait scale. The real and the simulated data results are very similar, which indicate that the real data results are not affected by model misfit. (PsycINFO Database Record (c) 2007 APA ) (journal abstract)10a2220 Tests & Testing10aAdaptive Testing10aAttitude Measurement10acomputer adaptive testing10aComputer Assisted Testing10aitems10aMotivation10apolytomous motivation10aStatistical Validity10aTest Administration10aTest Forms10aTest Items1 aHol, A M1 aVorst, H C M1 aMellenbergh, G J uhttp://mail.iacat.org/content/computerized-adaptive-testing-polytomous-motivation-items-administration-mode-effects-and01799nas a2200217 4500008004100000020004600041245017800087210006900265260002500334300001200359490000700371520094900378653001201327653004301339653001801382653000901400100001701409700001501426700001501441856012501456 2007 eng d a1062-7197 (Print); 1532-6977 (Electronic)00aThe effect of including pretest items in an operational computerized adaptive test: Do different ability examinees spend different amounts of time on embedded pretest items?0 aeffect of including pretest items in an operational computerized bLawrence Erlbaum: US a161-1730 v123 aThe purpose of this study was to examine the effect of pretest items on response time in an operational, fixed-length, time-limited computerized adaptive test (CAT). These pretest items are embedded within the CAT, but unlike the operational items, are not tailored to the examinee's ability level. If examinees with higher ability levels need less time to complete these items than do their counterparts with lower ability levels, they will have more time to devote to the operational test questions. Data were from a graduate admissions test that was administered worldwide. Data from both quantitative and verbal sections of the test were considered. For the verbal section, examinees in the lower ability groups spent systematically more time on their pretest items than did those in the higher ability groups, though for the quantitative section the differences were less clear. (PsycINFO Database Record (c) 2007 APA, all rights reserved)10aability10aoperational computerized adaptive test10apretest items10atime1 aFerdous, A A1 aPlake, B S1 aChang, S-R uhttp://mail.iacat.org/content/effect-including-pretest-items-operational-computerized-adaptive-test-do-different-ability02413nas a2200289 4500008004100000020002200041245010800063210006900171260004500240300001000285490000700295520141000302653003201712653002501744653002501769653002501794653002601819653001801845653003701863653002101900653001301921100001501934700001401949700001901963700002001982856012102002 2007 eng d a1015-5759 (Print)00aPsychometric properties of an emotional adjustment measure: An application of the graded response model0 aPsychometric properties of an emotional adjustment measure An ap bHogrefe & Huber Publishers GmbH: Germany a39-460 v233 aItem response theory (IRT) provides valuable methods for the analysis of the psychometric properties of a psychological measure. However, IRT has been mainly used for assessing achievements and ability rather than personality factors. This paper presents an application of the IRT to a personality measure. Thus, the psychometric properties of a new emotional adjustment measure that consists of a 28-six graded response items is shown. Classical test theory (CTT) analyses as well as IRT analyses are carried out. Samejima's (1969) graded-response model has been used for estimating item parameters. Results show that the bank of items fulfills model assumptions and fits the data reasonably well, demonstrating the suitability of the IRT models for the description and use of data originating from personality measures. In this sense, the model fulfills the expectations that IRT has undoubted advantages: (1) The invariance of the estimated parameters, (2) the treatment given to the standard error of measurement, and (3) the possibilities offered for the construction of computerized adaptive tests (CAT). The bank of items shows good reliability. It also shows convergent validity compared to the Eysenck Personality Inventory (EPQ-A; Eysenck & Eysenck, 1975) and the Big Five Questionnaire (BFQ; Caprara, Barbaranelli, & Borgogni, 1993). (PsycINFO Database Record (c) 2007 APA, all rights reserved)10acomputerized adaptive tests10aEmotional Adjustment10aItem Response Theory10aPersonality Measures10apersonnel recruitment10aPsychometrics10aSamejima's graded response model10atest reliability10avalidity1 aRubio, V J1 aAguado, D1 aHontangas, P M1 aHernández, J M uhttp://mail.iacat.org/content/psychometric-properties-emotional-adjustment-measure-application-graded-response-model01689nas a2200217 4500008004100000020002200041245008000063210006900143260002600212300001200238490000700250520095600257653003401213653002701247653002301274653002901297100001601326700001801342700001301360856009801373 2007 eng d a0146-6216 (Print)00aTest design optimization in CAT early stage with the nominal response model0 aTest design optimization in CAT early stage with the nominal res bSage Publications: US a213-2320 v313 aThe early stage of computerized adaptive testing (CAT) refers to the phase of the trait estimation during the administration of only a few items. This phase can be characterized by bias and instability of estimation. In this study, an item selection criterion is introduced in an attempt to lessen this instability: the D-optimality criterion. A polytomous unconstrained CAT simulation is carried out to evaluate this criterion's performance under different test premises. The simulation shows that the extent of early stage instability depends primarily on the quality of the item pool information and its size and secondarily on the item selection criteria. The efficiency of the D-optimality criterion is similar to the efficiency of other known item selection criteria. Yet, it often yields estimates that, at the beginning of CAT, display a more robust performance against instability. (PsycINFO Database Record (c) 2007 APA, all rights reserved)10acomputerized adaptive testing10anominal response model10arobust performance10atest design optimization1 aPassos, V L1 aBerger, M P F1 aTan, F E uhttp://mail.iacat.org/content/test-design-optimization-cat-early-stage-nominal-response-model01964nas a2200265 4500008004100000020002200041245008200063210006900145260002600214300001000240490000700250520112900257653001501386653003401401653001401435653001701449653002401466653001501490653002201505653001501527100002301542700001301565700001801578856010201596 2006 eng d a1076-9986 (Print)00aAssembling a computerized adaptive testing item pool as a set of linear tests0 aAssembling a computerized adaptive testing item pool as a set of bSage Publications: US a81-990 v313 aTest-item writing efforts typically results in item pools with an undesirable correlational structure between the content attributes of the items and their statistical information. If such pools are used in computerized adaptive testing (CAT), the algorithm may be forced to select items with less than optimal information, that violate the content constraints, and/or have unfavorable exposure rates. Although at first sight somewhat counterintuitive, it is shown that if the CAT pool is assembled as a set of linear test forms, undesirable correlations can be broken down effectively. It is proposed to assemble such pools using a mixed integer programming model with constraints that guarantee that each test meets all content specifications and an objective function that requires them to have maximal information at a well-chosen set of ability values. An empirical example with a previous master pool from the Law School Admission Test (LSAT) yielded a CAT with nearly uniform bias and mean-squared error functions for the ability estimator and item-exposure rates that satisfied the target for all items in the pool. 10aAlgorithms10acomputerized adaptive testing10aitem pool10alinear tests10amathematical models10astatistics10aTest Construction10aTest Items1 avan der Linden, WJ1 aAriel, A1 aVeldkamp, B P uhttp://mail.iacat.org/content/assembling-computerized-adaptive-testing-item-pool-set-linear-tests01948nas a2200241 4500008004100000020002200041245011200063210006900175260004100244300001200285490000700297520102300304653003401327653002401361653004701385653002401432653002101456653007001477100001301547700001401560700001001574856012201584 2006 eng d a0022-0655 (Print)00aComparing methods of assessing differential item functioning in a computerized adaptive testing environment0 aComparing methods of assessing differential item functioning in  bBlackwell Publishing: United Kingdom a245-2640 v433 aMantel-Haenszel and SIBTEST, which have known difficulty in detecting non-unidirectional differential item functioning (DIF), have been adapted with some success for computerized adaptive testing (CAT). This study adapts logistic regression (LR) and the item-response-theory-likelihood-ratio test (IRT-LRT), capable of detecting both unidirectional and non-unidirectional DIF, to the CAT environment in which pretest items are assumed to be seeded in CATs but not used for trait estimation. The proposed adaptation methods were evaluated with simulated data under different sample size ratios and impact conditions in terms of Type I error, power, and specificity in identifying the form of DIF. The adapted LR and IRT-LRT procedures are more powerful than the CAT version of SIBTEST for non-unidirectional DIF detection. The good Type I error control provided by IRT-LRT under extremely unequal sample sizes and large impact is encouraging. Implications of these and other findings are discussed. all rights reserved)10acomputerized adaptive testing10aeducational testing10aitem response theory likelihood ratio test10alogistic regression10atrait estimation10aunidirectional & non-unidirectional differential item functioning1 aLei, P-W1 aChen, S-Y1 aYu, L uhttp://mail.iacat.org/content/comparing-methods-assessing-differential-item-functioning-computerized-adaptive-testing01471nas a2200205 4500008004100000245002700041210002600068260006000094300001100154490000800165520087300173653005101046653003001097653002001127653001801147653001301165100001501178700001501193856005701208 2006 eng d00aComputer-based testing0 aComputerbased testing aWashington D.C. USAbAmerican Psychological Association a87-1000 vxiv3 a(From the chapter) There has been a proliferation of research designed to explore and exploit opportunities provided by computer-based assessment. This chapter provides an overview of the diverse efforts by researchers in this area. It begins by describing how paper-and-pencil tests can be adapted for administration by computers. Computerization provides the important advantage that items can be selected so they are of appropriate difficulty for each examinee. Some of the psychometric theory needed for computerized adaptive testing is reviewed. Then research on innovative computerized assessments is summarized. These assessments go beyond multiple-choice items by using formats made possible by computerization. Then some hardware and software issues are described, and finally, directions for future work are outlined. (PsycINFO Database Record (c) 2006 APA )10aAdaptive Testing computerized adaptive testing10aComputer Assisted Testing10aExperimentation10aPsychometrics10aTheories1 aDrasgow, F1 aChuah, S C uhttp://mail.iacat.org/content/computer-based-testing01585nas a2200205 4500008004100000020002200041245005000063210005000113260002600163300001200189490000700201520094200208653003401150653002801184653001901212653002001231653003301251100002301284856007201307 2006 eng d a0146-6216 (Print)00aEquating scores from adaptive to linear tests0 aEquating scores from adaptive to linear tests bSage Publications: US a493-5080 v303 aTwo local methods for observed-score equating are applied to the problem of equating an adaptive test to a linear test. In an empirical study, the methods were evaluated against a method based on the test characteristic function (TCF) of the linear test and traditional equipercentile equating applied to the ability estimates on the adaptive test for a population of test takers. The two local methods were generally best. Surprisingly, the TCF method performed slightly worse than the equipercentile method. Both methods showed strong bias and uniformly large inaccuracy, but the TCF method suffered from extra error due to the lower asymptote of the test characteristic function. It is argued that the worse performances of the two methods are a consequence of the fact that they use a single equating transformation for an entire population of test takers and therefore have to compromise between the individual score distributions. 10acomputerized adaptive testing10aequipercentile equating10alocal equating10ascore reporting10atest characteristic function1 avan der Linden, WJ uhttp://mail.iacat.org/content/equating-scores-adaptive-linear-tests02399nas a2200301 4500008004100000020002200041245010800063210006900171260002400240300000900264490000600273520142200279653002201701653003401723653002301757653003201780653001801812653002101830653001801851100001401869700001301883700001401896700001901910700001501929700001701944700001301961856012301974 2006 eng d a1529-7713 (Print)00aExpansion of a physical function item bank and development of an abbreviated form for clinical research0 aExpansion of a physical function item bank and development of an bRichard M Smith: US a1-150 v73 aWe expanded an existing 33-item physical function (PF) item bank with a sufficient number of items to enable computerized adaptive testing (CAT). Ten items were written to expand the bank and the new item pool was administered to 295 people with cancer. For this analysis of the new pool, seven poorly performing items were identified for further examination. This resulted in a bank with items that define an essentially unidimensional PF construct, cover a wide range of that construct, reliably measure the PF of persons with cancer, and distinguish differences in self-reported functional performance levels. We also developed a 5-item (static) assessment form ("BriefPF") that can be used in clinical research to express scores on the same metric as the overall bank. The BriefPF was compared to the PF-10 from the Medical Outcomes Study SF-36. Both short forms significantly differentiated persons across functional performance levels. While the entire bank was more precise across the PF continuum than either short form, there were differences in the area of the continuum in which each short form was more precise: the BriefPF was more precise than the PF-10 at the lower functional levels and the PF-10 was more precise than the BriefPF at the higher levels. Future research on this bank will include the development of a CAT version, the PF-CAT. (PsycINFO Database Record (c) 2007 APA, all rights reserved)10aclinical research10acomputerized adaptive testing10aperformance levels10aphysical function item bank10aPsychometrics10atest reliability10aTest Validity1 aBode, R K1 aLai, J-S1 aDineen, K1 aHeinemann, A W1 aShevrin, D1 aVon Roenn, J1 aCella, D uhttp://mail.iacat.org/content/expansion-physical-function-item-bank-and-development-abbreviated-form-clinical-research02434nas a2200241 4500008004100000020004600041245008600087210006900173260002500242300001200267490000700279520159900286653001801885653002401903653002001927653002801947653002101975653004001996653001402036100001802050700001202068856011202080 2006 eng d a0895-7347 (Print); 1532-4818 (Electronic)00aOptimal and nonoptimal computer-based test designs for making pass-fail decisions0 aOptimal and nonoptimal computerbased test designs for making pas bLawrence Erlbaum: US a221-2390 v193 aNow that many credentialing exams are being routinely administered by computer, new computer-based test designs, along with item response theory models, are being aggressively researched to identify specific designs that can increase the decision consistency and accuracy of pass-fail decisions. The purpose of this study was to investigate the impact of optimal and nonoptimal multistage test (MST) designs, linear parallel-form test designs (LPFT), and computer adaptive test (CAT) designs on the decision consistency and accuracy of pass-fail decisions. Realistic testing situations matching those of one of the large credentialing agencies were simulated to increase the generalizability of the findings. The conclusions were clear: (a) With the LPFTs, matching test information functions (TIFs) to the mean of the proficiency distribution produced slightly better results than matching them to the passing score; (b) all of the test designs worked better than test construction using random selection of items, subject to content constraints only; (c) CAT performed better than the other test designs; and (d) if matching a TIP to the passing score, the MST design produced a bit better results than the LPFT design. If an argument for the MST design is to be made, it can be made on the basis of slight improvements over the LPFT design and better expected item bank utilization, candidate preference, and the potential for improved diagnostic feedback, compared with the feedback that is possible with fixed linear test forms. (PsycINFO Database Record (c) 2007 APA, all rights reserved)10aadaptive test10acredentialing exams10aDecision Making10aEducational Measurement10amultistage tests10aoptimal computer-based test designs10atest form1 aHambleton, RK1 aXing, D uhttp://mail.iacat.org/content/optimal-and-nonoptimal-computer-based-test-designs-making-pass-fail-decisions02112nas a2200229 4500008004100000245013800041210006900179300001400248490000700262520127000269653003101539653003401570653002501604653001701629653001901646653002401665100001401689700001801703700001701721700001901738856012501757 2006 eng d00aSimulated computerized adaptive test for patients with lumbar spine impairments was efficient and produced valid measures of function0 aSimulated computerized adaptive test for patients with lumbar sp a947–9560 v593 aObjective: To equate physical functioning (PF) items with Back Pain Functional Scale (BPFS) items, develop a computerized adaptive test (CAT) designed to assess lumbar spine functional status (LFS) in people with lumbar spine impairments, and compare discriminant validity of LFS measures (qIRT) generated using all items analyzed with a rating scale Item Response Theory model (RSM) and measures generated using the simulated CAT (qCAT). Methods: We performed a secondary analysis of retrospective intake rehabilitation data. Results: Unidimensionality and local independence of 25 BPFS and PF items were supported. Differential item functioning was negligible for levels of symptom acuity, gender, age, and surgical history. The RSM fit the data well. A lumbar spine specific CAT was developed that was 72% more efficient than using all 25 items to estimate LFS measures. qIRT and qCAT measures did not discriminate patients by symptom acuity, age, or gender, but discriminated patients by surgical history in similar clinically logical ways. qCAT measures were as precise as qIRT measures. Conclusion: A body part specific simulated CAT developed from an LFS item bank was efficient and produced precise measures of LFS without eroding discriminant validity.10aBack Pain Functional Scale10acomputerized adaptive testing10aItem Response Theory10aLumbar spine10aRehabilitation10aTrue-score equating1 aHart, D L1 aMioduski, J E1 aWerneke, M W1 aStratford, P W uhttp://mail.iacat.org/content/simulated-computerized-adaptive-test-patients-lumbar-spine-impairments-was-efficient-and-002654nas a2200409 4500008004100000245013400041210006900175300001000244490000700254520123100261653002501492653003201517653003101549653001001580653000901590653002201599653003301621653001101654653001101665653000901676653001601685653002401701653003101725653004101756653004501797653006801842653006101910653003001971653002802001653002202029100001402051700001302065700001802078700001402096700001502110856011902125 2006 eng d00aSimulated computerized adaptive test for patients with shoulder impairments was efficient and produced valid measures of function0 aSimulated computerized adaptive test for patients with shoulder  a290-80 v593 aBACKGROUND AND OBJECTIVE: To test unidimensionality and local independence of a set of shoulder functional status (SFS) items, develop a computerized adaptive test (CAT) of the items using a rating scale item response theory model (RSM), and compare discriminant validity of measures generated using all items (theta(IRT)) and measures generated using the simulated CAT (theta(CAT)). STUDY DESIGN AND SETTING: We performed a secondary analysis of data collected prospectively during rehabilitation of 400 patients with shoulder impairments who completed 60 SFS items. RESULTS: Factor analytic techniques supported that the 42 SFS items formed a unidimensional scale and were locally independent. Except for five items, which were deleted, the RSM fit the data well. The remaining 37 SFS items were used to generate the CAT. On average, 6 items were needed to estimate precise measures of function using the SFS CAT, compared with all 37 SFS items. The theta(IRT) and theta(CAT) measures were highly correlated (r = .96) and resulted in similar classifications of patients. CONCLUSION: The simulated SFS CAT was efficient and produced precise, clinically relevant measures of functional status with good discriminating ability.10a*Computer Simulation10a*Range of Motion, Articular10aActivities of Daily Living10aAdult10aAged10aAged, 80 and over10aFactor Analysis, Statistical10aFemale10aHumans10aMale10aMiddle Aged10aProspective Studies10aReproducibility of Results10aResearch Support, N.I.H., Extramural10aResearch Support, U.S. Gov't, Non-P.H.S.10aShoulder Dislocation/*physiopathology/psychology/rehabilitation10aShoulder Pain/*physiopathology/psychology/rehabilitation10aShoulder/*physiopathology10aSickness Impact Profile10aTreatment Outcome1 aHart, D L1 aCook, KF1 aMioduski, J E1 aTeal, C R1 aCrane, P K uhttp://mail.iacat.org/content/simulated-computerized-adaptive-test-patients-shoulder-impairments-was-efficient-and02252nas a2200301 4500008004500000020001400045245009800059210007100157300001200228490000700240520127200247653003201519653002601551653002801577653001801605653002501623653001601648653001201664653001501676653001801691653001601709653001801725653002701743653001101770100001901781700001601800856013401816 2006 Spandsh  a0212-972800aTécnicas para detectar patrones de respuesta atípicos [Aberrant patterns detection methods]0 aTécnicas para detectar patrones de respuesta atípicos Aberrant p a143-1540 v223 aLa identificación de patrones de respuesta atípicos es de gran utilidad para la construcción de tests y de bancos de ítems con propiedades psicométricas así como para el análisis de validez de los mismos. En este trabajo de revisión se han recogido los más relevantes y novedosos métodos de ajuste de personas que se han elaborado dentro de cada uno de los principales ámbitos de trabajo de la Psicometría: el escalograma de Guttman, la Teoría Clásica de Tests (TCT), la Teoría de la Generalizabilidad (TG), la Teoría de Respuesta al Ítem (TRI), los Modelos de Respuesta al Ítem No Paramétricos (MRINP), los Modelos de Clase Latente de Orden Restringido (MCL-OR) y el Análisis de Estructura de Covarianzas (AEC).Aberrant patterns detection has a great usefulness in order to make tests and item banks with psychometric characteristics and validity analysis of tests and items. The most relevant and newest person-fit methods have been reviewed. All of them have been made in each one of main areas of Psychometry: Guttman's scalogram, Classical Test Theory (CTT), Generalizability Theory (GT), Item Response Theory (IRT), Non-parametric Response Models (NPRM), Order-Restricted Latent Class Models (OR-LCM) and Covariance Structure Analysis (CSA).10aaberrant patterns detection10aClassical Test Theory10ageneralizability theory10aItem Response10aItem Response Theory10aMathematics10amethods10aperson-fit10aPsychometrics10apsychometry10aTest Validity10atest validity analysis10aTheory1 aNúñez, R M N1 aPina, J A L uhttp://mail.iacat.org/content/t%C3%A9cnicas-para-detectar-patrones-de-respuesta-at%C3%ADpicos-aberrant-patterns-detection-methods01470nas a2200217 4500008004100000245015400041210006900195260004600264300001200310520063100322653003000953653001100983653002500994653001601019653002201035653002301057100001801080700001501098700001401113856012501127 2005 eng d00aApplications of item response theory to improve health outcomes assessment: Developing item banks, linking instruments, and computer-adaptive testing0 aApplications of item response theory to improve health outcomes  aCambridge, UKbCambridge University Press a445-4643 a(From the chapter) The current chapter builds on Reise's introduction to the basic concepts, assumptions, popular models, and important features of IRT and discusses the applications of item response theory (IRT) modeling to health outcomes assessment. In particular, we highlight the critical role of IRT modeling in: developing an instrument to match a study's population; linking two or more instruments measuring similar constructs on a common metric; and creating item banks that provide the foundation for tailored short-form instruments or for computerized adaptive assessments. (PsycINFO Database Record (c) 2005 APA )10aComputer Assisted Testing10aHealth10aItem Response Theory10aMeasurement10aTest Construction10aTreatment Outcomes1 aHambleton, RK1 aGotay, C C1 aSnyder, C uhttp://mail.iacat.org/content/applications-item-response-theory-improve-health-outcomes-assessment-developing-item-banks02353nas a2200193 4500008004100000245008700041210006900128300001200197490000700209520170500216653003001921653002201951653001401973653002801987653001302015100001002028700001302038856010802051 2005 eng d00aA computer-assisted test design and diagnosis system for use by classroom teachers0 acomputerassisted test design and diagnosis system for use by cla a419-4290 v213 aComputer-assisted assessment (CAA) has become increasingly important in education in recent years. A variety of computer software systems have been developed to help assess the performance of students at various levels. However, such systems are primarily designed to provide objective assessment of students and analysis of test items, and focus has been mainly placed on higher and further education. Although there are commercial professional systems available for use by primary and secondary educational institutions, such systems are generally expensive and require skilled expertise to operate. In view of the rapid progress made in the use of computer-based assessment for primary and secondary students by education authorities here in the UK and elsewhere, there is a need to develop systems which are economic and easy to use and can provide the necessary information that can help teachers improve students' performance. This paper presents the development of a software system that provides a range of functions including generating items and building item banks, designing tests, conducting tests on computers and analysing test results. Specifically, the system can generate information on the performance of students and test items that can be easily used to identify curriculum areas where students are under performing. A case study based on data collected from five secondary schools in Hong Kong involved in the Curriculum, Evaluation and Management Centre's Middle Years Information System Project, Durham University, UK, has been undertaken to demonstrate the use of the system for diagnostic and performance analysis. (PsycINFO Database Record (c) 2006 APA ) (journal abstract)10aComputer Assisted Testing10aComputer Software10aDiagnosis10aEducational Measurement10aTeachers1 aHe, Q1 aTymms, P uhttp://mail.iacat.org/content/computer-assisted-test-design-and-diagnosis-system-use-classroom-teachers02021nas a2200193 4500008004100000245009300041210006900134300001200203490000700215520136400222653001501586653002401601653001101625653002201636100001801658700001801676700001901694856011401713 2005 eng d00aInfeasibility in automated test assembly models: A comparison study of different methods0 aInfeasibility in automated test assembly models A comparison stu a223-2430 v423 aSeveral techniques exist to automatically put together a test meeting a number of specifications. In an item bank, the items are stored with their characteristics. A test is constructed by selecting a set of items that fulfills the specifications set by the test assembler. Test assembly problems are often formulated in terms of a model consisting of restrictions and an objective to be maximized or minimized. A problem arises when it is impossible to construct a test from the item pool that meets all specifications, that is, when the model is not feasible. Several methods exist to handle these infeasibility problems. In this article, test assembly models resulting from two practical testing programs were reconstructed to be infeasible. These models were analyzed using methods that forced a solution (Goal Programming, Multiple-Goal Programming, Greedy Heuristic), that analyzed the causes (Relaxed and Ordered Deletion Algorithm (RODA), Integer Randomized Deletion Algorithm (IRDA), Set Covering (SC), and Item Sampling), or that analyzed the causes and used this information to force a solution (Irreducible Infeasible Set-Solver). Specialized methods such as the IRDA and the Irreducible Infeasible Set-Solver performed best. Recommendations about the use of different methods are given. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10aAlgorithms10aItem Content (Test)10aModels10aTest Construction1 aHuitzing, H A1 aVeldkamp, B P1 aVerschoor, A J uhttp://mail.iacat.org/content/infeasibility-automated-test-assembly-models-comparison-study-different-methods02301nas a2200205 4500008004100000245009000041210007100131300000900202490000700211520161900218653002001837653001601857653001801873653002201891653001601913653001501929653001801944100001401962856011901976 2005 eng d00aLa Validez desde una óptica psicométrica [Validity from a psychometric perspective]0 aLa Validez desde una óptica psicométrica Validity from a psychom a9-200 v133 aEl estudio de la validez constituye el eje central de los análisis psicométricos de los instrumentos de medida. En esta comunicación se traza una breve nota histórica de los distintos modos de concebir la validez a lo largo de los tiempos, se comentan las líneas actuales, y se tratan de vislumbrar posibles vías futuras, teniendo en cuenta el impacto que las nuevas tecnologías informáticas están ejerciendo sobre los propios instrumentos de medida en Psicología y Educación. Cuestiones como los nuevos formatos multimedia de los ítems, la evaluación a distancia, el uso intercultural de las pruebas, las consecuencias de su uso, o los tests adaptativos informatizados, reclaman nuevas formas de evaluar y conceptualizar la validez. También se analizan críticamente algunos planteamientos recientes sobre el concepto de validez. The study of validity constitutes a central axis of psychometric analyses of measurement instruments. This paper presents a historical sketch of different modes of conceiving validity, with commentary on current views, and it attempts to predict future lines of research by considering the impact of new computerized technologies on measurement instruments in psychology and education. Factors such as the new multimedia format of items, distance assessment, the intercultural use of tests, the consequences of the latter, or the development of computerized adaptive tests demand new ways of conceiving and evaluating validity. Some recent thoughts about the concept of validity are also critically analyzed. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10aFactor Analysis10aMeasurement10aPsychometrics10aScaling (Testing)10aStatistical10aTechnology10aTest Validity1 aMuñiz, J uhttp://mail.iacat.org/content/la-validez-desde-una-%C3%B3ptica-psicom%C3%A9trica-validity-psychometric-perspective02903nas a2200409 4500008004100000245012300041210006900164260000800233300001000241490000700251520159700258653004701855653001001902653000901912653001901921653003101940653002601971653001101997653002902008653001102037653000902048653001602057653003902073653001402112653002502126653002702151653003002178653003202208653002802240653002202268100001502290700001602305700001702321700001602338700001502354856012402369 2005 eng d00aMeasuring physical function in patients with complex medical and postsurgical conditions: a computer adaptive approach0 aMeasuring physical function in patients with complex medical and cOct a741-80 v843 aOBJECTIVE: To examine whether the range of disability in the medically complex and postsurgical populations receiving rehabilitation is adequately sampled by the new Activity Measure--Post-Acute Care (AM-PAC), and to assess whether computer adaptive testing (CAT) can derive valid patient scores using fewer questions. DESIGN: Observational study of 158 subjects (mean age 67.2 yrs) receiving skilled rehabilitation services in inpatient (acute rehabilitation hospitals, skilled nursing facility units) and community (home health services, outpatient departments) settings for recent-onset or worsening disability from medical (excluding neurological) and surgical (excluding orthopedic) conditions. Measures were interviewer-administered activity questions (all patients) and physical functioning portion of the SF-36 (outpatients) and standardized chart items (11 Functional Independence Measure (FIM), 19 Standardized Outcome and Assessment Information Set (OASIS) items, and 22 Minimum Data Set (MDS) items). Rasch modeling analyzed all data and the relationship between person ability estimates and average item difficulty. CAT assessed the ability to derive accurate patient scores using a sample of questions. RESULTS: The 163-item activity item pool covered the range of physical movement and personal and instrumental activities. CAT analysis showed comparable scores between estimates using 10 items or the total item pool. CONCLUSION: The AM-PAC can assess a broad range of function in patients with complex medical illness. CAT achieves valid patient scores using fewer questions.10aActivities of Daily Living/*classification10aAdult10aAged10aCohort Studies10aContinuity of Patient Care10aDisability Evaluation10aFemale10aHealth Services Research10aHumans10aMale10aMiddle Aged10aPostoperative Care/*rehabilitation10aPrognosis10aRecovery of Function10aRehabilitation Centers10aRehabilitation/*standards10aSensitivity and Specificity10aSickness Impact Profile10aTreatment Outcome1 aSiebens, H1 aAndres, P L1 aPengsheng, N1 aCoster, W J1 aHaley, S M uhttp://mail.iacat.org/content/measuring-physical-function-patients-complex-medical-and-postsurgical-conditions-computer02855nas a2200241 4500008004100000245018600041210007000227300001200297490000700309520195000316653003002266653002502296653001802321653002502339653001802364653001802382653001102400100001402411700001502425700001902440700002002459856013402479 2005 eng d00aPropiedades psicométricas de un test Adaptativo Informatizado para la medición del ajuste emocional [Psychometric properties of an Emotional Adjustment Computerized Adaptive Test]0 aPropiedades psicométricas de un test Adaptativo Informatizado pa a484-4910 v173 aEn el presente trabajo se describen las propiedades psicométricas de un Test Adaptativo Informatizado para la medición del ajuste emocional de las personas. La revisión de la literatura acerca de la aplicación de los modelos de la teoría de la respuesta a los ítems (TRI) muestra que ésta se ha utilizado más en el trabajo con variables aptitudinales que para la medición de variables de personalidad, sin embargo diversos estudios han mostrado la eficacia de la TRI para la descripción psicométrica de dichasvariables. Aun así, pocos trabajos han explorado las características de un Test Adaptativo Informatizado, basado en la TRI, para la medición de una variable de personalidad como es el ajuste emocional. Nuestros resultados muestran la eficiencia del TAI para la evaluación del ajuste emocional, proporcionando una medición válida y precisa, utilizando menor número de elementos de medida encomparación con las escalas de ajuste emocional de instrumentos fuertemente implantados. Psychometric properties of an emotional adjustment computerized adaptive test. In the present work it was described the psychometric properties of an emotional adjustment computerized adaptive test. An examination of Item Response Theory (IRT) research literature indicates that IRT has been mainly used for assessing achievements and ability rather than personality factors. Nevertheless last years have shown several studies wich have successfully used IRT to personality assessment instruments. Even so, a few amount of works has inquired the computerized adaptative test features, based on IRT, for the measurement of a personality traits as it’s the emotional adjustment. Our results show the CAT efficiency for the emotional adjustment assessment so this provides a valid and accurate measurement; by using a less number of items in comparison with the emotional adjustment scales from the most strongly established questionnaires.10aComputer Assisted Testing10aEmotional Adjustment10aItem Response10aPersonality Measures10aPsychometrics10aTest Validity10aTheory1 aAguado, D1 aRubio, V J1 aHontangas, P M1 aHernández, J M uhttp://mail.iacat.org/content/propiedades-psicom%C3%A9tricas-de-un-test-adaptativo-informatizado-para-la-medici%C3%B3n-del-ajuste01738nas a2200181 4500008004100000245014500041210006900186300001200255490000700267520104600274653003001320653002401350653001501374100001301389700001701402700002101419856011601440 2005 eng d00aA randomized experiment to compare conventional, computerized, and computerized adaptive administration of ordinal polytomous attitude items0 arandomized experiment to compare conventional computerized and c a159-1830 v293 aA total of 520 high school students were randomly assigned to a paper-and-pencil test (PPT), a computerized standard test (CST), or a computerized adaptive test (CAT) version of the Dutch School Attitude Questionnaire (SAQ), consisting of ordinal polytomous items. The CST administered items in the same order as the PPT. The CAT administered all items of three SAQ subscales in adaptive order using Samejima's graded response model, so that six different stopping rule settings could be applied afterwards. School marks were used as external criteria. Results showed significant but small multivariate administration mode effects on conventional raw scores and small to medium effects on maximum likelihood latent trait estimates. When the precision of CAT latent trait estimates decreased, correlations with grade point average in general decreased. However, the magnitude of the decrease was not very large as compared to the PPT, the CST, and the CAT without the stopping rule. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10aComputer Assisted Testing10aTest Administration10aTest Items1 aHol, A M1 aVorst, H C M1 aMellenbergh, G J uhttp://mail.iacat.org/content/randomized-experiment-compare-conventional-computerized-and-computerized-adaptive01473nas a2200217 4500008004100000245004600041210004600087300001200133490000700145520085100152653001801003653002501021653003201046653001301078653002201091653002401113653001501137100001601152700001501168856007201183 2005 eng d00aTest construction for cognitive diagnosis0 aTest construction for cognitive diagnosis a262-2770 v293 aAlthough cognitive diagnostic models (CDMs) can be useful in the analysis and interpretation of existing tests, little has been developed to specify how one might construct a good test using aspects of the CDMs. This article discusses the derivation of a general CDM index based on Kullback-Leibler information that will serve as a measure of how informative an item is for the classification of examinees. The effectiveness of the index is examined for items calibrated using the deterministic input noisy "and" gate model (DINA) and the reparameterized unified model (RUM) by implementing a simple heuristic to construct a test from an item bank. When compared to randomly constructed tests from the same item bank, the heuristic shows significant improvement in classification rates. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10a(Measurement)10aCognitive Assessment10aItem Analysis (Statistical)10aProfiles10aTest Construction10aTest Interpretation10aTest Items1 aHenson, R K1 aDouglas, J uhttp://mail.iacat.org/content/test-construction-cognitive-diagnosis01669nas a2200229 4500008004100000245005500041210005300096300000800149490000700157520102900164653002101193653001201214653003001226653001801256653000901274100001701283700001201300700001501312700001601327700001401343856008201357 2004 eng d00aAssisted self-adapted testing: A comparative study0 aAssisted selfadapted testing A comparative study a2-90 v203 aA new type of self-adapted test (S-AT), called Assisted Self-Adapted Test (AS-AT), is presented. It differs from an ordinary S-AT in that prior to selecting the difficulty category, the computer advises examinees on their best difficulty category choice, based on their previous performance. Three tests (computerized adaptive test, AS-AT, and S-AT) were compared regarding both their psychometric (precision and efficiency) and psychological (anxiety) characteristics. Tests were applied in an actual assessment situation, in which test scores determined 20% of term grades. A sample of 173 high school students participated. Neither differences in posttest anxiety nor ability were obtained. Concerning precision, AS-AT was as precise as CAT, and both revealed more precision than S-AT. It was concluded that AS-AT acted as a CAT concerning precision. Some hints, but not conclusive support, of the psychological similarity between AS-AT and S-AT was also found. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10aAdaptive Testing10aAnxiety10aComputer Assisted Testing10aPsychometrics10aTest1 aHontangas, P1 aOlea, J1 aPonsoda, V1 aRevuelta, J1 aWise, S L uhttp://mail.iacat.org/content/assisted-self-adapted-testing-comparative-study01802nas a2200361 4500008004100000020002200041245009500063210006900158250001500227260001100242300001000253490000700263520066300270653002500933653002900958653001000987653000900997653002201006653004501028653003701073653001101110653001101121653000901132653001601141653003601157653003001193653003401223100001601257700002401273700001001297700001501307856011801322 2004 eng d a1074-9357 (Print)00aComputer adaptive testing: a strategy for monitoring stroke rehabilitation across settings0 aComputer adaptive testing a strategy for monitoring stroke rehab a2004/05/01 cSpring a33-390 v113 aCurrent functional assessment instruments in stroke rehabilitation are often setting-specific and lack precision, breadth, and/or feasibility. Computer adaptive testing (CAT) offers a promising potential solution by providing a quick, yet precise, measure of function that can be used across a broad range of patient abilities and in multiple settings. CAT technology yields a precise score by selecting very few relevant items from a large and diverse item pool based on each individual's responses. We demonstrate the potential usefulness of a CAT assessment model with a cross-sectional sample of persons with stroke from multiple rehabilitation settings.10a*Computer Simulation10a*User-Computer Interface10aAdult10aAged10aAged, 80 and over10aCerebrovascular Accident/*rehabilitation10aDisabled Persons/*classification10aFemale10aHumans10aMale10aMiddle Aged10aMonitoring, Physiologic/methods10aSeverity of Illness Index10aTask Performance and Analysis1 aAndres, P L1 aBlack-Schaffer, R M1 aNi, P1 aHaley, S M uhttp://mail.iacat.org/content/computer-adaptive-testing-strategy-monitoring-stroke-rehabilitation-across-settings01544nas a2200229 4500008004100000020002200041245006400063210006300127260002600190300001200216490000700228520081800235653003401053653003001087653002801117653001301145100001901158700001501177700001601192700001701208856008901225 2004 eng d a0146-6216 (Print)00aComputerized adaptive testing with multiple-form structures0 aComputerized adaptive testing with multipleform structures bSage Publications: US a147-1640 v283 aA multiple-form structure (MFS) is an ordered collection or network of testlets (i.e., sets of items). An examinee's progression through the network of testlets is dictated by the correctness of an examinee's answers, thereby adapting the test to his or her trait level. The collection of paths through the network yields the set of all possible test forms, allowing test specialists the opportunity to review them before they are administered. Also, limiting the exposure of an individual MFS to a specific period of time can enhance test security. This article provides an overview of methods that have been developed to generate parallel MFSs. The approach is applied to the assembly of an experimental computerized Law School Admission Test (LSAT). (PsycINFO Database Record (c) 2007 APA, all rights reserved)10acomputerized adaptive testing10aLaw School Admission Test10amultiple-form structure10atestlets1 aArmstrong, R D1 aJones, D H1 aKoppel, N B1 aPashley, P J uhttp://mail.iacat.org/content/computerized-adaptive-testing-multiple-form-structures02473nas a2200193 4500008004100000245010700041210007100148300001200219490000700231520172200238653002101960653003401981653001602015653003002031653002302061653004502084100001502129856013502144 2004 eng d00aÉvaluation et multimédia dans l'apprentissage d'une L2 [Assessment and multimedia in learning an L2]0 aÉvaluation et multimédia dans lapprentissage dune L2 Assessment  a475-4870 v163 aIn the first part of this paper different areas where technology may be used for second language assessment are described. First, item banking operations, which are generally based on item Response Theory but not necessarily restricted to dichotomously scored items, facilitate assessment task organization and require technological support. Second, technology may help to design more authentic assessment tasks or may be needed in some direct testing situations. Third, the assessment environment may be more adapted and more stimulating when technology is used to give the student more control. The second part of the paper presents different functions of assessment. The monitoring function (often called formative assessment) aims at adapting the classroom activities to students and to provide continuous feedback. Technology may be used to train the teachers in monitoring techniques, to organize data or to produce diagnostic information; electronic portfolios or quizzes that are built in some educational software may also be used for monitoring. The placement function is probably the one in which the application of computer adaptive testing procedures (e.g. French CAPT) is the most appropriate. Automatic scoring devices may also be used for placement purposes. Finally the certification function requires more valid and more reliable tools. Technology may be used to enhance the testing situation (to make it more authentic) or to facilitate data processing during the construction of a test. Almond et al. (2002) propose a four component model (Selection, Presentation, Scoring and Response) for designing assessment systems. Each component must be planned taking into account the assessment function. 10aAdaptive Testing10aComputer Assisted Instruction10aEducational10aForeign Language Learning10aProgram Evaluation10aTechnology computerized adaptive testing1 aLaurier, M uhttp://mail.iacat.org/content/%C3%A9valuation-et-multim%C3%A9dia-dans-lapprentissage-dune-l2-assessment-and-multimedia-learning-l201774nas a2200193 4500008004100000245023000041210006900271300000900340490000700349520093900356653002101295653003001316653001801346653001601364653004201380100001201422700001901434856012701453 2004 eng d00aKann die Konfundierung von Konzentrationsleistung und Aktivierung durch adaptives Testen mit dern FAKT vermieden werden? [Avoiding the confounding of concentration performance and activation by adaptive testing with the FACT]0 aKann die Konfundierung von Konzentrationsleistung und Aktivierun a1-170 v253 aThe study investigates the effect of computerized adaptive testing strategies on the confounding of concentration performance with activation. A sample of 54 participants was administered 1 out of 3 versions (2 adaptive, 1 non-adaptive) of the computerized Frankfurt Adaptive Concentration Test FACT (Moosbrugger & Heyden, 1997) at three subsequent points in time. During the test administration changes in activation (electrodermal activity) were recorded. The results pinpoint a confounding of concentration performance with activation for the non-adaptive test version, but not for the adaptive test versions (p = .01). Thus, adaptive FACT testing strategies can remove the confounding of concentration performance with activation, thereby increasing the discriminant validity. In conclusion, an attention-focusing-hypothesis is formulated to explain the observed effect. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10aAdaptive Testing10aComputer Assisted Testing10aConcentration10aPerformance10aTesting computerized adaptive testing1 aFrey, A1 aMoosbrugger, H uhttp://mail.iacat.org/content/kann-die-konfundierung-von-konzentrationsleistung-und-aktivierung-durch-adaptives-testen-mit01880nas a2200169 4500008004100000245013100041210006900172300001200241490000700253520122700260653003001487653002501517653001501542653001601557100001601573856012101589 2004 eng d00aUsing patterns of summed scores in paper-and-pencil tests and computer-adaptive tests to detect misfitting item score patterns0 aUsing patterns of summed scores in paperandpencil tests and comp a119-1360 v413 aTwo new methods have been proposed to determine unexpected sum scores on subtests (testlets) both for paper-and-pencil tests and computer adaptive tests. A method based on a conservative bound using the hypergeometric distribution, denoted ρ, was compared with a method where the probability for each score combination was calculated using a highest density region (HDR). Furthermore, these methods were compared with the standardized log-likelihood statistic with and without a correction for the estimated latent trait value (denoted as l-super(*)-sub(z) and l-sub(z), respectively). Data were simulated on the basis of the one-parameter logistic model, and both parametric and nonparametric logistic regression was used to obtain estimates of the latent trait. Results showed that it is important to take the trait level into account when comparing subtest scores. In a nonparametric item response theory (IRT) context, on adapted version of the HDR method was a powerful alterative to ρ. In a parametric IRT context, results showed that l-super(*)-sub(z) had the highest power when the data were simulated conditionally on the estimated latent trait level. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10aComputer Assisted Testing10aItem Response Theory10aperson Fit10aTest Scores1 aMeijer, R R uhttp://mail.iacat.org/content/using-patterns-summed-scores-paper-and-pencil-tests-and-computer-adaptive-tests-detect01228nas a2200193 4500008004100000245002900041210002900070260003200099300001200131520066800143653003000811653003200841653001400873653004500887100001200932700001500944700001600959856005900975 2003 eng d00aAssessing question banks0 aAssessing question banks aLondon, UKbKogan Page Ltd. a171-2303 aIn Chapter 14, Joanna Bull and James Daziel provide a comprehensive treatment of the issues surrounding the use of Question Banks and Computer Assisted Assessment, and provide a number of excellent examples of implementations. In their review of the technologies employed in Computer Assisted Assessment the authors include Computer Adaptive Testing and data generation. The authors reveal significant issues involving the impact of Intellectual Property rights and computer assisted assessment and make important suggestions for strategies to overcome these obstacles. (PsycINFO Database Record (c) 2005 APA )http://www-jime.open.ac.uk/2003/1/ (journal abstract)10aComputer Assisted Testing10aCurriculum Based Assessment10aEducation10aTechnology computerized adaptive testing1 aBull, J1 aDalziel, J1 aVreeland, T uhttp://mail.iacat.org/content/assessing-question-banks01762nas a2200313 4500008004100000245007700041210006900118300001200187490000700199520083400206653002101040653001501061653001701076653001601093653003001109653001701139653001501156653002501171653002001196653001501216653002501231653001801256653000901274100001801283700001201301700001601313700001601329856010301345 2003 eng d00aComputerized adaptive rating scales for measuring managerial performance0 aComputerized adaptive rating scales for measuring managerial per a237-2460 v113 aComputerized adaptive rating scales (CARS) had been developed to measure contextual or citizenship performance. This rating format used a paired-comparison protocol, presenting pairs of behavioral statements scaled according to effectiveness levels, and an iterative item response theory algorithm to obtain estimates of ratees' citizenship performance (W. C. Borman et al, 2001). In the present research, we developed CARS to measure the entire managerial performance domain, including task and citizenship performance, thus addressing a major limitation of the earlier CARS. The paper describes this development effort, including an adjustment to the algorithm that reduces substantially the number of item pairs required to obtain almost as much precision in the performance estimates. (PsycINFO Database Record (c) 2005 APA )10aAdaptive Testing10aAlgorithms10aAssociations10aCitizenship10aComputer Assisted Testing10aConstruction10aContextual10aItem Response Theory10aJob Performance10aManagement10aManagement Personnel10aRating Scales10aTest1 aSchneider, RJ1 aGoff, M1 aAnderson, S1 aBorman, W C uhttp://mail.iacat.org/content/computerized-adaptive-rating-scales-measuring-managerial-performance01437nas a2200205 4500008004100000245008800041210007000129300001200199490000700211520067700218653002100895653003000916653002400946653002500970653002600995653005201021100001901073700002301092856011601115 2003 eng d00aOptimal stratification of item pools in α-stratified computerized adaptive testing0 aOptimal stratification of item pools in αstratified computerized a262-2740 v273 aA method based on 0-1 linear programming (LP) is presented to stratify an item pool optimally for use in α-stratified adaptive testing. Because the 0-1 LP model belongs to the subclass of models with a network flow structure, efficient solutions are possible. The method is applied to a previous item pool from the computerized adaptive testing (CAT) version of the Graduate Record Exams (GRE) Quantitative Test. The results indicate that the new method performs well in practical situations. It improves item exposure control, reduces the mean squared error in the θ estimates, and increases test reliability. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10aAdaptive Testing10aComputer Assisted Testing10aItem Content (Test)10aItem Response Theory10aMathematical Modeling10aTest Construction computerized adaptive testing1 aChang, Hua-Hua1 avan der Linden, WJ uhttp://mail.iacat.org/content/optimal-stratification-item-pools-%CE%B1-stratified-computerized-adaptive-testing01802nas a2200241 4500008004100000245009300041210006900134300001200203490000700215520098200222653001801204653002101222653003001243653001901273653004601292653001801338653002501356653001501381100001401396700001901410700001501429856011601444 2003 eng d00aThe relationship between item exposure and test overlap in computerized adaptive testing0 arelationship between item exposure and test overlap in computeri a129-1450 v403 aThe purpose of this article is to present an analytical derivation for the mathematical form of an average between-test overlap index as a function of the item exposure index, for fixed-length computerized adaptive tests (CATs). This algebraic relationship is used to investigate the simultaneous control of item exposure at both the item and test levels. The results indicate that, in fixed-length CATs, control of the average between-test overlap is achieved via the mean and variance of the item exposure rates of the items that constitute the CAT item pool. The mean of the item exposure rates is easily manipulated. Control over the variance of the item exposure rates can be achieved via the maximum item exposure rate (r-sub(max)). Therefore, item exposure control methods which implement a specification of r-sub(max) (e.g., J. B. Sympson and R. D. Hetter, 1985) provide the most direct control at both the item and test levels. (PsycINFO Database Record (c) 2005 APA )10a(Statistical)10aAdaptive Testing10aComputer Assisted Testing10aHuman Computer10aInteraction computerized adaptive testing10aItem Analysis10aItem Analysis (Test)10aTest Items1 aChen, S-Y1 aAnkenmann, R D1 aSpray, J A uhttp://mail.iacat.org/content/relationship-between-item-exposure-and-test-overlap-computerized-adaptive-testing01521nas a2200157 4500008004100000245009500041210006900136300001200205490000700217520090100224653002101125653003001146653004501176100002301221856011901244 2003 eng d00aSome alternatives to Sympson-Hetter item-exposure control in computerized adaptive testing0 aSome alternatives to SympsonHetter itemexposure control in compu a249-2650 v283 aTheHetter and Sympson (1997; 1985) method is a method of probabilistic item-exposure control in computerized adaptive testing. Setting its control parameters to admissible values requires an iterative process of computer simulations that has been found to be time consuming, particularly if the parameters have to be set conditional on a realistic set of values for the examinees’ ability parameter. Formal properties of the method are identified that help us explain why this iterative process can be slow and does not guarantee admissibility. In addition, some alternatives to the SH method are introduced. The behavior of these alternatives was estimated for an adaptive test from an item pool from the Law School Admission Test (LSAT). Two of the alternatives showed attractive behavior and converged smoothly to admissibility for all items in a relatively small number of iteration steps. 10aAdaptive Testing10aComputer Assisted Testing10aTest Items computerized adaptive testing1 avan der Linden, WJ uhttp://mail.iacat.org/content/some-alternatives-sympson-hetter-item-exposure-control-computerized-adaptive-testing01779nas a2200217 4500008004100000245006200041210006200103260005600165300001200221520108400233653002401317653001601341653001401357653002201371653001501393653001801408653001301426100001901439700001601458856008701474 2002 eng d00aGenerating abstract reasoning items with cognitive theory0 aGenerating abstract reasoning items with cognitive theory aMahwah, N.J. USAbLawrence Erlbaum Associates, Inc. a219-2503 a(From the chapter) Developed and evaluated a generative system for abstract reasoning items based on cognitive theory. The cognitive design system approach was applied to generate matrix completion problems. Study 1 involved developing the cognitive theory with 191 college students who were administered Set I and Set II of the Advanced Progressive Matrices. Study 2 examined item generation by cognitive theory. Study 3 explored the psychometric properties and construct representation of abstract reasoning test items with 728 young adults. Five structurally equivalent forms of Abstract Reasoning Test (ART) items were prepared from the generated item bank and administered to the Ss. In Study 4, the nomothetic span of construct validity of the generated items was examined with 728 young adults who were administered ART items, and 217 young adults who were administered ART items and the Advanced Progressive Matrices. Results indicate the matrix completion items were effectively generated by the cognitive design system approach. (PsycINFO Database Record (c) 2005 APA )10aCognitive Processes10aMeasurement10aReasoning10aTest Construction10aTest Items10aTest Validity10aTheories1 aEmbretson, S E1 aKyllomen, P uhttp://mail.iacat.org/content/generating-abstract-reasoning-items-cognitive-theory02014nas a2200205 4500008004100000245008200041210006900123300001200192490000700204520132300211653002101534653001501555653003001570653001101600653000901611653004701620100001901667700001301686856010901699 2002 eng d00aHypergeometric family and item overlap rates in computerized adaptive testing0 aHypergeometric family and item overlap rates in computerized ada a387-3980 v673 aA computerized adaptive test (CAT) is usually administered to small groups of examinees at frequent time intervals. It is often the case that examinees who take the test earlier share information with examinees who will take the test later, thus increasing the risk that many items may become known. Item overlap rate for a group of examinees refers to the number of overlapping items encountered by these examinees divided by the test length. For a specific item pool, different item selection algorithms may yield different item overlap rates. An important issue in designing a good CAT item selection algorithm is to keep item overlap rate below a preset level. In doing so, it is important to investigate what the lowest rate could be for all possible item selection algorithms. In this paper we rigorously prove that if every item had an equal possibility to be selected from the pool in a fixed-length CAT, the number of overlapping item among any α randomly sampled examinees follows the hypergeometric distribution family for α ≥ 1. Thus, the expected values of the number of overlapping items among any randomly sampled α examinee can be calculated precisely. These values may serve as benchmarks in controlling item overlap rates for fixed-length adaptive tests. (PsycINFO Database Record (c) 2005 APA )10aAdaptive Testing10aAlgorithms10aComputer Assisted Testing10aTaking10aTest10aTime On Task computerized adaptive testing1 aChang, Hua-Hua1 aZhang, J uhttp://mail.iacat.org/content/hypergeometric-family-and-item-overlap-rates-computerized-adaptive-testing01218nas a2200217 4500008004100000245005100041210005100092300001200143490000700155520060500162653002600767653003000793653001600823653001300839653001300852653001100865653001200876653001500888100001600903856008100919 2002 eng d00aInformation technology and literacy assessment0 aInformation technology and literacy assessment a369-3730 v183 aThis column discusses information technology and literacy assessment in the past and present. The author also describes computer-based assessments today including the following topics: computer-scored testing, computer-administered formal assessment, Internet formal assessment, computerized adaptive tests, placement tests, informal assessment, electronic portfolios, information management, and Internet information dissemination. A model of the major present-day applications of information technologies in reading and literacy assessment is also included. (PsycINFO Database Record (c) 2005 APA )10aComputer Applications10aComputer Assisted Testing10aInformation10aInternet10aLiteracy10aModels10aSystems10aTechnology1 aBalajthy, E uhttp://mail.iacat.org/content/information-technology-and-literacy-assessment01694nas a2200277 4500008004100000020001000041245006500051210006400116260009700180300001100277520074500288653002101033653002201054653002501076653002801101653002501129653001601154653001801170653005501188653001501243653001201258100001801270700002301288700001301311856009201324 2002 eng d a02-0900aMathematical-programming approaches to test item pool design0 aMathematicalprogramming approaches to test item pool design aTwente, The NetherlandsbUniversity of Twente, Faculty of Educational Science and Technology a93-1083 a(From the chapter) This paper presents an approach to item pool design that has the potential to improve on the quality of current item pools in educational and psychological testing and hence to increase both measurement precision and validity. The approach consists of the application of mathematical programming techniques to calculate optimal blueprints for item pools. These blueprints can be used to guide the item-writing process. Three different types of design problems are discussed, namely for item pools for linear tests, item pools computerized adaptive testing (CAT), and systems of rotating item pools for CAT. The paper concludes with an empirical example of the problem of designing a system of rotating item pools for CAT.10aAdaptive Testing10aComputer Assisted10aComputer Programming10aEducational Measurement10aItem Response Theory10aMathematics10aPsychometrics10aStatistical Rotation computerized adaptive testing10aTest Items10aTesting1 aVeldkamp, B P1 avan der Linden, WJ1 aAriel, A uhttp://mail.iacat.org/content/mathematical-programming-approaches-test-item-pool-design01632nas a2200241 4500008004100000245005900041210005800100300001200158490000700170520087100177653002101048653003401069653002801103653002001131653003201151653002501183653001501208653002701223653002201250653001601272100001601288856008601304 2002 eng d00aOutlier detection in high-stakes certification testing0 aOutlier detection in highstakes certification testing a219-2330 v393 aDiscusses recent developments of person-fit analysis in computerized adaptive testing (CAT). Methods from statistical process control are presented that have been proposed to classify an item score pattern as fitting or misfitting the underlying item response theory model in CAT Most person-fit research in CAT is restricted to simulated data. In this study, empirical data from a certification test were used. Alternatives are discussed to generate norms so that bounds can be determined to classify an item score pattern as fitting or misfitting. Using bounds determined from a sample of a high-stakes certification test, the empirical analysis showed that different types of misfit can be distinguished Further applications using statistical process control methods to detect misfitting item score patterns are discussed. (PsycINFO Database Record (c) 2005 APA )10aAdaptive Testing10acomputerized adaptive testing10aEducational Measurement10aGoodness of Fit10aItem Analysis (Statistical)10aItem Response Theory10aperson Fit10aStatistical Estimation10aStatistical Power10aTest Scores1 aMeijer, R R uhttp://mail.iacat.org/content/outlier-detection-high-stakes-certification-testing02035nas a2200253 4500008004100000245010900041210006900150300000900219490000600228520114100234653002101375653001501396653003901411653002201450653002501472653001801497653002201515653005501537653001501592653001201607100001701619700002501636856012001661 2002 eng d00aA structure-based approach to psychological measurement: Matching measurement models to latent structure0 astructurebased approach to psychological measurement Matching me a4-160 v93 aThe present article sets forth the argument that psychological assessment should be based on a construct's latent structure. The authors differentiate dimensional (continuous) and taxonic (categorical) structures at the latent and manifest levels and describe the advantages of matching the assessment approach to the latent structure of a construct. A proper match will decrease measurement error, increase statistical power, clarify statistical relationships, and facilitate the location of an efficient cutting score when applicable. Thus, individuals will be placed along a continuum or assigned to classes more accurately. The authors briefly review the methods by which latent structure can be determined and outline a structure-based approach to assessment that builds on dimensional scaling models, such as item response theory, while incorporating classification methods as appropriate. Finally, the authors empirically demonstrate the utility of their approach and discuss its compatibility with traditional assessment methods and with computerized adaptive testing. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10aAdaptive Testing10aAssessment10aClassification (Cognitive Process)10aComputer Assisted10aItem Response Theory10aPsychological10aScaling (Testing)10aStatistical Analysis computerized adaptive testing10aTaxonomies10aTesting1 aRuscio, John1 aRuscio, Ayelet Meron uhttp://mail.iacat.org/content/structure-based-approach-psychological-measurement-matching-measurement-models-latent01700nas a2200229 4500008004100000245007800041210006900119300001200188490000700200520094500207653002501152653005101177653003001228653001801258653001101276653002701287653001101314100001701325700001101342700001801353856009901371 2001 eng d00aComputerized adaptive testing with the generalized graded unfolding model0 aComputerized adaptive testing with the generalized graded unfold a177-1960 v253 aExamined the use of the generalized graded unfolding model (GGUM) in computerized adaptive testing. The objective was to minimize the number of items required to produce equiprecise estimates of person locations. Simulations based on real data about college student attitudes toward abortion and on data generated to fit the GGUM were used. It was found that as few as 7 or 8 items were needed to produce accurate and precise person estimates using an expected a posteriori procedure. The number items in the item bank (20, 40, or 60 items) and their distribution on the continuum (uniform locations or item clusters in moderately extreme locations) had only small effects on the accuracy and precision of the estimates. These results suggest that adaptive testing with the GGUM is a good method for achieving estimates with an approximately uniform level of precision using a small number of items. (PsycINFO Database Record (c) 2005 APA )10aAttitude Measurement10aCollege Students computerized adaptive testing10aComputer Assisted Testing10aItem Response10aModels10aStatistical Estimation10aTheory1 aRoberts, J S1 aLin, Y1 aLaughlin, J E uhttp://mail.iacat.org/content/computerized-adaptive-testing-generalized-graded-unfolding-model01618nas a2200193 4500008004100000245008600041210006900127300001200196490000700208520094500215653002101160653003001181653004101211653000901252653001701261100001601278700001701294856011301311 2001 eng d00aDifferences between self-adapted and computerized adaptive tests: A meta-analysis0 aDifferences between selfadapted and computerized adaptive tests  a235-2470 v383 aSelf-adapted testing has been described as a variation of computerized adaptive testing that reduces test anxiety and thereby enhances test performance. The purpose of this study was to gain a better understanding of these proposed effects of self-adapted tests (SATs); meta-analysis procedures were used to estimate differences between SATs and computerized adaptive tests (CATs) in proficiency estimates and post-test anxiety levels across studies in which these two types of tests have been compared. After controlling for measurement error the results showed that SATs yielded proficiency estimates that were 0.12 standard deviation units higher and post-test anxiety levels that were 0.19 standard deviation units lower than those yielded by CATs. The authors speculate about possible reasons for these differences and discuss advantages and disadvantages of using SATs in operational settings. (PsycINFO Database Record (c) 2005 APA )10aAdaptive Testing10aComputer Assisted Testing10aScores computerized adaptive testing10aTest10aTest Anxiety1 aPitkin, A K1 aVispoel, W P uhttp://mail.iacat.org/content/differences-between-self-adapted-and-computerized-adaptive-tests-meta-analysis01841nas a2200229 4500008004100000245007400041210006900115300001000184490000700194520110700201653002101308653000901329653004801338653001801386653002801404653002401432653001501456100001601471700001701487700001601504856009101520 2001 eng d00aEvaluation of an MMPI-A short form: Implications for adaptive testing0 aEvaluation of an MMPIA short form Implications for adaptive test a76-890 v763 aReports some psychometric properties of an MMPI-Adolescent version (MMPI-A; J. N. Butcher et al, 1992) short form based on administration of the 1st 150 items of this test instrument. The authors report results for both the MMPI-A normative sample of 1,620 adolescents (aged 14-18 yrs) and a clinical sample of 565 adolescents (mean age 15.2 yrs) in a variety of treatment settings. The authors summarize results for the MMPI-A basic scales in terms of Pearson product-moment correlations generated between full administration and short-form administration formats and mean T score elevations for the basic scales generated by each approach. In this investigation, the authors also examine single-scale and 2-point congruences found for the MMPI-A basic clinical scales as derived from standard and short-form administrations. The authors present the relative strengths and weaknesses of the MMPI-A short form and discuss the findings in terms of implications for attempts to shorten the item pool through the use of computerized adaptive assessment approaches. (PsycINFO Database Record (c) 2005 APA )10aAdaptive Testing10aMean10aMinnesota Multiphasic Personality Inventory10aPsychometrics10aStatistical Correlation10aStatistical Samples10aTest Forms1 aArcher, R P1 aTirrell, C A1 aElkins, D E uhttp://mail.iacat.org/content/evaluation-mmpi-short-form-implications-adaptive-testing02102nas a2200337 4500008004100000245014400041210006900185300001200254490000700266520096600273653002501239653003601264653002501300653001001325653003001335653001101365653001001376653000901386653003101395653003201426653003601458653003401494653002001528100001601548700001401564700001601578700001901594700001301613700001501626856012301641 2001 eng d00aAn examination of the comparative reliability, validity, and accuracy of performance ratings made using computerized adaptive rating scales0 aexamination of the comparative reliability validity and accuracy a965-9730 v863 aThis laboratory research compared the reliability, validity, and accuracy of a computerized adaptive rating scale (CARS) format and 2 relatively common and representative rating formats. The CARS is a paired-comparison rating task that uses adaptive testing principles to present pairs of scaled behavioral statements to the rater to iteratively estimate a ratee's effectiveness on 3 dimensions of contextual performance. Videotaped vignettes of 6 office workers were prepared, depicting prescripted levels of contextual performance, and 112 subjects rated these vignettes using the CARS format and one or the other competing format. Results showed 23%-37% lower standard errors of measurement for the CARS format. In addition, validity was significantly higher for the CARS format (d = .18), and Cronbach's accuracy coefficients showed significantly higher accuracy, with a median effect size of .08. The discussion focuses on possible reasons for the results.10a*Computer Simulation10a*Employee Performance Appraisal10a*Personnel Selection10aAdult10aAutomatic Data Processing10aFemale10aHuman10aMale10aReproducibility of Results10aSensitivity and Specificity10aSupport, U.S. Gov't, Non-P.H.S.10aTask Performance and Analysis10aVideo Recording1 aBorman, W C1 aBuck, D E1 aHanson, M A1 aMotowidlo, S J1 aStark, S1 aDrasgow, F uhttp://mail.iacat.org/content/examination-comparative-reliability-validity-and-accuracy-performance-ratings-made-using01466nas a2200253 4500008004100000245013900041210006900180260005000249300001200299520053700311653002100848653002500869653001200894653002900906653002200935653002700957653002600984653001501010653001601025100001501041700001601056700001701072856012301089 2001 eng d00aItem response theory applied to combinations of multiple-choice and constructed-response items--approximation methods for scale scores0 aItem response theory applied to combinations of multiplechoice a aMahwah, N.J. USAbLawrence Erlbaum Associates a289-3153 a(From the chapter) The authors develop approximate methods that replace the scoring tables with weighted linear combinations of the component scores. Topics discussed include: a linear approximation for the extension to combinations of scores; the generalization of two or more scores; potential applications of linear approximations to item response theory in computerized adaptive tests; and evaluation of the pattern-of-summed-scores, and Gaussian approximation, estimates of proficiency. (PsycINFO Database Record (c) 2005 APA )10aAdaptive Testing10aItem Response Theory10aMethod)10aMultiple Choice (Testing10aScoring (Testing)10aStatistical Estimation10aStatistical Weighting10aTest Items10aTest Scores1 aThissen, D1 aNelson, L A1 aSwygert, K A uhttp://mail.iacat.org/content/item-response-theory-applied-combinations-multiple-choice-and-constructed-response-items01986nas a2200205 4500008004100000245010100041210006900142300001200211490000700223520124900230653001201479653002101491653003001512653001501542653001601557653004501573100001701618700001901635856012601654 2001 eng d00aItem selection in computerized adaptive testing: Should more discriminating items be used first?0 aItem selection in computerized adaptive testing Should more disc a249-2660 v383 aDuring computerized adaptive testing (CAT), items are selected continuously according to the test-taker's estimated ability. Test security has become a problem because high-discrimination items are more likely to be selected and become overexposed. So, there seems to be a tradeoff between high efficiency in ability estimations and balanced usage of items. This series of four studies addressed the dilemma by focusing on the notion of whether more or less discriminating items should be used first in CAT. The first study demonstrated that the common maximum information method with J. B. Sympson and R. D. Hetter (1985) control resulted in the use of more discriminating items first. The remaining studies showed that using items in the reverse order, as described in H. Chang and Z. Yings (1999) stratified method had potential advantages: (a) a more balanced item usage and (b) a relatively stable resultant item pool structure with easy and inexpensive management. This stratified method may have ability-estimation efficiency better than or close to that of other methods. It is argued that the judicious selection of items, as in the stratified method, is a more active control of item exposure. (PsycINFO Database Record (c) 2005 APA )10aability10aAdaptive Testing10aComputer Assisted Testing10aEstimation10aStatistical10aTest Items computerized adaptive testing1 aHau, Kit-Tai1 aChang, Hua-Hua uhttp://mail.iacat.org/content/item-selection-computerized-adaptive-testing-should-more-discriminating-items-be-used-first01706nas a2200181 4500008004100000245007300041210006900114300001100183490000700194520110100201653002101302653003001323653002501353653001501378100001701393700001501410856009901425 2001 eng d00aOutlier measures and norming methods for computerized adaptive tests0 aOutlier measures and norming methods for computerized adaptive t a85-1040 v263 aNotes that the problem of identifying outliers has 2 important aspects: the choice of outlier measures and the method to assess the degree of outlyingness (norming) of those measures. Several classes of measures for identifying outliers in Computerized Adaptive Tests (CATs) are introduced. Some of these measures are constructed to take advantage of CATs' sequential choice of items; other measures are taken directly from paper and pencil (P&P) tests and are used for baseline comparisons. Assessing the degree of outlyingness of CAT responses, however, can not be applied directly from P&P tests because stopping rules associated with CATs yield examinee responses of varying lengths. Standard outlier measures are highly correlated with the varying lengths which makes comparison across examinees impossible. Therefore, 4 methods are presented and compared which map outlier statistics to a familiar probability scale (a p value). The methods are explored in the context of CAT data from a 1995 Nationally Administered Computerized Examination (NACE). (PsycINFO Database Record (c) 2005 APA )10aAdaptive Testing10aComputer Assisted Testing10aStatistical Analysis10aTest Norms1 aBradlow, E T1 aWeiss, R E uhttp://mail.iacat.org/content/outlier-measures-and-norming-methods-computerized-adaptive-tests01349nas a2200181 4500008004100000245007300041210006900114260005600183300001200239520069300251653002100944653003000965653002200995653002001017100001601037700001701053856009701070 2001 eng d00aPractical issues in setting standards on computerized adaptive tests0 aPractical issues in setting standards on computerized adaptive t aMahwah, N.J. USAbLawrence Erlbaum Associates, Inc. a355-3693 a(From the chapter) Examples of setting standards on computerized adaptive tests (CATs) are hard to find. Some examples of CATs involving performance standards include the registered nurse exam and the Novell systems engineer exam. Although CATs do not require separate standard setting-methods, there are special issues to be addressed by test specialist who set performance standards on CATs. Setting standards on a CAT will typical require modifications on the procedures used with more traditional, fixed-form, paper-and -pencil examinations. The purpose of this chapter is to illustrate why CATs pose special challenges to the standard setter. (PsycINFO Database Record (c) 2005 APA )10aAdaptive Testing10aComputer Assisted Testing10aPerformance Tests10aTesting Methods1 aSireci, S G1 aClauser, B E uhttp://mail.iacat.org/content/practical-issues-setting-standards-computerized-adaptive-tests01592nas a2200205 4500008004100000245016500041210006900206300001200275490000700287520076900294653002101063653002601084653003001110653002501140653005101165100001301216700001701229700002101246856011901267 2001 eng d00aToepassing van een computergestuurde adaptieve testprocedure op persoonlijkheidsdata [Application of a computerised adaptive test procedure on personality data]0 aToepassing van een computergestuurde adaptieve testprocedure op  a119-1330 v563 aStudied the applicability of a computerized adaptive testing procedure to an existing personality questionnaire within the framework of item response theory. The procedure was applied to the scores of 1,143 male and female university students (mean age 21.8 yrs) in the Netherlands on the Neuroticism scale of the Amsterdam Biographical Questionnaire (G. J. Wilde, 1963). The graded response model (F. Samejima, 1969) was used. The quality of the adaptive test scores was measured based on their correlation with test scores for the entire item bank and on their correlation with scores on other scales from the personality test. The results indicate that computerized adaptive testing can be applied to personality scales. (PsycINFO Database Record (c) 2005 APA )10aAdaptive Testing10aComputer Applications10aComputer Assisted Testing10aPersonality Measures10aTest Reliability computerized adaptive testing1 aHol, A M1 aVorst, H C M1 aMellenbergh, G J uhttp://mail.iacat.org/content/toepassing-van-een-computergestuurde-adaptieve-testprocedure-op-persoonlijkheidsdata02845nas a2200193 4500008004100000245013800041210006900179300000900248490000700257520208000264653003002344653002002374653005202394653002202446653001802468653002402486100001602510856012502526 2000 eng d00aThe development of a computerized version of Vandenberg's mental rotation test and the effect of visuo-spatial working memory loading0 adevelopment of a computerized version of Vandenbergs mental rota a39380 v603 aThis dissertation focused on the generation and evaluation of web-based versions of Vandenberg's Mental Rotation Test. Memory and spatial visualization theory were explored in relation to the addition of a visuo-spatial working memory component. Analysis of the data determined that there was a significant difference between scores on the MRT Computer and MRT Memory test. The addition of a visuo-spatial working memory component did significantly affect results at the .05 alpha level. Reliability and discrimination estimates were higher on the MRT Memory version. The computerization of the paper and pencil version on the MRT did not significantly effect scores but did effect the time required to complete the test. The population utilized in the quasi-experiment consisted of 107 university students from eight institutions in engineering graphics related courses. The subjects completed two researcher developed, Web-based versions of Vandenberg's Mental Rotation Test and the original paper and pencil version of the Mental Rotation Test. One version of the test included a visuo-spatial working memory loading. Significant contributions of this study included developing and evaluating computerized versions of Vandenberg's Mental Rotation Test. Previous versions of Vandenberg's Mental Rotation Test did not take advantage of the ability of the computer to incorporate an interaction factor, such as a visuo-spatial working memory loading, into the test. The addition of an interaction factor results in a more discriminate test which will lend itself well to computerized adaptive testing practices. Educators in engineering graphics related disciplines should strongly consider the use of spatial visualization tests to aid in establishing the effects of modern computer systems on fundamental design/drafting skills. Regular testing of spatial visualization skills will result assist in the creation of a more relevant curriculum. Computerized tests which are valid and reliable will assist in making this task feasible. (PsycINFO Database Record (c) 2005 APA )10aComputer Assisted Testing10aMental Rotation10aShort Term Memory computerized adaptive testing10aTest Construction10aTest Validity10aVisuospatial Memory1 aStrong, S D uhttp://mail.iacat.org/content/development-computerized-version-vandenbergs-mental-rotation-test-and-effect-visuo-spatial00717nas a2200193 4500008004100000245008400041210006900125300001400194490000700208653003000215653001100245653002500256653001600281653005500297653002200352653002300374100001800397856010800415 2000 eng d00aEmergence of item response modeling in instrument development and data analysis0 aEmergence of item response modeling in instrument development an aII60-II650 v3810aComputer Assisted Testing10aHealth10aItem Response Theory10aMeasurement10aStatistical Validity computerized adaptive testing10aTest Construction10aTreatment Outcomes1 aHambleton, RK uhttp://mail.iacat.org/content/emergence-item-response-modeling-instrument-development-and-data-analysis02316nas a2200205 4500008004100000245012100041210006900162300000800231490000700239520159200246653002101838653003001859653002201889653001801911653001601929653000901945653001801954100001401972856012401986 2000 eng d00aAn examination of the reliability and validity of performance ratings made using computerized adaptive rating scales0 aexamination of the reliability and validity of performance ratin a5700 v613 aThis study compared the psychometric properties of performance ratings made using recently-developed computerized adaptive rating scales (CARS) to the psyc hometric properties of ratings made using more traditional paper-and-pencil rati ng formats, i.e., behaviorally-anchored and graphic rating scales. Specifically, the reliability, validity and accuracy of the performance ratings from each for mat were examined. One hundred twelve participants viewed six 5-minute videotape s of office situations and rated the performance of a target person in each vide otape on three contextual performance dimensions-Personal Support, Organizationa l Support, and Conscientious Initiative-using CARS and either behaviorally-ancho red or graphic rating scales. Performance rating properties were measured using Shrout and Fleiss's intraclass correlation (2, 1), Borman's differential accurac y measure, and Cronbach's accuracy components as indexes of rating reliability, validity, and accuracy, respectively. Results found that performance ratings mad e using the CARS were significantly more reliable and valid than performance rat ings made using either of the other formats. Additionally, CARS yielded more acc urate performance ratings than the paper-and-pencil formats. The nature of the C ARS system (i.e., its adaptive nature and scaling methodology) and its paired co mparison judgment task are offered as possible reasons for the differences found in the psychometric properties of the performance ratings made using the variou s rating formats. (PsycINFO Database Record (c) 2005 APA )10aAdaptive Testing10aComputer Assisted Testing10aPerformance Tests10aRating Scales10aReliability10aTest10aTest Validity1 aBuck, D E uhttp://mail.iacat.org/content/examination-reliability-and-validity-performance-ratings-made-using-computerized-adaptive01173nas a2200205 4500008004100000245005600041210005300097300001200150490000700162520055300169653002200722653002500744653002500769653002200794653001500816100002300831700001800854700001500872856008000887 2000 eng d00aAn integer programming approach to item bank design0 ainteger programming approach to item bank design a139-1500 v243 aAn integer programming approach to item bank design is presented that can be used to calculate an optimal blueprint for an item bank, in order to support an existing testing program. The results are optimal in that they minimize the effort involved in producing the items as revealed by current item writing patterns. Also presented is an adaptation of the models, which can be used as a set of monitoring tools in item bank management. The approach is demonstrated empirically for an item bank that was designed for the Law School Admission Test. 10aAptitude Measures10aItem Analysis (Test)10aItem Response Theory10aTest Construction10aTest Items1 avan der Linden, WJ1 aVeldkamp, B P1 aReese, L M uhttp://mail.iacat.org/content/integer-programming-approach-item-bank-design