00733nas a2200193 4500008004500000245009100045210006900136300001000205490000700215653003500222653003400257653002900291653002600320100001900346700002700365700002400392700002100416856010200437 2023 Engldsh 00aHow Do Trait Change Patterns Affect the Performance of Adaptive Measurement of Change?0 aHow Do Trait Change Patterns Affect the Performance of Adaptive  a32-580 v1010aadaptive measurement of change10acomputerized adaptive testing10alongitudinal measurement10atrait change patterns1 aTai, Ming, Him1 aCooperman, Allison, W.1 aDeWeese, Joseph, N.1 aWeiss, David, J. uhttp://mail.iacat.org/how-do-trait-change-patterns-affect-performance-adaptive-measurement-change01973nas a2200145 4500008003900000245012800039210006900167300001200236490000700248520147000255100002001725700001801745700001901763856004501782 2020 d00aA Dynamic Stratification Method for Improving Trait Estimation in Computerized Adaptive Testing Under Item Exposure Control0 aDynamic Stratification Method for Improving Trait Estimation in  a182-1960 v443 aWhen computerized adaptive testing (CAT) is under stringent item exposure control, the precision of trait estimation will substantially decrease. A new item selection method, the dynamic Stratification method based on Dominance Curves (SDC), which is aimed at improving trait estimation, is proposed to mitigate this problem. The objective function of the SDC in item selection is to maximize the sum of test information for all examinees rather than maximizing item information for individual examinees at a single-item administration, as in conventional CAT. To achieve this objective, the SDC uses dominance curves to stratify an item pool into strata with the number being equal to the test length to precisely and accurately increase the quality of the administered items as the test progresses, reducing the likelihood that a high-discrimination item will be administered to an examinee whose ability is not close to the item difficulty. Furthermore, the SDC incorporates a dynamic process for on-the-fly item–stratum adjustment to optimize the use of quality items. Simulation studies were conducted to investigate the performance of the SDC in CAT under item exposure control at different levels of severity. According to the results, the SDC can efficiently improve trait estimation in CAT through greater precision and more accurate trait estimation than those generated by other methods (e.g., the maximum Fisher information method) in most conditions.1 aChen, Jyun-Hong1 aChao, Hsiu-Yi1 aChen, Shu-Ying uhttps://doi.org/10.1177/014662161984382002111nas a2200193 4500008003900000245007100039210006900110300000900179490000700188520153100195653000701726653003201733653001701765653002301782100001501805700001501820700001901835856006301854 2020 d00aItem Calibration Methods With Multiple Subscale Multistage Testing0 aItem Calibration Methods With Multiple Subscale Multistage Testi a3-280 v573 aAbstract Many large-scale educational surveys have moved from linear form design to multistage testing (MST) design. One advantage of MST is that it can provide more accurate latent trait (θ) estimates using fewer items than required by linear tests. However, MST generates incomplete response data by design; hence, questions remain as to how to calibrate items using the incomplete data from MST design. Further complication arises when there are multiple correlated subscales per test, and when items from different subscales need to be calibrated according to their respective score reporting metric. The current calibration-per-subscale method produced biased item parameters, and there is no available method for resolving the challenge. Deriving from the missing data principle, we showed when calibrating all items together the Rubin's ignorability assumption is satisfied such that the traditional single-group calibration is sufficient. When calibrating items per subscale, we proposed a simple modification to the current calibration-per-subscale method that helps reinstate the missing-at-random assumption and therefore corrects for the estimation bias that is otherwise existent. Three mainstream calibration methods are discussed in the context of MST, they are the marginal maximum likelihood estimation, the expectation maximization method, and the fixed parameter calibration. An extensive simulation study is conducted and a real data example from NAEP is analyzed to provide convincing empirical evidence.10aEM10amarginal maximum likelihood10amissing data10amultistage testing1 aWang, Chun1 aChen, Ping1 aJiang, Shengyu uhttps://onlinelibrary.wiley.com/doi/abs/10.1111/jedm.1224101507nas a2200157 4500008003900000245011800039210006900157300001200226490000700238520096800245100001901213700002001232700002101252700001301273856006301286 2020 d00aItem Selection and Exposure Control Methods for Computerized Adaptive Testing with Multidimensional Ranking Items0 aItem Selection and Exposure Control Methods for Computerized Ada a343-3690 v573 aAbstract The use of computerized adaptive testing algorithms for ranking items (e.g., college preferences, career choices) involves two major challenges: unacceptably high computation times (selecting from a large item pool with many dimensions) and biased results (enhanced preferences or intensified examinee responses because of repeated statements across items). To address these issues, we introduce subpool partition strategies for item selection and within-person statement exposure control procedures. Simulations showed that the multinomial method reduces computation time while maintaining measurement precision. Both the freeze and revised Sympson-Hetter online (RSHO) methods controlled the statement exposure rate; RSHO sacrificed some measurement precision but increased pool use. Furthermore, preventing a statement's repetition on consecutive items neither hindered the effectiveness of the freeze or RSHO method nor reduced measurement precision.1 aChen, Chia-Wen1 aWang, Wen-Chung1 aChiu, Ming, Ming1 aRo, Sage uhttps://onlinelibrary.wiley.com/doi/abs/10.1111/jedm.1225201760nas a2200145 4500008003900000245008000039210006900119300000900188490000700197520132100204100001601525700001501541700001301556856004501569 2020 d00aNew Efficient and Practicable Adaptive Designs for Calibrating Items Online0 aNew Efficient and Practicable Adaptive Designs for Calibrating I a3-160 v443 aWhen calibrating new items online, it is practicable to first compare all new items according to some criterion and then assign the most suitable one to the current examinee who reaches a seeding location. The modified D-optimal design proposed by van der Linden and Ren (denoted as D-VR design) works within this practicable framework with the aim of directly optimizing the estimation of item parameters. However, the optimal design point for a given new item should be obtained by comparing all examinees in a static examinee pool. Thus, D-VR design still has room for improvement in calibration efficiency from the view of traditional optimal design. To this end, this article incorporates the idea of traditional optimal design into D-VR design and proposes a new online calibration design criterion, namely, excellence degree (ED) criterion. Four different schemes are developed to measure the information provided by the current examinee when implementing this new criterion, and four new ED designs equipped with them are put forward accordingly. Simulation studies were conducted under a variety of conditions to compare the D-VR design and the four proposed ED designs in terms of calibration efficiency. Results showed that the four ED designs outperformed D-VR design in almost all simulation conditions.1 aHe, Yinhong1 aChen, Ping1 aLi, Yong uhttps://doi.org/10.1177/014662161882485402064nas a2200157 4500008003900000245009100039210006900130300001200199490000700211520157600218100001501794700001901809700001401828700001901842856004501861 2020 d00aStratified Item Selection Methods in Cognitive Diagnosis Computerized Adaptive Testing0 aStratified Item Selection Methods in Cognitive Diagnosis Compute a346-3610 v443 aCognitive diagnostic computerized adaptive testing (CD-CAT) aims to obtain more useful diagnostic information by taking advantages of computerized adaptive testing (CAT). Cognitive diagnosis models (CDMs) have been developed to classify examinees into the correct proficiency classes so as to get more efficient remediation, whereas CAT tailors optimal items to the examinee’s mastery profile. The item selection method is the key factor of the CD-CAT procedure. In recent years, a large number of parametric/nonparametric item selection methods have been proposed. In this article, the authors proposed a series of stratified item selection methods in CD-CAT, which are combined with posterior-weighted Kullback–Leibler (PWKL), nonparametric item selection (NPS), and weighted nonparametric item selection (WNPS) methods, and named S-PWKL, S-NPS, and S-WNPS, respectively. Two different types of stratification indices were used: original versus novel. The performances of the proposed item selection methods were evaluated via simulation studies and compared with the PWKL, NPS, and WNPS methods without stratification. Manipulated conditions included calibration sample size, item quality, number of attributes, number of strata, and data generation models. Results indicated that the S-WNPS and S-NPS methods performed similarly, and both outperformed the S-PWKL method. And item selection methods with novel stratification indices performed slightly better than the ones with original stratification indices, and those without stratification performed the worst.1 aYang, Jing1 aChang, Hua-Hua1 aTao, Jian1 aShi, Ningzhong uhttps://doi.org/10.1177/014662161989378301598nas a2200157 4500008003900000245012200039210006900161300001200230490000700242520104300249100002401292700001601316700002001332700002501352856006301377 2019 d00aComputerized Adaptive Testing in Early Education: Exploring the Impact of Item Position Effects on Ability Estimation0 aComputerized Adaptive Testing in Early Education Exploring the I a437-4510 v563 aAbstract Studies have shown that item difficulty can vary significantly based on the context of an item within a test form. In particular, item position may be associated with practice and fatigue effects that influence item parameter estimation. The purpose of this research was to examine the relevance of item position specifically for assessments used in early education, an area of testing that has received relatively limited psychometric attention. In an initial study, multilevel item response models fit to data from an early literacy measure revealed statistically significant increases in difficulty for items appearing later in a 20-item form. The estimated linear change in logits for an increase of 1 in position was .024, resulting in a predicted change of .46 logits for a shift from the beginning to the end of the form. A subsequent simulation study examined impacts of item position effects on person ability estimation within computerized adaptive testing. Implications and recommendations for practice are discussed.1 aAlbano, Anthony, D.1 aCai, Liuhan1 aLease, Erin, M.1 aMcConnell, Scott, R. uhttps://onlinelibrary.wiley.com/doi/abs/10.1111/jedm.1221501544nas a2200145 4500008003900000245009800039210006900137300001200206490000700218520104000225100002901265700002501294700003401319856004501353 2019 d00aImputation Methods to Deal With Missing Responses in Computerized Adaptive Multistage Testing0 aImputation Methods to Deal With Missing Responses in Computerize a495-5110 v793 aRouting examinees to modules based on their ability level is a very important aspect in computerized adaptive multistage testing. However, the presence of missing responses may complicate estimation of examinee ability, which may result in misrouting of individuals. Therefore, missing responses should be handled carefully. This study investigated multiple missing data methods in computerized adaptive multistage testing, including two imputation techniques, the use of full information maximum likelihood and the use of scoring missing data as incorrect. These methods were examined under the missing completely at random, missing at random, and missing not at random frameworks, as well as other testing conditions. Comparisons were made to baseline conditions where no missing data were present. The results showed that imputation and the full information maximum likelihood methods outperformed incorrect scoring methods in terms of average bias, average root mean square error, and correlation between estimated and true thetas.1 aCetin-Berber, Dee, Duygu1 aSari, Halil, Ibrahim1 aHuggins-Manley, Anne, Corinne uhttps://doi.org/10.1177/001316441880553201341nas a2200133 4500008003900000245010900039210006900148300001200217490000700229520088900236100001801125700001901143856004501162 2019 d00aItem Selection Criteria With Practical Constraints in Cognitive Diagnostic Computerized Adaptive Testing0 aItem Selection Criteria With Practical Constraints in Cognitive  a335-3570 v793 aFor item selection in cognitive diagnostic computerized adaptive testing (CD-CAT), ideally, a single item selection index should be created to simultaneously regulate precision, exposure status, and attribute balancing. For this purpose, in this study, we first proposed an attribute-balanced item selection criterion, namely, the standardized weighted deviation global discrimination index (SWDGDI), and subsequently formulated the constrained progressive index (CP\_SWDGDI) by casting the SWDGDI in a progressive algorithm. A simulation study revealed that the SWDGDI method was effective in balancing attribute coverage and the CP\_SWDGDI method was able to simultaneously balance attribute coverage and item pool usage while maintaining acceptable estimation precision. This research also demonstrates the advantage of a relatively low number of attributes in CD-CAT applications.1 aLin, Chuan-Ju1 aChang, Hua-Hua uhttps://doi.org/10.1177/001316441879063401915nas a2200145 4500008003900000245007200039210006900111300001200180490000700192520146600199100002001665700001801685700002101703856004501724 2019 d00aNonparametric CAT for CD in Educational Settings With Small Samples0 aNonparametric CAT for CD in Educational Settings With Small Samp a543-5610 v433 aCognitive diagnostic computerized adaptive testing (CD-CAT) has been suggested by researchers as a diagnostic tool for assessment and evaluation. Although model-based CD-CAT is relatively well researched in the context of large-scale assessment systems, this type of system has not received the same degree of research and development in small-scale settings, such as at the course-based level, where this system would be the most useful. The main obstacle is that the statistical estimation techniques that are successfully applied within the context of a large-scale assessment require large samples to guarantee reliable calibration of the item parameters and an accurate estimation of the examinees’ proficiency class membership. Such samples are simply not obtainable in course-based settings. Therefore, the nonparametric item selection (NPS) method that does not require any parameter calibration, and thus, can be used in small educational programs is proposed in the study. The proposed nonparametric CD-CAT uses the nonparametric classification (NPC) method to estimate an examinee’s attribute profile and based on the examinee’s item responses, the item that can best discriminate the estimated attribute profile and the other attribute profiles is then selected. The simulation results show that the NPS method outperformed the compared parametric CD-CAT algorithms and the differences were substantial when the calibration samples were small.1 aChang, Yuan-Pei1 aChiu, Chia-Yi1 aTsai, Rung-Ching uhttps://doi.org/10.1177/014662161881311301674nas a2200157 4500008003900000245014600039210006900185300001200254490000700266520111500273100001801388700001701406700001301423700001701436856006301453 2018 d00aEvaluation of a New Method for Providing Full Review Opportunities in Computerized Adaptive Testing—Computerized Adaptive Testing With Salt0 aEvaluation of a New Method for Providing Full Review Opportuniti a582-5940 v553 aAbstract Allowing item review in computerized adaptive testing (CAT) is getting more attention in the educational measurement field as more and more testing programs adopt CAT. The research literature has shown that allowing item review in an educational test could result in more accurate estimates of examinees’ abilities. The practice of item review in CAT, however, is hindered by the potential danger of test-manipulation strategies. To provide review opportunities to examinees while minimizing the effect of test-manipulation strategies, researchers have proposed different algorithms to implement CAT with restricted revision options. In this article, we propose and evaluate a new method that implements CAT without any restriction on item review. In particular, we evaluate the new method in terms of the accuracy on ability estimates and the robustness against test-manipulation strategies. This study shows that the newly proposed method is promising in a win-win situation: examinees have full freedom to review and change answers, and the impacts of test-manipulation strategies are undermined.1 aCui, Zhongmin1 aLiu, Chunyan1 aHe, Yong1 aChen, Hanwei uhttps://onlinelibrary.wiley.com/doi/abs/10.1111/jedm.1219301331nas a2200121 4500008003900000245010900039210006900148300002100217520088900238100001801127700001901145856004501164 2018 d00aItem Selection Criteria With Practical Constraints in Cognitive Diagnostic Computerized Adaptive Testing0 aItem Selection Criteria With Practical Constraints in Cognitive  a00131644187906343 aFor item selection in cognitive diagnostic computerized adaptive testing (CD-CAT), ideally, a single item selection index should be created to simultaneously regulate precision, exposure status, and attribute balancing. For this purpose, in this study, we first proposed an attribute-balanced item selection criterion, namely, the standardized weighted deviation global discrimination index (SWDGDI), and subsequently formulated the constrained progressive index (CP\_SWDGDI) by casting the SWDGDI in a progressive algorithm. A simulation study revealed that the SWDGDI method was effective in balancing attribute coverage and the CP\_SWDGDI method was able to simultaneously balance attribute coverage and item pool usage while maintaining acceptable estimation precision. This research also demonstrates the advantage of a relatively low number of attributes in CD-CAT applications.1 aLin, Chuan-Ju1 aChang, Hua-Hua uhttps://doi.org/10.1177/001316441879063401840nas a2200157 4500008003900000245010800039210006900147300001200216490000700228520134100235100001501576700001601591700001301607700001701620856004501637 2018 d00aItem Selection Methods in Multidimensional Computerized Adaptive Testing With Polytomously Scored Items0 aItem Selection Methods in Multidimensional Computerized Adaptive a677-6940 v423 aMultidimensional computerized adaptive testing (MCAT) has been developed over the past decades, and most of them can only deal with dichotomously scored items. However, polytomously scored items have been broadly used in a variety of tests for their advantages of providing more information and testing complicated abilities and skills. The purpose of this study is to discuss the item selection algorithms used in MCAT with polytomously scored items (PMCAT). Several promising item selection algorithms used in MCAT are extended to PMCAT, and two new item selection methods are proposed to improve the existing selection strategies. Two simulation studies are conducted to demonstrate the feasibility of the extended and proposed methods. The simulation results show that most of the extended item selection methods for PMCAT are feasible and the new proposed item selection methods perform well. Combined with the security of the pool, when two dimensions are considered (Study 1), the proposed modified continuous entropy method (MCEM) is the ideal of all in that it gains the lowest item exposure rate and has a relatively high accuracy. As for high dimensions (Study 2), results show that mutual information (MUI) and MCEM keep relatively high estimation accuracy, and the item exposure rates decrease as the correlation increases.1 aTu, Dongbo1 aHan, Yuting1 aCai, Yan1 aGao, Xuliang uhttps://doi.org/10.1177/014662161876274801532nas a2200169 4500008003900000245007100039210006900110300001200179490000700191520103000198100001601228700001801244700001801262700001801280700001901298856004501317 2018 d00aLatent Class Analysis of Recurrent Events in Problem-Solving Items0 aLatent Class Analysis of Recurrent Events in ProblemSolving Item a478-4980 v423 aComputer-based assessment of complex problem-solving abilities is becoming more and more popular. In such an assessment, the entire problem-solving process of an examinee is recorded, providing detailed information about the individual, such as behavioral patterns, speed, and learning trajectory. The problem-solving processes are recorded in a computer log file which is a time-stamped documentation of events related to task completion. As opposed to cross-sectional response data from traditional tests, process data in log files are massive and irregularly structured, calling for effective exploratory data analysis methods. Motivated by a specific complex problem-solving item “Climate Control” in the 2012 Programme for International Student Assessment, the authors propose a latent class analysis approach to analyzing the events occurred in the problem-solving processes. The exploratory latent class analysis yields meaningful latent classes. Simulation studies are conducted to evaluate the proposed approach.1 aXu, Haochen1 aFang, Guanhua1 aChen, Yunxiao1 aLiu, Jingchen1 aYing, Zhiliang uhttps://doi.org/10.1177/014662161774832501848nas a2200145 4500008003900000245011000039210006900149300001200218490000700230520135600237100001801593700001301611700001501624856006301639 2018 d00aOn-the-Fly Constraint-Controlled Assembly Methods for Multistage Adaptive Testing for Cognitive Diagnosis0 aOntheFly ConstraintControlled Assembly Methods for Multistage Ad a595-6130 v553 aAbstract This study applied the mode of on-the-fly assembled multistage adaptive testing to cognitive diagnosis (CD-OMST). Several and several module assembly methods for CD-OMST were proposed and compared in terms of measurement precision, test security, and constrain management. The module assembly methods in the study included the maximum priority index method (MPI), the revised maximum priority index (RMPI), the weighted deviation model (WDM), and the two revised Monte Carlo methods (R1-MC, R2-MC). Simulation results showed that on the whole the CD-OMST performs well in that it not only has acceptable attribute pattern correct classification rates but also satisfies both statistical and nonstatistical constraints; the RMPI method was generally better than the MPI method, the R2-MC method was generally better than the R1-MC method, and the two revised Monte Carlo methods performed best in terms of test security and constraint management, whereas the RMPI and WDM methods worked best in terms of measurement precision. The study is not only expected to provide information about how to combine MST and CD using an on-the-fly method and how do these assembled methods in CD-OMST perform relative to each other but also offer guidance for practitioners to assemble modules in CD-OMST with both statistical and nonstatistical constraints.1 aLiu, Shuchang1 aCai, Yan1 aTu, Dongbo uhttps://onlinelibrary.wiley.com/doi/abs/10.1111/jedm.1219402101nas a2200181 4500008004100000245004600041210004600087260005500133520153800188653002501726653000801751100002801759700001901787700001301806700002001819700001301839856006701852 2017 eng d00aBayesian Perspectives on Adaptive Testing0 aBayesian Perspectives on Adaptive Testing aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

Although adaptive testing is usually treated from the perspective of maximum-likelihood parameter estimation and maximum-informaton item selection, a Bayesian pespective is more natural, statistically efficient, and computationally tractable. This observation not only holds for the core process of ability estimation but includes such processes as item calibration, and real-time monitoring of item security as well. Key elements of the approach are parametric modeling of each relevant process, updating of the parameter estimates after the arrival of each new response, and optimal design of the next step.

The purpose of the symposium is to illustrates the role of Bayesian statistics in this approach. The first presentation discusses a basic Bayesian algorithm for the sequential update of any parameter in adaptive testing and illustrates the idea of Bayesian optimal design for the two processes of ability estimation and online item calibration. The second presentation generalizes the ideas to the case of 62 IACAT 2017 ABSTRACTS BOOKLET adaptive testing with polytomous items. The third presentation uses the fundamental Bayesian idea of sampling from updated posterior predictive distributions (“multiple imputations”) to deal with the problem of scoring incomplete adaptive tests.

Session Video 1

Session Video 2

 

10aBayesian Perspective10aCAT1 avan der Linden, Wim, J.1 aJiang, Bingnan1 aRen, Hao1 aChoi, Seung, W.1 aDiao, Qi uhttp://mail.iacat.org/bayesian-perspectives-adaptive-testing-004447nas a2200157 4500008004100000245009700041210006900138260005500207520381800262653001104080653002804091100002004119700001804139700002104157856011104178 2017 eng d00aComputerized Adaptive Testing for Cognitive Diagnosis in Classroom: A Nonparametric Approach0 aComputerized Adaptive Testing for Cognitive Diagnosis in Classro aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

In the past decade, CDMs of educational test performance have received increasing attention among educational researchers (for details, see Fu & Li, 2007, and Rupp, Templin, & Henson, 2010). CDMs of educational test performance decompose the ability domain of a given test into specific skills, called attributes, each of which an examinee may or may not have mastered. The resulting attribute profile documents the individual’s strengths and weaknesses within the ability domain. The Cognitive Diagnostic Computerized Adaptive Testing (CD-CAT) has been suggested by researchers as a diagnostic tool for assessment and evaluation (e.g., Cheng & Chang, 2007; Cheng, 2009; Liu, You, Wang, Ding, & Chang, 2013; Tatsuoka & Tatsuoka, 1997). While model-based CD-CAT is relatively well-researched in the context of large-scale assessments, this type of system has not received the same degree of development in small-scale settings, where it would be most useful. The main challenge is that the statistical estimation techniques successfully applied to the parametric CD-CAT require large samples to guarantee the reliable calibration of item parameters and accurate estimation of examinees’ attribute profiles. In response to the challenge, a nonparametric approach that does not require any parameter calibration, and thus can be used in small educational programs, is proposed. The proposed nonparametric CD-CAT relies on the same principle as the regular CAT algorithm, but uses the nonparametric classification method (Chiu & Douglas, 2013) to assess and update the student’s ability state while the test proceeds. Based on a student’s initial responses, 2 a neighborhood of candidate proficiency classes is identified, and items not characteristic of the chosen proficiency classes are precluded from being chosen next. The response to the next item then allows for an update of the skill profile, and the set of possible proficiency classes is further narrowed. In this manner, the nonparametric CD-CAT cycles through item administration and update stages until the most likely proficiency class has been pinpointed. The simulation results show that the proposed method outperformed the compared parametric CD-CAT algorithms and the differences were significant when the item parameter calibration was not optimal.

References

Cheng, Y. (2009). When cognitive diagnosis meets computerized adaptive testing: CD-CAT. Psychometrika, 74, 619-632.

Cheng, Y., & Chang, H. (2007). The modified maximum global discrimination index method for cognitive diagnostic CAT. In D. Weiss (Ed.) Proceedings of the 2007 GMAC Computerized Adaptive Testing Conference.

Chiu, C.-Y., & Douglas, J. A. (2013). A nonparametric approach to cognitive diagnosis by proximity to ideal response patterns. Journal of Classification, 30, 225-250.

Fu, J., & Li, Y. (2007). An integrative review of cognitively diagnostic psychometric models. Paper presented at the Annual Meeting of the National Council on Measurement in Education. Chicago, Illinois.

Liu, H., You, X., Wang, W., Ding, S., & Chang, H. (2013). The development of computerized adaptive testing with cognitive diagnosis for an English achievement test in China. Journal of Classification, 30, 152-172.

Rupp, A. A., & Templin, J. L., & Henson, R. A. (2010). Diagnostic Measurement. Theory, Methods, and Applications. New York: Guilford.

Tatsuoka, K.K., & Tatsuoka, M.M. (1997), Computerized cognitive diagnostic adaptive testing: Effect on remedial instruction as empirical validation. Journal of Educational Measurement, 34, 3–20.

Session Video

10aCD-CAT10anon-parametric approach1 aChang, Yuan-Pei1 aChiu, Chia-Yi1 aTsai, Rung-Ching uhttp://mail.iacat.org/computerized-adaptive-testing-cognitive-diagnosis-classroom-nonparametric-approach-003453nas a2200157 4500008004100000245008400041210006900125260005500194520283600249653001203085653002503097653002803122100002303150700002003173856010203193 2017 eng d00aConsiderations in Performance Evaluations of Computerized Formative Assessments0 aConsiderations in Performance Evaluations of Computerized Format aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

Computerized adaptive instruments have been widely established and used in the context of summative assessments for purposes including licensure, admissions and proficiency testing. The benefits of examinee tailored examinations, which can provide estimates of performance that are more reliable and valid, have in recent years attracted a greater audience (i.e. patient oriented outcomes, test prep, etc.). Formative assessment, which are most widely understood in their implementation as diagnostic tools, have recently started to expand to lesser known areas of computerized testing such as in implementations of instructional designs aiming to maximize examinee learning through targeted practice.

Using a CAT instrument within the framework of evaluating repetitious examinee performances (in such settings as a Quiz Bank practices for example) poses unique challenges not germane to summative assessments. The scale on which item parameters (and subsequently examinee performance estimates such as Maximum Likelihood Estimates) are determined usually do not take change over time under consideration. While vertical scaling features resolve the learning acquisition problem, most content practice engines do not make use of explicit practice windows which could be vertically aligned. Alternatively, the Multidimensional (MIRT)- and Hierarchical Item Response Theory (HIRT) models allow for the specification of random effects associated with change over time in examinees’ skills, but are often complex and require content and usage resources not often observed.

The research submitted for consideration simulated examinees’ repeated variable length Quiz Bank practice in algebra using a 500 1-PL operational item pool. The stability simulations sought to determine with which rolling item interval size ability estimates would provide the most informative insight into the examinees’ learning progression over time. Estimates were evaluated in terms of reduction in estimate uncertainty, bias and RMSD with the true and total item based ability estimates. It was found that rolling item intervals between 20-25 items provided the best reduction of uncertainty around the estimate without compromising the ability to provide informed performance estimates to students. However, while asymptotically intervals of 20-25 items tended to provide adequate estimates of performance, changes over shorter periods of time assessed with shorter quizzes could not be detected as those changes would be suppressed in lieu of the performance based on the full interval considered. Implications for infrastructure (such as recommendation engines, etc.), product and scale development are discussed.

Session video

10aalgebra10aFormative Assessment10aPerformance Evaluations1 aChajewski, Michael1 aHarnisher, John uhttp://mail.iacat.org/considerations-performance-evaluations-computerized-formative-assessments-001179nas a2200157 4500008003900000245008400039210006900123300001200192490000700204520068700211100002000898700001600918700001800934700002200952856004700974 2017 d00aThe Development of MST Test Information for the Prediction of Test Performances0 aDevelopment of MST Test Information for the Prediction of Test P a570-5860 v773 aThe current study proposes novel methods to predict multistage testing (MST) performance without conducting simulations. This method, called MST test information, is based on analytic derivation of standard errors of ability estimates across theta levels. We compared standard errors derived analytically to the simulation results to demonstrate the validity of the proposed method in both measurement precision and classification accuracy. The results indicate that the MST test information effectively predicted the performance of MST. In addition, the results of the current study highlighted the relationship among the test construction, MST design factors, and MST performance.1 aPark, Ryoungsun1 aKim, Jiseon1 aChung, Hyewon1 aDodd, Barbara, G. uhttp://dx.doi.org/10.1177/001316441666296001541nas a2200157 4500008003900000022001400039245009700053210006900150300001400219490000700233520104800240100001901288700001601307700001901323856004101342 2017 d a1745-398400aDual-Objective Item Selection Criteria in Cognitive Diagnostic Computerized Adaptive Testing0 aDualObjective Item Selection Criteria in Cognitive Diagnostic Co a165–1830 v543 aThe development of cognitive diagnostic-computerized adaptive testing (CD-CAT) has provided a new perspective for gaining information about examinees' mastery on a set of cognitive attributes. This study proposes a new item selection method within the framework of dual-objective CD-CAT that simultaneously addresses examinees' attribute mastery status and overall test performance. The new procedure is based on the Jensen-Shannon (JS) divergence, a symmetrized version of the Kullback-Leibler divergence. We show that the JS divergence resolves the noncomparability problem of the dual information index and has close relationships with Shannon entropy, mutual information, and Fisher information. The performance of the JS divergence is evaluated in simulation studies in comparison with the methods available in the literature. Results suggest that the JS divergence achieves parallel or more precise recovery of latent trait variables compared to the existing methods and maintains practical advantages in computation and item pool usage.1 aKang, Hyeon-Ah1 aZhang, Susu1 aChang, Hua-Hua uhttp://dx.doi.org/10.1111/jedm.1213903610nas a2200169 4500008004100000245006900041210006600110260005500176520302300231653000803254653002403262653003303286100001503319700001803334700001703352856007103369 2017 eng d00aAn Imputation Approach to Handling Incomplete Computerized Tests0 aImputation Approach to Handling Incomplete Computerized Tests aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

As technology advances, computerized adaptive testing (CAT) is becoming increasingly popular as it allows tests to be tailored to an examinee’s ability.  Nevertheless, examinees might devise testing strategies to use CAT to their advantage.  For instance, if only the items that examinees answer count towards their score, then a higher theta score might be obtained by spending more time on items at the beginning of the test and skipping items at the end if time runs out. This type of gaming can be discouraged if examinees’ scores are lowered or “penalized” based on the amount of non-response.

The goal of this study was to devise a penalty function that would meet two criteria: 1) the greater the omit rate, the greater the penalty, and 2) examinees with the same ability and the same omit rate should receive the same penalty. To create the penalty, theta was calculated based on only the items the examinee responded to ( ).  Next, the expected number correct score (EXR) was obtained using  and the test characteristic curve. A penalized expected number correct score (E ) was obtained by multiplying EXR by the proportion of items the examinee responded to. Finally, the penalized theta ( ) was identified using the test characteristic curve. Based on   and the item parameters ( ) of an unanswered item, the likelihood of a correct response,  , is computed and employed to estimate the imputed score ( ) for the unanswered item.

Two datasets were used to generate tests with completion rates of 50%, 80%, and 90%.  The first dataset included real data where approximately 4,500 examinees responded to a 21 -item test which provided a baseline/truth. Sampling was done to achieve the three completion rate conditions. The second dataset consisted of simulated item scores for 50,000 simulees under a 1-2-4 multi-stage CAT design where each stage contained seven items. Imputed item scores for unanswered items were computed using a variety of values for G (and therefore T).  Three other approaches to handling unanswered items were also considered: all correct (i.e., T = 0), all incorrect (i.e., T = 1), and random scoring (i.e., T = 0.5).

The current study investigated the impact on theta estimates resulting from the proposed approach to handling unanswered items in a fixed-length CAT. In real testing situations, when examinees do not finish a test, it is hard to tell whether they tried diligently but ran out of time or whether they attempted to manipulate the scoring engine.  To handle unfinished tests with penalties, the proposed approach considers examinees’ abilities and incompletion rates. The results of this study provide direction for psychometric practitioners when considering penalties for omitted responses.

Session Video

10aCAT10aimputation approach10aincomplete computerized test1 aChen, Troy1 aHuang, Chi-Yu1 aLiu, Chunyan uhttps://drive.google.com/open?id=1vznZeO3nsZZK0k6_oyw5c9ZTP8uyGnXh02155nas a2200145 4500008004100000245005100041210005100092260005500143520166100198653002301859653002001882100001901902700001301921856007501934 2017 eng d00aItem Parameter Drifting and Online Calibration0 aItem Parameter Drifting and Online Calibration aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

Item calibration is a part of the most important topics in item response theory (IRT). Since many largescale testing programs have switched from paper and pencil (P&P) testing mode to computerized adaptive testing (CAT) mode, developing methods for efficiently calibrating new items have become vital. Among many proposed item calibration processes in CAT, online calibration is the most cost-effective. This presentation introduces an online (re)calibration design to detect item parameter drift for computerized adaptive testing (CAT) in both unidimensional and multidimensional environments. Specifically, for online calibration optimal design in unidimensional computerized adaptive testing model, a two-stage design is proposed by implementing a proportional density index algorithm. For a multidimensional computerized adaptive testing model, a four-quadrant online calibration pretest item selection design with proportional density index algorithm is proposed. Comparisons were made between different online calibration item selection strategies. Results showed that under unidimensional computerized adaptive testing, the proposed modified two-stage item selection criterion with the proportional density algorithm outperformed the other existing methods in terms of item parameter calibration and item parameter drift detection, and under multidimensional computerized adaptive testing, the online (re)calibration technique with the proposed four-quadrant item selection design with proportional density index outperformed other methods.

Session Video

10aonline calibration10aParameter Drift1 aChang, Hua-Hua1 aGuo, Rui uhttp://mail.iacat.org/item-parameter-drifting-and-online-calibration-001708nas a2200145 4500008004100000245006200041210006200103260005500165520122100220653000801441653001401449653003001463100002101493856004801514 2017 eng d00aItem Selection Strategies for Developing CAT in Indonesia0 aItem Selection Strategies for Developing CAT in Indonesia aNiigata JapanbNiiagata Seiryo Universityc08/20173 a

Recently, development of computerized testing in Indonesia is quiet promising for the future. Many government institutions used the technology for recruitment. Starting from Indonesian Army acknowledged the benefits of computerized adaptive testing (CAT) over conventional test administration, ones of the issues of selection the first item have taken place of attention. Due to CAT’s basic philosophy, several methods can be used to select the first item such as educational level, ability estimation from item simulation, or other methods. In this case, the question is remains how apply the methods most effective in the context of constrained adaptive testing. This paper reviews such strategies that appeared in the relevant literature. The focus of this paper is on studies that have been conducted in order to evaluate the effectiveness of item selection strategies for dichotomous scoring. In this paper, also discusses the strength and weaknesses of each strategy group using examples from simulation studies. No new research is presented but rather a compendium of models is reviewed in term of learning in the newcomer context, a wide view of first item selection strategies.

 

10aCAT10aIndonesia10aitem selection strategies1 aChandra, Istiani uhttps://www.youtube.com/watch?v=2KuFrRATq9Q03826nas a2200157 4500008004100000245008500041210006900126260005500195520325800250653000803508653002203516653002303538100001603561700002003577856007103597 2017 eng d00aA Large-Scale Progress Monitoring Application with Computerized Adaptive Testing0 aLargeScale Progress Monitoring Application with Computerized Ada aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

Many conventional assessment tools are available to teachers in schools for monitoring student progress in a formative manner. The outcomes of these assessment tools are essential to teachers’ instructional modifications and schools’ data-driven educational strategies, such as using remedial activities and planning instructional interventions for students with learning difficulties. When measuring student progress toward instructional goals or outcomes, assessments should be not only considerably precise but also sensitive to individual change in learning. Unlike conventional paper-pencil assessments that are usually not appropriate for every student, computerized adaptive tests (CATs) are highly capable of estimating growth consistently with minimum and consistent error. Therefore, CATs can be used as a progress monitoring tool in measuring student growth.

This study focuses on an operational CAT assessment that has been used for measuring student growth in reading during the academic school year. The sample of this study consists of nearly 7 million students from the 1st grade to the 12th grade in the US. The students received a CAT-based reading assessment periodically during the school year. The purpose of these periodical assessments is to measure the growth in students’ reading achievement and identify the students who may need additional instructional support (e.g., academic interventions). Using real data, this study aims to address the following research questions: (1) How many CAT administrations are necessary to make psychometrically sound decisions about the need for instructional changes in the classroom or when to provide academic interventions?; (2) What is the ideal amount of time between CAT administrations to capture student growth for the purpose of producing meaningful decisions from assessment results?

To address these research questions, we first used the Theil-Sen estimator for robustly fitting a regression line to each student’s test scores obtained from a series of CAT administrations. Next, we used the conditional standard error of measurement (cSEM) from the CAT administrations to create an error band around the Theil-Sen slope (i.e., student growth rate). This process resulted in the normative slope values across all the grade levels. The optimal number of CAT administrations was established from grade-level regression results. The amount of time needed for progress monitoring was determined by calculating the amount of time required for a student to show growth beyond the median cSEM value for each grade level. The results showed that the normative slope values were the highest for lower grades and declined steadily as grade level increased. The results also suggested that the CAT-based reading assessment is most useful for grades 1 through 4, since most struggling readers requiring an intervention appear to be within this grade range. Because CAT yielded very similar cSEM values across administrations, the amount of error in the progress monitoring decisions did not seem to depend on the number of CAT administrations.

Session Video

10aCAT10aLarge-Scale tests10aProcess monitoring1 aBulut, Okan1 aCormier, Damien uhttps://drive.google.com/open?id=1uGbCKenRLnqTxImX1fZicR2c7GRV6Udc02109nas a2200169 4500008004100000245005200041210005100093260005500144520156900199653002101768653000801789653002301797100001501820700001901835700001301854856007201867 2017 eng d00aMHK-MST Design and the Related Simulation Study0 aMHKMST Design and the Related Simulation Study aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

The MHK is a national standardized exam that tests and rates Chinese language proficiency. It assesses non-native Chinese minorities’ abilities in using the Chinese language in their daily, academic and professional lives; Computerized multistage adaptive testing (MST) is a combination of conventional paper-and-pencil (P&P) and item level computerized adaptive test (CAT), it is a kind of test forms based on computerized technology, take the item set as the scoring unit. It can be said that, MST estimate the Ability extreme value more accurate than conventional paper-and-pencil (P&P), also used the CAT auto-adapted characteristic to reduce the examination length and the score time of report. At present, MST has used in some large test, like Uniform CPA Examination and Graduate Record Examination(GRE). Therefore, it is necessary to develop the MST of application in China.

Based on consideration of the MHK characteristics and its future development, the researchers start with design of MHK-MST. This simulation study is conducted to validate the performance of the MHK -MST system. Real difficulty parameters of MHK items and the simulated ability parameters of the candidates are used to generate the original score matrix and the item modules are delivered to the candidates following the adaptive procedures set according to the path rules. This simulation study provides a sound basis for the implementation of MHK-MST.

Session Video

10alanguage testing10aMHK10amultistage testing1 aYuyu, Ling1 aChenglin, Zhou1 aJie, Ren uhttp://mail.iacat.org/mhk-mst-design-and-related-simulation-study-002365nas a2200277 4500008004100000245007100041210006900112260005500181520152900236653000801765653001501773653002801788100001501816700002101831700001501852700001401867700001601881700001501897700001601912700001601928700001601944700001801960700001901978700001901997856007102016 2017 eng d00aNew Challenges (With Solutions) and Innovative Applications of CAT0 aNew Challenges With Solutions and Innovative Applications of CAT aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

Over the past several decades, computerized adaptive testing (CAT) has profoundly changed the administration of large-scale aptitude tests, state-wide achievement tests, professional licensure exams, and health outcome measures. While many challenges of CAT have been successfully addressed due to the continual efforts of researchers in the field, there are still many remaining, longstanding challenges that have yet to be resolved. This symposium will begin with three presentations, each of which provides a sound solution to one of the unresolved challenges. They are (1) item calibration when responses are “missing not at random” from CAT administration; (2) online calibration of new items when person traits have non-ignorable measurement error; (3) establishing consistency and asymptotic normality of latent trait estimation when allowing item response revision in CAT. In addition, this symposium also features innovative applications of CAT. In particular, there is emerging interest in using cognitive diagnostic CAT to monitor and detect learning progress (4th presentation). Last but not least, the 5th presentation illustrates the power of multidimensional polytomous CAT that permits rapid identification of hospitalized patients’ rehabilitative care needs in health outcomes measurement. We believe this symposium covers a wide range of interesting and important topics in CAT.

Session Video

10aCAT10achallenges10ainnovative applications1 aWang, Chun1 aWeiss, David, J.1 aZhang, Xue1 aTao, Jian1 aHe, Yinhong1 aChen, Ping1 aWang, Shiyu1 aZhang, Susu1 aLin, Haiyan1 aGao, Xiaohong1 aChang, Hua-Hua1 aShang, Zhuoran uhttps://drive.google.com/open?id=1Wvgxw7in_QCq_F7kzID6zCZuVXWcFDPa02584nas a2200157 4500008004100000245011700041210006900158260005500227520193000282653001102212653001902223653002702242100001802269700001902287856012002306 2017 eng d00aA New Cognitive Diagnostic Computerized Adaptive Testing for Simultaneously Diagnosing Skills and Misconceptions0 aNew Cognitive Diagnostic Computerized Adaptive Testing for Simul aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

In education diagnoses, diagnosing misconceptions is important as well as diagnosing skills. However, traditional cognitive diagnostic computerized adaptive testing (CD-CAT) is usually developed to diagnose skills. This study aims to propose a new CD-CAT that can simultaneously diagnose skills and misconceptions. The proposed CD-CAT is based on a recently published new CDM, called the simultaneously identifying skills and misconceptions (SISM) model (Kuo, Chen, & de la Torre, in press). A new item selection algorithm is also proposed in the proposed CD-CAT for achieving high adaptive testing performance. In simulation studies, we compare our new item selection algorithm with three existing item selection methods, including the Kullback–Leibler (KL) and posterior-weighted KL (PWKL) proposed by Cheng (2009) and the modified PWKL (MPWKL) proposed by Kaplan, de la Torre, and Barrada (2015). The results show that our proposed CD-CAT can efficiently diagnose skills and misconceptions; the accuracy of our new item selection algorithm is close to the MPWKL but less computational burden; and our new item selection algorithm outperforms the KL and PWKL methods on diagnosing skills and misconceptions.

References

Cheng, Y. (2009). When cognitive diagnosis meets computerized adaptive testing: CD-CAT. Psychometrika, 74(4), 619–632. doi: 10.1007/s11336-009-9123-2

Kaplan, M., de la Torre, J., & Barrada, J. R. (2015). New item selection methods for cognitive diagnosis computerized adaptive testing. Applied Psychological Measurement, 39(3), 167–188. doi:10.1177/0146621614554650

Kuo, B.-C., Chen, C.-H., & de la Torre, J. (in press). A cognitive diagnosis model for identifying coexisting skills and misconceptions. Applied Psychological Measurement.

Session Video

10aCD-CAT10aMisconceptions10aSimultaneous diagnosis1 aKuo, Bor-Chen1 aChen, Chun-Hua uhttp://mail.iacat.org/new-cognitive-diagnostic-computerized-adaptive-testing-simultaneously-diagnosing-skills-and-003809nas a2200169 4500008004100000245008600041210006900127260005500196520318900251653000903440653000803449653002503457100002503482700001703507700001703524856009803541 2017 eng d00aNew Results on Bias in Estimates due to Discontinue Rules in Intelligence Testing0 aNew Results on Bias in Estimates due to Discontinue Rules in Int aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

The presentation provides new results on a form of adaptive testing that is used frequently in intelligence testing. In these tests, items are presented in order of increasing difficulty, and the presentation of items is adaptive in the sense that each subtest session is discontinued once a test taker produces a certain number of incorrect responses in sequence. The subsequent (not observed) responses are commonly scored as wrong for that subtest, even though the test taker has not seen these. Discontinuation rules allow a certain form of adaptiveness both in paper-based and computerbased testing, and help reducing testing time.

Two lines of research that are relevant are studies that directly assess the impact of discontinuation rules, and studies that more broadly look at the impact of scoring rules on test results with a large number of not administered or not reached items. He & Wolf (2012) compared different ability estimation methods for this type of discontinuation rule adaptation of test length in a simulation study. However, to our knowledge there has been no rigorous analytical study of the underlying distributional changes of the response variables under discontinuation rules. It is important to point out that the results obtained by He & Wolf (2012) agree with results presented by, for example, DeAyala, Plake & Impara (2001) as well as Rose, von Davier & Xu (2010) and Rose, von Davier & Nagengast (2016) in that ability estimates are biased most when scoring the not observed responses as wrong. Discontinuation rules combined with scoring the non-administered items as wrong is used operationally in several major intelligence tests, so more research is needed in order to improve this particular type of adaptiveness in the testing practice.

The presentation extends existing research on adaptiveness by discontinue-rules in intelligence tests in multiple ways: First, a rigorous analytical study of the distributional properties of discontinue-rule scored items is presented. Second, an extended simulation is presented that includes additional alternative scoring rules as well as bias-corrected ability estimators that may be suitable to improve results for discontinue-rule scored intelligence tests.

References: DeAyala, R. J., Plake, B. S., & Impara, J. C. (2001). The impact of omitted responses on the accuracy of ability estimation in item response theory. Journal of Educational Measurement, 38, 213-234.

He, W. & Wolfe, E. W. (2012). Treatment of Not-Administered Items on Individually Administered Intelligence Tests. Educational and Psychological Measurement, Vol 72, Issue 5, pp. 808 – 826. DOI: 10.1177/0013164412441937

Rose, N., von Davier, M., & Xu, X. (2010). Modeling non-ignorable missing data with item response theory (IRT; ETS RR-10-11). Princeton, NJ: Educational Testing Service.

Rose, N., von Davier, M., & Nagengast, B. (2016) Modeling omitted and not-reached items in irt models. Psychometrika. doi:10.1007/s11336-016-9544-7

Session Video

10aBias10aCAT10aIntelligence Testing1 avon Davier, Matthias1 aCho, Youngmi1 aPan, Tianshu uhttp://mail.iacat.org/new-results-bias-estimates-due-discontinue-rules-intelligence-testing-003598nas a2200169 4500008004100000245004300041210004100084260005500125520306600180653000803246653002303254653002303277100001703300700002003317700002003337856007103357 2017 eng d00aScripted On-the-fly Multistage Testing0 aScripted Onthefly Multistage Testing aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

On-the-fly multistage testing (OMST) was introduced recently as a promising alternative to preassembled MST. A decidedly appealing feature of both is the reviewability of items within the current stage. However, the fundamental difference is that, instead of routing to a preassembled module, OMST adaptively assembles a module at each stage according to an interim ability estimate. This produces more individualized forms with finer measurement precision, but imposing nonstatistical constraints and controlling item exposure become more cumbersome. One recommendation is to use the maximum priority index followed by a remediation step to satisfy content constraints, and the Sympson-Hetter method with a stratified item bank for exposure control.

However, these methods can be computationally expensive, thereby impeding practical implementation. Therefore, this study investigated the script method as a simpler solution to the challenge of strict content balancing and effective item exposure control in OMST. The script method was originally devised as an item selection algorithm for CAT and generally proceeds as follows: For a test with m items, there are m slots to be filled, and an item is selected according to pre-defined rules for each slot. For the first slot, randomly select an item from a designated content area (collection). For each subsequent slot, 1) Discard any enemies of items already administered in previous slots; 2) Draw a designated number of candidate items (selection length) from the designated collection according to the current ability estimate; 3) Randomly select one item from the set of candidates. There are two distinct features of the script method. First, a predetermined sequence of collections guarantees meeting content specifications. The specific ordering may be determined either randomly or deliberately by content experts. Second, steps 2 and 3 depict a method of exposure control, in which selection length balances item usage at the possible expense of ability estimation accuracy. The adaptation of the script method to OMST is straightforward. For the first module, randomly select each item from a designated collection. For each subsequent module, the process is the same as in scripted CAT (SCAT) except the same ability estimate is used for the selection of all items within the module. A series of simulations was conducted to evaluate the performance of scripted OMST (SOMST, with 3 or 4 evenly divided stages) relative to SCAT under various item exposure restrictions. In all conditions, reliability was maximized by programming an optimization algorithm that searches for the smallest possible selection length for each slot within the constraints. Preliminary results indicated that SOMST is certainly a capable design with performance comparable to that of SCAT. The encouraging findings and ease of implementation highly motivate the prospect of operational use for large-scale assessments.

Presentation Video

10aCAT10amultistage testing10aOn-the-fly testing1 aChoe, Edison1 aWilliams, Bruce1 aLee, Sung-Hyuck uhttps://drive.google.com/open?id=1wKuAstITLXo6BM4APf2mPsth1BymNl-y01840nas a2200145 4500008004100000245010900041210006900150260005500219520128100274100001501555700001401570700001901584700002001603856007101623 2017 eng d00aA Simulation Study to Compare Classification Method in Cognitive Diagnosis Computerized Adaptive Testing0 aSimulation Study to Compare Classification Method in Cognitive D aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

Cognitive Diagnostic Computerized Adaptive Testing (CD-CAT) combines the strengths of both CAT and cognitive diagnosis. Cognitive diagnosis models that can be viewed as restricted latent class models have been developed to classify the examinees into the correct profile of skills that have been mastered and those that have not so as to get more efficient remediation. Chiu & Douglas (2013) introduces a nonparametric procedure that only requires specification of Q-matrix to classify by proximity to ideal response pattern. In this article, we compare nonparametric procedure with common profile estimation method like maximum a posterior (MAP) in CD-CAT. Simulation studies consider a variety of Q-matrix structure, the number of attributes, ways to generate attribute profiles, and item quality. Results indicate that nonparametric procedure consistently gets the higher pattern and attribute recovery rate in nearly all conditions.

References

Chiu, C.-Y., & Douglas, J. (2013). A nonparametric approach to cognitive diagnosis by proximity to ideal response patterns. Journal of Classification, 30, 225-250. doi: 10.1007/s00357-013-9132-9

Session Video

1 aYang, Jing1 aTao, Jian1 aChang, Hua-Hua1 aShi, Ning-Zhong uhttps://drive.google.com/open?id=1jCL3fPZLgzIdwvEk20D-FliZ15OTUtpr02153nas a2200157 4500008004100000245008800041210006900129260005400198520153900252653002901791653001101820100001901831700002001850700001801870856010701888 2017 eng d00aUsing Bayesian Decision Theory in Cognitive Diagnosis Computerized Adaptive Testing0 aUsing Bayesian Decision Theory in Cognitive Diagnosis Computeriz aNiigata JapanbNiigata Seiryo Universityc08/20173 a

Cognitive diagnosis computerized adaptive testing (CD-CAT) purports to provide each individual a profile about the strengths and weaknesses of attributes or skills with computerized adaptive testing. In the CD-CAT literature, researchers dedicated to evolving item selection algorithms to improve measurement efficiency, and most algorithms were developed based on information theory. By the discontinuous nature of the latent variables in CD-CAT, this study introduced an alternative for item selection, called the minimum expected cost (MEC) method, which was derived based on Bayesian decision theory. Using simulations, the MEC method was evaluated against the posterior weighted Kullback-Leibler (PWKL) information, the modified PWKL (MPWKL), and the mutual information (MI) methods by manipulating item bank quality, item selection algorithm, and termination rule. Results indicated that, regardless of item quality and termination criterion, the MEC, MPWKL, and MI methods performed very similarly and they all outperformed the PWKL method in classification accuracy and test efficiency, especially in short tests; the MEC method had more efficient item bank usage than the MPWKL and MI methods. Moreover, the MEC method could consider the costs of incorrect decisions and improve classification accuracy and test efficiency when a particular profile was of concern. All the results suggest the practicability of the MEC method in CD-CAT.

Session Video

10aBayesian Decision Theory10aCD-CAT1 aHsu, Chia-Ling1 aWang, Wen-Chung1 aChen, ShuYing uhttp://mail.iacat.org/using-bayesian-decision-theory-cognitive-diagnosis-computerized-adaptive-testing01177nas a2200121 4500008003900000245007200039210006900111300000900180490000700189520078000196100002800976856005101004 2016 d00aBayesian Networks in Educational Assessment: The State of the Field0 aBayesian Networks in Educational Assessment The State of the Fie a3-210 v403 aBayesian networks (BN) provide a convenient and intuitive framework for specifying complex joint probability distributions and are thus well suited for modeling content domains of educational assessments at a diagnostic level. BN have been used extensively in the artificial intelligence community as student models for intelligent tutoring systems (ITS) but have received less attention among psychometricians. This critical review outlines the existing research on BN in educational assessment, providing an introduction to the ITS literature for the psychometric community, and points out several promising research paths. The online appendix lists 40 assessment systems that serve as empirical examples of the use of BN for educational assessment in a variety of domains.1 aCulbertson, Michael, J. uhttp://apm.sagepub.com/content/40/1/3.abstract00723nas a2200205 4500008004500000022001500045245013200060210006900192300000900261490000600270653002100276653003000297653003000327653001600357653002300373100002100396700002000417700002000437856006000457 2016 Engldsh  a2165-6592 00aEffect of Imprecise Parameter Estimation on Ability Estimation in a Multistage Test in an Automatic Item Generation Context 0 aEffect of Imprecise Parameter Estimation on Ability Estimation i a1-180 v410aAdaptive Testing10aautomatic item generation10aerrors in item parameters10aitem clones10amultistage testing1 aColvin, Kimberly1 aKeller, Lisa, A1 aRobin, Frederic uhttp://iacat.org/jcat/index.php/jcat/article/view/59/2702164nas a2200133 4500008003900000245014400039210007100183300001200254490000700266520166600273100001901939700001901958856005301977 2016 d00aHigh-Efficiency Response Distribution–Based Item Selection Algorithms for Short-Length Cognitive Diagnostic Computerized Adaptive Testing0 aHighEfficiency Response Distribution–Based Item Selection Algori a608-6240 v403 aCognitive diagnostic computerized adaptive testing (CD-CAT) purports to obtain useful diagnostic information with great efficiency brought by CAT technology. Most of the existing CD-CAT item selection algorithms are evaluated when test length is fixed and relatively long, but some applications of CD-CAT, such as in interim assessment, require to obtain the cognitive pattern with a short test. The mutual information (MI) algorithm proposed by Wang is the first endeavor to accommodate this need. To reduce the computational burden, Wang provided a simplified scheme, but at the price of scale/sign change in the original index. As a result, it is very difficult to combine it with some popular constraint management methods. The current study proposes two high-efficiency algorithms, posterior-weighted cognitive diagnostic model (CDM) discrimination index (PWCDI) and posterior-weighted attribute-level CDM discrimination index (PWACDI), by modifying the CDM discrimination index. They can be considered as an extension of the Kullback–Leibler (KL) and posterior-weighted KL (PWKL) methods. A pre-calculation strategy has also been developed to address the computational issue. Simulation studies indicate that the newly developed methods can produce results comparable with or better than the MI and PWKL in both short and long tests. The other major advantage is that the computational issue has been addressed more elegantly than MI. PWCDI and PWACDI can run as fast as PWKL. More importantly, they do not suffer from the problem of scale/sign change as MI and, thus, can be used with constraint management methods together in a straightforward manner.1 aZheng, Chanjin1 aChang, Hua-Hua uhttp://apm.sagepub.com/content/40/8/608.abstract01608nas a2200169 4500008003900000022001400039245009800053210006900151300001200220490000700232520108900239100001601328700001601344700001901360700001801379856004101397 2016 d a1745-398400aHybrid Computerized Adaptive Testing: From Group Sequential Design to Fully Sequential Design0 aHybrid Computerized Adaptive Testing From Group Sequential Desig a45–620 v533 aComputerized adaptive testing (CAT) and multistage testing (MST) have become two of the most popular modes in large-scale computer-based sequential testing.  Though most designs of CAT and MST exhibit strength and weakness in recent large-scale implementations, there is no simple answer to the question of which design is better because different modes may fit different practical situations. This article proposes a hybrid adaptive framework to combine both CAT and MST, inspired by an analysis of the history of CAT and MST. The proposed procedure is a design which transitions from a group sequential design to a fully sequential design. This allows for the robustness of MST in early stages, but also shares the advantages of CAT in later stages with fine tuning of the ability estimator once its neighborhood has been identified. Simulation results showed that hybrid designs following our proposed principles provided comparable or even better estimation accuracy and efficiency than standard CAT and MST designs, especially for examinees at the two ends of the ability range.1 aWang, Shiyu1 aLin, Haiyan1 aChang, Hua-Hua1 aDouglas, Jeff uhttp://dx.doi.org/10.1111/jedm.1210001483nas a2200157 4500008003900000245004600039210004600085300001200131490000700143520104600150100001901196700002601215700001201241700001901253856005301272 2016 d00aOptimal Reassembly of Shadow Tests in CAT0 aOptimal Reassembly of Shadow Tests in CAT a469-4850 v403 aEven in the age of abundant and fast computing resources, concurrency requirements for large-scale online testing programs still put an uninterrupted delivery of computer-adaptive tests at risk. In this study, to increase the concurrency for operational programs that use the shadow-test approach to adaptive testing, we explored various strategies aiming for reducing the number of reassembled shadow tests without compromising the measurement quality. Strategies requiring fixed intervals between reassemblies, a certain minimal change in the interim ability estimate since the last assembly before triggering a reassembly, and a hybrid of the two strategies yielded substantial reductions in the number of reassemblies without degradation in the measurement accuracy. The strategies effectively prevented unnecessary reassemblies due to adapting to the noise in the early test stages. They also highlighted the practicality of the shadow-test approach by minimizing the computational load involved in its use of mixed-integer programming.1 aChoi, Seung, W1 aMoellering, Karin, T.1 aLi, Jie1 aLinden, Wim, J uhttp://apm.sagepub.com/content/40/7/469.abstract01438nas a2200133 4500008003900000245013200039210006900171300001200240490000700252520095400259100001901213700001901232856005301251 2016 d00aParameter Drift Detection in Multidimensional Computerized Adaptive Testing Based on Informational Distance/Divergence Measures0 aParameter Drift Detection in Multidimensional Computerized Adapt a534-5500 v403 aAn informational distance/divergence-based approach is proposed to detect the presence of parameter drift in multidimensional computerized adaptive testing (MCAT). The study presents significance testing procedures for identifying changes in multidimensional item response functions (MIRFs) over time based on informational distance/divergence measures that capture the discrepancy between two probability functions. To approximate the MIRFs from the observed response data, the k-nearest neighbors algorithm is used with the random search method. A simulation study suggests that the distance/divergence-based drift measures perform effectively in identifying the instances of parameter drift in MCAT. They showed moderate power with small samples of 500 examinees and excellent power when the sample size was as large as 1,000. The proposed drift measures also adequately controlled for Type I error at the nominal level under the null hypothesis.1 aKang, Hyeon-Ah1 aChang, Hua-Hua uhttp://apm.sagepub.com/content/40/7/534.abstract01559nas a2200169 4500008003900000022001400039245007700053210006900130300001300199490000700212520105800219100002101277700001401298700001901312700001701331856004101348 2015 d a1745-398400aAssessing Individual-Level Impact of Interruptions During Online Testing0 aAssessing IndividualLevel Impact of Interruptions During Online  a80–1050 v523 aWith an increase in the number of online tests, the number of interruptions during testing due to unexpected technical issues seems to be on the rise. For example, interruptions occurred during several recent state tests. When interruptions occur, it is important to determine the extent of their impact on the examinees' scores. Researchers such as Hill and Sinharay et al. examined the impact of interruptions at an aggregate level. However, there is a lack of research on the assessment of impact of interruptions at an individual level. We attempt to fill that void. We suggest four methodological approaches, primarily based on statistical hypothesis testing, linear regression, and item response theory, which can provide evidence on the individual-level impact of interruptions. We perform a realistic simulation study to compare the Type I error rate and power of the suggested approaches. We then apply the approaches to data from the 2013 Indiana Statewide Testing for Educational Progress-Plus (ISTEP+) test that experienced interruptions.1 aSinharay, Sandip1 aWan, Ping1 aChoi, Seung, W1 aKim, Dong-In uhttp://dx.doi.org/10.1111/jedm.1206401780nas a2200145 4500008003900000245008400039210006900123300001200192490000700204520131700211100001601528700002301544700001401567856005301581 2015 d00aa-Stratified Computerized Adaptive Testing in the Presence of Calibration Error0 aaStratified Computerized Adaptive Testing in the Presence of Cal a260-2830 v753 aa-Stratified computerized adaptive testing with b-blocking (AST), as an alternative to the widely used maximum Fisher information (MFI) item selection method, can effectively balance item pool usage while providing accurate latent trait estimates in computerized adaptive testing (CAT). However, previous comparisons of these methods have treated item parameter estimates as if they are the true population parameter values. Consequently, capitalization on chance may occur. In this article, we examined the performance of the AST method under more realistic conditions where item parameter estimates instead of true parameter values are used in the CAT. Its performance was compared against that of the MFI method when the latter is used in conjunction with Sympson–Hetter or randomesque exposure control. Results indicate that the MFI method, even when combined with exposure control, is susceptible to capitalization on chance. This is particularly true when the calibration sample size is small. On the other hand, AST is more robust to capitalization on chance. Consistent with previous investigations using true item parameter values, AST yields much more balanced item pool usage, with a small loss in the precision of latent trait estimates. The loss is negligible when the test is as long as 40 items.1 aCheng, Ying1 aPatton, Jeffrey, M1 aShao, Can uhttp://epm.sagepub.com/content/75/2/260.abstract01791nas a2200133 4500008003900000245009200039210006900131300001200200490000700212520135400219100001601573700001501589856005301604 2015 d00aThe Effect of Upper and Lower Asymptotes of IRT Models on Computerized Adaptive Testing0 aEffect of Upper and Lower Asymptotes of IRT Models on Computeriz a551-5650 v393 aIn this article, the effect of the upper and lower asymptotes in item response theory models on computerized adaptive testing is shown analytically. This is done by deriving the step size between adjacent latent trait estimates under the four-parameter logistic model (4PLM) and two models it subsumes, the usual three-parameter logistic model (3PLM) and the 3PLM with upper asymptote (3PLMU). The authors show analytically that the large effect of the discrimination parameter on the step size holds true for the 4PLM and the two models it subsumes under both the maximum information method and the b-matching method for item selection. Furthermore, the lower asymptote helps reduce the positive bias of ability estimates associated with early guessing, and the upper asymptote helps reduce the negative bias induced by early slipping. Relative step size between modeling versus not modeling the upper or lower asymptote under the maximum Fisher information method (MI) and the b-matching method is also derived. It is also shown analytically why the gain from early guessing is smaller than the loss from early slipping when the lower asymptote is modeled, and vice versa when the upper asymptote is modeled. The benefit to loss ratio is quantified under both the MI and the b-matching method. Implications of the analytical results are discussed.1 aCheng, Ying1 aLiu, Cheng uhttp://apm.sagepub.com/content/39/7/551.abstract01322nas a2200145 4500008003900000245005100039210004900090300000900139490000700148520091500155100001801070700001801088700001901106856005101125 2015 d00aOnline Item Calibration for Q-Matrix in CD-CAT0 aOnline Item Calibration for QMatrix in CDCAT a5-150 v393 a

Item replenishment is important for maintaining a large-scale item bank. In this article, the authors consider calibrating new items based on pre-calibrated operational items under the deterministic inputs, noisy-and-gate model, the specification of which includes the so-called -matrix, as well as the slipping and guessing parameters. Making use of the maximum likelihood and Bayesian estimators for the latent knowledge states, the authors propose two methods for the calibration. These methods are applicable to both traditional paper–pencil–based tests, for which the selection of operational items is prefixed, and computerized adaptive tests, for which the selection of operational items is sequential and random. Extensive simulations are done to assess and to compare the performance of these approaches. Extensions to other diagnostic classification models are also discussed.

1 aChen, Yunxiao1 aLiu, Jingchen1 aYing, Zhiliang uhttp://apm.sagepub.com/content/39/1/5.abstract01517nas a2200133 4500008003900000245005300039210005100092300001200143490000700155520113500162100001401297700001901311856005301330 2015 d00aOn-the-Fly Assembled Multistage Adaptive Testing0 aOntheFly Assembled Multistage Adaptive Testing a104-1180 v393 a

Recently, multistage testing (MST) has been adopted by several important large-scale testing programs and become popular among practitioners and researchers. Stemming from the decades of history of computerized adaptive testing (CAT), the rapidly growing MST alleviates several major problems of earlier CAT applications. Nevertheless, MST is only one among all possible solutions to these problems. This article presents a new adaptive testing design, “on-the-fly assembled multistage adaptive testing” (OMST), which combines the benefits of CAT and MST and offsets their limitations. Moreover, OMST also provides some unique advantages over both CAT and MST. A simulation study was conducted to compare OMST with MST and CAT, and the results demonstrated the promising features of OMST. Finally, the “Discussion” section provides suggestions on possible future adaptive testing designs based on the OMST framework, which could provide great flexibility for adaptive tests in the digital future and open an avenue for all types of hybrid designs based on the different needs of specific tests.

1 aZheng, Yi1 aChang, Hua-Hua uhttp://apm.sagepub.com/content/39/2/104.abstract00532nas a2200145 4500008004500000245013100045210006900176300001000245490000600255100001900261700001100280700001600291700001500307856006400322 2014 Engldsh 00aCognitive Diagnostic Models and Computerized Adaptive Testing: Two New Item-Selection Methods That Incorporate Response Times0 aCognitive Diagnostic Models and Computerized Adaptive Testing Tw a59-760 v21 aFinkelman, M D1 aKim, W1 aWeissman, A1 aCook, R.J. uhttp://www.iacat.org/jcat/index.php/jcat/article/view/43/2101373nas a2200133 4500008003900000245008400039210006900123300001200192490000700204520092900211100002201140700002401162856005301186 2014 d00aComputerized Adaptive Testing for the Random Weights Linear Logistic Test Model0 aComputerized Adaptive Testing for the Random Weights Linear Logi a415-4310 v383 a

This article discusses four-item selection rules to design efficient individualized tests for the random weights linear logistic test model (RWLLTM): minimum posterior-weighted -error minimum expected posterior-weighted -error maximum expected Kullback–Leibler divergence between subsequent posteriors (KLP), and maximum mutual information (MUI). The RWLLTM decomposes test items into a set of subtasks or cognitive features and assumes individual-specific effects of the features on the difficulty of the items. The model extends and improves the well-known linear logistic test model in which feature effects are only estimated at the aggregate level. Simulations show that the efficiencies of the designs obtained with the different criteria appear to be equivalent. However, KLP and MUI are given preference over and due to their lesser complexity, which significantly reduces the computational burden.

1 aCrabbe, Marjolein1 aVandebroek, Martina uhttp://apm.sagepub.com/content/38/6/415.abstract01617nas a2200193 4500008003900000022001400039245007400053210006900127300001400196490000700210520105700217100002101274700001401295700001901309700001701328700001801345700001901363856004101382 2014 d a1745-398400aDetermining the Overall Impact of Interruptions During Online Testing0 aDetermining the Overall Impact of Interruptions During Online Te a419–4400 v513 a

With an increase in the number of online tests, interruptions during testing due to unexpected technical issues seem unavoidable. For example, interruptions occurred during several recent state tests. When interruptions occur, it is important to determine the extent of their impact on the examinees’ scores. There is a lack of research on this topic due to the novelty of the problem. This article is an attempt to fill that void. Several methods, primarily based on propensity score matching, linear regression, and item response theory, were suggested to determine the overall impact of the interruptions on the examinees’ scores. A realistic simulation study shows that the suggested methods have satisfactory Type I error rate and power. Then the methods were applied to data from the Indiana Statewide Testing for Educational Progress-Plus (ISTEP+) test that experienced interruptions in 2013. The results indicate that the interruptions did not have a significant overall impact on the student scores for the ISTEP+ test.

1 aSinharay, Sandip1 aWan, Ping1 aWhitaker, Mike1 aKim, Dong-In1 aZhang, Litong1 aChoi, Seung, W uhttp://dx.doi.org/10.1111/jedm.1205201835nas a2200157 4500008003900000022001400039245010200053210006900155300001400224490000700238520133800245100001501583700001901598700001901617856004101636 2014 d a1745-398400aAn Enhanced Approach to Combine Item Response Theory With Cognitive Diagnosis in Adaptive Testing0 aEnhanced Approach to Combine Item Response Theory With Cognitive a358–3800 v513 a

Computerized adaptive testing offers the possibility of gaining information on both the overall ability and cognitive profile in a single assessment administration. Some algorithms aiming for these dual purposes have been proposed, including the shadow test approach, the dual information method (DIM), and the constraint weighted method. The current study proposed two new methods, aggregate ranked information index (ARI) and aggregate standardized information index (ASI), which appropriately addressed the noncompatibility issue inherent in the original DIM method. More flexible weighting schemes that put different emphasis on information about general ability (i.e., θ in item response theory) and information about cognitive profile (i.e., α in cognitive diagnostic modeling) were also explored. Two simulation studies were carried out to investigate the effectiveness of the new methods and weighting schemes. Results showed that the new methods with the flexible weighting schemes could produce more accurate estimation of both overall ability and cognitive profile than the original DIM. Among them, the ASI with both empirical and theoretical weights is recommended, and attribute-level weighting scheme is preferred if some attributes are considered more important from a substantive perspective.

1 aWang, Chun1 aZheng, Chanjin1 aChang, Hua-Hua uhttp://dx.doi.org/10.1111/jedm.1205701425nas a2200157 4500008003900000245009200039210006900131300001200200490000700212520092000219100002001139700001601159700001801175700002101193856005301214 2014 d00aEnhancing Pool Utilization in Constructing the Multistage Test Using Mixed-Format Tests0 aEnhancing Pool Utilization in Constructing the Multistage Test U a268-2800 v383 a

This study investigated a new pool utilization method of constructing multistage tests (MST) using the mixed-format test based on the generalized partial credit model (GPCM). MST simulations of a classification test were performed to evaluate the MST design. A linear programming (LP) model was applied to perform MST reassemblies based on the initial MST construction. Three subsequent MST reassemblies were performed. For each reassembly, three test unit replacement ratios (TRRs; 0.22, 0.44, and 0.66) were investigated. The conditions of the three passing rates (30%, 50%, and 70%) were also considered in the classification testing. The results demonstrated that various MST reassembly conditions increased the overall pool utilization rates, while maintaining the desired MST construction. All MST testing conditions performed equally well in terms of the precision of the classification decision.

1 aPark, Ryoungsun1 aKim, Jiseon1 aChung, Hyewon1 aDodd, Barbara, G uhttp://apm.sagepub.com/content/38/4/268.abstract01889nas a2200157 4500008003900000245006900039210006800108300001200176490000700188520141000195100001901605700001601624700002001640700001801660856005301678 2014 d00aGeneral Test Overlap Control: Improved Algorithm for CAT and CCT0 aGeneral Test Overlap Control Improved Algorithm for CAT and CCT a229-2440 v383 a

This article proposed a new online test overlap control algorithm that is an improvement of Chen’s algorithm in controlling general test overlap rate for item pooling among a group of examinees. Chen’s algorithm is not very efficient in that not only item pooling between current examinee and prior examinees is controlled for but also item pooling between previous examinees, which would have been controlled for when they were current examinees. The proposed improvement increases efficiency by only considering item pooling between current and previous examinees, and its improved performance over Chen is demonstrated in a simulated computerized adaptive testing (CAT) environment. Moreover, the proposed algorithm is adapted for computerized classification testing (CCT) using the sequential probability ratio test procedure and is evaluated against some existing exposure control procedures. The proposed algorithm appears to work best in controlling general test overlap rate among the exposure control procedures examined without sacrificing much classification precision, though longer tests might be required for more stringent control of item pooling among larger groups. Given the capability of the proposed algorithm in controlling item pooling among a group of examinees of any size and its ease of implementation, it appears to be a good test overlap control method.

1 aChen, Shu-Ying1 aLei, Pui-Wa1 aChen, Jyun-Hong1 aLiu, Tzu-Chen uhttp://apm.sagepub.com/content/38/3/229.abstract00965nas a2200121 4500008003900000245008900039210006900128300001200197490000700209520055500216100001900771856005300790 2014 d00aA Numerical Investigation of the Recovery of Point Patterns With Minimal Information0 aNumerical Investigation of the Recovery of Point Patterns With M a329-3350 v383 a

A method has been proposed (Tsogo et al. 2001) in order to reconstruct the geometrical configuration of a large point set using minimal information. This paper employs numerical examples to investigate the proposed procedure. The suggested method has two great advantages. It reduces the volume of the data collection exercise and eases the computational effort involved in analyzing the data. It is suggested, however, that the method while possibly providing a useful starting point for a solution, does not provide a universal panacea.

1 aCox, M., A. A. uhttp://apm.sagepub.com/content/38/4/329.abstract02198nas a2200145 4500008003900000245007900039210006900118300001100187490000700198520173800205100001501943700001901958700002301977856005202000 2013 d00aDeriving Stopping Rules for Multidimensional Computerized Adaptive Testing0 aDeriving Stopping Rules for Multidimensional Computerized Adapti a99-1220 v373 a

Multidimensional computerized adaptive testing (MCAT) is able to provide a vector of ability estimates for each examinee, which could be used to provide a more informative profile of an examinee’s performance. The current literature on MCAT focuses on the fixed-length tests, which can generate less accurate results for those examinees whose abilities are quite different from the average difficulty level of the item bank when there are only a limited number of items in the item bank. Therefore, instead of stopping the test with a predetermined fixed test length, the authors use a more informative stopping criterion that is directly related to measurement accuracy. Specifically, this research derives four stopping rules that either quantify the measurement precision of the ability vector (i.e., minimum determinant rule [D-rule], minimum eigenvalue rule [E-rule], and maximum trace rule [T-rule]) or quantify the amount of available information carried by each item (i.e., maximum Kullback–Leibler divergence rule [K-rule]). The simulation results showed that all four stopping rules successfully terminated the test when the mean squared error of ability estimation is within a desired range, regardless of examinees’ true abilities. It was found that when using the D-, E-, or T-rule, examinees with extreme abilities tended to have tests that were twice as long as the tests received by examinees with moderate abilities. However, the test length difference with K-rule is not very dramatic, indicating that K-rule may not be very sensitive to measurement precision. In all cases, the cutoff value for each stopping rule needs to be adjusted on a case-by-case basis to find an optimal solution.

1 aWang, Chun1 aChang, Hua-Hua1 aBoughton, Keith, A uhttp://apm.sagepub.com/content/37/2/99.abstract00533nas a2200145 4500008004500000245008500045210006900130300001000199490000600209100001700215700001700232700001700249700001000266856011100276 2013 Engldsh 00aEstimating Measurement Precision in Reduced-Length Multi-Stage Adaptive Testing 0 aEstimating Measurement Precision in ReducedLength MultiStage Ada a67-870 v11 aCrotts, K.M.1 aZenisky, A L1 aSireci, S.G.1 aLi, X uhttp://mail.iacat.org/content/estimating-measurement-precision-reduced-length-multi-stage-adaptive-testing01630nas a2200157 4500008003900000245010100039210006900140300001200209490000700221520111200228100001501340700001601355700001901371700002501390856005701415 2013 d00aA Semiparametric Model for Jointly Analyzing Response Times and Accuracy in Computerized Testing0 aSemiparametric Model for Jointly Analyzing Response Times and Ac a381-4170 v383 a

The item response times (RTs) collected from computerized testing represent an underutilized type of information about items and examinees. In addition to knowing the examinees’ responses to each item, we can investigate the amount of time examinees spend on each item. Current models for RTs mainly focus on parametric models, which have the advantage of conciseness, but may suffer from reduced flexibility to fit real data. We propose a semiparametric approach, specifically, the Cox proportional hazards model with a latent speed covariate to model the RTs, embedded within the hierarchical framework proposed by van der Linden to model the RTs and response accuracy simultaneously. This semiparametric approach combines the flexibility of nonparametric modeling and the brevity and interpretability of the parametric modeling. A Markov chain Monte Carlo method for parameter estimation is given and may be used with sparse data obtained by computerized adaptive testing. Both simulation studies and real data analysis are carried out to demonstrate the applicability of the new model.

1 aWang, Chun1 aFan, Zhewen1 aChang, Hua-Hua1 aDouglas, Jeffrey, A. uhttp://jeb.sagepub.com/cgi/content/abstract/38/4/38101716nas a2200145 4500008003900000245008600039210006900125300001200194490000700206520124600213100001901459700002001478700001901498856005301517 2013 d00aVariable-Length Computerized Adaptive Testing Based on Cognitive Diagnosis Models0 aVariableLength Computerized Adaptive Testing Based on Cognitive  a563-5820 v373 a

Interest in developing computerized adaptive testing (CAT) under cognitive diagnosis models (CDMs) has increased recently. CAT algorithms that use a fixed-length termination rule frequently lead to different degrees of measurement precision for different examinees. Fixed precision, in which the examinees receive the same degree of measurement precision, is a major advantage of CAT over nonadaptive testing. In addition to the precision issue, test security is another important issue in practical CAT programs. In this study, the authors implemented two termination criteria for the fixed-precision rule and evaluated their performance under two popular CDMs using simulations. The results showed that using the two criteria with the posterior-weighted Kullback–Leibler information procedure for selecting items could achieve the prespecified measurement precision. A control procedure was developed to control item exposure and test overlap simultaneously among examinees. The simulation results indicated that in contrast to no method of controlling exposure, the control procedure developed in this study could maintain item exposure and test overlap at the prespecified level at the expense of only a few more items.

1 aHsu, Chia-Ling1 aWang, Wen-Chung1 aChen, Shu-Ying uhttp://apm.sagepub.com/content/37/7/563.abstract02777nas a2200229 4500008004100000022001400041245015400055210006900209260000900278300000800287490000700295520193000302653001802232653003702250653001102287653002702298653002302325653003702348100002002385700001902405856012302424 2012 eng d a1471-228800aComparison of two Bayesian methods to detect mode effects between paper-based and computerized adaptive assessments: a preliminary Monte Carlo study.0 aComparison of two Bayesian methods to detect mode effects betwee c2012 a1240 v123 a

BACKGROUND: Computerized adaptive testing (CAT) is being applied to health outcome measures developed as paper-and-pencil (P&P) instruments. Differences in how respondents answer items administered by CAT vs. P&P can increase error in CAT-estimated measures if not identified and corrected.

METHOD: Two methods for detecting item-level mode effects are proposed using Bayesian estimation of posterior distributions of item parameters: (1) a modified robust Z (RZ) test, and (2) 95% credible intervals (CrI) for the CAT-P&P difference in item difficulty. A simulation study was conducted under the following conditions: (1) data-generating model (one- vs. two-parameter IRT model); (2) moderate vs. large DIF sizes; (3) percentage of DIF items (10% vs. 30%), and (4) mean difference in θ estimates across modes of 0 vs. 1 logits. This resulted in a total of 16 conditions with 10 generated datasets per condition.

RESULTS: Both methods evidenced good to excellent false positive control, with RZ providing better control of false positives and with slightly higher power for CrI, irrespective of measurement model. False positives increased when items were very easy to endorse and when there with mode differences in mean trait level. True positives were predicted by CAT item usage, absolute item difficulty and item discrimination. RZ outperformed CrI, due to better control of false positive DIF.

CONCLUSIONS: Whereas false positives were well controlled, particularly for RZ, power to detect DIF was suboptimal. Research is needed to examine the robustness of these methods under varying prior assumptions concerning the distribution of item and person parameters and when data fail to conform to prior assumptions. False identification of DIF when items were very easy to endorse is a problem warranting additional investigation.

10aBayes Theorem10aData Interpretation, Statistical10aHumans10aMathematical Computing10aMonte Carlo Method10aOutcome Assessment (Health Care)1 aRiley, Barth, B1 aCarle, Adam, C uhttp://mail.iacat.org/content/comparison-two-bayesian-methods-detect-mode-effects-between-paper-based-and-computerized01240nas a2200145 4500008003900000245009000039210006900129300001200198490000700210520076800217100001900985700001701004700002001021856005301041 2012 d00aComputerized Adaptive Testing Using a Class of High-Order Item Response Theory Models0 aComputerized Adaptive Testing Using a Class of HighOrder Item Re a689-7060 v363 a

In the human sciences, a common assumption is that latent traits have a hierarchical structure. Higher order item response theory models have been developed to account for this hierarchy. In this study, computerized adaptive testing (CAT) algorithms based on these kinds of models were implemented, and their performance under a variety of situations was examined using simulations. The results showed that the CAT algorithms were very effective. The progressive method for item selection, the Sympson and Hetter method with online and freeze procedure for item exposure control, and the multinomial model for content balancing can simultaneously maintain good measurement precision, item exposure control, content balance, test security, and pool usage.

1 aHuang, Hung-Yu1 aChen, Po-Hsi1 aWang, Wen-Chung uhttp://apm.sagepub.com/content/36/8/689.abstract01556nas a2200169 4500008003900000245012400039210006900163300001000232490000700242520099400249100001901243700001801262700001801280700001601298700002001314856005201334 2012 d00aAn Empirical Evaluation of the Slip Correction in the Four Parameter Logistic Models With Computerized Adaptive Testing0 aEmpirical Evaluation of the Slip Correction in the Four Paramete a75-870 v363 a

In a selected response test, aberrant responses such as careless errors and lucky guesses might cause error in ability estimation because these responses do not actually reflect the knowledge that examinees possess. In a computerized adaptive test (CAT), these aberrant responses could further cause serious estimation error due to dynamic item administration. To enhance the robust performance of CAT against aberrant responses, Barton and Lord proposed the four-parameter logistic (4PL) item response theory (IRT) model. However, most studies relevant to the 4PL IRT model were conducted based on simulation experiments. This study attempts to investigate the performance of the 4PL IRT model as a slip-correction mechanism with an empirical experiment. The results showed that the 4PL IRT model could not only reduce the problematic underestimation of the examinees’ ability introduced by careless mistakes in practical situations but also improve measurement efficiency.

1 aYen, Yung-Chin1 aHo, Rong-Guey1 aLaio, Wen-Wei1 aChen, Li-Ju1 aKuo, Ching-Chin uhttp://apm.sagepub.com/content/36/2/75.abstract01089nas a2200157 4500008003900000022001400039245007000053210006900123300001400192490000700206520061300213100001600826700001700842700001700859856005500876 2012 d a1745-398400aInvestigating the Effect of Item Position in Computer-Based Tests0 aInvestigating the Effect of Item Position in ComputerBased Tests a362–3790 v493 a

Computer-based tests (CBTs) often use random ordering of items in order to minimize item exposure and reduce the potential for answer copying. Little research has been done, however, to examine item position effects for these tests. In this study, different versions of a Rasch model and different response time models were examined and applied to data from a CBT administration of a medical licensure examination. The models specifically were used to investigate whether item position affected item difficulty and item intensity estimates. Results indicated that the position effect was negligible.

1 aLi, Feiming1 aCohen, Allan1 aShen, Linjun uhttp://dx.doi.org/10.1111/j.1745-3984.2012.00181.x01899nas a2200157 4500008003900000245009300039210007100132300001200203490000700215520139800222100001401620700002101634700001601655700001701671856005301688 2012 d00aA Mixture Rasch Model–Based Computerized Adaptive Test for Latent Class Identification0 aMixture Rasch Model–Based Computerized Adaptive Test for Latent  a469-4930 v363 a

This study explored a computerized adaptive test delivery algorithm for latent class identification based on the mixture Rasch model. Four item selection methods based on the Kullback–Leibler (KL) information were proposed and compared with the reversed and the adaptive KL information under simulated testing conditions. When item separation was large, all item selection methods did not differ evidently in terms of accuracy in classifying examinees into different latent classes and estimating latent ability. However, when item separation was small, two methods with class-specific ability estimates performed better than the other two methods based on a single latent ability estimate across all latent classes. The three types of KL information distributions were compared. The KL and the reversed KL information could be the same or different depending on the ability level and the item difficulty difference between latent classes. Although the KL information and the reversed KL information were different at some ability levels and item difficulty difference levels, the use of the KL, the reversed KL, or the adaptive KL information did not affect the results substantially due to the symmetric distribution of item difficulty differences between latent classes in the simulated item pools. Item pool usage and classification convergence points were examined as well.

1 aHong Jiao1 aMacready, George1 aLiu, Junhui1 aCho, Youngmi uhttp://apm.sagepub.com/content/36/6/469.abstract01421nas a2200157 4500008003900000245008000039210006900119300001200188490000700200520092800207100001601135700001801151700002101169700002001190856005301210 2012 d00aPanel Design Variations in the Multistage Test Using the Mixed-Format Tests0 aPanel Design Variations in the Multistage Test Using the MixedFo a574-5880 v723 a

This study compared various panel designs of the multistage test (MST) using mixed-format tests in the context of classification testing. Simulations varied the design of the first-stage module. The first stage was constructed according to three levels of test information functions (TIFs) with three different TIF centers. Additional computerized adaptive test (CAT) conditions provided baseline comparisons. Three passing rate conditions were also included. The various MST conditions using mixed-format tests were constructed properly and performed well. When the levels of TIFs at the first stage were higher, the simulations produced a greater number of correct classifications. CAT with the randomesque-10 procedure yielded comparable results to the MST with increased levels of TIFs. Finally, all MST conditions achieved better test security results compared with CAT’s maximum information conditions.

1 aKim, Jiseon1 aChung, Hyewon1 aDodd, Barbara, G1 aPark, Ryoungsun uhttp://epm.sagepub.com/content/72/4/574.abstract01222nas a2200157 4500008003900000245013900039210006900178300001200247490000700259520066600266100002100932700001900953700001500972700002400987856005301011 2012 d00aOn the Reliability and Validity of a Numerical Reasoning Speed Dimension Derived From Response Times Collected in Computerized Testing0 aReliability and Validity of a Numerical Reasoning Speed Dimensio a245-2630 v723 a

Data from 181 college students were used to assess whether math reasoning item response times in computerized testing can provide valid and reliable measures of a speed dimension. The alternate forms reliability of the speed dimension was .85. A two-dimensional structural equation model suggests that the speed dimension is related to the accuracy of speeded responses. Speed factor scores were significantly correlated with performance on the ACT math scale. Results suggest that the speed dimension underlying response times can be reliably measured and that the dimension is related to the accuracy of performance under the pressure of time limits.

1 aDavison, Mark, L1 aSemmes, Robert1 aHuang, Lan1 aClose, Catherine, N uhttp://epm.sagepub.com/content/72/2/245.abstract00494nas a2200133 4500008003900000245008200039210006900121300001400190490000800204100001300212700001600225700001500241856010400256 2011 d00aApplying computerized adaptive testing to the CES-D scale: A simulation study0 aApplying computerized adaptive testing to the CESD scale A simul a147–1550 v1881 aSmits, N1 aCuijpers, P1 aStraten, A uhttp://mail.iacat.org/content/applying-computerized-adaptive-testing-ces-d-scale-simulation-study-001667nas a2200169 4500008004100000020004100041022001300082245008200095210006900177250001500246260001000261520108000271100001301351700001601364700001501380856010201395 2011 Eng d a0165-1781 (Print)0165-1781 (Linking) a2120866000aApplying computerized adaptive testing to the CES-D scale: A simulation study0 aApplying computerized adaptive testing to the CESD scale A simul a2011/01/07 cJan 33 aIn this paper we studied the appropriateness of developing an adaptive version of the Center of Epidemiological Studies-Depression (CES-D, Radloff, 1977) scale. Computerized Adaptive Testing (CAT) involves the computerized administration of a test in which each item is dynamically selected from a pool of items until a pre-specified measurement precision is reached. Two types of analyses were performed using the CES-D responses of a large sample of adolescents (N=1392). First, it was shown that the items met the psychometric requirements needed for CAT. Second, CATs were simulated by using the existing item responses as if they had been collected adaptively. CATs selecting only a small number of items gave results which, in terms of depression measurement and criterion validity, were only marginally different from the results of full CES-D assessment. It was concluded that CAT is a very fruitful way of improving the efficiency of the CES-D questionnaire. The discussion addresses the strengths and limitations of the application of CAT in mental health research.1 aSmits, N1 aCuijpers, P1 aStraten, A uhttp://mail.iacat.org/content/applying-computerized-adaptive-testing-ces-d-scale-simulation-study00495nas a2200121 4500008004100000245009500041210006900136653001800205653000800223653000900231100001900240856011400259 2011 eng d00aBuilding Affordable CD-CAT Systems for Schools To Address Today's Challenges In Assessment0 aBuilding Affordable CDCAT Systems for Schools To Address Todays 10aaffordability10aCAT10acost1 aChang, Hua-Hua uhttp://mail.iacat.org/content/building-affordable-cd-cat-systems-schools-address-todays-challenges-assessment00466nas a2200121 4500008003900000245009900039210006900138300001200207490000700219100001900226700003100245856006800276 2011 d00aComputerized Adaptive Testing with the Zinnes and Griggs Pairwise Preference Ideal Point Model0 aComputerized Adaptive Testing with the Zinnes and Griggs Pairwis a231-2470 v111 aStark, Stephen1 aChernyshenko, Oleksandr, S uhttp://www.tandfonline.com/doi/abs/10.1080/15305058.2011.56145902593nas a2200313 4500008004100000020004600041245011200087210006900199250001500268260001900283300000700302490000600309520160100315100001901916700001801935700001701953700001801970700001701988700001402005700001602019700001502035700001402050700002602064700001602090700001302106700001502119700001902134856012602153 2011 Eng d a1477-7525 (Electronic)1477-7525 (Linking)00aCross-cultural development of an item list for computer-adaptive testing of fatigue in oncological patients0 aCrosscultural development of an item list for computeradaptive t a2011/03/31 cMarch 29, 2011 a100 v93 aABSTRACT: INTRODUCTION: Within an ongoing project of the EORTC Quality of Life Group, we are developing computerized adaptive test (CAT) measures for the QLQ-C30 scales. These new CAT measures are conceptualised to reflect the same constructs as the QLQ-C30 scales. Accordingly, the Fatigue-CAT is intended to capture physical and general fatigue. METHODS: The EORTC approach to CAT development comprises four phases (literature search, operationalisation, pre-testing, and field testing). Phases I-III are described in detail in this paper. A literature search for fatigue items was performed in major medical databases. After refinement through several expert panels, the remaining items were used as the basis for adapting items and/or formulating new items fitting the EORTC item style. To obtain feedback from patients with cancer, these English items were translated into Danish, French, German, and Spanish and tested in the respective countries. RESULTS: Based on the literature search a list containing 588 items was generated. After a comprehensive item selection procedure focusing on content, redundancy, item clarity and item difficulty a list of 44 fatigue items was generated. Patient interviews (n=52) resulted in 12 revisions of wording and translations. DISCUSSION: The item list developed in phases I-III will be further investigated within a field-testing phase (IV) to examine psychometric characteristics and to fit an item response theory model. The Fatigue CAT based on this item bank will provide scores that are backward-compatible to the original QLQ-C30 fatigue scale.1 aGiesinger, J M1 aPetersen, M A1 aGroenvold, M1 aAaronson, N K1 aArraras, J I1 aConroy, T1 aGamper, E M1 aKemmler, G1 aKing, M T1 aOberguggenberger, A S1 aVelikova, G1 aYoung, T1 aHolzner, B1 aEortc-Qlg, E O uhttp://mail.iacat.org/content/cross-cultural-development-item-list-computer-adaptive-testing-fatigue-oncological-patients01232nas a2200193 4500008004100000245009400041210006900135260001200204520052700216653002600743653000800769653000800777653003100785653003200816653003000848100002000878700001900898856012100917 2011 eng d00aDetecting DIF between Conventional and Computerized Adaptive Testing: A Monte Carlo Study0 aDetecting DIF between Conventional and Computerized Adaptive Tes c10/20113 a

A comparison od two procedures, Modified Robust Z and 95% Credible Interval, were compared in a Monte Carlo study. Both procedures evidenced adequate control of false positive DIF results.

10a95% Credible Interval10aCAT10aDIF10adifferential item function10amodified robust Z statistic10aMonte Carlo methodologies1 aRiley, Barth, B1 aCarle, Adam, C uhttp://mail.iacat.org/content/detecting-dif-between-conventional-and-computerized-adaptive-testing-monte-carlo-study00483nas a2200145 4500008004100000245006100041210005900102653000800161653001600169653001300185100001900198700001600217700002200233856008200255 2011 eng d00aA Heuristic Of CAT Item Selection Procedure For Testlets0 aHeuristic Of CAT Item Selection Procedure For Testlets10aCAT10ashadow test10atestlets1 aChien, Yuehmei1 aShin, David1 aWay, Walter Denny uhttp://mail.iacat.org/content/heuristic-cat-item-selection-procedure-testlets00654nas a2200157 4500008004100000020001400041245015700055210006900212100001800281700001400299700001500313700001600328700001500344700001300359856012400372 2011 eng d a1073-191100aItem banks for measuring emotional distress from the Patient-Reported Outcomes Measurement Information System (PROMIS®): depression, anxiety, and anger0 aItem banks for measuring emotional distress from the PatientRepo1 aPilkonis, P A1 aChoi, S W1 aReise, S P1 aStover, A M1 aRiley, W T1 aCella, D uhttp://mail.iacat.org/content/item-banks-measuring-emotional-distress-patient-reported-outcomes-measurement-information00769nas a2200157 4500008004500000245008000045210006900125490000700194520022700201100001500428700001600443700001700459700001600476700001500492856010400507 2011 Engldsh 00aA new adaptive testing algorithm for shortening health literacy assessments0 anew adaptive testing algorithm for shortening health literacy as0 v113 a

 

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3178473/?tool=pmcentrez
1 aKandula, S1 aAncker, J S1 aKaufman, D R1 aCurrie, L M1 aQing, Z -T uhttp://mail.iacat.org/content/new-adaptive-testing-algorithm-shortening-health-literacy-assessments01203nas a2200145 4500008003900000245005800039210005600097300001000153490000700163520077300170100001900943700002200962700002100984856005201005 2011 d00aA New Stopping Rule for Computerized Adaptive Testing0 aNew Stopping Rule for Computerized Adaptive Testing a37-530 v713 a

The goal of the current study was to introduce a new stopping rule for computerized adaptive testing (CAT). The predicted standard error reduction (PSER) stopping rule uses the predictive posterior variance to determine the reduction in standard error that would result from the administration of additional items. The performance of the PSER was compared with that of the minimum standard error stopping rule and a modified version of the minimum information stopping rule in a series of simulated adaptive tests, drawn from a number of item pools. Results indicate that the PSER makes efficient use of CAT item pools, administering fewer items when predictive gains in information are small and increasing measurement precision when information is abundant.

1 aChoi, Seung, W1 aGrady, Matthew, W1 aDodd, Barbara, G uhttp://epm.sagepub.com/content/71/1/37.abstract01369nas a2200157 4500008003900000022001400039245010400053210006900157300001400226490000700240520085700247100001501104700001901119700001801138856005501156 2011 d a1745-398400aRestrictive Stochastic Item Selection Methods in Cognitive Diagnostic Computerized Adaptive Testing0 aRestrictive Stochastic Item Selection Methods in Cognitive Diagn a255–2730 v483 a

This paper proposes two new item selection methods for cognitive diagnostic computerized adaptive testing: the restrictive progressive method and the restrictive threshold method. They are built upon the posterior weighted Kullback-Leibler (KL) information index but include additional stochastic components either in the item selection index or in the item selection procedure. Simulation studies show that both methods are successful at simultaneously suppressing overexposed items and increasing the usage of underexposed items. Compared to item selection based upon (1) pure KL information and (2) the Sympson-Hetter method, the two new methods strike a better balance between item exposure control and measurement accuracy. The two new methods are also compared with Barrada et al.'s (2008) progressive method and proportional method.

1 aWang, Chun1 aChang, Hua-Hua1 aHuebner, Alan uhttp://dx.doi.org/10.1111/j.1745-3984.2011.00145.x01768nas a2200157 4500008004100000020002300041245020500064210006900269300001200338490000700350520108200357100001501439700001601454700001601470856012401486 2010 eng d a0146-62161552-349700aA comparison of content-balancing procedures for estimating multiple clinical domains in computerized adaptive testing: Relative precision, validity, and detection of persons with misfitting responses0 acomparison of contentbalancing procedures for estimating multipl a410-4230 v343 aThis simulation study sought to compare four different computerized adaptive testing (CAT) content-balancing procedures designed for use in a multidimensional assessment with respect to measurement precision, symptom severity classification, validity of clinical diagnostic recommendations, and sensitivity to atypical responding. The four content-balancing procedures were (a) no content balancing, (b) screener-based, (c) mixed (screener plus content balancing), and (d) full content balancing. In full content balancing and in mixed content balancing following administration of the screener items, item selection was based on (a) whether the target numberof items for the item’s subscale was reached and (b) the item’s information function. Mixed and full content balancing provided the best representation of items from each of the main subscales of the Internal Mental Distress Scale. These procedures also resulted in higher CAT to full-scale correlations for the Trauma and Homicidal/Suicidal Thought subscales and improved detection of atypical responding.Keywords1 aRiley, B B1 aDennis, M L1 aConrad, K J uhttp://mail.iacat.org/content/comparison-content-balancing-procedures-estimating-multiple-clinical-domains-computerized01688nas a2200145 4500008003900000245020500039210006900244300001200313490000700325520109200332100002001424700002301444700002201467856005301489 2010 d00aA Comparison of Content-Balancing Procedures for Estimating Multiple Clinical Domains in Computerized Adaptive Testing: Relative Precision, Validity, and Detection of Persons With Misfitting Responses0 aComparison of ContentBalancing Procedures for Estimating Multipl a410-4230 v343 a

This simulation study sought to compare four different computerized adaptive testing (CAT) content-balancing procedures designed for use in a multidimensional assessment with respect to measurement precision, symptom severity classification, validity of clinical diagnostic recommendations, and sensitivity to atypical responding. The four content-balancing procedures were (a) no content balancing, (b) screener-based, (c) mixed (screener plus content balancing), and (d) full content balancing. In full content balancing and in mixed content balancing following administration of the screener items, item selection was based on (a) whether the target number of items for the item’s subscale was reached and (b) the item’s information function. Mixed and full content balancing provided the best representation of items from each of the main subscales of the Internal Mental Distress Scale. These procedures also resulted in higher CAT to full-scale correlations for the Trauma and Homicidal/Suicidal Thought subscales and improved detection of atypical responding.

1 aRiley, Barth, B1 aDennis, Michael, L1 aConrad, Kendon, J uhttp://apm.sagepub.com/content/34/6/410.abstract00558nas a2200157 4500008004100000245008700041210006900128300001400197490001000211100001300221700001200234700001400246700001400260700001400274856011200288 2010 eng d00aDevelopment and evaluation of a confidence-weighting computerized adaptive testing0 aDevelopment and evaluation of a confidenceweighting computerized a163–1760 v13(3)1 aYen, Y C1 aHo, R G1 aChen, L J1 aChou, K Y1 aChen, Y L uhttp://mail.iacat.org/content/development-and-evaluation-confidence-weighting-computerized-adaptive-testing02224nas a2200289 4500008004100000020004600041245010800087210006900195250001500264300001200279490000700291520131000298100001801608700001701626700001801643700001401661700001401675700001801689700001401707700001701721700001501738700001301753700001401766700001601780700001301796856012501809 2010 Eng d a1573-2649 (Electronic)0962-9343 (Linking)00aDevelopment of computerized adaptive testing (CAT) for the EORTC QLQ-C30 physical functioning dimension0 aDevelopment of computerized adaptive testing CAT for the EORTC Q a2010/10/26 a479-4900 v203 aPURPOSE: Computerized adaptive test (CAT) methods, based on item response theory (IRT), enable a patient-reported outcome instrument to be adapted to the individual patient while maintaining direct comparability of scores. The EORTC Quality of Life Group is developing a CAT version of the widely used EORTC QLQ-C30. We present the development and psychometric validation of the item pool for the first of the scales, physical functioning (PF). METHODS: Initial developments (including literature search and patient and expert evaluations) resulted in 56 candidate items. Responses to these items were collected from 1,176 patients with cancer from Denmark, France, Germany, Italy, Taiwan, and the United Kingdom. The items were evaluated with regard to psychometric properties. RESULTS: Evaluations showed that 31 of the items could be included in a unidimensional IRT model with acceptable fit and good content coverage, although the pool may lack items at the upper extreme (good PF). There were several findings of significant differential item functioning (DIF). However, the DIF findings appeared to have little impact on the PF estimation. CONCLUSIONS: We have established an item pool for CAT measurement of PF and believe that this CAT instrument will clearly improve the EORTC measurement of PF.1 aPetersen, M A1 aGroenvold, M1 aAaronson, N K1 aChie, W C1 aConroy, T1 aCostantini, A1 aFayers, P1 aHelbostad, J1 aHolzner, B1 aKaasa, S1 aSinger, S1 aVelikova, G1 aYoung, T uhttp://mail.iacat.org/content/development-computerized-adaptive-testing-cat-eortc-qlq-c30-physical-functioning-dimension00606nas a2200157 4500008004100000245011500041210006900156300001400225490001000239100001200249700001500261700001800276700001400294700001300308856012700321 2010 eng d00aEfficiency of static and computer adaptive short forms compared to full-length measures of depressive symptoms0 aEfficiency of static and computer adaptive short forms compared  a125–1360 v19(1)1 aChoi, S1 aReise, S P1 aPilkonis, P A1 aHays, R D1 aCella, D uhttp://mail.iacat.org/content/efficiency-static-and-computer-adaptive-short-forms-compared-full-length-measures-depressive01300nas a2200181 4500008004100000020004100041245005800082210005600140250001500196260001000211300000900221490000700230520075600237100001400993700001501007700001401022856008201036 2010 Eng d a0013-1644 (Print)0013-1644 (Linking)00aA new stopping rule for computerized adaptive testing0 anew stopping rule for computerized adaptive testing a2011/02/01 cDec 1 a1-170 v703 aThe goal of the current study was to introduce a new stopping rule for computerized adaptive testing. The predicted standard error reduction stopping rule (PSER) uses the predictive posterior variance to determine the reduction in standard error that would result from the administration of additional items. The performance of the PSER was compared to that of the minimum standard error stopping rule and a modified version of the minimum information stopping rule in a series of simulated adaptive tests, drawn from a number of item pools. Results indicate that the PSER makes efficient use of CAT item pools, administering fewer items when predictive gains in information are small and increasing measurement precision when information is abundant.1 aChoi, S W1 aGrady, M W1 aDodd, B G uhttp://mail.iacat.org/content/new-stopping-rule-computerized-adaptive-testing01594nas a2200145 4500008004100000020001400041245007300055210006900128300001200197490000700209520110600216100001501322700001201337856009901349 2010 eng d a0033-312300aOnline calibration via variable length computerized adaptive testing0 aOnline calibration via variable length computerized adaptive tes a140-1570 v753 aItem calibration is an essential issue in modern item response theory based psychological or educational testing. Due to the popularity of computerized adaptive testing, methods to efficiently calibrate new items have become more important than that in the time when paper and pencil test administration is the norm. There are many calibration processes being proposed and discussed from both theoretical and practical perspectives. Among them, the online calibration may be one of the most cost effective processes. In this paper, under a variable length computerized adaptive testing scenario, we integrate the methods of adaptive design, sequential estimation, and measurement error models to solve online item calibration problems. The proposed sequential estimate of item parameters is shown to be strongly consistent and asymptotically normally distributed with a prechosen accuracy. Numerical results show that the proposed method is very promising in terms of both estimation accuracy and efficiency. The results of using calibrated items to estimate the latent trait levels are also reported.1 aChang, Y I1 aLu, H Y uhttp://mail.iacat.org/content/online-calibration-variable-length-computerized-adaptive-testing01445nas a2200121 4500008003900000245008600039210006900125300001200194490000700206520103800213100001901251856005301270 2010 d00aA Procedure for Controlling General Test Overlap in Computerized Adaptive Testing0 aProcedure for Controlling General Test Overlap in Computerized A a393-4090 v343 a

To date, exposure control procedures that are designed to control test overlap in computerized adaptive tests (CATs) are based on the assumption of item sharing between pairs of examinees. However, in practice, examinees may obtain test information from more than one previous test taker. This larger scope of information sharing needs to be considered in conducting test overlap control. The purpose of this study is to propose a test overlap control method such that the proportion of overlapping items encountered by an examinee with a group of previous examinees (described as general test overlap rate) can be controlled. Results indicated that item exposure rate and general test overlap rate could be simultaneously controlled by implementing the procedure. In addition, these two indices were controlled on the fly without any iterative simulations conducted prior to operational CATs. Thus, the proposed procedure would be an efficient method for controlling both the item exposure and general test overlap in CATs.

1 aChen, Shu-Ying uhttp://apm.sagepub.com/content/34/6/393.abstract01719nas a2200157 4500008003900000022001400039245009400053210006900147300001400216490000700230520121600237100001401453700002001467700001901487856005501506 2010 d a1745-398400aStratified and Maximum Information Item Selection Procedures in Computer Adaptive Testing0 aStratified and Maximum Information Item Selection Procedures in  a202–2260 v473 a

In this study we evaluated and compared three item selection procedures: the maximum Fisher information procedure (F), the a-stratified multistage computer adaptive testing (CAT) (STR), and a refined stratification procedure that allows more items to be selected from the high a strata and fewer items from the low a strata (USTR), along with completely random item selection (RAN). The comparisons were with respect to error variances, reliability of ability estimates and item usage through CATs simulated under nine test conditions of various practical constraints and item selection space. The results showed that F had an apparent precision advantage over STR and USTR under unconstrained item selection, but with very poor item usage. USTR reduced error variances for STR under various conditions, with small compromises in item usage. Compared to F, USTR enhanced item usage while achieving comparable precision in ability estimates; it achieved a precision level similar to F with improved item usage when items were selected under exposure control and with limited item selection space. The results provide implications for choosing an appropriate item selection procedure in applied settings.

1 aDeng, Hui1 aAnsley, Timothy1 aChang, Hua-Hua uhttp://dx.doi.org/10.1111/j.1745-3984.2010.00109.x00520nas a2200133 4500008004100000245009400041210006900135300001200204490000700216100001200223700001400235700001600249856012100265 2010 Eng d00aStratified and maximum information item selection procedures in computer adaptive testing0 aStratified and maximum information item selection procedures in  a202-2260 v471 aDeng, H1 aAnsley, T1 aChang, H -H uhttp://mail.iacat.org/content/stratified-and-maximum-information-item-selection-procedures-computer-adaptive-testing01858nas a2200181 4500008004100000020001400041245011000055210006900165300001200234490000700246520122300253100001601476700001601492700001601508700001201524700001301536856012701549 2010 eng d a1529-771300aThe use of PROMIS and assessment center to deliver patient-reported outcome measures in clinical research0 ause of PROMIS and assessment center to deliver patientreported o a304-3140 v113 aThe Patient-Reported Outcomes Measurement Information System (PROMIS) was developed as one of the first projects funded by the NIH Roadmap for Medical Research Initiative to re-engineer the clinical research enterprise. The primary goal of PROMIS is to build item banks and short forms that measure key health outcome domains that are manifested in a variety of chronic diseases which could be used as a "common currency" across research projects. To date, item banks, short forms and computerized adaptive tests (CAT) have been developed for 13 domains with relevance to pediatric and adult subjects. To enable easy delivery of these new instruments, PROMIS built a web-based resource (Assessment Center) for administering CATs and other self-report data, tracking item and instrument development, monitoring accrual, managing data, and storing statistical analysis results. Assessment Center can also be used to deliver custom researcher developed content, and has numerous features that support both simple and complicated accrual designs (branching, multiple arms, multiple time points, etc.). This paper provides an overview of the development of the PROMIS item banks and details Assessment Center functionality.1 aGershon, RC1 aRothrock, N1 aHanrahan, R1 aBass, M1 aCella, D uhttp://mail.iacat.org/content/use-promis-and-assessment-center-deliver-patient-reported-outcome-measures-clinical-research02758nas a2200241 4500008004100000020004600041245009400087210006900181250001500250300000800265490000600273520198300279100001502262700001602277700001402293700001302307700002002320700001902340700001702359700001302376700001402389856011302403 2010 eng d a1477-7525 (Electronic)1477-7525 (Linking)00aValidation of a computer-adaptive test to evaluate generic health-related quality of life0 aValidation of a computeradaptive test to evaluate generic health a2010/12/07 a1470 v83 aBACKGROUND: Health Related Quality of Life (HRQoL) is a relevant variable in the evaluation of health outcomes. Questionnaires based on Classical Test Theory typically require a large number of items to evaluate HRQoL. Computer Adaptive Testing (CAT) can be used to reduce tests length while maintaining and, in some cases, improving accuracy. This study aimed at validating a CAT based on Item Response Theory (IRT) for evaluation of generic HRQoL: the CAT-Health instrument. METHODS: Cross-sectional study of subjects aged over 18 attending Primary Care Centres for any reason. CAT-Health was administered along with the SF-12 Health Survey. Age, gender and a checklist of chronic conditions were also collected. CAT-Health was evaluated considering: 1) feasibility: completion time and test length; 2) content range coverage, Item Exposure Rate (IER) and test precision; and 3) construct validity: differences in the CAT-Health scores according to clinical variables and correlations between both questionnaires. RESULTS: 396 subjects answered CAT-Health and SF-12, 67.2% females, mean age (SD) 48.6 (17.7) years. 36.9% did not report any chronic condition. Median completion time for CAT-Health was 81 seconds (IQ range = 59-118) and it increased with age (p < 0.001). The median number of items administered was 8 (IQ range = 6-10). Neither ceiling nor floor effects were found for the score. None of the items in the pool had an IER of 100% and it was over 5% for 27.1% of the items. Test Information Function (TIF) peaked between levels -1 and 0 of HRQoL. Statistically significant differences were observed in the CAT-Health scores according to the number and type of conditions. CONCLUSIONS: Although domain-specific CATs exist for various areas of HRQoL, CAT-Health is one of the first IRT-based CATs designed to evaluate generic HRQoL and it has proven feasible, valid and efficient, when administered to a broad sample of individuals attending primary care settings.1 aRebollo, P1 aCastejon, I1 aCuervo, J1 aVilla, G1 aGarcia-Cueto, E1 aDiaz-Cuervo, H1 aZardain, P C1 aMuniz, J1 aAlonso, J uhttp://mail.iacat.org/content/validation-computer-adaptive-test-evaluate-generic-health-related-quality-life02652nas a2200181 4500008004100000020001400041245007900055210006900134300001000203490000700213520201300220653005202233653004002285100001502325700001302340700001502353856010202368 2009 eng d a0360-131500aAn adaptive testing system for supporting versatile educational assessment0 aadaptive testing system for supporting versatile educational ass a53-670 v523 aWith the rapid growth of computer and mobile technology, it is a challenge to integrate computer based test (CBT) with mobile learning (m-learning) especially for formative assessment and self-assessment. In terms of self-assessment, computer adaptive test (CAT) is a proper way to enable students to evaluate themselves. In CAT, students are assessed through a process that uses item response theory (IRT), a well-founded psychometric theory. Furthermore, a large item bank is indispensable to a test, but when a CAT system has a large item bank, the test item selection of IRT becomes more tedious. Besides the large item bank, item exposure mechanism is also essential to a testing system. However, IRT all lack the above-mentioned points. These reasons have motivated the authors to carry out this study. This paper describes a design issue aimed at the development and implementation of an adaptive testing system. The system can support several assessment functions and different devices. Moreover, the researchers apply a novel approach, particle swarm optimization (PSO) to alleviate the computational complexity and resolve the problem of item exposure. Throughout the development of the system, a formative evaluation was embedded into an integral part of the design methodology that was used for improving the system. After the system was formally released onto the web, some questionnaires and experiments were conducted to evaluate the usability, precision, and efficiency of the system. The results of these evaluations indicated that the system provides an adaptive testing for different devices and supports versatile assessment functions. Moreover, the system can estimate students' ability reliably and validly and conduct an adaptive test efficiently. Furthermore, the computational complexity of the system was alleviated by the PSO approach. By the approach, the test item selection procedure becomes efficient and the average best fitness values are very close to the optimal solutions.10aArchitectures for educational technology system10aDistance education and telelearning1 aHuang, Y-M1 aLin, Y-T1 aCheng, S-C uhttp://mail.iacat.org/content/adaptive-testing-system-supporting-versatile-educational-assessment00591nas a2200121 4500008004100000245010000041210006900141260009700210100001600307700001500323700001500338856011600353 2009 eng d00aAdequacy of an item pool measuring proficiency in English language to implement a CAT procedure0 aAdequacy of an item pool measuring proficiency in English langua aD. J. Weiss (Ed.), Proceedings of the 2009 GMAC Conference on Computerized Adaptive Testing.1 aKarino, C A1 aCosta, D R1 aLaros, J A uhttp://mail.iacat.org/content/adequacy-item-pool-measuring-proficiency-english-language-implement-cat-procedure00637nas a2200145 4500008004100000245010000041210006900141260009700210100001300307700001300320700001300333700001400346700001700360856011400377 2009 eng d00aApplications of CAT in admissions to higher education in Israel: Twenty-two years of experience0 aApplications of CAT in admissions to higher education in Israel  aD. J. Weiss (Ed.), Proceedings of the 2009 GMAC Conference on Computerized Adaptive Testing.1 aGafni, N1 aCohen, Y1 aRoded, K1 aBaumer, M1 aMoshinsky, A uhttp://mail.iacat.org/content/applications-cat-admissions-higher-education-israel-twenty-two-years-experience01941nas a2200121 4500008004100000245010300041210006900144260010000213520135600313100001601669700001401685856012001699 2009 eng d00aA burdened CAT: Incorporating response burden with maximum Fisher's information for item selection0 aburdened CAT Incorporating response burden with maximum Fishers  aIn D. J. Weiss (Ed.), Proceedings of the 2009 GMAC Conference on Computerized Adaptive Testing.3 aWidely used in various educational and vocational assessment applications, computerized adaptive testing (CAT) has recently begun to be used to measure patient-reported outcomes Although successful in reducing respondent burden, most current CAT algorithms do not formally consider it as part of the item selection process. This study used a loss function approach motivated by decision theory to develop an item selection method that incorporates respondent burden into the item selection process based on maximum Fisher information item selection. Several different loss functions placing varying degrees of importance on respondent burden were compared, using an item bank of 62 polytomous items measuring depressive symptoms. One dataset consisted of the real responses from the 730 subjects who responded to all the items. A second dataset consisted of simulated responses to all the items based on a grid of latent trait scores with replicates at each grid point. The algorithm enables a CAT administrator to more efficiently control the respondent burden without severely affecting the measurement precision than when using MFI alone. In particular, the loss function incorporating respondent burden protected respondents from receiving longer tests when their estimated trait score fell in a region where there were few informative items. 1 aSwartz, R J1 aChoi, S W uhttp://mail.iacat.org/content/burdened-cat-incorporating-response-burden-maximum-fishers-information-item-selection00573nas a2200097 4500008004100000245013000041210006900171260009700240100001400337856012400351 2009 eng d00aComparison of adaptive Bayesian estimation and weighted Bayesian estimation in multidimensional computerized adaptive testing0 aComparison of adaptive Bayesian estimation and weighted Bayesian aD. J. Weiss (Ed.), Proceedings of the 2009 GMAC Conference on Computerized Adaptive Testing.1 aChen, P H uhttp://mail.iacat.org/content/comparison-adaptive-bayesian-estimation-and-weighted-bayesian-estimation-multidimensional01148nas a2200133 4500008003900000245006700039210006700106300001200173490000700185520072700192100001900919700002300938856005300961 2009 d00aComparison of CAT Item Selection Criteria for Polytomous Items0 aComparison of CAT Item Selection Criteria for Polytomous Items a419-4400 v333 a

Item selection is a core component in computerized adaptive testing (CAT). Several studies have evaluated new and classical selection methods; however, the few that have applied such methods to the use of polytomous items have reported conflicting results. To clarify these discrepancies and further investigate selection method properties, six different selection methods are compared systematically. The results showed no clear benefit from more sophisticated selection criteria and showed one method previously believed to be superior—the maximum expected posterior weighted information (MEPWI)—to be mathematically equivalent to a simpler method, the maximum posterior weighted information (MPWI).

1 aChoi, Seung, W1 aSwartz, Richard, J uhttp://apm.sagepub.com/content/33/6/419.abstract00438nas a2200121 4500008004100000245006700041210006700108300001400175490000700189100001400196700001600210856009000226 2009 eng d00aComparison of CAT item selection criteria for polytomous items0 aComparison of CAT item selection criteria for polytomous items a419–4400 v331 aChoi, S W1 aSwartz, R J uhttp://mail.iacat.org/content/comparison-cat-item-selection-criteria-polytomous-items00596nas a2200133 4500008004100000245008600041210006900127260009700196100001500293700001600308700001700324700001700341856010400358 2009 eng d00aA comparison of three methods of item selection for computerized adaptive testing0 acomparison of three methods of item selection for computerized a aD. J. Weiss (Ed.), Proceedings of the 2009 GMAC Conference on Computerized Adaptive Testing.1 aCosta, D R1 aKarino, C A1 aMoura, F A S1 aAndrade, D F uhttp://mail.iacat.org/content/comparison-three-methods-item-selection-computerized-adaptive-testing00449nas a2200097 4500008004100000245005800041210005800099260009700157100001300254856008400267 2009 eng d00aComputerized adaptive testing for cognitive diagnosis0 aComputerized adaptive testing for cognitive diagnosis aD. J. Weiss (Ed.), Proceedings of the 2009 GMAC Conference on Computerized Adaptive Testing.1 aCheng, Y uhttp://mail.iacat.org/content/computerized-adaptive-testing-cognitive-diagnosis01651nas a2200157 4500008003900000245010700039210006900146300001000215490000700225520113900232100001501371700001901386700002101405700001501426856005201441 2009 d00aConstraint-Weighted a-Stratification for Computerized Adaptive Testing With Nonstatistical Constraints0 aConstraintWeighted aStratification for Computerized Adaptive Tes a35-490 v693 a

a-stratification is a method that utilizes items with small discrimination (a) parameters early in an exam and those with higher a values when more is learned about the ability parameter. It can achieve much better item usage than the maximum information criterion (MIC). To make a-stratification more practical and more widely applicable, a method for weighting the item selection process in a-stratification as a means of satisfying multiple test constraints is proposed. This method is studied in simulation against an analogous method without stratification as well as a-stratification using descending-rather than ascending-a procedures. In addition, a variation of a-stratification that allows for unbalanced usage of a parameters is included in the study to examine the trade-off between efficiency and exposure control. Finally, MIC and randomized item selection are included as baseline measures. Results indicate that the weighting mechanism successfully addresses the constraints, that stratification helps to a great extent balancing exposure rates, and that the ascending-a design improves measurement precision.

1 aYing Cheng1 aChang, Hua-Hua1 aDouglas, Jeffrey1 aFanmin Guo uhttp://epm.sagepub.com/content/69/1/35.abstract00619nas a2200145 4500008004100000245016200041210006900203300001000272490000700282100001300289700001900302700001500321700001100336856012600347 2009 eng d00aConstraint-weighted a-stratification for computerized adaptive testing with nonstatistical constraints: Balancing measurement efficiency and exposure control0 aConstraintweighted astratification for computerized adaptive tes a35-490 v691 aCheng, Y1 aChang, Hua-Hua1 aDouglas, J1 aGuo, F uhttp://mail.iacat.org/content/constraint-weighted-stratification-computerized-adaptive-testing-nonstatistical-constraints00513nas a2200121 4500008003900000245011000039210006900149300001000218490000600228100001200234700002000246856012500266 2009 d00aDiagnostic classification models and multidimensional adaptive testing: A commentary on Rupp and Templin.0 aDiagnostic classification models and multidimensional adaptive t a58-610 v71 aFrey, A1 aCarstensen, C H uhttp://mail.iacat.org/content/diagnostic-classification-models-and-multidimensional-adaptive-testing-commentary-rupp-and00563nas a2200145 4500008004100000020004600041245008900087210006900176250001500245260001000260300001200270490000700282100001400289856011400303 2009 Eng d a1552-3497 (Electronic)0146-6216 (Linking)00aFirestar: Computerized adaptive testing simulation program for polytomous IRT models0 aFirestar Computerized adaptive testing simulation program for po a2009/12/17 cNov 1 a644-6450 v331 aChoi, S W uhttp://mail.iacat.org/content/firestar-computerized-adaptive-testing-simulation-program-polytomous-irt-models00460nas a2200109 4500008004100000245008900041210006900130300001400199490000700213100001400220856011600234 2009 eng d00aFirestar: Computerized adaptive testing simulation program for polytomous IRT models0 aFirestar Computerized adaptive testing simulation program for po a644–6450 v331 aChoi, S W uhttp://mail.iacat.org/content/firestar-computerized-adaptive-testing-simulation-program-polytomous-irt-models-002036nas a2200121 4500008004100000245009400041210006900135260009700204520146200301100001201763700001901775856012001794 2009 eng d00aKullback-Leibler information in multidimensional adaptive testing: theory and application0 aKullbackLeibler information in multidimensional adaptive testing aD. J. Weiss (Ed.), Proceedings of the 2009 GMAC Conference on Computerized Adaptive Testing.3 aBuilt on multidimensional item response theory (MIRT), multidimensional adaptive testing (MAT) can, in principle, provide a promising choice to ensuring efficient estimation of each ability dimension in a multidimensional vector. Currently, two item selection procedures have been developed for MAT, one based on Fisher information embedded within a Bayesian framework, and the other powered by Kullback-Leibler (KL) information. It is well-known that in unidimensional IRT that the second derivative of KL information (also termed “global information”) is Fisher information evaluated atθ 0. This paper first generalizes the relationship between these two types of information in two ways—the analytical result is given as well as the graphical representation, to enhance interpretation and understanding. Second, a KL information index is constructed for MAT, which represents the integration of KL nformation over all of the ability dimensions. This paper further discusses how this index correlates with the item discrimination parameters. The analytical results would lay foundation for future development of item selection methods in MAT which can help equalize the item exposure rate. Finally, a simulation study is conducted to verify the above results. The connection between the item parameters, item KL information, and item exposure rate is demonstrated for empirical MAT delivered by an item bank calibrated under two-dimensional IRT.1 aWang, C1 aChang, Hua-Hua uhttp://mail.iacat.org/content/kullback-leibler-information-multidimensional-adaptive-testing-theory-and-application01655nas a2200289 4500008004100000020004100041245011100082210006900193250001500262260000800277300001100285490000700296520053700303653004800840653006200888653005700950653001101007653002701018653002401045653005101069653004701120653003101167653001301198100001301211700001901224856012201243 2009 eng d a0007-1102 (Print)0007-1102 (Linking)00aThe maximum priority index method for severely constrained item selection in computerized adaptive testing0 amaximum priority index method for severely constrained item sele a2008/06/07 cMay a369-830 v623 aThis paper introduces a new heuristic approach, the maximum priority index (MPI) method, for severely constrained item selection in computerized adaptive testing. Our simulation study shows that it is able to accommodate various non-statistical constraints simultaneously, such as content balancing, exposure control, answer key balancing, and so on. Compared with the weighted deviation modelling method, it leads to fewer constraint violations and better exposure control while maintaining the same level of measurement precision.10aAptitude Tests/*statistics & numerical data10aDiagnosis, Computer-Assisted/*statistics & numerical data10aEducational Measurement/*statistics & numerical data10aHumans10aMathematical Computing10aModels, Statistical10aPersonality Tests/*statistics & numerical data10aPsychometrics/*statistics & numerical data10aReproducibility of Results10aSoftware1 aCheng, Y1 aChang, Hua-Hua uhttp://mail.iacat.org/content/maximum-priority-index-method-severely-constrained-item-selection-computerized-adaptive00545nas a2200121 4500008004100000245007000041210006900111260009700180100001200277700001900289700001500308856010000323 2009 eng d00aObtaining reliable diagnostic information through constrained CAT0 aObtaining reliable diagnostic information through constrained CA aD. J. Weiss (Ed.), Proceedings of the 2009 GMAC Conference on Computerized Adaptive Testing.1 aWang, C1 aChang, Hua-Hua1 aDouglas, J uhttp://mail.iacat.org/content/obtaining-reliable-diagnostic-information-through-constrained-cat00582nas a2200109 4500008004100000245011800041210006900159260009700228100001700325700001300342856011700355 2009 eng d00aOptimizing item exposure control algorithms for polytomous computerized adaptive tests with restricted item banks0 aOptimizing item exposure control algorithms for polytomous compu aD. J. Weiss (Ed.), Proceedings of the 2009 GMAC Conference on Computerized Adaptive Testing.1 aChajewski, M1 aLewis, C uhttp://mail.iacat.org/content/optimizing-item-exposure-control-algorithms-polytomous-computerized-adaptive-tests02905nas a2200289 4500008004100000020004100041245011100082210006900193250001500262260000800277300001400285490000700299520193300306653002702239653003802266653004102304653001902345653001102364653001402375653003102389100001502420700001302435700001202448700001602460700001302476856012602489 2009 eng d a0315-162X (Print)0315-162X (Linking)00aProgress in assessing physical function in arthritis: PROMIS short forms and computerized adaptive testing0 aProgress in assessing physical function in arthritis PROMIS shor a2009/09/10 cSep a2061-20660 v363 aOBJECTIVE: Assessing self-reported physical function/disability with the Health Assessment Questionnaire Disability Index (HAQ) and other instruments has become central in arthritis research. Item response theory (IRT) and computerized adaptive testing (CAT) techniques can increase reliability and statistical power. IRT-based instruments can improve measurement precision substantially over a wider range of disease severity. These modern methods were applied and the magnitude of improvement was estimated. METHODS: A 199-item physical function/disability item bank was developed by distilling 1865 items to 124, including Legacy Health Assessment Questionnaire (HAQ) and Physical Function-10 items, and improving precision through qualitative and quantitative evaluation in over 21,000 subjects, which included about 1500 patients with rheumatoid arthritis and osteoarthritis. Four new instruments, (A) Patient-Reported Outcomes Measurement Information (PROMIS) HAQ, which evolved from the original (Legacy) HAQ; (B) "best" PROMIS 10; (C) 20-item static (short) forms; and (D) simulated PROMIS CAT, which sequentially selected the most informative item, were compared with the HAQ. RESULTS: Online and mailed administration modes yielded similar item and domain scores. The HAQ and PROMIS HAQ 20-item scales yielded greater information content versus other scales in patients with more severe disease. The "best" PROMIS 20-item scale outperformed the other 20-item static forms over a broad range of 4 standard deviations. The 10-item simulated PROMIS CAT outperformed all other forms. CONCLUSION: Improved items and instruments yielded better information. The PROMIS HAQ is currently available and considered validated. The new PROMIS short forms, after validation, are likely to represent further improvement. CAT-based physical function/disability assessment offers superior performance over static forms of equal length.10a*Disability Evaluation10a*Outcome Assessment (Health Care)10aArthritis/diagnosis/*physiopathology10aHealth Surveys10aHumans10aPrognosis10aReproducibility of Results1 aFries, J F1 aCella, D1 aRose, M1 aKrishnan, E1 aBruce, B uhttp://mail.iacat.org/content/progress-assessing-physical-function-arthritis-promis-short-forms-and-computerized-adaptive02598nas a2200337 4500008004100000020004600041245012800087210006900215250001500284300000700299490000600306520149200312653003201804653002301836653002501859653003401884653001101918653001101929653000901940653002601949653003101975653002702006653001102033653001802044100001502062700001202077700001402089700001802103700001202121856012702133 2009 eng d a1477-7525 (Electronic)1477-7525 (Linking)00aReduction in patient burdens with graphical computerized adaptive testing on the ADL scale: tool development and simulation0 aReduction in patient burdens with graphical computerized adaptiv a2009/05/07 a390 v73 aBACKGROUND: The aim of this study was to verify the effectiveness and efficacy of saving time and reducing burden for patients, nurses, and even occupational therapists through computer adaptive testing (CAT). METHODS: Based on an item bank of the Barthel Index (BI) and the Frenchay Activities Index (FAI) for assessing comprehensive activities of daily living (ADL) function in stroke patients, we developed a visual basic application (VBA)-Excel CAT module, and (1) investigated whether the averaged test length via CAT is shorter than that of the traditional all-item-answered non-adaptive testing (NAT) approach through simulation, (2) illustrated the CAT multimedia on a tablet PC showing data collection and response errors of ADL clinical functional measures in stroke patients, and (3) demonstrated the quality control of endorsing scale with fit statistics to detect responding errors, which will be further immediately reconfirmed by technicians once patient ends the CAT assessment. RESULTS: The results show that endorsed items could be shorter on CAT (M = 13.42) than on NAT (M = 23) at 41.64% efficiency in test length. However, averaged ability estimations reveal insignificant differences between CAT and NAT. CONCLUSION: This study found that mobile nursing services, placed at the bedsides of patients could, through the programmed VBA-Excel CAT module, reduce the burden to patients and save time, more so than the traditional NAT paper-and-pencil testing appraisals.10a*Activities of Daily Living10a*Computer Graphics10a*Computer Simulation10a*Diagnosis, Computer-Assisted10aFemale10aHumans10aMale10aPoint-of-Care Systems10aReproducibility of Results10aStroke/*rehabilitation10aTaiwan10aUnited States1 aChien, T W1 aWu, H M1 aWang, W-C1 aCastillo, R V1 aChou, W uhttp://mail.iacat.org/content/reduction-patient-burdens-graphical-computerized-adaptive-testing-adl-scale-tool-development00427nas a2200109 4500008004100000245007300041210006900114300001200183490000700195100001300202856010200215 2009 eng d00aWhen cognitive diagnosis meets computerized adaptive testing: CD-CAT0 aWhen cognitive diagnosis meets computerized adaptive testing CDC a619-6320 v741 aCheng, Y uhttp://mail.iacat.org/content/when-cognitive-diagnosis-meets-computerized-adaptive-testing-cd-cat03437nas a2200481 4500008004100000020004600041245013800087210006900225250001500294260000800309300001200317490000700329520191400336653002702250653002302277653003102300653001502331653001602346653001002362653002102372653002402393653002302417653003802440653001102478653002202489653001102511653001102522653000902533653003702542653002102579653003102600653002602631653001702657653003202674653001602706653002802722100001602750700001502766700001002781700001502791700002502806856012402831 2008 eng d a1532-821X (Electronic)0003-9993 (Linking)00aAssessing self-care and social function using a computer adaptive testing version of the pediatric evaluation of disability inventory0 aAssessing selfcare and social function using a computer adaptive a2008/04/01 cApr a622-6290 v893 aOBJECTIVE: To examine score agreement, validity, precision, and response burden of a prototype computer adaptive testing (CAT) version of the self-care and social function scales of the Pediatric Evaluation of Disability Inventory compared with the full-length version of these scales. DESIGN: Computer simulation analysis of cross-sectional and longitudinal retrospective data; cross-sectional prospective study. SETTING: Pediatric rehabilitation hospital, including inpatient acute rehabilitation, day school program, outpatient clinics; community-based day care, preschool, and children's homes. PARTICIPANTS: Children with disabilities (n=469) and 412 children with no disabilities (analytic sample); 38 children with disabilities and 35 children without disabilities (cross-validation sample). INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Summary scores from prototype CAT applications of each scale using 15-, 10-, and 5-item stopping rules; scores from the full-length self-care and social function scales; time (in seconds) to complete assessments and respondent ratings of burden. RESULTS: Scores from both computer simulations and field administration of the prototype CATs were highly consistent with scores from full-length administration (r range, .94-.99). Using computer simulation of retrospective data, discriminant validity, and sensitivity to change of the CATs closely approximated that of the full-length scales, especially when the 15- and 10-item stopping rules were applied. In the cross-validation study the time to administer both CATs was 4 minutes, compared with over 16 minutes to complete the full-length scales. CONCLUSIONS: Self-care and social function score estimates from CAT administration are highly comparable with those obtained from full-length scale administration, with small losses in validity and precision and substantial decreases in administration time.10a*Disability Evaluation10a*Social Adjustment10aActivities of Daily Living10aAdolescent10aAge Factors10aChild10aChild, Preschool10aComputer Simulation10aCross-Over Studies10aDisabled Children/*rehabilitation10aFemale10aFollow-Up Studies10aHumans10aInfant10aMale10aOutcome Assessment (Health Care)10aReference Values10aReproducibility of Results10aRetrospective Studies10aRisk Factors10aSelf Care/*standards/trends10aSex Factors10aSickness Impact Profile1 aCoster, W J1 aHaley, S M1 aNi, P1 aDumas, H M1 aFragala-Pinkham, M A uhttp://mail.iacat.org/content/assessing-self-care-and-social-function-using-computer-adaptive-testing-version-pediatric02583nas a2200241 4500008004100000020002200041245009000063210006900153250001500222260000800237300001100245490000700256520178300263653001502046653001502061653002502076653002902101653005002130653001102180100001602191700001902207856011502226 2008 eng d a1554-351X (Print)00aCombining computer adaptive testing technology with cognitively diagnostic assessment0 aCombining computer adaptive testing technology with cognitively  a2008/08/14 cAug a808-210 v403 aA major advantage of computerized adaptive testing (CAT) is that it allows the test to home in on an examinee's ability level in an interactive manner. The aim of the new area of cognitive diagnosis is to provide information about specific content areas in which an examinee needs help. The goal of this study was to combine the benefit of specific feedback from cognitively diagnostic assessment with the advantages of CAT. In this study, three approaches to combining these were investigated: (1) item selection based on the traditional ability level estimate (theta), (2) item selection based on the attribute mastery feedback provided by cognitively diagnostic assessment (alpha), and (3) item selection based on both the traditional ability level estimate (theta) and the attribute mastery feedback provided by cognitively diagnostic assessment (alpha). The results from these three approaches were compared for theta estimation accuracy, attribute mastery estimation accuracy, and item exposure control. The theta- and alpha-based condition outperformed the alpha-based condition regarding theta estimation, attribute mastery pattern estimation, and item exposure control. Both the theta-based condition and the theta- and alpha-based condition performed similarly with regard to theta estimation, attribute mastery estimation, and item exposure control, but the theta- and alpha-based condition has an additional advantage in that it uses the shadow test method, which allows the administrator to incorporate additional constraints in the item selection process, such as content balancing, item type constraints, and so forth, and also to select items on the basis of both the current theta and alpha estimates, which can be built on top of existing 3PL testing programs.10a*Cognition10a*Computers10a*Models, Statistical10a*User-Computer Interface10aDiagnosis, Computer-Assisted/*instrumentation10aHumans1 aMcGlohen, M1 aChang, Hua-Hua uhttp://mail.iacat.org/content/combining-computer-adaptive-testing-technology-cognitively-diagnostic-assessment03042nas a2200481 4500008004100000020004600041245012200087210006900209250001500278260000800293300001200301490000700313520155700320653003201877653003101909653002201940653002001962653001001982653000901992653002202001653002802023653003302051653001102084653001102095653002502106653000902131653001602140653004602156653002202202653002402224653003002248653002902278100001502307700001402322700001502336700002402351700001802375700001102393700001602404700001002420700001502430856011502445 2008 eng d a1532-821X (Electronic)0003-9993 (Linking)00aComputerized adaptive testing for follow-up after discharge from inpatient rehabilitation: II. Participation outcomes0 aComputerized adaptive testing for followup after discharge from  a2008/01/30 cFeb a275-2830 v893 aOBJECTIVES: To measure participation outcomes with a computerized adaptive test (CAT) and compare CAT and traditional fixed-length surveys in terms of score agreement, respondent burden, discriminant validity, and responsiveness. DESIGN: Longitudinal, prospective cohort study of patients interviewed approximately 2 weeks after discharge from inpatient rehabilitation and 3 months later. SETTING: Follow-up interviews conducted in patient's home setting. PARTICIPANTS: Adults (N=94) with diagnoses of neurologic, orthopedic, or medically complex conditions. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Participation domains of mobility, domestic life, and community, social, & civic life, measured using a CAT version of the Participation Measure for Postacute Care (PM-PAC-CAT) and a 53-item fixed-length survey (PM-PAC-53). RESULTS: The PM-PAC-CAT showed substantial agreement with PM-PAC-53 scores (intraclass correlation coefficient, model 3,1, .71-.81). On average, the PM-PAC-CAT was completed in 42% of the time and with only 48% of the items as compared with the PM-PAC-53. Both formats discriminated across functional severity groups. The PM-PAC-CAT had modest reductions in sensitivity and responsiveness to patient-reported change over a 3-month interval as compared with the PM-PAC-53. CONCLUSIONS: Although continued evaluation is warranted, accurate estimates of participation status and responsiveness to change for group-level analyses can be obtained from CAT administrations, with a sizeable reduction in respondent burden.10a*Activities of Daily Living10a*Adaptation, Physiological10a*Computer Systems10a*Questionnaires10aAdult10aAged10aAged, 80 and over10aChi-Square Distribution10aFactor Analysis, Statistical10aFemale10aHumans10aLongitudinal Studies10aMale10aMiddle Aged10aOutcome Assessment (Health Care)/*methods10aPatient Discharge10aProspective Studies10aRehabilitation/*standards10aSubacute Care/*standards1 aHaley, S M1 aGandek, B1 aSiebens, H1 aBlack-Schaffer, R M1 aSinclair, S J1 aTao, W1 aCoster, W J1 aNi, P1 aJette, A M uhttp://mail.iacat.org/content/computerized-adaptive-testing-follow-after-discharge-inpatient-rehabilitation-ii01923nas a2200217 4500008004100000020004100041245009100082210006900173250001500242260000800257300001100265490000700276520119200283653004001475653002701515653001101542100001401553700001301567700001401580856011101594 2008 eng d a0007-1102 (Print)0007-1102 (Linking)00aControlling item exposure and test overlap on the fly in computerized adaptive testing0 aControlling item exposure and test overlap on the fly in compute a2007/07/26 cNov a471-920 v613 aThis paper proposes an on-line version of the Sympson and Hetter procedure with test overlap control (SHT) that can provide item exposure control at both the item and test levels on the fly without iterative simulations. The on-line procedure is similar to the SHT procedure in that exposure parameters are used for simultaneous control of item exposure rates and test overlap rate. The exposure parameters for the on-line procedure, however, are updated sequentially on the fly, rather than through iterative simulations conducted prior to operational computerized adaptive tests (CATs). Unlike the SHT procedure, the on-line version can control item exposure rate and test overlap rate without time-consuming iterative simulations even when item pools or examinee populations have been changed. Moreover, the on-line procedure was found to perform better than the SHT procedure in controlling item exposure and test overlap for examinees who take tests earlier. Compared with two other on-line alternatives, this proposed on-line method provided the best all-around test security control. Thus, it would be an efficient procedure for controlling item exposure and test overlap in CATs.10a*Decision Making, Computer-Assisted10a*Models, Psychological10aHumans1 aChen, S-Y1 aLei, P W1 aLiao, W H uhttp://mail.iacat.org/content/controlling-item-exposure-and-test-overlap-fly-computerized-adaptive-testing00463nas a2200121 4500008004100000245008400041210006900125300000900194490000700203100001300210700001400223856010400237 2008 eng d00aInvestigating item exposure control on the fly in computerized adaptive testing0 aInvestigating item exposure control on the fly in computerized a a1-320 v551 aWu, M -L1 aChen, S-Y uhttp://mail.iacat.org/content/investigating-item-exposure-control-fly-computerized-adaptive-testing00442nas a2200121 4500008004100000245007200041210006900113300001200182490000700194100000800201700001400209856009700223 2008 eng d00aItem exposure control in a-stratified computerized adaptive testing0 aItem exposure control in astratified computerized adaptive testi a793-8110 v551 aJhu1 aChen, S-Y uhttp://mail.iacat.org/content/item-exposure-control-stratified-computerized-adaptive-testing03233nas a2200397 4500008004100000020002700041245014200068210006900210250001500279260001100294300001200305490000700317520193600324653002702260653003002287653001002317653000902327653002202336653003602358653001602394653002402410653004402434653001102478653001602489653002602505653003002531653003002561653003102591100001302622700001402635700001502649700001402664700001702678700001502695856012502710 2008 eng d a1528-1159 (Electronic)00aLetting the CAT out of the bag: Comparing computer adaptive tests and an 11-item short form of the Roland-Morris Disability Questionnaire0 aLetting the CAT out of the bag Comparing computer adaptive tests a2008/05/23 cMay 20 a1378-830 v333 aSTUDY DESIGN: A post hoc simulation of a computer adaptive administration of the items of a modified version of the Roland-Morris Disability Questionnaire. OBJECTIVE: To evaluate the effectiveness of adaptive administration of back pain-related disability items compared with a fixed 11-item short form. SUMMARY OF BACKGROUND DATA: Short form versions of the Roland-Morris Disability Questionnaire have been developed. An alternative to paper-and-pencil short forms is to administer items adaptively so that items are presented based on a person's responses to previous items. Theoretically, this allows precise estimation of back pain disability with administration of only a few items. MATERIALS AND METHODS: Data were gathered from 2 previously conducted studies of persons with back pain. An item response theory model was used to calibrate scores based on all items, items of a paper-and-pencil short form, and several computer adaptive tests (CATs). RESULTS: Correlations between each CAT condition and scores based on a 23-item version of the Roland-Morris Disability Questionnaire ranged from 0.93 to 0.98. Compared with an 11-item short form, an 11-item CAT produced scores that were significantly more highly correlated with scores based on the 23-item scale. CATs with even fewer items also produced scores that were highly correlated with scores based on all items. For example, scores from a 5-item CAT had a correlation of 0.93 with full scale scores. Seven- and 9-item CATs correlated at 0.95 and 0.97, respectively. A CAT with a standard-error-based stopping rule produced scores that correlated at 0.95 with full scale scores. CONCLUSION: A CAT-based back pain-related disability measure may be a valuable tool for use in clinical and research contexts. Use of CAT for other common measures in back pain research, such as other functional scales or measures of psychological distress, may offer similar advantages.10a*Disability Evaluation10a*Health Status Indicators10aAdult10aAged10aAged, 80 and over10aBack Pain/*diagnosis/psychology10aCalibration10aComputer Simulation10aDiagnosis, Computer-Assisted/*standards10aHumans10aMiddle Aged10aModels, Psychological10aPredictive Value of Tests10aQuestionnaires/*standards10aReproducibility of Results1 aCook, KF1 aChoi, S W1 aCrane, P K1 aDeyo, R A1 aJohnson, K L1 aAmtmann, D uhttp://mail.iacat.org/content/letting-cat-out-bag-comparing-computer-adaptive-tests-and-11-item-short-form-roland-morris02142nas a2200289 4500008004100000020004600041245007200087210006400159250001500223260001100238300000700249490000700256520118400263653002901447653003501476653002601511653002601537653001101563653006101574653001801635653004501653653001301698100001601711700001301727700001801740856009401758 2008 eng d a1553-6467 (Electronic)0002-9459 (Linking)00aThe NAPLEX: evolution, purpose, scope, and educational implications0 aNAPLEX evolution purpose scope and educational implications a2008/05/17 cApr 15 a330 v723 aSince 2004, passing the North American Pharmacist Licensure Examination (NAPLEX) has been a requirement for earning initial pharmacy licensure in all 50 United States. The creation and evolution from 1952-2005 of the particular pharmacy competency testing areas and quantities of questions are described for the former paper-and-pencil National Association of Boards of Pharmacy Licensure Examination (NABPLEX) and the current candidate-specific computer adaptive NAPLEX pharmacy licensure examinations. A 40% increase in the weighting of NAPLEX Blueprint Area 2 in May 2005, compared to that in the preceding 1997-2005 Blueprint, has implications for candidates' NAPLEX performance and associated curricular content and instruction. New pharmacy graduates' scores on the NAPLEX are neither intended nor validated to serve as a criterion for assessing or judging the quality or effectiveness of pharmacy curricula and instruction. The newest cycle of NAPLEX Blueprint revision, a continual process to ensure representation of nationwide contemporary practice, began in early 2008. It may take up to 2 years, including surveying several thousand national pharmacists, to complete.10a*Educational Measurement10aEducation, Pharmacy/*standards10aHistory, 20th Century10aHistory, 21st Century10aHumans10aLicensure, Pharmacy/history/*legislation & jurisprudence10aNorth America10aPharmacists/*legislation & jurisprudence10aSoftware1 aNewton, D W1 aBoyle, M1 aCatizone, C A uhttp://mail.iacat.org/content/naplex-evolution-purpose-scope-and-educational-implications01948nas a2200277 4500008004100000020004100041245007300082210006900155250001500224260000800239300001000247490000700257520099700264653001601261653002901277653004801306653006201354653001101416653002401427653004601451653003101497653001301528100001401541700001501555856010001570 2008 eng d a0007-1102 (Print)0007-1102 (Linking)00aPredicting item exposure parameters in computerized adaptive testing0 aPredicting item exposure parameters in computerized adaptive tes a2008/05/17 cMay a75-910 v613 aThe purpose of this study is to find a formula that describes the relationship between item exposure parameters and item parameters in computerized adaptive tests by using genetic programming (GP) - a biologically inspired artificial intelligence technique. Based on the formula, item exposure parameters for new parallel item pools can be predicted without conducting additional iterative simulations. Results show that an interesting formula between item exposure parameters and item parameters in a pool can be found by using GP. The item exposure parameters predicted based on the found formula were close to those observed from the Sympson and Hetter (1985) procedure and performed well in controlling item exposure rates. Similar results were observed for the Stocking and Lewis (1998) multinomial model for item selection and the Sympson and Hetter procedure with content balancing. The proposed GP approach has provided a knowledge-based solution for finding item exposure parameters.10a*Algorithms10a*Artificial Intelligence10aAptitude Tests/*statistics & numerical data10aDiagnosis, Computer-Assisted/*statistics & numerical data10aHumans10aModels, Statistical10aPsychometrics/statistics & numerical data10aReproducibility of Results10aSoftware1 aChen, S-Y1 aDoong, S H uhttp://mail.iacat.org/content/predicting-item-exposure-parameters-computerized-adaptive-testing01559nas a2200145 4500008003900000245009000039210006900129300001200198490000700210520109400217100001201311700001801323700001901341856005301360 2008 d00aSeverity of Organized Item Theft in Computerized Adaptive Testing: A Simulation Study0 aSeverity of Organized Item Theft in Computerized Adaptive Testin a543-5580 v323 a

Criteria had been proposed for assessing the severity of possible test security violations for computerized tests with high-stakes outcomes. However, these criteria resulted from theoretical derivations that assumed uniformly randomized item selection. This study investigated potential damage caused by organized item theft in computerized adaptive testing (CAT) for two realistic item selection methods, maximum item information and a-stratified with content blocking, using the randomized method as a baseline for comparison. Damage caused by organized item theft was evaluated by the number of compromised items each examinee could encounter and the impact of the compromised items on examinees' ability estimates. Severity of test security violation was assessed under self-organized and organized item theft simulation scenarios. Results indicated that though item theft could cause severe damage to CAT with either item selection method, the maximum item information method was more vulnerable to the organized item theft simulation than was the a-stratified method.

1 aQing Yi1 aJinming Zhang1 aChang, Hua-Hua uhttp://apm.sagepub.com/content/32/7/543.abstract01428nas a2200133 4500008003900000245008900039210006900128300001200197490000700209520094400216100001601160700001201176856010601188 2008 d00aTo Weight Or Not To Weight? Balancing Influence Of Initial Items In Adaptive Testing0 aTo Weight Or Not To Weight Balancing Influence Of Initial Items  a441-4500 v733 a

It has been widely reported that in computerized adaptive testing some examinees may get much lower scores than they would normally if an alternative paper-and-pencil version were given. The main purpose of this investigation is to quantitatively reveal the cause for the underestimation phenomenon. The logistic models, including the 1PL, 2PL, and 3PL models, are used to demonstrate our assertions. Our analytical derivation shows that, under the maximum information item selection strategy, if an examinee failed a few items at the beginning of the test, easy but more discriminating items are likely to be administered. Such items are ineffective to move the estimate close to the true theta, unless the test is sufficiently long or a variable-length test is used. Our results also indicate that a certain weighting mechanism is necessary to make the algorithm rely less on the items administered at the beginning of the test.

1 aChang, H -H1 aYing, Z uhttp://mail.iacat.org/content/weight-or-not-weight-balancing-influence-initial-items-adaptive-testing00527nas a2200109 4500008004100000245007700041210006900118260009700187100001300284700002200297856009800319 2007 eng d00aAdaptive testing with the multi-unidimensional pairwise preference model0 aAdaptive testing with the multiunidimensional pairwise preferenc aD. J. Weiss (Ed.), Proceedings of the 2007 GMAC Conference on Computerized Adaptive Testing.1 aStark, S1 aChernyshenko, O S uhttp://mail.iacat.org/content/adaptive-testing-multi-unidimensional-pairwise-preference-model00495nas a2200109 4500008004100000245011700041210006900158300001200227490000700239100001400246856012500260 2007 eng d00aThe comparison of maximum likelihood estimation and expected a posteriori in CAT using the graded response model0 acomparison of maximum likelihood estimation and expected a poste a339-3710 v191 aChen, S-K uhttp://mail.iacat.org/content/comparison-maximum-likelihood-estimation-and-expected-posteriori-cat-using-graded-response00609nas a2200121 4500008004100000245012500041210006900166260009700235100001200332700001500344700001100359856011700370 2007 eng d00aComputerized attribute-adaptive testing: A new computerized adaptive testing approach incorporating cognitive psychology0 aComputerized attributeadaptive testing A new computerized adapti aD. J. Weiss (Ed.), Proceedings of the 2007 GMAC Conference on Computerized Adaptive Testing.1 aZhou, J1 aGierl, M J1 aCui, Y uhttp://mail.iacat.org/content/computerized-attribute-adaptive-testing-new-computerized-adaptive-testing-approach01549nas a2200169 4500008003900000022001400039245006100053210006100114300001400175490000700189520104600196100001901242700002301261700002201284700001801306856005501324 2007 d a1745-398400aDetecting Differential Speededness in Multistage Testing0 aDetecting Differential Speededness in Multistage Testing a117–1300 v443 a

A potential undesirable effect of multistage testing is differential speededness, which happens if some of the test takers run out of time because they receive subtests with items that are more time intensive than others. This article shows how a probabilistic response-time model can be used for estimating differences in time intensities and speed between subtests and test takers and detecting differential speededness. An empirical data set for a multistage test in the computerized CPA Exam was used to demonstrate the procedures. Although the more difficult subtests appeared to have items that were more time intensive than the easier subtests, an analysis of the residual response times did not reveal any significant differential speededness because the time limit appeared to be appropriate. In a separate analysis, within each of the subtests, we found minor but consistent patterns of residual times that are believed to be due to a warm-up effect, that is, use of more time on the initial items than they actually need.

1 aLinden, Wim, J1 aBreithaupt, Krista1 aChuah, Siang Chee1 aZhang, Yanwei uhttp://dx.doi.org/10.1111/j.1745-3984.2007.00030.x02419nas a2200325 4500008004100000020002200041245008700063210006900150250001500219300001100234490000700245520140100252653001901653653003001672653001901702653003801721653002101759653002001780653001401800653001501814653003301829653001101862653002401873653001801897100001701915700001501932700001501947700001501962856011601977 2007 eng d a0962-9343 (Print)00aDeveloping tailored instruments: item banking and computerized adaptive assessment0 aDeveloping tailored instruments item banking and computerized ad a2007/05/29 a95-1080 v163 aItem banks and Computerized Adaptive Testing (CAT) have the potential to greatly improve the assessment of health outcomes. This review describes the unique features of item banks and CAT and discusses how to develop item banks. In CAT, a computer selects the items from an item bank that are most relevant for and informative about the particular respondent; thus optimizing test relevance and precision. Item response theory (IRT) provides the foundation for selecting the items that are most informative for the particular respondent and for scoring responses on a common metric. The development of an item bank is a multi-stage process that requires a clear definition of the construct to be measured, good items, a careful psychometric analysis of the items, and a clear specification of the final CAT. The psychometric analysis needs to evaluate the assumptions of the IRT model such as unidimensionality and local independence; that the items function the same way in different subgroups of the population; and that there is an adequate fit between the data and the chosen item response models. Also, interpretation guidelines need to be established to help the clinical application of the assessment. Although medical research can draw upon expertise from educational testing in the development of item banks and CAT, the medical field also encounters unique opportunities and challenges.10a*Health Status10a*Health Status Indicators10a*Mental Health10a*Outcome Assessment (Health Care)10a*Quality of Life10a*Questionnaires10a*Software10aAlgorithms10aFactor Analysis, Statistical10aHumans10aModels, Statistical10aPsychometrics1 aBjorner, J B1 aChang, C-H1 aThissen, D1 aReeve, B B uhttp://mail.iacat.org/content/developing-tailored-instruments-item-banking-and-computerized-adaptive-assessment01799nas a2200217 4500008004100000020004600041245017800087210006900265260002500334300001200359490000700371520094900378653001201327653004301339653001801382653000901400100001701409700001501426700001501441856012501456 2007 eng d a1062-7197 (Print); 1532-6977 (Electronic)00aThe effect of including pretest items in an operational computerized adaptive test: Do different ability examinees spend different amounts of time on embedded pretest items?0 aeffect of including pretest items in an operational computerized bLawrence Erlbaum: US a161-1730 v123 aThe purpose of this study was to examine the effect of pretest items on response time in an operational, fixed-length, time-limited computerized adaptive test (CAT). These pretest items are embedded within the CAT, but unlike the operational items, are not tailored to the examinee's ability level. If examinees with higher ability levels need less time to complete these items than do their counterparts with lower ability levels, they will have more time to devote to the operational test questions. Data were from a graduate admissions test that was administered worldwide. Data from both quantitative and verbal sections of the test were considered. For the verbal section, examinees in the lower ability groups spent systematically more time on their pretest items than did those in the higher ability groups, though for the quantitative section the differences were less clear. (PsycINFO Database Record (c) 2007 APA, all rights reserved)10aability10aoperational computerized adaptive test10apretest items10atime1 aFerdous, A A1 aPlake, B S1 aChang, S-R uhttp://mail.iacat.org/content/effect-including-pretest-items-operational-computerized-adaptive-test-do-different-ability02894nas a2200169 4500008004100000020001400041245011300055210006900168300001200237490000700249520229000256100001302546700001602559700001302575700001402588856012202602 2007 eng d a0962-934300aThe future of outcomes measurement: item banking, tailored short-forms, and computerized adaptive assessment0 afuture of outcomes measurement item banking tailored shortforms  a133-1410 v163 aThe use of item banks and computerized adaptive testing (CAT) begins with clear definitions of important outcomes, and references those definitions to specific questions gathered into large and well-studied pools, or “banks” of items. Items can be selected from the bank to form customized short scales, or can be administered in a sequence and length determined by a computer programmed for precision and clinical relevance. Although far from perfect, such item banks can form a common definition and understanding of human symptoms and functional problems such as fatigue, pain, depression, mobility, social function, sensory function, and many other health concepts that we can only measure by asking people directly. The support of the National Institutes of Health (NIH), as witnessed by its cooperative agreement with measurement experts through the NIH Roadmap Initiative known as PROMIS (www.nihpromis.org), is a big step in that direction. Our approach to item banking and CAT is practical; as focused on application as it is on science or theory. From a practical perspective, we frequently must decide whether to re-write and retest an item, add more items to fill gaps (often at the ceiling of the measure), re-test a bank after some modifications, or split up a bank into units that are more unidimensional, yet less clinically relevant or complete. These decisions are not easy, and yet they are rarely unforgiving. We encourage people to build practical tools that are capable of producing multiple short form measures and CAT administrations from common banks, and to further our understanding of these banks with various clinical populations and ages, so that with time the scores that emerge from these many activities begin to have not only a common metric and range, but a shared meaning and understanding across users. In this paper, we provide an overview of item banking and CAT, discuss our approach to item banking and its byproducts, describe testing options, discuss an example of CAT for fatigue, and discuss models for long term sustainability of an entity such as PROMIS. Some barriers to success include limitations in the methods themselves, controversies and disagreements across approaches, and end-user reluctance to move away from the familiar. 1 aCella, D1 aGershon, RC1 aLai, J-S1 aChoi, S W uhttp://mail.iacat.org/content/future-outcomes-measurement-item-banking-tailored-short-forms-and-computerized-adaptive02876nas a2200313 4500008004100000020002200041245010100063210006900164250001500233260000800248300001200256490000700268520179500275653005102070653002002121653003702141653002602178653001902204653001102223653003002234653004602264653003502310653002802345653001302373100002102386700001702407700001502424856012302439 2007 eng d a0315-162X (Print)00aImproving patient reported outcomes using item response theory and computerized adaptive testing0 aImproving patient reported outcomes using item response theory a a2007/06/07 cJun a1426-310 v343 aOBJECTIVE: Patient reported outcomes (PRO) are considered central outcome measures for both clinical trials and observational studies in rheumatology. More sophisticated statistical models, including item response theory (IRT) and computerized adaptive testing (CAT), will enable critical evaluation and reconstruction of currently utilized PRO instruments to improve measurement precision while reducing item burden on the individual patient. METHODS: We developed a domain hierarchy encompassing the latent trait of physical function/disability from the more general to most specific. Items collected from 165 English-language instruments were evaluated by a structured process including trained raters, modified Delphi expert consensus, and then patient evaluation. Each item in the refined data bank will undergo extensive analysis using IRT to evaluate response functions and measurement precision. CAT will allow for real-time questionnaires of potentially smaller numbers of questions tailored directly to each individual's level of physical function. RESULTS: Physical function/disability domain comprises 4 subdomains: upper extremity, trunk, lower extremity, and complex activities. Expert and patient review led to consensus favoring use of present-tense "capability" questions using a 4- or 5-item Likert response construct over past-tense "performance"items. Floor and ceiling effects, attribution of disability, and standardization of response categories were also addressed. CONCLUSION: By applying statistical techniques of IRT through use of CAT, existing PRO instruments may be improved to reduce questionnaire burden on the individual patients while increasing measurement precision that may ultimately lead to reduced sample size requirements for costly clinical trials.10a*Rheumatic Diseases/physiopathology/psychology10aClinical Trials10aData Interpretation, Statistical10aDisability Evaluation10aHealth Surveys10aHumans10aInternational Cooperation10aOutcome Assessment (Health Care)/*methods10aPatient Participation/*methods10aResearch Design/*trends10aSoftware1 aChakravarty, E F1 aBjorner, J B1 aFries, J F uhttp://mail.iacat.org/content/improving-patient-reported-outcomes-using-item-response-theory-and-computerized-adaptive03105nas a2200445 4500008004100000020002200041245007100063210006900134250001500203300001200218490000700230520183100237653003802068653001902106653002102125653002002146653001402166653001102180653003002191653001102221653000902232653002502241653004602266653001802312653002602330100001302356700001402369700001702383700001302400700001502413700001502428700001702443700001402460700001802474700002302492700001602515700001602531700001502547856009702562 2007 eng d a0962-9343 (Print)00aIRT health outcomes data analysis project: an overview and summary0 aIRT health outcomes data analysis project an overview and summar a2007/03/14 a121-1320 v163 aBACKGROUND: In June 2004, the National Cancer Institute and the Drug Information Association co-sponsored the conference, "Improving the Measurement of Health Outcomes through the Applications of Item Response Theory (IRT) Modeling: Exploration of Item Banks and Computer-Adaptive Assessment." A component of the conference was presentation of a psychometric and content analysis of a secondary dataset. OBJECTIVES: A thorough psychometric and content analysis was conducted of two primary domains within a cancer health-related quality of life (HRQOL) dataset. RESEARCH DESIGN: HRQOL scales were evaluated using factor analysis for categorical data, IRT modeling, and differential item functioning analyses. In addition, computerized adaptive administration of HRQOL item banks was simulated, and various IRT models were applied and compared. SUBJECTS: The original data were collected as part of the NCI-funded Quality of Life Evaluation in Oncology (Q-Score) Project. A total of 1,714 patients with cancer or HIV/AIDS were recruited from 5 clinical sites. MEASURES: Items from 4 HRQOL instruments were evaluated: Cancer Rehabilitation Evaluation System-Short Form, European Organization for Research and Treatment of Cancer Quality of Life Questionnaire, Functional Assessment of Cancer Therapy and Medical Outcomes Study Short-Form Health Survey. RESULTS AND CONCLUSIONS: Four lessons learned from the project are discussed: the importance of good developmental item banks, the ambiguity of model fit results, the limits of our knowledge regarding the practical implications of model misfit, and the importance in the measurement of HRQOL of construct definition. With respect to these lessons, areas for future research are suggested. The feasibility of developing item banks for broad definitions of health is discussed.10a*Data Interpretation, Statistical10a*Health Status10a*Quality of Life10a*Questionnaires10a*Software10aFemale10aHIV Infections/psychology10aHumans10aMale10aNeoplasms/psychology10aOutcome Assessment (Health Care)/*methods10aPsychometrics10aStress, Psychological1 aCook, KF1 aTeal, C R1 aBjorner, J B1 aCella, D1 aChang, C-H1 aCrane, P K1 aGibbons, L E1 aHays, R D1 aMcHorney, C A1 aOcepek-Welikson, K1 aRaczek, A E1 aTeresi, J A1 aReeve, B B uhttp://mail.iacat.org/content/irt-health-outcomes-data-analysis-project-overview-and-summary01120nas a2200169 4500008004100000020002300041245008300064210006900147300001400216490000700230520054200237100001500779700001500794700001700809700001500826856010900841 2007 eng d a0962-93431573-264900aMethodological issues for building item banks and computerized adaptive scales0 aMethodological issues for building item banks and computerized a a109-119, 0 v163 aAbstract This paper reviews important methodological considerations for developing item banks and computerized adaptive scales (commonly called computerized adaptive tests in the educational measurement literature, yielding the acronym CAT), including issues of the reference population, dimensionality, dichotomous versus polytomous response scales, differential item functioning (DIF) and conditional scoring, mode effects, the impact of local dependence, and innovative approaches to assessment using CATs in health outcomes research.1 aThissen, D1 aReeve, B B1 aBjorner, J B1 aChang, C-H uhttp://mail.iacat.org/content/methodological-issues-building-item-banks-and-computerized-adaptive-scales00589nas a2200109 4500008004100000245011500041210006900156260010200225100001300327700001900340856012000359 2007 eng d00aThe modified maximum global discrimination index method for cognitive diagnostic computerized adaptive testing0 amodified maximum global discrimination index method for cognitiv a  D. J. Weiss (Ed.).  Proceedings of the 2007 GMAC Conference on Computerized Adaptive Testing.1 aCheng, Y1 aChang, Hua-Hua uhttp://mail.iacat.org/content/modified-maximum-global-discrimination-index-method-cognitive-diagnostic-computerized01800nas a2200265 4500008004100000020004100041245010400082210006900186250001500255300001100270490001500281520085700296653001901153653003801172653002101210653001401231653002901245653005601274653001101330653002501341653001901366653001801385100001501403856011601418 2007 eng d a0962-9343 (Print)0962-9343 (Linking)00aPatient-reported outcomes measurement and management with innovative methodologies and technologies0 aPatientreported outcomes measurement and management with innovat a2007/05/29 a157-660 v16 Suppl 13 aSuccessful integration of modern psychometrics and advanced informatics in patient-reported outcomes (PRO) measurement and management can potentially maximize the value of health outcomes research and optimize the delivery of quality patient care. Unlike the traditional labor-intensive paper-and-pencil data collection method, item response theory-based computerized adaptive testing methodologies coupled with novel technologies provide an integrated environment to collect, analyze and present ready-to-use PRO data for informed and shared decision-making. This article describes the needs, challenges and solutions for accurate, efficient and cost-effective PRO data acquisition and dissemination means in order to provide critical and timely PRO information necessary to actively support and enhance routine patient care in busy clinical settings.10a*Health Status10a*Outcome Assessment (Health Care)10a*Quality of Life10a*Software10aComputer Systems/*trends10aHealth Insurance Portability and Accountability Act10aHumans10aPatient Satisfaction10aQuestionnaires10aUnited States1 aChang, C-H uhttp://mail.iacat.org/content/patient-reported-outcomes-measurement-and-management-innovative-methodologies-and01228nas a2200229 4500008004100000245014700041210006900188300001100257490000700268520046800275100001300743700001300756700001600769700001600785700001300801700001300814700001200827700001500839700001300854700001200867856011900879 2007 eng d00aThe Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years0 aPatientReported Outcomes Measurement Information System PROMIS p aS3-S110 v453 aThe National Institutes of Health (NIH) Patient-Reported Outcomes Measurement Information System (PROMIS) Roadmap initiative (www.nihpromis.org) is a 5-year cooperative group program of research designed to develop, validate, and standardize item banks to measure patient-reported outcomes (PROs) relevant across common medical conditions. In this article, we will summarize the organization and scientific activity of the PROMIS network during its first 2 years.1 aCella, D1 aYount, S1 aRothrock, N1 aGershon, RC1 aCook, KF1 aReeve, B1 aAder, D1 aFries, J F1 aBruce, B1 aRose, M uhttp://mail.iacat.org/content/patient-reported-outcomes-measurement-information-system-promis-progress-nih-roadmap02745nas a2200541 4500008004100000020002200041245017000063210006900233250001500302260000800317300001100325490000700336520116200343653001901505653002501524653002101549653002101570653001501591653001001606653000901616653001601625653002301641653003201664653001101696653001101707653000901718653001601727653004601743653001801789653002901807653001801836100001501854700001401869700001701883700001301900700001501913700001601928700001501944700001701959700001401976700001801990700001102008700001602019700001502035700001302050700001302063856012702076 2007 eng d a0025-7079 (Print)00aPsychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS)0 aPsychometric evaluation and calibration of healthrelated quality a2007/04/20 cMay aS22-310 v453 aBACKGROUND: The construction and evaluation of item banks to measure unidimensional constructs of health-related quality of life (HRQOL) is a fundamental objective of the Patient-Reported Outcomes Measurement Information System (PROMIS) project. OBJECTIVES: Item banks will be used as the foundation for developing short-form instruments and enabling computerized adaptive testing. The PROMIS Steering Committee selected 5 HRQOL domains for initial focus: physical functioning, fatigue, pain, emotional distress, and social role participation. This report provides an overview of the methods used in the PROMIS item analyses and proposed calibration of item banks. ANALYSES: Analyses include evaluation of data quality (eg, logic and range checking, spread of response distribution within an item), descriptive statistics (eg, frequencies, means), item response theory model assumptions (unidimensionality, local independence, monotonicity), model fit, differential item functioning, and item calibration for banking. RECOMMENDATIONS: Summarized are key analytic issues; recommendations are provided for future evaluations of item banks in HRQOL assessment.10a*Health Status10a*Information Systems10a*Quality of Life10a*Self Disclosure10aAdolescent10aAdult10aAged10aCalibration10aDatabases as Topic10aEvaluation Studies as Topic10aFemale10aHumans10aMale10aMiddle Aged10aOutcome Assessment (Health Care)/*methods10aPsychometrics10aQuestionnaires/standards10aUnited States1 aReeve, B B1 aHays, R D1 aBjorner, J B1 aCook, KF1 aCrane, P K1 aTeresi, J A1 aThissen, D1 aRevicki, D A1 aWeiss, DJ1 aHambleton, RK1 aLiu, H1 aGershon, RC1 aReise, S P1 aLai, J S1 aCella, D uhttp://mail.iacat.org/content/psychometric-evaluation-and-calibration-health-related-quality-life-item-banks-plans-patient01723nas a2200169 4500008004100000245016600041210006900207300001000276490000600286520105700292653001501349100001501364700001601379700001701395700001601412856012501428 2007 eng d00aRelative precision, efficiency and construct validity of different starting and stopping rules for a computerized adaptive test: The GAIN Substance Problem Scale0 aRelative precision efficiency and construct validity of differen a48-650 v83 aSubstance abuse treatment programs are being pressed to measure and make clinical decisions more efficiently about an increasing array of problems. This computerized adaptive testing (CAT) simulation examined the relative efficiency, precision and construct validity of different starting and stopping rules used to shorten the Global Appraisal of Individual Needs’ (GAIN) Substance Problem Scale (SPS) and facilitate diagnosis based on it. Data came from 1,048 adolescents and adults referred to substance abuse treatment centers in 5 sites. CAT performance was evaluated using: (1) average standard errors, (2) average number of items, (3) bias in personmeasures, (4) root mean squared error of person measures, (5) Cohen’s kappa to evaluate CAT classification compared to clinical classification, (6) correlation between CAT and full-scale measures, and (7) construct validity of CAT classification vs. clinical classification using correlations with five theoretically associated instruments. Results supported both CAT efficiency and validity.10aMy article1 aRiley, B B1 aConrad, K J1 aBezruczko, N1 aDennis, M L uhttp://mail.iacat.org/content/relative-precision-efficiency-and-construct-validity-different-starting-and-stopping-rules02043nas a2200253 4500008004100000020002200041245007400063210006900137250001500206300001100221490000700232520121000239653002201449653001101471653005501482653005101537100001501588700002001603700002301623700001801646700001301664700001701677856009501694 2007 eng d a0885-3924 (Print)00aA system for interactive assessment and management in palliative care0 asystem for interactive assessment and management in palliative c a2007/03/16 a745-550 v333 aThe availability of psychometrically sound and clinically relevant screening, diagnosis, and outcome evaluation tools is essential to high-quality palliative care assessment and management. Such data will enable us to improve patient evaluations, prognoses, and treatment selections, and to increase patient satisfaction and quality of life. To accomplish these goals, medical care needs more precise, efficient, and comprehensive tools for data acquisition, analysis, interpretation, and management. We describe a system for interactive assessment and management in palliative care (SIAM-PC), which is patient centered, model driven, database derived, evidence based, and technology assisted. The SIAM-PC is designed to reliably measure the multiple dimensions of patients' needs for palliative care, and then to provide information to clinicians, patients, and the patients' families to achieve optimal patient care, while improving our capacity for doing palliative care research. This system is innovative in its application of the state-of-the-science approaches, such as item response theory and computerized adaptive testing, to many of the significant clinical problems related to palliative care.10a*Needs Assessment10aHumans10aMedical Informatics/*organization & administration10aPalliative Care/*organization & administration1 aChang, C-H1 aBoni-Saenz, A A1 aDurazo-Arvizu, R A1 aDesHarnais, S1 aLau, D T1 aEmanuel, L L uhttp://mail.iacat.org/content/system-interactive-assessment-and-management-palliative-care00483nas a2200133 4500008004100000245007700041210006900118300001400187490000600201100001300207700001900220700001000239856010000249 2007 eng d00aTwo-phase item selection procedure for flexible content balancing in CAT0 aTwophase item selection procedure for flexible content balancing a467–4820 v31 aCheng, Y1 aChang, Hua-Hua1 aYi, Q uhttp://mail.iacat.org/content/two-phase-item-selection-procedure-flexible-content-balancing-cat01435nas a2200145 4500008003900000245007700039210006900116300001200185490000700197520098600204100001501190700001901205700001201224856005301236 2007 d00aTwo-Phase Item Selection Procedure for Flexible Content Balancing in CAT0 aTwoPhase Item Selection Procedure for Flexible Content Balancing a467-4820 v313 a

Content balancing is an important issue in the design and implementation of computerized adaptive testing (CAT). Content-balancing techniques that have been applied in fixed content balancing, where the number of items from each content area is fixed, include constrained CAT (CCAT), the modified multinomial model (MMM), modified constrained CAT (MCCAT), and others. In this article, four methods are proposed to address the flexible content-balancing issue with the a-stratification design, named STR_C. The four methods are MMM+, an extension of MMM; MCCAT+, an extension of MCCAT; the TPM method, a two-phase content-balancing method using MMM in both phases; and the TPF method, a two-phase content-balancing method using MMM in the first phase and MCCAT in the second. Simulation results show that all of the methods work well in content balancing, and TPF performs the best in item exposure control and item pool utilization while maintaining measurement precision.

1 aYing Cheng1 aChang, Hua-Hua1 aQing Yi uhttp://apm.sagepub.com/content/31/6/467.abstract00587nas a2200121 4500008004100000245011800041210006900159260006500228100001600293700001500309700001600324856012500340 2007 eng d00aThe use of computerized adaptive testing to assess psychopathology using the Global Appraisal of Individual Needs0 ause of computerized adaptive testing to assess psychopathology u aPortland, OR USAbAmerican Evaluation Association cNovember1 aConrad, K J1 aRiley, B B1 aDennis, M L uhttp://mail.iacat.org/content/use-computerized-adaptive-testing-assess-psychopathology-using-global-appraisal-individual00374nas a2200121 4500008004100000245004100041210004100082300001200123490001000135100001700145700001900162856007100181 2006 eng d00aAssessing CAT Test Security Severity0 aAssessing CAT Test Security Severity a62–630 v30(1)1 aYi, Zhang, J1 aChang, Hua-Hua uhttp://mail.iacat.org/content/assessing-cat-test-security-severity01948nas a2200241 4500008004100000020002200041245011200063210006900175260004100244300001200285490000700297520102300304653003401327653002401361653004701385653002401432653002101456653007001477100001301547700001401560700001001574856012201584 2006 eng d a0022-0655 (Print)00aComparing methods of assessing differential item functioning in a computerized adaptive testing environment0 aComparing methods of assessing differential item functioning in  bBlackwell Publishing: United Kingdom a245-2640 v433 aMantel-Haenszel and SIBTEST, which have known difficulty in detecting non-unidirectional differential item functioning (DIF), have been adapted with some success for computerized adaptive testing (CAT). This study adapts logistic regression (LR) and the item-response-theory-likelihood-ratio test (IRT-LRT), capable of detecting both unidirectional and non-unidirectional DIF, to the CAT environment in which pretest items are assumed to be seeded in CATs but not used for trait estimation. The proposed adaptation methods were evaluated with simulated data under different sample size ratios and impact conditions in terms of Type I error, power, and specificity in identifying the form of DIF. The adapted LR and IRT-LRT procedures are more powerful than the CAT version of SIBTEST for non-unidirectional DIF detection. The good Type I error control provided by IRT-LRT under extremely unequal sample sizes and large impact is encouraging. Implications of these and other findings are discussed. all rights reserved)10acomputerized adaptive testing10aeducational testing10aitem response theory likelihood ratio test10alogistic regression10atrait estimation10aunidirectional & non-unidirectional differential item functioning1 aLei, P-W1 aChen, S-Y1 aYu, L uhttp://mail.iacat.org/content/comparing-methods-assessing-differential-item-functioning-computerized-adaptive-testing01526nas a2200157 4500008003900000022001400039245011200053210006900165300001400234490000700248520101100255100001601266700001901282700001201301856005501313 2006 d a1745-398400aComparing Methods of Assessing Differential Item Functioning in a Computerized Adaptive Testing Environment0 aComparing Methods of Assessing Differential Item Functioning in  a245–2640 v433 a

Mantel-Haenszel and SIBTEST, which have known difficulty in detecting non-unidirectional differential item functioning (DIF), have been adapted with some success for computerized adaptive testing (CAT). This study adapts logistic regression (LR) and the item-response-theory-likelihood-ratio test (IRT-LRT), capable of detecting both unidirectional and non-unidirectional DIF, to the CAT environment in which pretest items are assumed to be seeded in CATs but not used for trait estimation. The proposed adaptation methods were evaluated with simulated data under different sample size ratios and impact conditions in terms of Type I error, power, and specificity in identifying the form of DIF. The adapted LR and IRT-LRT procedures are more powerful than the CAT version of SIBTEST for non-unidirectional DIF detection. The good Type I error control provided by IRT-LRT under extremely unequal sample sizes and large impact is encouraging. Implications of these and other findings are discussed.

1 aLei, Pui-Wa1 aChen, Shu-Ying1 aYu, Lan uhttp://dx.doi.org/10.1111/j.1745-3984.2006.00015.x01471nas a2200205 4500008004100000245002700041210002600068260006000094300001100154490000800165520087300173653005101046653003001097653002001127653001801147653001301165100001501178700001501193856005701208 2006 eng d00aComputer-based testing0 aComputerbased testing aWashington D.C. USAbAmerican Psychological Association a87-1000 vxiv3 a(From the chapter) There has been a proliferation of research designed to explore and exploit opportunities provided by computer-based assessment. This chapter provides an overview of the diverse efforts by researchers in this area. It begins by describing how paper-and-pencil tests can be adapted for administration by computers. Computerization provides the important advantage that items can be selected so they are of appropriate difficulty for each examinee. Some of the psychometric theory needed for computerized adaptive testing is reviewed. Then research on innovative computerized assessments is summarized. These assessments go beyond multiple-choice items by using formats made possible by computerization. Then some hardware and software issues are described, and finally, directions for future work are outlined. (PsycINFO Database Record (c) 2006 APA )10aAdaptive Testing computerized adaptive testing10aComputer Assisted Testing10aExperimentation10aPsychometrics10aTheories1 aDrasgow, F1 aChuah, S C uhttp://mail.iacat.org/content/computer-based-testing03330nas a2200469 4500008004100000020002200041245011600063210006900179250001500248260000800263300001200271490000700283520189400290653003202184653003102216653002202247653002002269653001002289653000902299653002202308653002802330653003302358653001102391653001102402653002502413653000902438653001602447653004602463653002202509653002402531653003002555653002902585100001502614700001502629700001602644700001102660700002402671700001402695700001802709700001002727856012302737 2006 eng d a0003-9993 (Print)00aComputerized adaptive testing for follow-up after discharge from inpatient rehabilitation: I. Activity outcomes0 aComputerized adaptive testing for followup after discharge from  a2006/08/01 cAug a1033-420 v873 aOBJECTIVE: To examine score agreement, precision, validity, efficiency, and responsiveness of a computerized adaptive testing (CAT) version of the Activity Measure for Post-Acute Care (AM-PAC-CAT) in a prospective, 3-month follow-up sample of inpatient rehabilitation patients recently discharged home. DESIGN: Longitudinal, prospective 1-group cohort study of patients followed approximately 2 weeks after hospital discharge and then 3 months after the initial home visit. SETTING: Follow-up visits conducted in patients' home setting. PARTICIPANTS: Ninety-four adults who were recently discharged from inpatient rehabilitation, with diagnoses of neurologic, orthopedic, and medically complex conditions. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Summary scores from AM-PAC-CAT, including 3 activity domains of movement and physical, personal care and instrumental, and applied cognition were compared with scores from a traditional fixed-length version of the AM-PAC with 66 items (AM-PAC-66). RESULTS: AM-PAC-CAT scores were in good agreement (intraclass correlation coefficient model 3,1 range, .77-.86) with scores from the AM-PAC-66. On average, the CAT programs required 43% of the time and 33% of the items compared with the AM-PAC-66. Both formats discriminated across functional severity groups. The standardized response mean (SRM) was greater for the movement and physical fixed form than the CAT; the effect size and SRM of the 2 other AM-PAC domains showed similar sensitivity between CAT and fixed formats. Using patients' own report as an anchor-based measure of change, the CAT and fixed length formats were comparable in responsiveness to patient-reported change over a 3-month interval. CONCLUSIONS: Accurate estimates for functional activity group-level changes can be obtained from CAT administrations, with a considerable reduction in administration time.10a*Activities of Daily Living10a*Adaptation, Physiological10a*Computer Systems10a*Questionnaires10aAdult10aAged10aAged, 80 and over10aChi-Square Distribution10aFactor Analysis, Statistical10aFemale10aHumans10aLongitudinal Studies10aMale10aMiddle Aged10aOutcome Assessment (Health Care)/*methods10aPatient Discharge10aProspective Studies10aRehabilitation/*standards10aSubacute Care/*standards1 aHaley, S M1 aSiebens, H1 aCoster, W J1 aTao, W1 aBlack-Schaffer, R M1 aGandek, B1 aSinclair, S J1 aNi, P uhttp://mail.iacat.org/content/computerized-adaptive-testing-follow-after-discharge-inpatient-rehabilitation-i-activity00536nas a2200121 4500008004100000245011700041210006900158260001800227100001300245700001900258700001400277856012300291 2006 eng d00aConstraints-weighted information method for item selection of severely constrained computerized adaptive testing0 aConstraintsweighted information method for item selection of sev aSan Francisco1 aCheng, Y1 aChang, Hua-Hua1 aWang, X B uhttp://mail.iacat.org/content/constraints-weighted-information-method-item-selection-severely-constrained-computerized02399nas a2200301 4500008004100000020002200041245010800063210006900171260002400240300000900264490000600273520142200279653002201701653003401723653002301757653003201780653001801812653002101830653001801851100001401869700001301883700001401896700001901910700001501929700001701944700001301961856012301974 2006 eng d a1529-7713 (Print)00aExpansion of a physical function item bank and development of an abbreviated form for clinical research0 aExpansion of a physical function item bank and development of an bRichard M Smith: US a1-150 v73 aWe expanded an existing 33-item physical function (PF) item bank with a sufficient number of items to enable computerized adaptive testing (CAT). Ten items were written to expand the bank and the new item pool was administered to 295 people with cancer. For this analysis of the new pool, seven poorly performing items were identified for further examination. This resulted in a bank with items that define an essentially unidimensional PF construct, cover a wide range of that construct, reliably measure the PF of persons with cancer, and distinguish differences in self-reported functional performance levels. We also developed a 5-item (static) assessment form ("BriefPF") that can be used in clinical research to express scores on the same metric as the overall bank. The BriefPF was compared to the PF-10 from the Medical Outcomes Study SF-36. Both short forms significantly differentiated persons across functional performance levels. While the entire bank was more precise across the PF continuum than either short form, there were differences in the area of the continuum in which each short form was more precise: the BriefPF was more precise than the PF-10 at the lower functional levels and the PF-10 was more precise than the BriefPF at the higher levels. Future research on this bank will include the development of a CAT version, the PF-CAT. (PsycINFO Database Record (c) 2007 APA, all rights reserved)10aclinical research10acomputerized adaptive testing10aperformance levels10aphysical function item bank10aPsychometrics10atest reliability10aTest Validity1 aBode, R K1 aLai, J-S1 aDineen, K1 aHeinemann, A W1 aShevrin, D1 aVon Roenn, J1 aCella, D uhttp://mail.iacat.org/content/expansion-physical-function-item-bank-and-development-abbreviated-form-clinical-research02161nas a2200289 4500008004100000245010000041210006900141260000800210300001200218490000700230520127700237653003401514653002101548653000901569653001201578653002201590653001101612653001101623653000901634653001601643653002901659653001901688100001301707700001501720700001301735856012301748 2006 eng d00aFactor analysis techniques for assessing sufficient unidimensionality of cancer related fatigue0 aFactor analysis techniques for assessing sufficient unidimension cSep a1179-900 v153 aBACKGROUND: Fatigue is the most common unrelieved symptom experienced by people with cancer. The purpose of this study was to examine whether cancer-related fatigue (CRF) can be summarized using a single score, that is, whether CRF is sufficiently unidimensional for measurement approaches that require or assume unidimensionality. We evaluated this question using factor analysis techniques including the theory-driven bi-factor model. METHODS: Five hundred and fifty five cancer patients from the Chicago metropolitan area completed a 72-item fatigue item bank, covering a range of fatigue-related concerns including intensity, frequency and interference with physical, mental, and social activities. Dimensionality was assessed using exploratory and confirmatory factor analysis (CFA) techniques. RESULTS: Exploratory factor analysis (EFA) techniques identified from 1 to 17 factors. The bi-factor model suggested that CRF was sufficiently unidimensional. CONCLUSIONS: CRF can be considered sufficiently unidimensional for applications that require unidimensionality. One such application, item response theory (IRT), will facilitate the development of short-form and computer-adaptive testing. This may further enable practical and accurate clinical assessment of CRF.10a*Factor Analysis, Statistical10a*Quality of Life10aAged10aChicago10aFatigue/*etiology10aFemale10aHumans10aMale10aMiddle Aged10aNeoplasms/*complications10aQuestionnaires1 aLai, J-S1 aCrane, P K1 aCella, D uhttp://mail.iacat.org/content/factor-analysis-techniques-assessing-sufficient-unidimensionality-cancer-related-fatigue00471nas a2200133 4500008003900000245008700039210006900126300001200195490000700207100002200214700001500236700002000251856006600271 2006 d00aHow Big Is Big Enough? Sample Size Requirements for CAST Item Parameter Estimation0 aHow Big Is Big Enough Sample Size Requirements for CAST Item Par a241-2550 v191 aChuah, Siang Chee1 aDrasgow, F1 aLuecht, Richard uhttp://www.tandfonline.com/doi/abs/10.1207/s15324818ame1903_502187nas a2200181 4500008004100000245009900041210006900140260000800209300001400217490000700231520157400238100001401812700001301826700001401839700001601853700001301869856012301882 2006 eng d00aItem banks and their potential applications to health status assessment in diverse populations0 aItem banks and their potential applications to health status ass cNov aS189-S1970 v443 aIn the context of an ethnically diverse, aging society, attention is increasingly turning to health-related quality of life measurement to evaluate healthcare and treatment options for chronic diseases. When evaluating and treating symptoms and concerns such as fatigue, pain, or physical function, reliable and accurate assessment is a priority. Modern psychometric methods have enabled us to move from long, static tests that provide inefficient and often inaccurate assessment of individual patients, to computerized adaptive tests (CATs) that can precisely measure individuals on health domains of interest. These modern methods, collectively referred to as item response theory (IRT), can produce calibrated "item banks" from larger pools of questions. From these banks, CATs can be conducted on individuals to produce their scores on selected domains. Item banks allow for comparison of patients across different question sets because the patient's score is expressed on a common scale. Other advantages of using item banks include flexibility in terms of the degree of precision desired; interval measurement properties under most circumstances; realistic capability for accurate individual assessment over time (using CAT); and measurement equivalence across different patient populations. This work summarizes the process used in the creation and evaluation of item banks and reviews their potential contributions and limitations regarding outcome assessment and patient care, particularly when they are applied across people of different cultural backgrounds.1 aHahn, E A1 aCella, D1 aBode, R K1 aGershon, RC1 aLai, J S uhttp://mail.iacat.org/content/item-banks-and-their-potential-applications-health-status-assessment-diverse-populations00404nas a2200121 4500008003900000245005500039210005300094300001200147490000700159100001900166700003100185856006600216 2006 d00aMultistage Testing: Widely or Narrowly Applicable?0 aMultistage Testing Widely or Narrowly Applicable a257-2600 v191 aStark, Stephen1 aChernyshenko, Oleksandr, S uhttp://www.tandfonline.com/doi/abs/10.1207/s15324818ame1903_602073nas a2200217 4500008004500000245013400045210006900179300001200248490000700260520127300267653003401540653004201574653002501616653001901641100001401660700001301674700001801687700001401705700001501719856012101734 2006 Engldsh 00aSimulated computerized adaptive test for patients with shoulder impairments was efficient and produced valid measures of function0 aSimulated computerized adaptive test for patients with shoulder  a290-2980 v593 a

Background and Objective: To test unidimensionality and local independence of a set of shoulder functional status (SFS) items,
develop a computerized adaptive test (CAT) of the items using a rating scale item response theory model (RSM), and compare discriminant validity of measures generated using all items (qIRT) and measures generated using the simulated CAT (qCAT).
Study Design and Setting: We performed a secondary analysis of data collected prospectively during rehabilitation of 400 patients
with shoulder impairments who completed 60 SFS items.
Results: Factor analytic techniques supported that the 42 SFS items formed a unidimensional scale and were locally independent. Except for five items, which were deleted, the RSM fit the data well. The remaining 37 SFS items were used to generate the CAT. On average, 6 items on were needed to estimate precise measures of function using the SFS CAT, compared with all 37 SFS items. The qIRT and qCAT measures were highly correlated (r 5 .96) and resulted in similar classifications of patients.
Conclusion: The simulated SFS CAT was efficient and produced precise, clinically relevant measures of functional status with good
discriminating ability. 

10acomputerized adaptive testing10aFlexilevel Scale of Shoulder Function10aItem Response Theory10aRehabilitation1 aHart, D L1 aCook, KF1 aMioduski, J E1 aTeal, C R1 aCrane, P K uhttp://mail.iacat.org/content/simulated-computerized-adaptive-test-patients-shoulder-impairments-was-efficient-and-002654nas a2200409 4500008004100000245013400041210006900175300001000244490000700254520123100261653002501492653003201517653003101549653001001580653000901590653002201599653003301621653001101654653001101665653000901676653001601685653002401701653003101725653004101756653004501797653006801842653006101910653003001971653002802001653002202029100001402051700001302065700001802078700001402096700001502110856011902125 2006 eng d00aSimulated computerized adaptive test for patients with shoulder impairments was efficient and produced valid measures of function0 aSimulated computerized adaptive test for patients with shoulder  a290-80 v593 aBACKGROUND AND OBJECTIVE: To test unidimensionality and local independence of a set of shoulder functional status (SFS) items, develop a computerized adaptive test (CAT) of the items using a rating scale item response theory model (RSM), and compare discriminant validity of measures generated using all items (theta(IRT)) and measures generated using the simulated CAT (theta(CAT)). STUDY DESIGN AND SETTING: We performed a secondary analysis of data collected prospectively during rehabilitation of 400 patients with shoulder impairments who completed 60 SFS items. RESULTS: Factor analytic techniques supported that the 42 SFS items formed a unidimensional scale and were locally independent. Except for five items, which were deleted, the RSM fit the data well. The remaining 37 SFS items were used to generate the CAT. On average, 6 items were needed to estimate precise measures of function using the SFS CAT, compared with all 37 SFS items. The theta(IRT) and theta(CAT) measures were highly correlated (r = .96) and resulted in similar classifications of patients. CONCLUSION: The simulated SFS CAT was efficient and produced precise, clinically relevant measures of functional status with good discriminating ability.10a*Computer Simulation10a*Range of Motion, Articular10aActivities of Daily Living10aAdult10aAged10aAged, 80 and over10aFactor Analysis, Statistical10aFemale10aHumans10aMale10aMiddle Aged10aProspective Studies10aReproducibility of Results10aResearch Support, N.I.H., Extramural10aResearch Support, U.S. Gov't, Non-P.H.S.10aShoulder Dislocation/*physiopathology/psychology/rehabilitation10aShoulder Pain/*physiopathology/psychology/rehabilitation10aShoulder/*physiopathology10aSickness Impact Profile10aTreatment Outcome1 aHart, D L1 aCook, KF1 aMioduski, J E1 aTeal, C R1 aCrane, P K uhttp://mail.iacat.org/content/simulated-computerized-adaptive-test-patients-shoulder-impairments-was-efficient-and03124nas a2200385 4500008004100000020002200041245012900063210006900192250001500261260000800276300001000284490000700294520188400301653002502185653002702210653001502237653001002252653002102262653002802283653003802311653001102349653001102360653001102371653000902382653004602391653002702437653003002464653003202494100001502526700001602541700001602557700001502573700002502588856012502613 2005 eng d a0003-9993 (Print)00aAssessing mobility in children using a computer adaptive testing version of the pediatric evaluation of disability inventory0 aAssessing mobility in children using a computer adaptive testing a2005/05/17 cMay a932-90 v863 aOBJECTIVE: To assess score agreement, validity, precision, and response burden of a prototype computerized adaptive testing (CAT) version of the Mobility Functional Skills Scale (Mob-CAT) of the Pediatric Evaluation of Disability Inventory (PEDI) as compared with the full 59-item version (Mob-59). DESIGN: Computer simulation analysis of cross-sectional and longitudinal retrospective data; and cross-sectional prospective study. SETTING: Pediatric rehabilitation hospital, including inpatient acute rehabilitation, day school program, outpatient clinics, community-based day care, preschool, and children's homes. PARTICIPANTS: Four hundred sixty-nine children with disabilities and 412 children with no disabilities (analytic sample); 41 children without disabilities and 39 with disabilities (cross-validation sample). INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Summary scores from a prototype Mob-CAT application and versions using 15-, 10-, and 5-item stopping rules; scores from the Mob-59; and number of items and time (in seconds) to administer assessments. RESULTS: Mob-CAT scores from both computer simulations (intraclass correlation coefficient [ICC] range, .94-.99) and field administrations (ICC=.98) were in high agreement with scores from the Mob-59. Using computer simulations of retrospective data, discriminant validity, and sensitivity to change of the Mob-CAT closely approximated that of the Mob-59, especially when using the 15- and 10-item stopping rule versions of the Mob-CAT. The Mob-CAT used no more than 15% of the items for any single administration, and required 20% of the time needed to administer the Mob-59. CONCLUSIONS: Comparable score estimates for the PEDI mobility scale can be obtained from CAT administrations, with losses in validity and precision for shorter forms, but with a considerable reduction in administration time.10a*Computer Simulation10a*Disability Evaluation10aAdolescent10aChild10aChild, Preschool10aCross-Sectional Studies10aDisabled Children/*rehabilitation10aFemale10aHumans10aInfant10aMale10aOutcome Assessment (Health Care)/*methods10aRehabilitation Centers10aRehabilitation/*standards10aSensitivity and Specificity1 aHaley, S M1 aRaczek, A E1 aCoster, W J1 aDumas, H M1 aFragala-Pinkham, M A uhttp://mail.iacat.org/content/assessing-mobility-children-using-computer-adaptive-testing-version-pediatric-evaluation-000640nas a2200169 4500008004100000020001300041245012900054210006900183300001200252490000700264100001300271700001400284700001400298700001300312700002200325856012300347 2005 eng d a0003999300aAssessing Mobility in Children Using a Computer Adaptive Testing Version of the Pediatric Evaluation of Disability Inventory0 aAssessing Mobility in Children Using a Computer Adaptive Testing a932-9390 v861 aHaley, S1 aRaczek, A1 aCoster, W1 aDumas, H1 aFragalapinkham, M uhttp://mail.iacat.org/content/assessing-mobility-children-using-computer-adaptive-testing-version-pediatric-evaluation01374nas a2200193 4500008004100000245005000041210004700091300001000138490000600148520081200154653001700966653002100983653002601004653002501030100001501055700001401070700002301084856007301107 2005 eng d00aAn Authoring Environment for Adaptive Testing0 aAuthoring Environment for Adaptive Testing a66-760 v83 a

SIETTE is a web-based adaptive testing system. It implements Computerized Adaptive Tests. These tests are tailor-made, theory-based tests, where questions shown to students, finalization of the test, and student knowledge estimation is accomplished adaptively. To construct these tests, SIETTE has an authoring environment comprising a suite of tools that helps teachers create questions and tests properly, and analyze students’ performance after taking a test. In this paper, we present this authoring environment in the
framework of adaptive testing. As will be shown, this set of visual tools, that contain some adaptable eatures, can be useful for teachers lacking skills in this kind of testing. Additionally, other systems that implement adaptive testing will be studied. 

10aAdaptability10aAdaptive Testing10aAuthoring environment10aItem Response Theory1 aGuzmán, E1 aConejo, R1 aGarcía-Hervás, E uhttp://mail.iacat.org/content/authoring-environment-adaptive-testing00535nas a2200133 4500008003900000245010500039210006900144300001200213490000700225100001700232700001600249700001500265856012100280 2005 d00aA closer look at using judgments of item difficulty to change answers on computerized adaptive tests0 acloser look at using judgments of item difficulty to change answ a331-3500 v421 aVispoel, W P1 aClough, S J1 aBleiler, T uhttp://mail.iacat.org/content/closer-look-using-judgments-item-difficulty-change-answers-computerized-adaptive-tests02792nas a2200469 4500008004100000020002200041245010400063210006900167250001500236260000800251300001200259490000700271520132800278653002201606653003101628653001501659653001601674653001001690653003401700653002101734653002401755653002501779653001501804653001101819653005301830653002901883653001101912653001101923653002001934653000901954653003101963653004601994653003102040653001402071653003202085100001502117700001002132700002502142700001702167700001302184856012502197 2005 eng d a0012-1622 (Print)00aA computer adaptive testing approach for assessing physical functioning in children and adolescents0 acomputer adaptive testing approach for assessing physical functi a2005/02/15 cFeb a113-1200 v473 aThe purpose of this article is to demonstrate: (1) the accuracy and (2) the reduction in amount of time and effort in assessing physical functioning (self-care and mobility domains) of children and adolescents using computer-adaptive testing (CAT). A CAT algorithm selects questions directly tailored to the child's ability level, based on previous responses. Using a CAT algorithm, a simulation study was used to determine the number of items necessary to approximate the score of a full-length assessment. We built simulated CAT (5-, 10-, 15-, and 20-item versions) for self-care and mobility domains and tested their accuracy in a normative sample (n=373; 190 males, 183 females; mean age 6y 11mo [SD 4y 2m], range 4mo to 14y 11mo) and a sample of children and adolescents with Pompe disease (n=26; 21 males, 5 females; mean age 6y 1mo [SD 3y 10mo], range 5mo to 14y 10mo). Results indicated that comparable score estimates (based on computer simulations) to the full-length tests can be achieved in a 20-item CAT version for all age ranges and for normative and clinical samples. No more than 13 to 16% of the items in the full-length tests were needed for any one administration. These results support further consideration of using CAT programs for accurate and efficient clinical assessments of physical functioning.10a*Computer Systems10aActivities of Daily Living10aAdolescent10aAge Factors10aChild10aChild Development/*physiology10aChild, Preschool10aComputer Simulation10aConfidence Intervals10aDemography10aFemale10aGlycogen Storage Disease Type II/physiopathology10aHealth Status Indicators10aHumans10aInfant10aInfant, Newborn10aMale10aMotor Activity/*physiology10aOutcome Assessment (Health Care)/*methods10aReproducibility of Results10aSelf Care10aSensitivity and Specificity1 aHaley, S M1 aNi, P1 aFragala-Pinkham, M A1 aSkrinar, A M1 aCorzo, D uhttp://mail.iacat.org/content/computer-adaptive-testing-approach-assessing-physical-functioning-children-and-adolescents02220nas a2200241 4500008004100000020004100041245009600082210006900178250001500247260000800262300001100270490000700281520139600288653002701684653003701711653002701748653001101775653002601786100001501812700001901827700001301846856011901859 2005 eng d a0007-1102 (Print)0007-1102 (Linking)00aComputerized adaptive testing: a mixture item selection approach for constrained situations0 aComputerized adaptive testing a mixture item selection approach  a2005/11/19 cNov a239-570 v583 aIn computerized adaptive testing (CAT), traditionally the most discriminating items are selected to provide the maximum information so as to attain the highest efficiency in trait (theta) estimation. The maximum information (MI) approach typically results in unbalanced item exposure and hence high item-overlap rates across examinees. Recently, Yi and Chang (2003) proposed the multiple stratification (MS) method to remedy the shortcomings of MI. In MS, items are first sorted according to content, then difficulty and finally discrimination parameters. As discriminating items are used strategically, MS offers a better utilization of the entire item pool. However, for testing with imposed non-statistical constraints, this new stratification approach may not maintain its high efficiency. Through a series of simulation studies, this research explored the possible benefits of a mixture item selection approach (MS-MI), integrating the MS and MI approaches, in testing with non-statistical constraints. In all simulation conditions, MS consistently outperformed the other two competing approaches in item pool utilization, while the MS-MI and the MI approaches yielded higher measurement efficiency and offered better conformity to the constraints. Furthermore, the MS-MI approach was shown to perform better than MI on all evaluation criteria when control of item exposure was imposed.10a*Computer-Aided Design10a*Educational Measurement/methods10a*Models, Psychological10aHumans10aPsychometrics/methods1 aLeung, C K1 aChang, Hua-Hua1 aHau, K T uhttp://mail.iacat.org/content/computerized-adaptive-testing-mixture-item-selection-approach-constrained-situations01320nas a2200133 4500008003900000245008000039210006900119300001200188490000700200520089100207100001901098700001601117856005301133 2005 d00aControlling Item Exposure and Test Overlap in Computerized Adaptive Testing0 aControlling Item Exposure and Test Overlap in Computerized Adapt a204-2170 v293 a

This article proposes an item exposure control method, which is the extension of the Sympson and Hetter procedure and can provide item exposure control at both the item and test levels. Item exposure rate and test overlap rate are two indices commonly used to track item exposure in computerized adaptive tests. By considering both indices, item exposure can be monitored at both the item and test levels. To control the item exposure rate and test overlap rate simultaneously, the modified procedure attempted to control not only the maximum value but also the variance of item exposure rates. Results indicated that the item exposure rate and test overlap rate could be controlled simultaneously by implementing the modified procedure. Item exposure control was improved and precision of trait estimation decreased when a prespecified maximum test overlap rate was stringent.

1 aChen, Shu-Ying1 aLei, Pui-Wa uhttp://apm.sagepub.com/content/29/3/204.abstract00472nas a2200121 4500008004100000245008000041210006900121300001400190490001000204100001400214700001300228856010900241 2005 eng d00aControlling item exposure and test overlap in computerized adaptive testing0 aControlling item exposure and test overlap in computerized adapt a204–2170 v29(2)1 aChen, S Y1 aLei, P W uhttp://mail.iacat.org/content/controlling-item-exposure-and-test-overlap-computerized-adaptive-testing-001560nas a2200169 4500008004100000245008000041210006900121300001200190490000700202520094200209653002101151653003001172653005401202100001401256700001301270856010701283 2005 eng d00aControlling item exposure and test overlap in computerized adaptive testing0 aControlling item exposure and test overlap in computerized adapt a204-2170 v293 aThis article proposes an item exposure control method, which is the extension of the Sympson and Hetter procedure and can provide item exposure control at both the item and test levels. Item exposure rate and test overlap rate are two indices commonly used to track item exposure in computerized adaptive tests. By considering both indices, item exposure can be monitored at both the item and test levels. To control the item exposure rate and test overlap rate simultaneously, the modified procedure attempted to control not only the maximum value but also the variance of item exposure rates. Results indicated that the item exposure rate and test overlap rate could be controlled simultaneously by implementing the modified procedure. Item exposure control was improved and precision of trait estimation decreased when a prespecified maximum test overlap rate was stringent. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10aAdaptive Testing10aComputer Assisted Testing10aItem Content (Test) computerized adaptive testing1 aChen, S-Y1 aLei, P-W uhttp://mail.iacat.org/content/controlling-item-exposure-and-test-overlap-computerized-adaptive-testing02215nas a2200373 4500008004100000245011500041210006900156300001100225490000700236520104400243653002101287653002001308653001001328653000901338653002801347653001101375653003801386653000901424653001601433653004101449653001801490653003701508653003001545100001401575700001301589700001301602700001501615700001701630700001501647700001901662700001601681700001801697856012601715 2005 eng d00aData pooling and analysis to build a preliminary item bank: an example using bowel function in prostate cancer0 aData pooling and analysis to build a preliminary item bank an ex a142-590 v283 aAssessing bowel function (BF) in prostate cancer can help determine therapeutic trade-offs. We determined the components of BF commonly assessed in prostate cancer studies as an initial step in creating an item bank for clinical and research application. We analyzed six archived data sets representing 4,246 men with prostate cancer. Thirty-one items from validated instruments were available for analysis. Items were classified into domains (diarrhea, rectal urgency, pain, bleeding, bother/distress, and other) then subjected to conventional psychometric and item response theory (IRT) analyses. Items fit the IRT model if the ratio between observed and expected item variance was between 0.60 and 1.40. Four of 31 items had inadequate fit in at least one analysis. Poorly fitting items included bleeding (2), rectal urgency (1), and bother/distress (1). A fifth item assessing hemorrhoids was poorly correlated with other items. Our analyses supported four related components of BF: diarrhea, rectal urgency, pain, and bother/distress.10a*Quality of Life10a*Questionnaires10aAdult10aAged10aData Collection/methods10aHumans10aIntestine, Large/*physiopathology10aMale10aMiddle Aged10aProstatic Neoplasms/*physiopathology10aPsychometrics10aResearch Support, Non-U.S. Gov't10aStatistics, Nonparametric1 aEton, D T1 aLai, J S1 aCella, D1 aReeve, B B1 aTalcott, J A1 aClark, J A1 aMcPherson, C P1 aLitwin, M S1 aMoinpour, C M uhttp://mail.iacat.org/content/data-pooling-and-analysis-build-preliminary-item-bank-example-using-bowel-function-prostate02001nas a2200205 4500008004100000020004600041245007900087210006900166260004100235300001400276490000700290520127200297653003001569653002501599653003401624100001301658700001801671700001601689856009001705 2005 eng d a0017-9124 (Print); 1475-6773 (Electronic)00aDynamic assessment of health outcomes: Time to let the CAT out of the bag?0 aDynamic assessment of health outcomes Time to let the CAT out of bBlackwell Publishing: United Kingdom a1694-17110 v403 aBackground: The use of item response theory (IRT) to measure self-reported outcomes has burgeoned in recent years. Perhaps the most important application of IRT is computer-adaptive testing (CAT), a measurement approach in which the selection of items is tailored for each respondent. Objective. To provide an introduction to the use of CAT in the measurement of health outcomes, describe several IRT models that can be used as the basis of CAT, and discuss practical issues associated with the use of adaptive scaling in research settings. Principal Points: The development of a CAT requires several steps that are not required in the development of a traditional measure including identification of "starting" and "stopping" rules. CAT's most attractive advantage is its efficiency. Greater measurement precision can be achieved with fewer items. Disadvantages of CAT include the high cost and level of technical expertise required to develop a CAT. Conclusions: Researchers, clinicians, and patients benefit from the availability of psychometrically rigorous measures that are not burdensome. CAT outcome measures hold substantial promise in this regard, but their development is not without challenges. (PsycINFO Database Record (c) 2007 APA, all rights reserved)10acomputer adaptive testing10aItem Response Theory10aself reported health outcomes1 aCook, KF1 aO'Malley, K J1 aRoddey, T S uhttp://mail.iacat.org/content/dynamic-assessment-health-outcomes-time-let-cat-out-bag00464nas a2200109 4500008004100000245008400041210006900125260003000194100001300224700001300237856010400250 2005 eng d00aThe effectiveness of using multiple item pools in computerized adaptive testing0 aeffectiveness of using multiple item pools in computerized adapt aMontreal, Canadac04/20051 aZhang, J1 aChang, H uhttp://mail.iacat.org/content/effectiveness-using-multiple-item-pools-computerized-adaptive-testing00453nas a2200121 4500008004100000245006700041210006700108260002100175100001000196700001300206700001900219856009300238 2005 eng d00aIdentifying practical indices for enhancing item pool security0 aIdentifying practical indices for enhancing item pool security aMontreal, Canada1 aYi, Q1 aZhang, J1 aChang, Hua-Hua uhttp://mail.iacat.org/content/identifying-practical-indices-enhancing-item-pool-security00554nas a2200109 4500008004100000245010300041210006900144260006800213100002300281700001900304856012100323 2005 eng d00aImplementing content constraints in alpha-stratified adaptive testing using a shadow test approach0 aImplementing content constraints in alphastratified adaptive tes aLaw School Admission Council, Computerized Testing Report 01-091 avan der Linden, WJ1 aChang, Hua-Hua uhttp://mail.iacat.org/content/implementing-content-constraints-alpha-stratified-adaptive-testing-using-shadow-test-002444nas a2200373 4500008004100000020004100041245008200082210006900164250001500233260000800248300001000256490000700266520136700273653001001640653000901650653002201659653003301681653003301714653001101747653001101758653000901769653001601778653004001794653001801834653001901852100001301871700001301884700001401897700001201911700001701923700001601940700001501956856009901971 2005 eng d a0895-4356 (Print)0895-4356 (Linking)00aAn item bank was created to improve the measurement of cancer-related fatigue0 aitem bank was created to improve the measurement of cancerrelate a2005/02/01 cFeb a190-70 v583 aOBJECTIVE: Cancer-related fatigue (CRF) is one of the most common unrelieved symptoms experienced by patients. CRF is underrecognized and undertreated due to a lack of clinically sensitive instruments that integrate easily into clinics. Modern computerized adaptive testing (CAT) can overcome these obstacles by enabling precise assessment of fatigue without requiring the administration of a large number of questions. A working item bank is essential for development of a CAT platform. The present report describes the building of an operational item bank for use in clinical settings with the ultimate goal of improving CRF identification and treatment. STUDY DESIGN AND SETTING: The sample included 301 cancer patients. Psychometric properties of items were examined by using Rasch analysis, an Item Response Theory (IRT) model. RESULTS AND CONCLUSION: The final bank includes 72 items. These 72 unidimensional items explained 57.5% of the variance, based on factor analysis results. Excellent internal consistency (alpha=0.99) and acceptable item-total correlation were found (range: 0.51-0.85). The 72 items covered a reasonable range of the fatigue continuum. No significant ceiling effects, floor effects, or gaps were found. A sample short form was created for demonstration purposes. The resulting bank is amenable to the development of a CAT platform.10aAdult10aAged10aAged, 80 and over10aFactor Analysis, Statistical10aFatigue/*etiology/psychology10aFemale10aHumans10aMale10aMiddle Aged10aNeoplasms/*complications/psychology10aPsychometrics10aQuestionnaires1 aLai, J-S1 aCella, D1 aDineen, K1 aBode, R1 aVon Roenn, J1 aGershon, RC1 aShevrin, D uhttp://mail.iacat.org/content/item-bank-was-created-improve-measurement-cancer-related-fatigue01744nas a2200229 4500008004100000245008300041210006900124300001100193490000700204520103000211653003401241100001301275700001401288700001501302700001701317700001501334700001501349700001401364700001301378700001301391856011001404 2005 eng d00aAn item response theory-based pain item bank can enhance measurement precision0 aitem response theorybased pain item bank can enhance measurement a278-880 v303 aCancer-related pain is often under-recognized and undertreated. This is partly due to the lack of appropriate assessments, which need to be comprehensive and precise yet easily integrated into clinics. Computerized adaptive testing (CAT) can enable precise-yet-brief assessments by only selecting the most informative items from a calibrated item bank. The purpose of this study was to create such a bank. The sample included 400 cancer patients who were asked to complete 61 pain-related items. Data were analyzed using factor analysis and the Rasch model. The final bank consisted of 43 items which satisfied the measurement requirement of factor analysis and the Rasch model, demonstrated high internal consistency and reasonable item-total correlations, and discriminated patients with differing degrees of pain. We conclude that this bank demonstrates good psychometric properties, is sensitive to pain reported by patients, and can be used as the foundation for a CAT pain-testing platform for use in clinical practice.10acomputerized adaptive testing1 aLai, J-S1 aDineen, K1 aReeve, B B1 aVon Roenn, J1 aShervin, D1 aMcGuire, M1 aBode, R K1 aPaice, J1 aCella, D uhttp://mail.iacat.org/content/item-response-theory-based-pain-item-bank-can-enhance-measurement-precision02903nas a2200409 4500008004100000245012300041210006900164260000800233300001000241490000700251520159700258653004701855653001001902653000901912653001901921653003101940653002601971653001101997653002902008653001102037653000902048653001602057653003902073653001402112653002502126653002702151653003002178653003202208653002802240653002202268100001502290700001602305700001702321700001602338700001502354856012402369 2005 eng d00aMeasuring physical function in patients with complex medical and postsurgical conditions: a computer adaptive approach0 aMeasuring physical function in patients with complex medical and cOct a741-80 v843 aOBJECTIVE: To examine whether the range of disability in the medically complex and postsurgical populations receiving rehabilitation is adequately sampled by the new Activity Measure--Post-Acute Care (AM-PAC), and to assess whether computer adaptive testing (CAT) can derive valid patient scores using fewer questions. DESIGN: Observational study of 158 subjects (mean age 67.2 yrs) receiving skilled rehabilitation services in inpatient (acute rehabilitation hospitals, skilled nursing facility units) and community (home health services, outpatient departments) settings for recent-onset or worsening disability from medical (excluding neurological) and surgical (excluding orthopedic) conditions. Measures were interviewer-administered activity questions (all patients) and physical functioning portion of the SF-36 (outpatients) and standardized chart items (11 Functional Independence Measure (FIM), 19 Standardized Outcome and Assessment Information Set (OASIS) items, and 22 Minimum Data Set (MDS) items). Rasch modeling analyzed all data and the relationship between person ability estimates and average item difficulty. CAT assessed the ability to derive accurate patient scores using a sample of questions. RESULTS: The 163-item activity item pool covered the range of physical movement and personal and instrumental activities. CAT analysis showed comparable scores between estimates using 10 items or the total item pool. CONCLUSION: The AM-PAC can assess a broad range of function in patients with complex medical illness. CAT achieves valid patient scores using fewer questions.10aActivities of Daily Living/*classification10aAdult10aAged10aCohort Studies10aContinuity of Patient Care10aDisability Evaluation10aFemale10aHealth Services Research10aHumans10aMale10aMiddle Aged10aPostoperative Care/*rehabilitation10aPrognosis10aRecovery of Function10aRehabilitation Centers10aRehabilitation/*standards10aSensitivity and Specificity10aSickness Impact Profile10aTreatment Outcome1 aSiebens, H1 aAndres, P L1 aPengsheng, N1 aCoster, W J1 aHaley, S M uhttp://mail.iacat.org/content/measuring-physical-function-patients-complex-medical-and-postsurgical-conditions-computer01915nas a2200157 4500008004100000245010500041210006900146300001000215490000700225520132900232653003401561100001501595700001301610700001301623856012101636 2005 eng d00aThe promise of PROMIS: using item response theory to improve assessment of patient-reported outcomes0 apromise of PROMIS using item response theory to improve assessme aS53-70 v233 aPROMIS (Patient-Reported-Outcomes Measurement Information System) is an NIH Roadmap network project intended to improve the reliability, validity, and precision of PROs and to provide definitive new instruments that will exceed the capabilities of classic instruments and enable improved outcome measurement for clinical research across all NIH institutes. Item response theory (IRT) measurement models now permit us to transition conventional health status assessment into an era of item banking and computerized adaptive testing (CAT). Item banking uses IRT measurement models and methods to develop item banks from large pools of items from many available questionnaires. IRT allows the reduction and improvement of items and assembles domains of items which are unidimensional and not excessively redundant. CAT provides a model-driven algorithm and software to iteratively select the most informative remaining item in a domain until a desired degree of precision is obtained. Through these approaches the number of patients required for a clinical trial may be reduced while holding statistical power constant. PROMIS tools, expected to improve precision and enable assessment at the individual patient level which should broaden the appeal of PROs, will begin to be available to the general medical community in 2008.10acomputerized adaptive testing1 aFries, J F1 aBruce, B1 aCella, D uhttp://mail.iacat.org/content/promise-promis-using-item-response-theory-improve-assessment-patient-reported-outcomes00343nas a2200109 4500008004100000245004000041210004000081260002100121100001500142700001300157856006300170 2005 eng d00aRescuing CAT by fixing the problems0 aRescuing CAT by fixing the problems aMontreal, Canada1 aChang, S-H1 aZhang, J uhttp://mail.iacat.org/content/rescuing-cat-fixing-problems10202nas a2200553 4500008004100000245016300041210006900204300001400273490000700287520862500294653001808919653003208937100001608969700001608985700001409001700001609015700001409031700001509045700001609060700001409076700001509090700001509105700001409120700001709134700001709151700001709168700001609185700001709201700001609218700001509234700001609249700001709265700002309282700001609305700002209321700001609343700001609359700001709375700001309392700001509405700001309420700001609433700001209449700001409461700001609475700002009491700001209511856012509523 2005 eng d00aToward efficient and comprehensive measurement of the alcohol problems continuum in college students: The Brief Young Adult Alcohol Consequences Questionnaire0 aToward efficient and comprehensive measurement of the alcohol pr a1180-11890 v293 aBackground: Although a number of measures of alcohol problems in college students have been studied, the psychometric development and validation of these scales have been limited, for the most part, to methods based on classical test theory. In this study, we conducted analyses based on item response theory to select a set of items for measuring the alcohol problem severity continuum in college students that balances comprehensiveness and efficiency and is free from significant gender bias., Method: We conducted Rasch model analyses of responses to the 48-item Young Adult Alcohol Consequences Questionnaire by 164 male and 176 female college students who drank on at least a weekly basis. An iterative process using item fit statistics, item severities, item discrimination parameters, model residuals, and analysis of differential item functioning by gender was used to pare the items down to those that best fit a Rasch model and that were most efficient in discriminating among levels of alcohol problems in the sample., Results: The process of iterative Rasch model analyses resulted in a final 24-item scale with the data fitting the unidimensional Rasch model very well. The scale showed excellent distributional properties, had items adequately matched to the severity of alcohol problems in the sample, covered a full range of problem severity, and appeared highly efficient in retaining all of the meaningful variance captured by the original set of 48 items., Conclusions: The use of Rasch model analyses to inform item selection produced a final scale that, in both its comprehensiveness and its efficiency, should be a useful tool for researchers studying alcohol problems in college students. To aid interpretation of raw scores, examples of the types of alcohol problems that are likely to be experienced across a range of selected scores are provided., (C)2005Research Society on AlcoholismAn important, sometimes controversial feature of all psychological phenomena is whether they are categorical or dimensional. A conceptual and psychometric framework is described for distinguishing whether the latent structure behind manifest categories (e.g., psychiatric diagnoses, attitude groups, or stages of development) is category-like or dimension-like. Being dimension-like requires (a) within-category heterogeneity and (b) between-category quantitative differences. Being category-like requires (a) within-category homogeneity and (b) between-category qualitative differences. The relation between this classification and abrupt versus smooth differences is discussed. Hybrid structures are possible. Being category-like is itself a matter of degree; the authors offer a formalized framework to determine this degree. Empirical applications to personality disorders, attitudes toward capital punishment, and stages of cognitive development illustrate the approach., (C) 2005 by the American Psychological AssociationThe authors conducted Rasch model ( G. Rasch, 1960) analyses of items from the Young Adult Alcohol Problems Screening Test (YAAPST; S. C. Hurlbut & K. J. Sher, 1992) to examine the relative severity and ordering of alcohol problems in 806 college students. Items appeared to measure a single dimension of alcohol problem severity, covering a broad range of the latent continuum. Items fit the Rasch model well, with less severe symptoms reliably preceding more severe symptoms in a potential progression toward increasing levels of problem severity. However, certain items did not index problem severity consistently across demographic subgroups. A shortened, alternative version of the YAAPST is proposed, and a norm table is provided that allows for a linking of total YAAPST scores to expected symptom expression., (C) 2004 by the American Psychological AssociationA didactic on latent growth curve modeling for ordinal outcomes is presented. The conceptual aspects of modeling growth with ordinal variables and the notion of threshold invariance are illustrated graphically using a hypothetical example. The ordinal growth model is described in terms of 3 nested models: (a) multivariate normality of the underlying continuous latent variables (yt) and its relationship with the observed ordinal response pattern (Yt), (b) threshold invariance over time, and (c) growth model for the continuous latent variable on a common scale. Algebraic implications of the model restrictions are derived, and practical aspects of fitting ordinal growth models are discussed with the help of an empirical example and Mx script ( M. C. Neale, S. M. Boker, G. Xie, & H. H. Maes, 1999). The necessary conditions for the identification of growth models with ordinal data and the methodological implications of the model of threshold invariance are discussed., (C) 2004 by the American Psychological AssociationRecent research points toward the viability of conceptualizing alcohol problems as arrayed along a continuum. Nevertheless, modern statistical techniques designed to scale multiple problems along a continuum (latent trait modeling; LTM) have rarely been applied to alcohol problems. This study applies LTM methods to data on 110 problems reported during in-person interviews of 1,348 middle-aged men (mean age = 43) from the general population. The results revealed a continuum of severity linking the 110 problems, ranging from heavy and abusive drinking, through tolerance and withdrawal, to serious complications of alcoholism. These results indicate that alcohol problems can be arrayed along a dimension of severity and emphasize the relevance of LTM to informing the conceptualization and assessment of alcohol problems., (C) 2004 by the American Psychological AssociationItem response theory (IRT) is supplanting classical test theory as the basis for measures development. This study demonstrated the utility of IRT for evaluating DSM-IV diagnostic criteria. Data on alcohol, cannabis, and cocaine symptoms from 372 adult clinical participants interviewed with the Composite International Diagnostic Interview-Expanded Substance Abuse Module (CIDI-SAM) were analyzed with Mplus ( B. Muthen & L. Muthen, 1998) and MULTILOG ( D. Thissen, 1991) software. Tolerance and legal problems criteria were dropped because of poor fit with a unidimensional model. Item response curves, test information curves, and testing of variously constrained models suggested that DSM-IV criteria in the CIDI-SAM discriminate between only impaired and less impaired cases and may not be useful to scale case severity. IRT can be used to study the construct validity of DSM-IV diagnoses and to identify diagnostic criteria with poor performance., (C) 2004 by the American Psychological AssociationThis study examined the psychometric characteristics of an index of substance use involvement using item response theory. The sample consisted of 292 men and 140 women who qualified for a Diagnostic and Statistical Manual of Mental Disorders (3rd ed., rev.; American Psychiatric Association, 1987) substance use disorder (SUD) diagnosis and 293 men and 445 women who did not qualify for a SUD diagnosis. The results indicated that men had a higher probability of endorsing substance use compared with women. The index significantly predicted health, psychiatric, and psychosocial disturbances as well as level of substance use behavior and severity of SUD after a 2-year follow-up. Finally, this index is a reliable and useful prognostic indicator of the risk for SUD and the medical and psychosocial sequelae of drug consumption., (C) 2002 by the American Psychological AssociationComparability, validity, and impact of loss of information of a computerized adaptive administration of the Minnesota Multiphasic Personality Inventory-2 (MMPI-2) were assessed in a sample of 140 Veterans Affairs hospital patients. The countdown method ( Butcher, Keller, & Bacon, 1985) was used to adaptively administer Scales L (Lie) and F (Frequency), the 10 clinical scales, and the 15 content scales. Participants completed the MMPI-2 twice, in 1 of 2 conditions: computerized conventional test-retest, or computerized conventional-computerized adaptive. Mean profiles and test-retest correlations across modalities were comparable. Correlations between MMPI-2 scales and criterion measures supported the validity of the countdown method, although some attenuation of validity was suggested for certain health-related items. Loss of information incurred with this mode of adaptive testing has minimal impact on test validity. Item and time savings were substantial., (C) 1999 by the American Psychological Association10aPsychometrics10aSubstance-Related Disorders1 aKahler, C W1 aStrong, D R1 aRead, J P1 aDe Boeck, P1 aWilson, M1 aActon, G S1 aPalfai, T P1 aWood, M D1 aMehta, P D1 aNeale, M C1 aFlay, B R1 aConklin, C A1 aClayton, R R1 aTiffany, S T1 aShiffman, S1 aKrueger, R F1 aNichol, P E1 aHicks, B M1 aMarkon, K E1 aPatrick, C J1 aIacono, William, G1 aMcGue, Matt1 aLangenbucher, J W1 aLabouvie, E1 aMartin, C S1 aSanjuan, P M1 aBavly, L1 aKirisci, L1 aChung, T1 aVanyukov, M1 aDunn, M1 aTarter, R1 aHandel, R W1 aBen-Porath, Y S1 aWatt, M uhttp://mail.iacat.org/content/toward-efficient-and-comprehensive-measurement-alcohol-problems-continuum-college-students01451nas a2200133 4500008004100000245012200041210006900163300001000232490001000242520092300252100001001175700001501185856011701200 2005 eng d00aValidation of a computerized adaptive testing version of the Schedule for Nonadaptive and Adaptive Personality (SNAP)0 aValidation of a computerized adaptive testing version of the Sch a28-430 v17(1)3 aThis is a validation study of a computerized adaptive (CAT) version of the Schedule for Nonadaptive and Adaptive Personality (SNAP) conducted with 413 undergraduates who completed the SNAP twice, 1 week apart. Participants were assigned randomly to 1 of 4 retest groups: (a) paper-and-pencil (P&P) SNAP, (b) CAT, (c) P&P/CAT, and (d) CAT/P&P. With number of items held constant, computerized administration had little effect on descriptive statistics, rank ordering of scores, reliability, and concurrent validity, but was preferred over P&P administration by most participants. CAT administration yielded somewhat lower precision and validity than P&P administration, but required 36% to 37% fewer items and 58% to 60% less time to complete. These results confirm not only key findings from previous CAT simulation studies of personality measures but extend them for the 1st time to a live assessment setting.1 aSimms1 aClark, L A uhttp://mail.iacat.org/content/validation-computerized-adaptive-testing-version-schedule-nonadaptive-and-adaptive01452nas a2200133 4500008004100000245011400041210006900155300001000224490000700234520092000241100001501161700001501176856012701191 2005 eng d00aValidation of a computerized adaptive version of the Schedule of Non-Adaptive and Adaptive Personality (SNAP)0 aValidation of a computerized adaptive version of the Schedule of a28-430 v173 a This is a validation study of a computerized adaptive (CAT) version of the Schedule for Nonadaptive and Adaptive Personality (SNAP) conducted with 413 undergraduates who completed the SNAP twice, 1 week apart. Participants were assigned randomly to 1 of 4 retest groups: (a) paper-and-pencil (P&P) SNAP, (b) CAT, (c) P&P/CAT, and (d) CAT/P&P. With number of items held constant, computerized administration had little effect on descriptive statistics, rank ordering of scores, reliability, and concurrent validity, but was preferred over P&P administration by most participants. CAT administration yielded somewhat lower precision and validity than P&P administration, but required 36% to 37% fewer items and 58% to 60% less time to complete. These results confirm not only key findings from previous CAT simulation studies of personality measures but extend them for the 1st time to a live assessment setting. 1 aSimms, L J1 aClark, L J uhttp://mail.iacat.org/content/validation-computerized-adaptive-version-schedule-non-adaptive-and-adaptive-personality-snap03708nas a2200481 4500008004100000245005200041210005200093300001200145490000700157520221100164653001902375653002902394653005802423653001002481653005302491653000902544653001102553653002502564653002602589653003302615653001102648653001002659653000902669653001602678653002402694653007402718653001802792653002902810653005802839653003102897653003202928653003602960653003202996100001503028700001603043700001603059700001603075700001003091700001403101700001803115700001503133856007803148 2004 eng d00aActivity outcome measurement for postacute care0 aActivity outcome measurement for postacute care aI49-1610 v423 aBACKGROUND: Efforts to evaluate the effectiveness of a broad range of postacute care services have been hindered by the lack of conceptually sound and comprehensive measures of outcomes. It is critical to determine a common underlying structure before employing current methods of item equating across outcome instruments for future item banking and computer-adaptive testing applications. OBJECTIVE: To investigate the factor structure, reliability, and scale properties of items underlying the Activity domains of the International Classification of Functioning, Disability and Health (ICF) for use in postacute care outcome measurement. METHODS: We developed a 41-item Activity Measure for Postacute Care (AM-PAC) that assessed an individual's execution of discrete daily tasks in his or her own environment across major content domains as defined by the ICF. We evaluated the reliability and discriminant validity of the prototype AM-PAC in 477 individuals in active rehabilitation programs across 4 rehabilitation settings using factor analyses, tests of item scaling, internal consistency reliability analyses, Rasch item response theory modeling, residual component analysis, and modified parallel analysis. RESULTS: Results from an initial exploratory factor analysis produced 3 distinct, interpretable factors that accounted for 72% of the variance: Applied Cognition (44%), Personal Care & Instrumental Activities (19%), and Physical & Movement Activities (9%); these 3 activity factors were verified by a confirmatory factor analysis. Scaling assumptions were met for each factor in the total sample and across diagnostic groups. Internal consistency reliability was high for the total sample (Cronbach alpha = 0.92 to 0.94), and for specific diagnostic groups (Cronbach alpha = 0.90 to 0.95). Rasch scaling, residual factor, differential item functioning, and modified parallel analyses supported the unidimensionality and goodness of fit of each unique activity domain. CONCLUSIONS: This 3-factor model of the AM-PAC can form the conceptual basis for common-item equating and computer-adaptive applications, leading to a comprehensive system of outcome instruments for postacute care settings.10a*Self Efficacy10a*Sickness Impact Profile10aActivities of Daily Living/*classification/psychology10aAdult10aAftercare/*standards/statistics & numerical data10aAged10aBoston10aCognition/physiology10aDisability Evaluation10aFactor Analysis, Statistical10aFemale10aHuman10aMale10aMiddle Aged10aMovement/physiology10aOutcome Assessment (Health Care)/*methods/statistics & numerical data10aPsychometrics10aQuestionnaires/standards10aRehabilitation/*standards/statistics & numerical data10aReproducibility of Results10aSensitivity and Specificity10aSupport, U.S. Gov't, Non-P.H.S.10aSupport, U.S. Gov't, P.H.S.1 aHaley, S M1 aCoster, W J1 aAndres, P L1 aLudlow, L H1 aNi, P1 aBond, T L1 aSinclair, S J1 aJette, A M uhttp://mail.iacat.org/content/activity-outcome-measurement-postacute-care00507nas a2200121 4500008004100000245009000041210006900131260001700200100001700217700001900234700001500253856011700268 2004 eng d00aCombining computer adaptive testing technology with cognitively diagnostic assessment0 aCombining computer adaptive testing technology with cognitively  aSan Diego CA1 aMcGlohen, MK1 aChang, Hua-Hua1 aWills, J T uhttp://mail.iacat.org/content/combining-computer-adaptive-testing-technology-cognitively-diagnostic-assessment-000490nas a2200097 4500008004100000245008700041210006900128260006200197100001500259856011800274 2004 eng d00aDeveloping tailored instruments: Item banking and computerized adaptive assessment0 aDeveloping tailored instruments Item banking and computerized ad aItem Banks, and Computer-Adaptive Testing,” Bethesda MD1 aChang, C-H uhttp://mail.iacat.org/content/developing-tailored-instruments-item-banking-and-computerized-adaptive-assessment-201994nas a2200193 4500008004100000020002200041245011400063210006900177260004100246300001200287490000700299520125600306653003401562653002501596653002601621100001401647700001901661856012001680 2004 eng d a0022-0655 (Print)00aEffects of practical constraints on item selection rules at the early stages of computerized adaptive testing0 aEffects of practical constraints on item selection rules at the  bBlackwell Publishing: United Kingdom a149-1740 v413 aThe purpose of this study was to compare the effects of four item selection rules--(1) Fisher information (F), (2) Fisher information with a posterior distribution (FP), (3) Kullback-Leibler information with a posterior distribution (KP), and (4) completely randomized item selection (RN)--with respect to the precision of trait estimation and the extent of item usage at the early stages of computerized adaptive testing. The comparison of the four item selection rules was carried out under three conditions: (1) using only the item information function as the item selection criterion; (2) using both the item information function and content balancing; and (3) using the item information function, content balancing, and item exposure control. When test length was less than 10 items, FP and KP tended to outperform F at extreme trait levels in Condition 1. However, in more realistic settings, it could not be concluded that FP and KP outperformed F, especially when item exposure control was imposed. When test length was greater than 10 items, the three nonrandom item selection procedures performed similarly no matter what the condition was, while F had slightly higher item usage. (PsycINFO Database Record (c) 2007 APA, all rights reserved)10acomputerized adaptive testing10aitem selection rules10apractical constraints1 aChen, S-Y1 aAnkenmann, R D uhttp://mail.iacat.org/content/effects-practical-constraints-item-selection-rules-early-stages-computerized-adaptive00525nas a2200121 4500008004100000245011300041210006900154300001200223490000700235100001500242700001900257856012700276 2004 eng d00affects of practical constraints on item selection rules at the early stages of computerized adaptive testing0 affects of practical constraints on item selection rules at the e a149-1740 v411 aChen, Y -Y1 aAnkenmann, R D uhttp://mail.iacat.org/content/ffects-practical-constraints-item-selection-rules-early-stages-computerized-adaptive-testing01192nas a2200133 4500008003900000245009600039210006900135300001200204490000700216520074500223100002000968700001700988856005301005 2004 d00aImplementation and Measurement Efficiency of Multidimensional Computerized Adaptive Testing0 aImplementation and Measurement Efficiency of Multidimensional Co a295-3160 v283 a

Multidimensional adaptive testing (MAT) procedures are proposed for the measurement of several latent traits by a single examination. Bayesian latent trait estimation and adaptive item selection are derived. Simulations were conducted to compare the measurement efficiency of MAT with those of unidimensional adaptive testing and random administration. The results showed that the higher the correlation between latent traits, the more latent traits there were, and the more scoring levels there were in the items, the more efficient MAT was than the other two procedures. For tests containing multidimensional items, only MAT is applicable, whereas unidimensional adaptive testing is not. Issues in implementing MAT are discussed.

1 aWang, Wen-Chung1 aChen, Po-Hsi uhttp://apm.sagepub.com/content/28/5/295.abstract00391nas a2200121 4500008004100000245004800041210004800089260001700137100001200154700001500166700001500181856007300196 2004 eng d00aItem parameter recovery with adaptive tests0 aItem parameter recovery with adaptive tests aSan Diego CA1 aDo, B-R1 aChuah, S C1 aDrasgow, F uhttp://mail.iacat.org/content/item-parameter-recovery-adaptive-tests00449nas a2200097 4500008004100000245009600041210006900137260001700206100001500223856011300238 2004 eng d00aProtecting the integrity of computer-adaptive licensure tests: Results of a legal challenge0 aProtecting the integrity of computeradaptive licensure tests Res aSan Diego CA1 aCizek, G J uhttp://mail.iacat.org/content/protecting-integrity-computer-adaptive-licensure-tests-results-legal-challenge04033nas a2200433 4500008004100000245012300041210006900164260000800233300001200241490000700253520252400260653001902784653002902803653005802832653001002890653000902900653002202909653002602931653003302957653001102990653001103001653000903012653001603021653007403037653003003111653003603141653005803177653003103235653004503266653004103311653003203352100001603384700001503400700001603415700001603431700001403447700001203461856012603473 2004 eng d00aRefining the conceptual basis for rehabilitation outcome measurement: personal care and instrumental activities domain0 aRefining the conceptual basis for rehabilitation outcome measure cJan aI62-1720 v423 aBACKGROUND: Rehabilitation outcome measures routinely include content on performance of daily activities; however, the conceptual basis for item selection is rarely specified. These instruments differ significantly in format, number, and specificity of daily activity items and in the measurement dimensions and type of scale used to specify levels of performance. We propose that a requirement for upper limb and hand skills underlies many activities of daily living (ADL) and instrumental activities of daily living (IADL) items in current instruments, and that items selected based on this definition can be placed along a single functional continuum. OBJECTIVE: To examine the dimensional structure and content coverage of a Personal Care and Instrumental Activities item set and to examine the comparability of items from existing instruments and a set of new items as measures of this domain. METHODS: Participants (N = 477) from 3 different disability groups and 4 settings representing the continuum of postacute rehabilitation care were administered the newly developed Activity Measure for Post-Acute Care (AM-PAC), the SF-8, and an additional setting-specific measure: FIM (in-patient rehabilitation); MDS (skilled nursing facility); MDS-PAC (postacute settings); OASIS (home care); or PF-10 (outpatient clinic). Rasch (partial-credit model) analyses were conducted on a set of 62 items covering the Personal Care and Instrumental domain to examine item fit, item functioning, and category difficulty estimates and unidimensionality. RESULTS: After removing 6 misfitting items, the remaining 56 items fit acceptably along the hypothesized continuum. Analyses yielded different difficulty estimates for the maximum score (eg, "Independent performance") for items with comparable content from different instruments. Items showed little differential item functioning across age, diagnosis, or severity groups, and 92% of the participants fit the model. CONCLUSIONS: ADL and IADL items from existing rehabilitation outcomes instruments that depend on skilled upper limb and hand use can be located along a single continuum, along with the new personal care and instrumental items of the AM-PAC addressing gaps in content. Results support the validity of the proposed definition of the Personal Care and Instrumental Activities dimension of function as a guide for future development of rehabilitation outcome instruments, such as linked, setting-specific short forms and computerized adaptive testing approaches.10a*Self Efficacy10a*Sickness Impact Profile10aActivities of Daily Living/*classification/psychology10aAdult10aAged10aAged, 80 and over10aDisability Evaluation10aFactor Analysis, Statistical10aFemale10aHumans10aMale10aMiddle Aged10aOutcome Assessment (Health Care)/*methods/statistics & numerical data10aQuestionnaires/*standards10aRecovery of Function/physiology10aRehabilitation/*standards/statistics & numerical data10aReproducibility of Results10aResearch Support, U.S. Gov't, Non-P.H.S.10aResearch Support, U.S. Gov't, P.H.S.10aSensitivity and Specificity1 aCoster, W J1 aHaley, S M1 aAndres, P L1 aLudlow, L H1 aBond, T L1 aNi, P S uhttp://mail.iacat.org/content/refining-conceptual-basis-rehabilitation-outcome-measurement-personal-care-and-instrumental02889nas a2200301 4500008004100000020002200041245013700063210006900200250001500269260000800284300001000292490000700302520186600309653001102175653003302186653001102219653004602230653002402276653002902300653003002329653002902359100001502388700001602403700001602419700001602435700001002451856012602461 2004 eng d a0003-9993 (Print)00aScore comparability of short forms and computerized adaptive testing: Simulation study with the activity measure for post-acute care0 aScore comparability of short forms and computerized adaptive tes a2004/04/15 cApr a661-60 v853 aOBJECTIVE: To compare simulated short-form and computerized adaptive testing (CAT) scores to scores obtained from complete item sets for each of the 3 domains of the Activity Measure for Post-Acute Care (AM-PAC). DESIGN: Prospective study. SETTING: Six postacute health care networks in the greater Boston metropolitan area, including inpatient acute rehabilitation, transitional care units, home care, and outpatient services. PARTICIPANTS: A convenience sample of 485 adult volunteers who were receiving skilled rehabilitation services. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Inpatient and community-based short forms and CAT applications were developed for each of 3 activity domains (physical & mobility, personal care & instrumental, applied cognition) using item pools constructed from new items and items from existing postacute care instruments. RESULTS: Simulated CAT scores correlated highly with score estimates from the total item pool in each domain (4- and 6-item CAT r range,.90-.95; 10-item CAT r range,.96-.98). Scores on the 10-item short forms constructed for inpatient and community settings also provided good estimates of the AM-PAC item pool scores for the physical & movement and personal care & instrumental domains, but were less consistent in the applied cognition domain. Confidence intervals around individual scores were greater in the short forms than for the CATs. CONCLUSIONS: Accurate scoring estimates for AM-PAC domains can be obtained with either the setting-specific short forms or the CATs. The strong relationship between CAT and item pool scores can be attributed to the CAT's ability to select specific items to match individual responses. The CAT may have additional advantages over short forms in practicality, efficiency, and the potential for providing more precise scoring estimates for individuals.10aBoston10aFactor Analysis, Statistical10aHumans10aOutcome Assessment (Health Care)/*methods10aProspective Studies10aQuestionnaires/standards10aRehabilitation/*standards10aSubacute Care/*standards1 aHaley, S M1 aCoster, W J1 aAndres, P L1 aKosinski, M1 aNi, P uhttp://mail.iacat.org/content/score-comparability-short-forms-and-computerized-adaptive-testing-simulation-study-activity01572nas a2200133 4500008004100000020001300041245007500054210006900129300001200198490000800210520110300218100001501321856010201336 2004 eng d a0378375800aSequential estimation in variable length computerized adaptive testing0 aSequential estimation in variable length computerized adaptive t a249-2640 v1213 aWith the advent of modern computer technology, there have been growing e3orts in recent years to computerize standardized tests, including the popular Graduate Record Examination (GRE), the Graduate Management Admission Test (GMAT) and the Test of English as a Foreign Language (TOEFL). Many of such computer-based tests are known as the computerized adaptive tests, a major feature of which is that, depending on their performance in the course of testing, di3erent examinees may be given with di3erent sets of items (questions). In doing so, items can be e>ciently utilized to yield maximum accuracy for estimation of examinees’ ability traits. We consider, in this article, one type of such tests where test lengths vary with examinees to yield approximately same predetermined accuracy for all ability traits. A comprehensive large sample theory is developed for the expected test length and the sequential point and interval estimates of the latent trait. Extensive simulations are conducted with results showing that the large sample approximations are adequate for realistic sample sizes. 1 aChang, I Y uhttp://mail.iacat.org/content/sequential-estimation-variable-length-computerized-adaptive-testing00542nas a2200181 4500008004100000245005000041210004800091300001000139490000700149653003400156100001400190700001500204700001500219700001400234700002600248700001300274856007300287 2004 eng d00aSiette: a web-based tool for adaptive testing0 aSiette a webbased tool for adaptive testing a29-610 v1410acomputerized adaptive testing1 aConejo, R1 aGuzmán, E1 aMillán, E1 aTrella, M1 aPérez-De-La-Cruz, JL1 aRíos, A uhttp://mail.iacat.org/content/siette-web-based-tool-adaptive-testing00544nas a2200097 4500008004100000245008700041210006900128260012200197100001900319856010800338 2004 eng d00aUnderstanding computerized adaptive testing: From Robbins-Munro to Lord and beyond0 aUnderstanding computerized adaptive testing From RobbinsMunro to aD. Kaplan (Ed.), The Sage handbook of quantitative methodology for the social sciences (pp. 117-133). New York: Sage.1 aChang, Hua-Hua uhttp://mail.iacat.org/content/understanding-computerized-adaptive-testing-robbins-munro-lord-and-beyond00475nas a2200121 4500008004100000245008000041210006900121300001200190490000700202100002300209700001900232856010200251 2003 eng d00aAlpha-stratified adaptive testing with large numbers of content constraints0 aAlphastratified adaptive testing with large numbers of content c a107-1200 v271 avan der Linden, WJ1 aChang, Hua-Hua uhttp://mail.iacat.org/content/alpha-stratified-adaptive-testing-large-numbers-content-constraints00407nas a2200109 4500008004100000245006200041210006200103260001500165100001900180700001300199856008500212 2003 eng d00aAssessing CAT security breaches by the item pooling index0 aAssessing CAT security breaches by the item pooling index aChicago IL1 aChang, Hua-Hua1 aZhang, J uhttp://mail.iacat.org/content/assessing-cat-security-breaches-item-pooling-index00412nas a2200121 4500008003900000245006100039210005900100300001400159490000700173100001000180700001600190856008400206 2003 d00aa-Stratified multistage CAT design with content-blocking0 aaStratified multistage CAT design with contentblocking a359–3780 v561 aYi, Q1 aChang, H -H uhttp://mail.iacat.org/content/stratified-multistage-cat-design-content-blocking00526nas a2200145 4500008004100000245008400041210006900125300001200194490000700206100001300213700001400226700001300240700001700253856011000270 2003 eng d00aCan an item response theory-based pain item bank enhance measurement precision?0 aCan an item response theorybased pain item bank enhance measurem aD34-D360 v251 aLai, J-S1 aDineen, K1 aCella, D1 aVon Roenn, J uhttp://mail.iacat.org/content/can-item-response-theory-based-pain-item-bank-enhance-measurement-precision01841nas a2200205 4500008004100000245009000041210006900131300001100200490000700211520111400218653002101332653003001353653001601383653003201399653001601431653004501447100001501492700001601507856011201523 2003 eng d00aA comparative study of item exposure control methods in computerized adaptive testing0 acomparative study of item exposure control methods in computeriz a71-1030 v403 aThis study compared the properties of five methods of item exposure control within the purview of estimating examinees' abilities in a computerized adaptive testing (CAT) context. Each exposure control algorithm was incorporated into the item selection procedure and the adaptive testing progressed based on the CAT design established for this study. The merits and shortcomings of these strategies were considered under different item pool sizes and different desired maximum exposure rates and were evaluated in light of the observed maximum exposure rates, the test overlap rates, and the conditional standard errors of measurement. Each method had its advantages and disadvantages, but no one possessed all of the desired characteristics. There was a clear and logical trade-off between item exposure control and measurement precision. The M. L. Stocking and C. Lewis conditional multinomial procedure and, to a slightly lesser extent, the T. Davey and C. G. Parshall method seemed to be the most promising considering all of the factors that this study addressed. (PsycINFO Database Record (c) 2005 APA )10aAdaptive Testing10aComputer Assisted Testing10aEducational10aItem Analysis (Statistical)10aMeasurement10aStrategies computerized adaptive testing1 aChang, S-W1 aAnsley, T N uhttp://mail.iacat.org/content/comparative-study-item-exposure-control-methods-computerized-adaptive-testing01115nas a2200145 4500008004100000245008300041210006900124300000900193490000600202520060700208100001500815700001900830700001300849856010700862 2003 eng d00aComputerized adaptive testing: A comparison of three content balancing methods0 aComputerized adaptive testing A comparison of three content bala a1-150 v23 aContent balancing is often a practical consideration in the design of computerized adaptive testing (CAT). This study compared three content balancing methods, namely, the constrained CAT (CCAT), the modified constrained CAT (MCCAT), and the modified multinomial model (MMM), under various conditions of test length and target maximum exposure rate. Results of a series of simulation studies indicate that there is no systematic effect of content balancing method in measurement efficiency and pool utilization. However, among the three methods, the MMM appears to consistently over-expose fewer items.1 aLeung, C-K1 aChang, Hua-Hua1 aHau, K-T uhttp://mail.iacat.org/content/computerized-adaptive-testing-comparison-three-content-balancing-methods00510nas a2200133 4500008004100000245008300041210006900124260001500193100001500208700001900223700001300242700001200255856010900267 2003 eng d00aComputerized adaptive testing: A comparison of three content balancing methods0 aComputerized adaptive testing A comparison of three content bala aChicago IL1 aLeung, C-K1 aChang, Hua-Hua1 aHau, K-T1 aWen, z. uhttp://mail.iacat.org/content/computerized-adaptive-testing-comparison-three-content-balancing-methods-000450nas a2200121 4500008004100000245007200041210006900113300001200182490000700194100001500201700001200216856010000228 2003 eng d00aComputerized adaptive testing using the nearest-neighbors criterion0 aComputerized adaptive testing using the nearestneighbors criteri a204-2160 v271 aCheng, P E1 aLiou, M uhttp://mail.iacat.org/content/computerized-adaptive-testing-using-nearest-neighbors-criterion-001932nas a2200229 4500008004100000245007200041210006900113300001200182490000700194520116000201653001801361653002101379653003001400653001801430653002501448653002501473653005701498653002201555100001501577700001201592856009801604 2003 eng d00aComputerized adaptive testing using the nearest-neighbors criterion0 aComputerized adaptive testing using the nearestneighbors criteri a204-2160 v273 aItem selection procedures designed for computerized adaptive testing need to accurately estimate every taker's trait level (θ) and, at the same time, effectively use all items in a bank. Empirical studies showed that classical item selection procedures based on maximizing Fisher or other related information yielded highly varied item exposure rates; with these procedures, some items were frequently used whereas others were rarely selected. In the literature, methods have been proposed for controlling exposure rates; they tend to affect the accuracy in θ estimates, however. A modified version of the maximum Fisher information (MFI) criterion, coined the nearest neighbors (NN) criterion, is proposed in this study. The NN procedure improves to a moderate extent the undesirable item exposure rates associated with the MFI criterion and keeps sufficient precision in estimates. The NN criterion will be compared with a few other existing methods in an empirical study using the mean squared errors in θ estimates and plots of item exposure rates associated with different distributions. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10a(Statistical)10aAdaptive Testing10aComputer Assisted Testing10aItem Analysis10aItem Response Theory10aStatistical Analysis10aStatistical Estimation computerized adaptive testing10aStatistical Tests1 aCheng, P E1 aLiou, M uhttp://mail.iacat.org/content/computerized-adaptive-testing-using-nearest-neighbors-criterion01805nas a2200277 4500008004100000245007600041210006900117300001100186490000600197520090300203653001501106653002901121653003001150653002001180653001101200653005001211653001801261653003201279653004101311653001801352100001401370700001301384700001301397700001901410856009801429 2003 eng d00aDeveloping an initial physical function item bank from existing sources0 aDeveloping an initial physical function item bank from existing  a124-360 v43 aThe objective of this article is to illustrate incremental item banking using health-related quality of life data collected from two samples of patients receiving cancer treatment. The kinds of decisions one faces in establishing an item bank for computerized adaptive testing are also illustrated. Pre-calibration procedures include: identifying common items across databases; creating a new database with data from each pool; reverse-scoring "negative" items; identifying rating scales used in items; identifying pivot points in each rating scale; pivot anchoring items at comparable rating scale categories; and identifying items in each instrument that measure the construct of interest. A series of calibrations were conducted in which a small proportion of new items were added to the common core and misfitting items were identified and deleted until an initial item bank has been developed.10a*Databases10a*Sickness Impact Profile10aAdaptation, Psychological10aData Collection10aHumans10aNeoplasms/*physiopathology/psychology/therapy10aPsychometrics10aQuality of Life/*psychology10aResearch Support, U.S. Gov't, P.H.S.10aUnited States1 aBode, R K1 aCella, D1 aLai, J S1 aHeinemann, A W uhttp://mail.iacat.org/content/developing-initial-physical-function-item-bank-existing-sources00510nas a2200121 4500008004100000245009900041210006900140100001300209700001600222700001800238700001500256856011700271 2003 eng d00aDevelopment and psychometric evaluation of the Flexilevel Scale of Shoulder Function (FLEX-SF)0 aDevelopment and psychometric evaluation of the Flexilevel Scale 1 aCook, KF1 aRoddey, T S1 aGartsman, G M1 aOlson, S L uhttp://mail.iacat.org/content/development-and-psychometric-evaluation-flexilevel-scale-shoulder-function-flex-sf00528nas a2200121 4500008003900000245008900039210007000128260004600198100001700244700001300261700001700274856011500291 2003 d00aEffect of extra time on GRE® Quantitative and Verbal Scores (Research Report 03-13)0 aEffect of extra time on GRE® Quantitative and Verbal Scores Rese aPrinceton NJ: Educational Testing service1 aBridgeman, B1 aCline, F1 aHessinger, J uhttp://mail.iacat.org/content/effect-extra-time-gre%C2%AE-quantitative-and-verbal-scores-research-report-03-1301817nas a2200253 4500008004100000245013100041210006900172300001000241490000600251520095700257653001501214653002901229653002501258653001501283653002001298653001101318653003101329100001401360700001601374700001401390700001401404700002101418856012401439 2003 eng d00aAn examination of exposure control and content balancing restrictions on item selection in CATs using the partial credit model0 aexamination of exposure control and content balancing restrictio a24-420 v43 aThe purpose of the present investigation was to systematically examine the effectiveness of the Sympson-Hetter technique and rotated content balancing relative to no exposure control and no content rotation conditions in a computerized adaptive testing system (CAT) based on the partial credit model. A series of simulated fixed and variable length CATs were run using two data sets generated to multiple content areas for three sizes of item pools. The 2 (exposure control) X 2 (content rotation) X 2 (test length) X 3 (item pool size) X 2 (data sets) yielded a total of 48 conditions. Results show that while both procedures can be used with no deleterious effect on measurement precision, the gains in exposure control, pool utilization, and item overlap appear quite modest. Difficulties involved with setting the exposure control parameters in small item pools make questionable the utility of the Sympson-Hetter technique with similar item pools.10a*Computers10a*Educational Measurement10a*Models, Theoretical10aAutomation10aDecision Making10aHumans10aReproducibility of Results1 aDavis, LL1 aPastor, D A1 aDodd, B G1 aChiang, C1 aFitzpatrick, S J uhttp://mail.iacat.org/content/examination-exposure-control-and-content-balancing-restrictions-item-selection-cats-using00515nas a2200121 4500008004100000245010300041210006900144300001200213490000700225100002300232700001900255856011900274 2003 eng d00aImplementing content constraints in alpha-stratified adaptive testing using a shadow test approach0 aImplementing content constraints in alphastratified adaptive tes a107-1200 v271 avan der Linden, WJ1 aChang, Hua-Hua uhttp://mail.iacat.org/content/implementing-content-constraints-alpha-stratified-adaptive-testing-using-shadow-test00962nas a2200157 4500008004100000245011200041210006900153300001100222490000700233520035900240653003400599100001500633700001900648700001300667856012400680 2003 eng d00aIncorporation of Content Balancing Requirements in Stratification Designs for Computerized Adaptive Testing0 aIncorporation of Content Balancing Requirements in Stratificatio a257-700 v633 aStudied three stratification designs for computerized adaptive testing in conjunction with three well-developed content balancing methods. Simulation study results show substantial differences in item overlap rate and pool utilization among different methods. Recommends an optimal combination of stratification design and content balancing method. (SLD)10acomputerized adaptive testing1 aLeung, C-K1 aChang, Hua-Hua1 aHau, K-T uhttp://mail.iacat.org/content/incorporation-content-balancing-requirements-stratification-designs-computerized-adaptive01586nas a2200145 4500008003900000245011200039210006900151300001200220490000700232520109100239100002101330700001901351700001701370856005301387 2003 d00aIncorporation Of Content Balancing Requirements In Stratification Designs For Computerized Adaptive Testing0 aIncorporation Of Content Balancing Requirements In Stratificatio a257-2700 v633 a

In computerized adaptive testing, the multistage a-stratified design advocates a new philosophy on pool management and item selection in which, contradictory to common practice, less discriminating items are used first. The method is effective in reducing item-overlap rate and enhancing pool utilization. This stratification method has been extended in different ways to deal with the practical issues of content constraints and the positive correlation between item difficulty and discrimination. Nevertheless, these modified designs on their own do not automatically satisfy content requirements. In this study, three stratification designs were examined in conjunction with three well developed content balancing methods. The performance of each of these nine combinational methods was evaluated in terms of their item security, measurement efficiency, and pool utilization. Results showed substantial differences in item-overlap rate and pool utilization among different methods. An optimal combination of stratification design and content balancing method is recommended.

1 aLeung, Chi-Keung1 aChang, Hua-Hua1 aHau, Kit-Tai uhttp://epm.sagepub.com/content/63/2/257.abstract02749nas a2200349 4500008004100000245015800041210006900199260000800268300001200276490000700288520160300295653003001898653002001928653001001948653003201958653001101990653001102001653000902012653001602021653002802037653001802065653003702083653004102120653002802161100001302189700001502202700001302217700001502230700001402245700001902259856012102278 2003 eng d00aItem banking to improve, shorten and computerized self-reported fatigue: an illustration of steps to create a core item bank from the FACIT-Fatigue Scale0 aItem banking to improve shorten and computerized selfreported fa cAug a485-5010 v123 aFatigue is a common symptom among cancer patients and the general population. Due to its subjective nature, fatigue has been difficult to effectively and efficiently assess. Modern computerized adaptive testing (CAT) can enable precise assessment of fatigue using a small number of items from a fatigue item bank. CAT enables brief assessment by selecting questions from an item bank that provide the maximum amount of information given a person's previous responses. This article illustrates steps to prepare such an item bank, using 13 items from the Functional Assessment of Chronic Illness Therapy Fatigue Subscale (FACIT-F) as the basis. Samples included 1022 cancer patients and 1010 people from the general population. An Item Response Theory (IRT)-based rating scale model, a polytomous extension of the Rasch dichotomous model was utilized. Nine items demonstrating acceptable psychometric properties were selected and positioned on the fatigue continuum. The fatigue levels measured by these nine items along with their response categories covered 66.8% of the general population and 82.6% of the cancer patients. Although the operational CAT algorithms to handle polytomously scored items are still in progress, we illustrated how CAT may work by using nine core items to measure level of fatigue. Using this illustration, a fatigue measure comparable to its full-length 13-item scale administration was obtained using four items. The resulting item bank can serve as a core to which will be added a psychometrically sound and operational item bank covering the entire fatigue continuum.10a*Health Status Indicators10a*Questionnaires10aAdult10aFatigue/*diagnosis/etiology10aFemale10aHumans10aMale10aMiddle Aged10aNeoplasms/complications10aPsychometrics10aResearch Support, Non-U.S. Gov't10aResearch Support, U.S. Gov't, P.H.S.10aSickness Impact Profile1 aLai, J-S1 aCrane, P K1 aCella, D1 aChang, C-H1 aBode, R K1 aHeinemann, A W uhttp://mail.iacat.org/content/item-banking-improve-shorten-and-computerized-self-reported-fatigue-illustration-steps01437nas a2200205 4500008004100000245008800041210007000129300001200199490000700211520067700218653002100895653003000916653002400946653002500970653002600995653005201021100001901073700002301092856011601115 2003 eng d00aOptimal stratification of item pools in α-stratified computerized adaptive testing0 aOptimal stratification of item pools in αstratified computerized a262-2740 v273 aA method based on 0-1 linear programming (LP) is presented to stratify an item pool optimally for use in α-stratified adaptive testing. Because the 0-1 LP model belongs to the subclass of models with a network flow structure, efficient solutions are possible. The method is applied to a previous item pool from the computerized adaptive testing (CAT) version of the Graduate Record Exams (GRE) Quantitative Test. The results indicate that the new method performs well in practical situations. It improves item exposure control, reduces the mean squared error in the θ estimates, and increases test reliability. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10aAdaptive Testing10aComputer Assisted Testing10aItem Content (Test)10aItem Response Theory10aMathematical Modeling10aTest Construction computerized adaptive testing1 aChang, Hua-Hua1 avan der Linden, WJ uhttp://mail.iacat.org/content/optimal-stratification-item-pools-%CE%B1-stratified-computerized-adaptive-testing00437nas a2200109 4500008004100000245007300041210006900114260001500183100001400198700001300212856010200225 2003 eng d00aPredicting item exposure parameters in computerized adaptive testing0 aPredicting item exposure parameters in computerized adaptive tes aChicago IL1 aChen, S-Y1 aDoong, H uhttp://mail.iacat.org/content/predicting-item-exposure-parameters-computerized-adaptive-testing-000520nas a2200133 4500008004100000245009300041210006900134300001200203490000700215100001200222700001900234700001500253856011800268 2003 eng d00aThe relationship between item exposure and test overlap in computerized adaptive testing0 arelationship between item exposure and test overlap in computeri a129-1450 v401 aChen, S1 aAnkenmann, R D1 aSpray, J A uhttp://mail.iacat.org/content/relationship-between-item-exposure-and-test-overlap-computerized-adaptive-testing-101802nas a2200241 4500008004100000245009300041210006900134300001200203490000700215520098200222653001801204653002101222653003001243653001901273653004601292653001801338653002501356653001501381100001401396700001901410700001501429856011601444 2003 eng d00aThe relationship between item exposure and test overlap in computerized adaptive testing0 arelationship between item exposure and test overlap in computeri a129-1450 v403 aThe purpose of this article is to present an analytical derivation for the mathematical form of an average between-test overlap index as a function of the item exposure index, for fixed-length computerized adaptive tests (CATs). This algebraic relationship is used to investigate the simultaneous control of item exposure at both the item and test levels. The results indicate that, in fixed-length CATs, control of the average between-test overlap is achieved via the mean and variance of the item exposure rates of the items that constitute the CAT item pool. The mean of the item exposure rates is easily manipulated. Control over the variance of the item exposure rates can be achieved via the maximum item exposure rate (r-sub(max)). Therefore, item exposure control methods which implement a specification of r-sub(max) (e.g., J. B. Sympson and R. D. Hetter, 1985) provide the most direct control at both the item and test levels. (PsycINFO Database Record (c) 2005 APA )10a(Statistical)10aAdaptive Testing10aComputer Assisted Testing10aHuman Computer10aInteraction computerized adaptive testing10aItem Analysis10aItem Analysis (Test)10aTest Items1 aChen, S-Y1 aAnkenmann, R D1 aSpray, J A uhttp://mail.iacat.org/content/relationship-between-item-exposure-and-test-overlap-computerized-adaptive-testing00520nas a2200133 4500008004100000245009300041210006900134300001200203490000700215100001200222700001900234700001500253856011800268 2003 eng d00aThe relationship between item exposure and test overlap in computerized adaptive testing0 arelationship between item exposure and test overlap in computeri a129-1450 v401 aChen, S1 aAnkenmann, R D1 aSpray, J A uhttp://mail.iacat.org/content/relationship-between-item-exposure-and-test-overlap-computerized-adaptive-testing-000458nas a2200121 4500008004100000245007300041210006900114260001500183100001000198700001900208700001500227856009400242 2003 eng d00aA simulation study to compare CAT strategies for cognitive diagnosis0 asimulation study to compare CAT strategies for cognitive diagnos aChicago IL1 aXu, X1 aChang, Hua-Hua1 aDouglas, J uhttp://mail.iacat.org/content/simulation-study-compare-cat-strategies-cognitive-diagnosis00501nas a2200109 4500008004100000245010900041210006900150260001500219100001900234700001200253856012600265 2003 eng d00aTest-score comparability, ability estimation, and item-exposure control in computerized adaptive testing0 aTestscore comparability ability estimation and itemexposure cont aChicago IL1 aChang, Hua-Hua1 aYing, Z uhttp://mail.iacat.org/content/test-score-comparability-ability-estimation-and-item-exposure-control-computerized-adaptive02860nas a2200265 4500008004100000245006600041210006600107260000800173300000900181490000700190520208800197653002102285653002902306653003002335653001202365653001102377653001302388653003102401653001902432100001302451700001502464700001302479700001502492856008702507 2002 eng d00aAdvances in quality of life measurements in oncology patients0 aAdvances in quality of life measurements in oncology patients cJun a60-80 v293 aAccurate assessment of the quality of life (QOL) of patients can provide important clinical information to physicians, especially in the area of oncology. Changes in QOL are important indicators of the impact of a new cytotoxic therapy, can affect a patient's willingness to continue treatment, and may aid in defining response in the absence of quantifiable endpoints such as tumor regression. Because QOL is becoming an increasingly important aspect in the management of patients with malignant disease, it is vital that the instruments used to measure QOL are reliable and accurate. Assessment of QOL involves a multidimensional approach that includes physical, functional, social, and emotional well-being, and the most comprehensive instruments measure at least three of these domains. Instruments to measure QOL can be generic (eg, the Nottingham Health Profile), targeted toward specific illnesses (eg, Functional Assessment of Cancer Therapy - Lung), or be a combination of generic and targeted. Two of the most widely used examples of the combination, or hybrid, instruments are the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire Core 30 Items and the Functional Assessment of Chronic Illness Therapy. A consequence of the increasing international collaboration in clinical trials has been the growing necessity for instruments that are valid across languages and cultures. To assure the continuing reliability and validity of QOL instruments in this regard, item response theory can be applied. Techniques such as item response theory may be used in the future to construct QOL item banks containing large sets of validated questions that represent various levels of QOL domains. As QOL becomes increasingly important in understanding and approaching the overall management of cancer patients, the tools available to clinicians and researchers to assess QOL will continue to evolve. While the instruments currently available provide reliable and valid measurement, further improvements in precision and application are anticipated.10a*Quality of Life10a*Sickness Impact Profile10aCross-Cultural Comparison10aCulture10aHumans10aLanguage10aNeoplasms/*physiopathology10aQuestionnaires1 aCella, D1 aChang, C-H1 aLai, J S1 aWebster, K uhttp://mail.iacat.org/content/advances-quality-life-measurements-oncology-patients03320nas a2200133 4500008004100000245005900041210005900100300001000159490000700169520289000176100001903066700001603085856008503101 2002 eng d00aApplicable adaptive testing models for school teachers0 aApplicable adaptive testing models for school teachers a55-590 v393 aThe purpose of this study was to investigate the attitudinal effects on SPRT adaptive testing environment for junior high school students. Subjects were 39 eighth graders from a selected junior high school. Major instruments for the study were the Junior High School Natural Sciences Adaptive Testing System driven by the SPRT algorithm, and a self-developed attitudinal questionnaire, factors examined include: test anxiety, examinee preference, adaptability of the test, and acceptance of the test result. The major findings were that overall, junior high school students" attitudes towards computerized adaptive tests were positive, no significant correlations existed between test attitude and the test length. The results indicated that junior high school students generally have positive attitudes towards adaptive testing.Modèles de tests d"adaptation à l"usage des enseignants. L"objectif de cette étude était d"enquêter sur les effets causés par une passation de tests d"adaptation ( selon l"algorithme "Sequential Probability Radio Test " (SPRT) ) dans une classe de trente-neuf élèves de huitième année du secondaire inférieur. Les principaux instruments utilisés ont été ceux du système de tests d"adaptation (avec le SPRT) et destiné aux classes de sciences naturelles du degré secondaire inférieur. Un questionnaire d"attitude, développé par nos soins, a également été utilisé pour examiner les facteurs suivants: test d"anxiété, préférence des candidats, adaptabilité du test et acceptation des résultats. Les principales conclusions ont été que, dans l"ensemble, l"attitude des élèves du secondaire inférieur face aux tests d"adaptation informatisés a été positive, aucune corrélation significative existant entre cette attitude et la longueur des tests. Les résultats démontrent aussi que les élèves du secondaire ont une attitude généralement positive envers les tests d"adaptation.Test Modelle zur Anwendung durch Klassenlehrer Zweck dieser Untersuchung war, die Auswirkungen über die Einstellung von Jun. High School Schülern im Zusammenhang mit dem SPRT Testumfeld zu untersuchen. 39 Achtklässler einer Jun. High School nahmen an dem Test teil. Die Untersuchung stützte sich hauptsächlich auf das Jun. High School Natural. Sciences Adaptive Testing System, das auf dem SPRT Rechnungsverfahren basiert sowie einem selbst erstellten Fragebogen mit folgenden Faktoren: Testängste, Präferenzen der Testperson, Geeignetheit des Tests, Anerkennung des Testergebnisses. Es stellte sich heraus, dass die Einstellung der Studenten zu den Computer adaptierten Tests im allgemeinen positiv waren; es ergaben sich keine bedeutsamen Wechselwirkungen zwischen persönlicher Testeinstellung und Testlänge. Die Ergebnisse belegen, dass Jun. High School Schüler im allgemeinen eine positive Haltung zu adaptierten Tests haben. 1 aChang-Hwa, W A1 aChuang, C-L uhttp://mail.iacat.org/content/applicable-adaptive-testing-models-school-teachers00619nas a2200157 4500008004100000245012800041210006900169300001300238490000700251100001700258700001600275700001500291700002100306700001300327856012100340 2002 eng d00aCan examinees use judgments of item difficulty to improve proficiency estimates on computerized adaptive vocabulary tests? 0 aCan examinees use judgments of item difficulty to improve profic a 311-3300 v391 aVispoel, W P1 aClough, S J1 aBleiler, T1 aHendrickson, A B1 aIhrig, D uhttp://mail.iacat.org/content/can-examinees-use-judgments-item-difficulty-improve-proficiency-estimates-computerized00537nas a2200121 4500008004100000245011500041210006900156260001900225100001500244700001900259700001300278856012400291 2002 eng d00aComparing three item selection approaches for computerized adaptive testing with content balancing requirement0 aComparing three item selection approaches for computerized adapt aNew Orleans LA1 aLeung, C-K1 aChang, Hua-Hua1 aHau, K-T uhttp://mail.iacat.org/content/comparing-three-item-selection-approaches-computerized-adaptive-testing-content-balancing02086nas a2200229 4500008004100000245012900041210006900170300001200239490000700251520124000258653001801498653002101516653004501537653003001582653001801612653002501630653002601655100001601681700001401697700001901711856012601730 2002 eng d00aA comparison of item selection techniques and exposure control mechanisms in CATs using the generalized partial credit model0 acomparison of item selection techniques and exposure control mec a147-1630 v263 aThe use of more performance items in large-scale testing has led to an increase in the research investigating the use of polytomously scored items in computer adaptive testing (CAT). Because this research has to be complemented with information pertaining to exposure control, the present research investigated the impact of using five different exposure control algorithms in two sized item pools calibrated using the generalized partial credit model. The results of the simulation study indicated that the a-stratified design, in comparison to a no-exposure control condition, could be used to reduce item exposure and overlap, increase pool utilization, and only minorly degrade measurement precision. Use of the more restrictive exposure control algorithms, such as the Sympson-Hetter and conditional Sympson-Hetter, controlled exposure to a greater extent but at the cost of measurement precision. Because convergence of the exposure control parameters was problematic for some of the more restrictive exposure control algorithms, use of the more simplistic exposure control mechanisms, particularly when the test length to item pool size ratio is large, is recommended. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10a(Statistical)10aAdaptive Testing10aAlgorithms computerized adaptive testing10aComputer Assisted Testing10aItem Analysis10aItem Response Theory10aMathematical Modeling1 aPastor, D A1 aDodd, B G1 aChang, Hua-Hua uhttp://mail.iacat.org/content/comparison-item-selection-techniques-and-exposure-control-mechanisms-cats-using-generalized00690nas a2200145 4500008004100000245003400041210003400075300001100109490000700120520029200127653003400419100001200453700001500465856006400480 2002 eng d00aComputerised adaptive testing0 aComputerised adaptive testing a619-220 v333 aConsiders the potential of computer adaptive testing (CAT). Discusses the use of CAT instead of traditional paper and pencil tests, identifies decisions that impact the efficacy of CAT, and concludes that CAT is beneficial when used to its full potential on certain types of tests. (LRW)10acomputerized adaptive testing1 aLatu, E1 aChapman, E uhttp://mail.iacat.org/content/computerised-adaptive-testing00408nas a2200109 4500008004100000245006200041210006200103260001900165100001700184700001300201856008400214 2002 eng d00aFairness issues in adaptive tests with strict time limits0 aFairness issues in adaptive tests with strict time limits aNew Orleans LA1 aBridgeman, B1 aCline, F uhttp://mail.iacat.org/content/fairness-issues-adaptive-tests-strict-time-limits00573nas a2200145 4500008004100000245011700041210006900158300000800227490001000235100001500245700001500260700001300275700001300288856012600301 2002 eng d00aFeasibility and acceptability of computerized adaptive testing (CAT) for fatigue monitoring in clinical practice0 aFeasibility and acceptability of computerized adaptive testing C a1340 v11(7)1 aDavis, K M1 aChang, C-H1 aLai, J-S1 aCella, D uhttp://mail.iacat.org/content/feasibility-and-acceptability-computerized-adaptive-testing-cat-fatigue-monitoring-clinical02014nas a2200205 4500008004100000245008200041210006900123300001200192490000700204520132300211653002101534653001501555653003001570653001101600653000901611653004701620100001901667700001301686856010901699 2002 eng d00aHypergeometric family and item overlap rates in computerized adaptive testing0 aHypergeometric family and item overlap rates in computerized ada a387-3980 v673 aA computerized adaptive test (CAT) is usually administered to small groups of examinees at frequent time intervals. It is often the case that examinees who take the test earlier share information with examinees who will take the test later, thus increasing the risk that many items may become known. Item overlap rate for a group of examinees refers to the number of overlapping items encountered by these examinees divided by the test length. For a specific item pool, different item selection algorithms may yield different item overlap rates. An important issue in designing a good CAT item selection algorithm is to keep item overlap rate below a preset level. In doing so, it is important to investigate what the lowest rate could be for all possible item selection algorithms. In this paper we rigorously prove that if every item had an equal possibility to be selected from the pool in a fixed-length CAT, the number of overlapping item among any α randomly sampled examinees follows the hypergeometric distribution family for α ≥ 1. Thus, the expected values of the number of overlapping items among any randomly sampled α examinee can be calculated precisely. These values may serve as benchmarks in controlling item overlap rates for fixed-length adaptive tests. (PsycINFO Database Record (c) 2005 APA )10aAdaptive Testing10aAlgorithms10aComputer Assisted Testing10aTaking10aTest10aTime On Task computerized adaptive testing1 aChang, Hua-Hua1 aZhang, J uhttp://mail.iacat.org/content/hypergeometric-family-and-item-overlap-rates-computerized-adaptive-testing00484nas a2200109 4500008004100000245009700041210006900138260001900207100001900226700001300245856011600258 2002 eng d00aIdentify the lower bounds for item sharing and item pooling in computerized adaptive testing0 aIdentify the lower bounds for item sharing and item pooling in c aNew Orleans LA1 aChang, Hua-Hua1 aZhang, J uhttp://mail.iacat.org/content/identify-lower-bounds-item-sharing-and-item-pooling-computerized-adaptive-testing00555nas a2200133 4500008004100000245012100041210006900162300001200231490000700243100001500250700001900265700001300284856012400297 2002 eng d00aItem selection in computerized adaptive testing: Improving the a-stratified design with the Sympson-Hetter algorithm0 aItem selection in computerized adaptive testing Improving the as a376-3920 v261 aLeung, C-K1 aChang, Hua-Hua1 aHau, K-T uhttp://mail.iacat.org/content/item-selection-computerized-adaptive-testing-improving-stratified-design-sympson-hetter-000579nas a2200145 4500008004100000020001400041245012100055210006900176300001200245490000700257100001500264700001900279700001300298856012200311 2002 eng d a0146-621600aItem selection in computerized adaptive testing: Improving the a-stratified design with the Sympson-Hetter algorithm0 aItem selection in computerized adaptive testing Improving the as a376-3920 v261 aLeung, C K1 aChang, Hua-Hua1 aHau, K T uhttp://mail.iacat.org/content/item-selection-computerized-adaptive-testing-improving-stratified-design-sympson-hetter02416nas a2200277 4500008004100000245012200041210006900163260000800232300001000240490000700250520148700257653002101744653002101765653002001786653001001806653002201816653002901838653001101867653001801878653001901896653004101915653003201956100001301988700001802001856011902019 2002 eng d00aMeasuring quality of life in chronic illness: the functional assessment of chronic illness therapy measurement system0 aMeasuring quality of life in chronic illness the functional asse cDec aS10-70 v833 aWe focus on quality of life (QOL) measurement as applied to chronic illness. There are 2 major types of health-related quality of life (HRQOL) instruments-generic health status and targeted. Generic instruments offer the opportunity to compare results across patient and population cohorts, and some can provide normative or benchmark data from which to interpret results. Targeted instruments ask questions that focus more on the specific condition or treatment under study and, as a result, tend to be more responsive to clinically important changes than generic instruments. Each type of instrument has a place in the assessment of HRQOL in chronic illness, and consideration of the relative advantages and disadvantages of the 2 options best drives choice of instrument. The Functional Assessment of Chronic Illness Therapy (FACIT) system of HRQOL measurement is a hybrid of the 2 approaches. The FACIT system combines a core general measure with supplemental measures targeted toward specific diseases, conditions, or treatments. Thus, it capitalizes on the strengths of each type of measure. Recently, FACIT questionnaires were administered to a representative sample of the general population with results used to derive FACIT norms. These normative data can be used for benchmarking and to better understand changes in HRQOL that are often seen in clinical trials. Future directions in HRQOL assessment include test equating, item banking, and computerized adaptive testing.10a*Chronic Disease10a*Quality of Life10a*Rehabilitation10aAdult10aComparative Study10aHealth Status Indicators10aHumans10aPsychometrics10aQuestionnaires10aResearch Support, U.S. Gov't, P.H.S.10aSensitivity and Specificity1 aCella, D1 aNowinski, C J uhttp://mail.iacat.org/content/measuring-quality-life-chronic-illness-functional-assessment-chronic-illness-therapy00461nas a2200121 4500008004100000245007300041210006900114260001900183100001400202700001900216700001300235856009100248 2002 eng d00aOptimum number of strata in the a-stratified adaptive testing design0 aOptimum number of strata in the astratified adaptive testing des aNew Orleans LA1 aWen, J -B1 aChang, Hua-Hua1 aHau, K-T uhttp://mail.iacat.org/content/optimum-number-strata-stratified-adaptive-testing-design00460nas a2200109 4500008004100000245008600041210006900127260001900196100001500215700001600230856010400246 2002 eng d00aRedeveloping the exposure control parameters of CAT items when a pool is modified0 aRedeveloping the exposure control parameters of CAT items when a aNew Orleans LA1 aChang, S-W1 aHarris, D J uhttp://mail.iacat.org/content/redeveloping-exposure-control-parameters-cat-items-when-pool-modified00569nas a2200133 4500008004100000245012500041210006900166260001900235100001400254700001800268700001600286700001200302856012100314 2002 eng d00aThe robustness of the unidimensional 3PL IRT model when applied to two-dimensional data in computerized adaptive testing0 arobustness of the unidimensional 3PL IRT model when applied to t aNew Orleans LA1 aZhao, J C1 aMcMorris, R F1 aPruzek, R M1 aChen, R uhttp://mail.iacat.org/content/robustness-unidimensional-3pl-irt-model-when-applied-two-dimensional-data-computerized00538nas a2200109 4500008004100000245005100041210005000092260017700142100001500319700001700334856007700351 2002 eng d00aTest models for complex computer-based testing0 aTest models for complex computerbased testing aC. N. Mille,. M. T. Potenza, J. J. Fremer, and W. C. Ward (Eds.). Computer-based testing: Building the foundation for future assessments (pp. 67-88). Hillsdale NJ: Erlbaum.1 aLuecht, RM1 aClauser, B E uhttp://mail.iacat.org/content/test-models-complex-computer-based-testing00472nas a2200109 4500008004100000245008900041210006900130260001900199100001900218700001200237856011300249 2002 eng d00aTo weight or not to weight – balancing influence of initial and later items in CAT0 aTo weight or not to weight balancing influence of initial and la aNew Orleans LA1 aChang, Hua-Hua1 aYing, Z uhttp://mail.iacat.org/content/weight-or-not-weight-%E2%80%93-balancing-influence-initial-and-later-items-cat00518nas a2200121 4500008004100000245009900041210006900140260001900209100001700228700001600245700001500261856012000276 2002 eng d00aUsing judgments of item difficulty to change answers on computerized adaptive vocabulary tests0 aUsing judgments of item difficulty to change answers on computer aNew Orleans LA1 aVispoel, W P1 aClough, S J1 aBleiler, T uhttp://mail.iacat.org/content/using-judgments-item-difficulty-change-answers-computerized-adaptive-vocabulary-tests00493nas a2200121 4500008004100000245008700041210006900128260001500197100001400212700001900226700001700245856010900262 2001 eng d00aAdaptation of a-stratified method in variable length computerized adaptive testing0 aAdaptation of astratified method in variable length computerized aSeattle WA1 aWen, J -B1 aChang, Hua-Hua1 aHau, K -T   uhttp://mail.iacat.org/content/adaptation-stratified-method-variable-length-computerized-adaptive-testing02225nas a2200145 4500008004100000245011600041210006900157300001100226490000600237520165200243653003401895100001301929700001601942856012101958 2001 eng d00aAssessment in the twenty-first century: A role of computerised adaptive testing in national curriculum subjects0 aAssessment in the twentyfirst century A role of computerised ada a241-570 v53 aWith the investment of large sums of money in new technologies forschools and education authorities and the subsequent training of teachers to integrate Information and Communications Technology (ICT) into their teaching strategies, it is remarkable that the old outdated models of assessment still remain. This article highlights the current problems associated with pen-and paper-testing and offers suggestions for an innovative and new approach to assessment for the twenty-first century. Based on the principle of the 'wise examiner' a computerised adaptive testing system which measures pupils' ability against the levels of the United Kingdom National Curriculum has been developed for use in mathematics. Using constructed response items, pupils are administered a test tailored to their ability with a reliability index of 0.99. Since the software administers maximally informative questions matched to each pupil's current ability estimate, no two pupils will receive the same set of items in the same order therefore removing opportunities for plagarism and teaching to the test. All marking is automated and a journal recording the outcome of the test and highlighting the areas of difficulty for each pupil is available for printing by the teacher. The current prototype of the system can be used on a school's network however the authors envisage a day when Examination Boards or the Qualifications and Assessment Authority (QCA) will administer Government tests from a central server to all United Kingdom schools or testing centres. Results will be issued at the time of testing and opportunities for resits will become more widespr10acomputerized adaptive testing1 aCowan, P1 aMorrison, H uhttp://mail.iacat.org/content/assessment-twenty-first-century-role-computerised-adaptive-testing-national-curriculum00375nas a2200109 4500008004100000245005000041210004800091260002400139100001000163700001900173856007300192 2001 eng d00aa-stratified CAT design with content-blocking0 aastratified CAT design with contentblocking aKing of Prussia, PA1 aYi, Q1 aChang, Hua-Hua uhttp://mail.iacat.org/content/stratified-cat-design-content-blocking00465nas a2200109 4500008004100000245008800041210006900129260001500198100001200213700001900225856011100244 2001 eng d00aa-stratified computerized adaptive testing with unequal item exposure across strata0 aastratified computerized adaptive testing with unequal item expo aSeattle WA1 aDeng, H1 aChang, Hua-Hua uhttp://mail.iacat.org/content/stratified-computerized-adaptive-testing-unequal-item-exposure-across-strata01099nas a2200157 4500008004100000020001400041245007400055210006900129300001200198490000700210520058200217100001900799700001200818700001200830856009900842 2001 eng d a0146-621600aa-Stratified multistage computerized adaptive testing with b blocking0 aaStratified multistage computerized adaptive testing with b bloc a333-3410 v253 aChang & Ying’s (1999) computerized adaptive testing item-selection procedure stratifies the item bank according to a parameter values and requires b parameter values to be evenly distributed across all strata. Thus, a and b parameter values must be incorporated into how strata are formed. A refinement is proposed, based on Weiss’ (1973) stratification of items according to b values. Simulation studies using a retired item bank of a Graduate Record Examination test indicate that the new approach improved control of item exposure rates and reduced mean squared errors. 1 aChang, Hua-Hua1 aQian, J1 aYing, Z uhttp://mail.iacat.org/content/stratified-multistage-computerized-adaptive-testing-b-blocking-000843nas a2200157 4500008004100000245007400041210006900115300001100184490000700195520030900202653003400511100001900545700001200564700001200576856009700588 2001 eng d00aa-stratified multistage computerized adaptive testing with b blocking0 aastratified multistage computerized adaptive testing with b bloc a333-410 v253 aProposed a refinement, based on the stratification of items developed by D. Weiss (1973), of the computerized adaptive testing item selection procedure of H. Chang and Z. Ying (1999). Simulation studies using an item bank from the Graduate Record Examination show the benefits of the new procedure. (SLD)10acomputerized adaptive testing1 aChang, Hua-Hua1 aQian, J1 aYang, Z uhttp://mail.iacat.org/content/stratified-multistage-computerized-adaptive-testing-b-blocking00573nas a2200133 4500008004100000245012700041210006900168260001600237100001700253700001600270700001700286700001300303856012300316 2001 eng d00aCan examinees use judgments of item difficulty to improve proficiency estimates on computerized adaptive vocabulary tests?0 aCan examinees use judgments of item difficulty to improve profic a Seattle WA1 aVispoel, W P1 aClough, S J1 aBleiler, A B1 aIhrig, D uhttp://mail.iacat.org/content/can-examinees-use-judgments-item-difficulty-improve-proficiency-estimates-computerized-000427nas a2200121 4500008004100000245005900041210005900100260001500159100001700174700001900191700001200210856008300222 2001 eng d00aDeriving a stopping rule for sequential adaptive tests0 aDeriving a stopping rule for sequential adaptive tests aSeattle WA1 aGrabovsky, I1 aChang, Hua-Hua1 aYing, Z uhttp://mail.iacat.org/content/deriving-stopping-rule-sequential-adaptive-tests00553nas a2200157 4500008004100000245008100041210006900122300001200191490000700203100002000210700001600230700001600246700001700262700001400279856010200293 2001 eng d00aDevelopment of an adaptive multimedia program to collect patient health data0 aDevelopment of an adaptive multimedia program to collect patient a320-3240 v211 aSutherland, L A1 aCampbell, M1 aOrnstein, K1 aWildemuth, B1 aLobach, D uhttp://mail.iacat.org/content/development-adaptive-multimedia-program-collect-patient-health-data00483nas a2200109 4500008003900000245010300039210006900142260001500211100001500226700001300241856011900254 2001 d00aEffects of changes in the examinees’ ability distribution on the exposure control methods in CAT0 aEffects of changes in the examinees ability distribution on the  aSeattle WA1 aChang, S-W1 aTwu, B-Y uhttp://mail.iacat.org/content/effects-changes-examinees%E2%80%99-ability-distribution-exposure-control-methods-cat00532nas a2200121 4500008004100000245011100041210006900152260001500221100001500236700001900251700001300270856012700283 2001 eng d00aAn examination of item selection rules by stratified CAT designs integrated with content balancing methods0 aexamination of item selection rules by stratified CAT designs in aSeattle WA1 aLeung, C-K1 aChang, Hua-Hua1 aHau, K-T uhttp://mail.iacat.org/content/examination-item-selection-rules-stratified-cat-designs-integrated-content-balancing-methods00683nas a2200133 4500008004100000245001800041210001700059300001000076490000800086520036100094653003400455100001300489856004700502 2001 eng d00aFinal answer?0 aFinal answer a24-260 v1883 aThe Northwest Evaluation Association helped an Indiana school district develop a computerized adaptive testing system that was aligned with its curriculum and geared toward measuring individual student growth. Now the district can obtain such information from semester to semester and year to year, get immediate results, and test students on demand. (MLH)10acomputerized adaptive testing1 aCoyle, J uhttp://mail.iacat.org/content/final-answer00590nas a2200109 4500008004100000245012400041210006900165260008200234100001900316700002300335856012200358 2001 eng d00aImplementing content constraints in a-stratified adaptive testing using a shadow test approach (Research Report 01-001)0 aImplementing content constraints in astratified adaptive testing aUniversity of Twente, Department of Educational Measurement and Data Analysis1 aChang, Hua-Hua1 avan der Linden, WJ uhttp://mail.iacat.org/content/implementing-content-constraints-stratified-adaptive-testing-using-shadow-test-approach00495nas a2200121 4500008004100000245008700041210006900128260001500197100001600212700001900228700001300247856011300260 2001 eng d00aIntegrating stratification and information approaches for multiple constrained CAT0 aIntegrating stratification and information approaches for multip aSeattle WA1 aLeung, C -I1 aChang, Hua-Hua1 aHau, K-T uhttp://mail.iacat.org/content/integrating-stratification-and-information-approaches-multiple-constrained-cat01986nas a2200205 4500008004100000245010100041210006900142300001200211490000700223520124900230653001201479653002101491653003001512653001501542653001601557653004501573100001701618700001901635856012601654 2001 eng d00aItem selection in computerized adaptive testing: Should more discriminating items be used first?0 aItem selection in computerized adaptive testing Should more disc a249-2660 v383 aDuring computerized adaptive testing (CAT), items are selected continuously according to the test-taker's estimated ability. Test security has become a problem because high-discrimination items are more likely to be selected and become overexposed. So, there seems to be a tradeoff between high efficiency in ability estimations and balanced usage of items. This series of four studies addressed the dilemma by focusing on the notion of whether more or less discriminating items should be used first in CAT. The first study demonstrated that the common maximum information method with J. B. Sympson and R. D. Hetter (1985) control resulted in the use of more discriminating items first. The remaining studies showed that using items in the reverse order, as described in H. Chang and Z. Yings (1999) stratified method had potential advantages: (a) a more balanced item usage and (b) a relatively stable resultant item pool structure with easy and inexpensive management. This stratified method may have ability-estimation efficiency better than or close to that of other methods. It is argued that the judicious selection of items, as in the stratified method, is a more active control of item exposure. (PsycINFO Database Record (c) 2005 APA )10aability10aAdaptive Testing10aComputer Assisted Testing10aEstimation10aStatistical10aTest Items computerized adaptive testing1 aHau, Kit-Tai1 aChang, Hua-Hua uhttp://mail.iacat.org/content/item-selection-computerized-adaptive-testing-should-more-discriminating-items-be-used-first00498nas a2200145 4500008004100000245007200041210006900113300001000182490000700192100001400199700001400213700001900227700001200246856009400258 2001 eng d00aOn maximizing item information and matching difficulty with ability0 amaximizing item information and matching difficulty with ability a69-770 v661 aBickel, P1 aBuyske, S1 aChang, Hua-Hua1 aYing, Z uhttp://mail.iacat.org/content/maximizing-item-information-and-matching-difficulty-ability00451nas a2200109 4500008004100000245007700041210006900118260002100187100001400208700001500222856010400237 2001 eng d00aMeasurement efficiency of multidimensional computerized adaptive testing0 aMeasurement efficiency of multidimensional computerized adaptive aSan Francisco CA1 aWang, W-C1 aChen, B -H uhttp://mail.iacat.org/content/measurement-efficiency-multidimensional-computerized-adaptive-testing00407nas a2200097 4500008004100000245007400041210006900115260001500184100001400199856009600213 2001 eng d00aA new approach to simulation studies in computerized adaptive testing0 anew approach to simulation studies in computerized adaptive test aSeattle WA1 aChen, S-Y uhttp://mail.iacat.org/content/new-approach-simulation-studies-computerized-adaptive-testing00522nas a2200121 4500008004100000245010800041210006900149260002400218100001100242700001300253700001200266856012200278 2001 eng d00aOn-line Calibration Using PARSCALE Item Specific Prior Method: Changing Test Population and Sample Size0 aOnline Calibration Using PARSCALE Item Specific Prior Method Cha aSeattle, Washington1 aGuo, F1 aStone, E1 aCruz, D uhttp://mail.iacat.org/content/line-calibration-using-parscale-item-specific-prior-method-changing-test-population-and01349nas a2200181 4500008004100000245007300041210006900114260005600183300001200239520069300251653002100944653003000965653002200995653002001017100001601037700001701053856009701070 2001 eng d00aPractical issues in setting standards on computerized adaptive tests0 aPractical issues in setting standards on computerized adaptive t aMahwah, N.J. USAbLawrence Erlbaum Associates, Inc. a355-3693 a(From the chapter) Examples of setting standards on computerized adaptive tests (CATs) are hard to find. Some examples of CATs involving performance standards include the registered nurse exam and the Novell systems engineer exam. Although CATs do not require separate standard setting-methods, there are special issues to be addressed by test specialist who set performance standards on CATs. Setting standards on a CAT will typical require modifications on the procedures used with more traditional, fixed-form, paper-and -pencil examinations. The purpose of this chapter is to illustrate why CATs pose special challenges to the standard setter. (PsycINFO Database Record (c) 2005 APA )10aAdaptive Testing10aComputer Assisted Testing10aPerformance Tests10aTesting Methods1 aSireci, S G1 aClauser, B E uhttp://mail.iacat.org/content/practical-issues-setting-standards-computerized-adaptive-tests01390nas a2200193 4500008004100000245009400041210006900135300001200204490000700216520067900223653002100902653003000923653002500953653005700978100001401035700001901049700001901068856010901087 2000 eng d00aA comparison of item selection rules at the early stages of computerized adaptive testing0 acomparison of item selection rules at the early stages of comput a241-2550 v243 aThe effects of 5 item selection rules--Fisher information (FI), Fisher interval information (FII), Fisher information with a posterior distribution (FIP), Kullback-Leibler information (KL), and Kullback-Leibler information with a posterior distribution (KLP)--were compared with respect to the efficiency and precision of trait (θ) estimation at the early stages of computerized adaptive testing (CAT). FII, FIP, KL, and KLP performed marginally better than FI at the early stages of CAT for θ=-3 and -2. For tests longer than 10 items, there appeared to be no precision advantage for any of the selection rules. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10aAdaptive Testing10aComputer Assisted Testing10aItem Analysis (Test)10aStatistical Estimation computerized adaptive testing1 aChen, S-Y1 aAnkenmann, R D1 aChang, Hua-Hua uhttp://mail.iacat.org/content/comparison-item-selection-rules-early-stages-computerized-adaptive-testing00520nas a2200133 4500008004100000245009400041210006900135300001200204490000700216100001400223700001900237700001900256856011100275 2000 eng d00aA comparison of item selection rules at the early stages of computerized adaptive testing0 acomparison of item selection rules at the early stages of comput a241-2550 v241 aChen, S Y1 aAnkenmann, R D1 aChang, Hua-Hua uhttp://mail.iacat.org/content/comparison-item-selection-rules-early-stages-computerized-adaptive-testing-000474nas a2200121 4500008004100000245007400041210006900115260002000184100001500204700001900219700001300238856010100251 2000 eng d00aContent balancing in stratified computerized adaptive testing designs0 aContent balancing in stratified computerized adaptive testing de aNew Orleans, LA1 aLeung, C-K1 aChang, Hua-Hua1 aHau, K-T uhttp://mail.iacat.org/content/content-balancing-stratified-computerized-adaptive-testing-designs00408nas a2200121 4500008004100000245005400041210005300095300001200148490000700160100001700167700001900184856008300203 2000 eng d00aDoes adaptive testing violate local independence?0 aDoes adaptive testing violate local independence a149-1560 v651 aMislevy, R J1 aChang, Hua-Hua uhttp://mail.iacat.org/content/does-adaptive-testing-violate-local-independence00588nas a2200133 4500008004100000245014200041210006900183260002700252100001600279700001600295700001400311700001300325856011600338 2000 eng d00aEstimating item parameters from classical indices for item pool development with a computerized classification test (ACT Research 2000-4)0 aEstimating item parameters from classical indices for item pool  aIowa City IA, ACT, Inc1 aChang, C -Y1 aKalohn, J C1 aLin, C -J1 aSpray, J uhttp://mail.iacat.org/content/estimating-item-parameters-classical-indices-item-pool-development-computerized-001433nas a2200193 4500008004100000245006300041210006300104300001200167490000700179520079500186653001800981653002100999653003001020653001801050653005701068100001501125700001201140856008701152 2000 eng d00aEstimation of trait level in computerized adaptive testing0 aEstimation of trait level in computerized adaptive testing a257-2650 v243 aNotes that in computerized adaptive testing (CAT), a examinee's trait level (θ) must be estimated with reasonable accuracy based on a small number of item responses. A successful implementation of CAT depends on (1) the accuracy of statistical methods used for estimating θ and (2) the efficiency of the item-selection criterion. Methods of estimating θ suitable for CAT are reviewed, and the differences between Fisher and Kullback-Leibler information criteria for selecting items are discussed. The accuracy of different CAT algorithms was examined in an empirical study. The results show that correcting θ estimates for bias was necessary at earlier stages of CAT, but most CAT algorithms performed equally well for tests of 10 or more items. (PsycINFO Database Record (c) 2005 APA )10a(Statistical)10aAdaptive Testing10aComputer Assisted Testing10aItem Analysis10aStatistical Estimation computerized adaptive testing1 aCheng, P E1 aLiou, M uhttp://mail.iacat.org/content/estimation-trait-level-computerized-adaptive-testing00390nas a2200109 4500008004100000245006200041210006200103300000800165490000700173100001500180856008500195 2000 eng d00aETS finds flaws in the way online GRE rates some students0 aETS finds flaws in the way online GRE rates some students aa470 v471 aCarlson, S uhttp://mail.iacat.org/content/ets-finds-flaws-way-online-gre-rates-some-students00610nas a2200145 4500008004100000245013100041210006900172260002000241100001400261700001600275700001400291700001400305700001900319856012600338 2000 eng d00aAn examination of exposure control and content balancing restrictions on item selection in CATs using the partial credit model0 aexamination of exposure control and content balancing restrictio aNew Orleans, LA1 aDavis, LL1 aPastor, D A1 aDodd, B G1 aChiang, C1 aFitzpatrick, S uhttp://mail.iacat.org/content/examination-exposure-control-and-content-balancing-restrictions-item-selection-cats-using-000415nas a2200109 4500008004100000245006100041210006100102260002700163100001000190700001900200856008600219 2000 eng d00aMultiple stratification CAT designs with content control0 aMultiple stratification CAT designs with content control aUnpublished manuscript1 aYi, Q1 aChang, Hua-Hua uhttp://mail.iacat.org/content/multiple-stratification-cat-designs-content-control00527nas a2200121 4500008004100000245010400041210006900145260002100214100001900235700001300254700001100267856012700278 2000 eng d00aPerformance of item exposure control methods in computerized adaptive testing: Further explorations0 aPerformance of item exposure control methods in computerized ada aNew Orleans , LA1 aChang, Hua-Hua1 aChang, S1 aAnsley uhttp://mail.iacat.org/content/performance-item-exposure-control-methods-computerized-adaptive-testing-further-explorations00497nas a2200121 4500008004100000245008600041210006900127260002100196100001500217700001900232700001300251856011100264 2000 eng d00aSolving complex constraints in a-stratified computerized adaptive testing designs0 aSolving complex constraints in astratified computerized adaptive aNew Orleans, USA1 aLeung, C-K1 aChang, Hua-Hua1 aHau, K-T uhttp://mail.iacat.org/content/solving-complex-constraints-stratified-computerized-adaptive-testing-designs00547nas a2200133 4500008004100000245005900041210005900100260009900159100001400258700001400272700002700286700001400313856008600327 2000 eng d00aUsing Bayesian Networks in Computerized Adaptive Tests0 aUsing Bayesian Networks in Computerized Adaptive Tests aM. Ortega and J. Bravo (Eds.),Computers and Education in the 21st Century. Kluwer, pp. 217228.1 aMillan, E1 aTrella, M1 aPerez-de-la-Cruz, J -L1 aConejo, R uhttp://mail.iacat.org/content/using-bayesian-networks-computerized-adaptive-tests00380nas a2200097 4500008004100000245006000041210006000101100001800161700001600179856008700195 2000 eng d00aUsing constraints to develop and deliver adaptive tests0 aUsing constraints to develop and deliver adaptive tests1 aAbdullah, S C1 aCooley, R E uhttp://mail.iacat.org/content/using-constraints-develop-and-deliver-adaptive-tests00597nas a2200109 4500008004100000245016200041210006900203260006200272100001700334700001300351856012300364 2000 eng d00aVariations in mean response times for questions on the computer-adaptive GRE general test: Implications for fair assessment (GRE Board Professional Report No0 aVariations in mean response times for questions on the computera a96-20P: Educational Testing Service Research Report 00-7)1 aBridgeman, B1 aCline, F uhttp://mail.iacat.org/content/variations-mean-response-times-questions-computer-adaptive-gre-general-test-implications01735nas a2200145 4500008004100000245005800041210005700099300001200156490000700168520126300175653003401438100001901472700001201491856008601503 1999 eng d00aa-stratified multistage computerized adaptive testing0 aastratified multistage computerized adaptive testing a211-2220 v233 aFor computerized adaptive tests (CAT) based on the three-parameter logistic mode it was found that administering items with low discrimination parameter (a) values early in the test and administering those with high a values later was advantageous; the skewness of item exposure distributions was reduced while efficiency was maintain in trait level estimation. Thus, a new multistage adaptive testing approach is proposed that factors a into the item selection process. In this approach, the items in the item bank are stratified into a number of levels based on their a values. The early stages of a test use items with lower as and later stages use items with higher as. At each stage, items are selected according to an optimization criterion from the corresponding level. Simulation studies were performed to compare a-stratified CATs with CATs based on the Sympson-Hetter method for controlling item exposure. Results indicated that this new strategy led to tests that were well-balanced, with respect to item exposure, and efficient. The a-stratified CATs achieved a lower average exposure rate than CATs based on Bayesian or information-based item selection and the Sympson-Hetter method. (PsycINFO Database Record (c) 2003 APA, all rights reserved).10acomputerized adaptive testing1 aChang, Hua-Hua1 aYing, Z uhttp://mail.iacat.org/content/stratified-multistage-computerized-adaptive-testing00418nas a2200121 4500008004100000245005800041210005700099300001400156490000700170100001900177700001200196856008800208 1999 eng d00aa-stratified multistage computerized adaptive testing0 aastratified multistage computerized adaptive testing a211–2220 v231 aChang, Hua-Hua1 aYing, Z uhttp://mail.iacat.org/content/stratified-multistage-computerized-adaptive-testing-000416nas a2200121 4500008004100000245005800041210005700099300001200156490000700168100001900175700001200194856008800206 1999 eng d00aa-stratified multistage computerized adaptive testing0 aastratified multistage computerized adaptive testing a211-2220 v231 aChang, Hua-Hua1 aYing, Z uhttp://mail.iacat.org/content/stratified-multistage-computerized-adaptive-testing-100447nas a2200121 4500008004100000245006400041210006100105260002100166100001500187700001900202700001300221856009100234 1999 eng d00aAn enhanced stratified computerized adaptive testing design0 aenhanced stratified computerized adaptive testing design aMontreal, Canada1 aLeung, C-K1 aChang, Hua-Hua1 aHau, K-T uhttp://mail.iacat.org/content/enhanced-stratified-computerized-adaptive-testing-design00622nas a2200121 4500008004100000245011300041210006900154260011200223100001200335700001900347700001500366856011900381 1999 eng d00aExploring the relationship between item exposure rate and test overlap rate in computerized adaptive testing0 aExploring the relationship between item exposure rate and test o aPaper presented at the annual meeting of the National Council on Measurement in Education, Montreal, Canada1 aChen, S1 aAnkenmann, R D1 aSpray, J A uhttp://mail.iacat.org/content/exploring-relationship-between-item-exposure-rate-and-test-overlap-rate-computerized00575nas a2200121 4500008004100000245014700041210006900188260002700257100001400284700001900298700001500317856012100332 1999 eng d00aExploring the relationship between item exposure rate and test overlap rate in computerized adaptive testing (ACT Research Report series 99-5)0 aExploring the relationship between item exposure rate and test o aIowa City IA: ACT, Inc1 aChen, S-Y1 aAnkenmann, R D1 aSpray, J A uhttp://mail.iacat.org/content/exploring-relationship-between-item-exposure-rate-and-test-overlap-rate-computerized-000379nas a2200121 4500008004100000245003900041210003800080260002100118100002000139700001700159700001500176856006600191 1999 eng d00aFairness in computer-based testing0 aFairness in computerbased testing aMontreal, Canada1 aGallagher, Aand1 aBridgeman, B1 aCalahan, C uhttp://mail.iacat.org/content/fairness-computer-based-testing00541nas a2200121 4500008004100000245012100041210006900162260001700231100001500248700001900263700001300282856012400295 1999 eng d00aItem selection in computerized adaptive testing: improving the a-stratified design with the Sympson-Hetter algorithm0 aItem selection in computerized adaptive testing improving the as aMontreal, CA1 aLeung, C-K1 aChang, Hua-Hua1 aHau, K-T uhttp://mail.iacat.org/content/item-selection-computerized-adaptive-testing-improving-stratified-design-sympson-hetter-100525nas a2200133 4500008004100000245009300041210006900134260002100203100001600224700001400240700001400254700001400268856010900282 1999 eng d00aPerformance of the Sympson-Hetter exposure control algorithm with a polytomous item bank0 aPerformance of the SympsonHetter exposure control algorithm with aMontreal, Canada1 aPastor, D A1 aChiang, C1 aDodd, B G1 aYockey, R uhttp://mail.iacat.org/content/performance-sympson-hetter-exposure-control-algorithm-polytomous-item-bank00357nas a2200097 4500008004100000245005900041210005200100260002100152100001500173856007100188 1999 eng d00aThe use of linear-on-the-fly testing for TOEFL Reading0 ause of linearonthefly testing for TOEFL Reading aMontreal, Canada1 aCarey, P A uhttp://mail.iacat.org/content/use-linear-fly-testing-toefl-reading00508nas a2200097 4500008004100000245013100041210006900172260003400241100001000275856012500285 1999 eng d00aWISCAT: Een computergestuurd toetspakket voor rekenen en wiskunde [A computerized test package for arithmetic and mathematics]0 aWISCAT Een computergestuurd toetspakket voor rekenen en wiskunde aCito: Arnhem, The Netherlands1 aCito. uhttp://mail.iacat.org/content/wiscat-een-computergestuurd-toetspakket-voor-rekenen-en-wiskunde-computerized-test-package01457nas a2200133 4500008004100000245006700041210006700108300000900175490000700184520098800191653003401179100001901213856009101232 1998 eng d00aApplications of network flows to computerized adaptive testing0 aApplications of network flows to computerized adaptive testing a08550 v593 aRecently, the concept of Computerized Adaptive Testing (CAT) has been receiving ever growing attention from the academic community. This is so because of both practical and theoretical considerations. Its practical importance lies in the advantages of CAT over the traditional (perhaps outdated) paper-and-pencil test in terms of time, accuracy, and money. The theoretical interest is sparked by its natural relationship to Item Response Theory (IRT). This dissertation offers a mathematical programming approach which creates a model that generates a CAT that takes care of many questions concerning the test, such as feasibility, accuracy and time of testing, as well as item pool security. The CAT generated is designed to obtain the most information about a single test taker. Several methods for eatimating the examinee's ability, based on the (dichotomous) responses to the items in the test, are also offered here. (PsycINFO Database Record (c) 2003 APA, all rights reserved).10acomputerized adaptive testing1 aClaudio, M J C uhttp://mail.iacat.org/content/applications-network-flows-computerized-adaptive-testing00487nam a2200097 4500008004100000245006700041210006700108260010400175100001700279856009300296 1998 eng d00aApplications of network flows to computerized adaptive testing0 aApplications of network flows to computerized adaptive testing aDissertation, Rutgers Center for Operations Research (RUTCOR), Rutgers University, New Brunswick NJ1 aCordova, M J uhttp://mail.iacat.org/content/applications-network-flows-computerized-adaptive-testing-001292nas a2200145 4500008004100000245007300041210006900114300001200183490000700195520080400202100001701006700001501023700001101038856009701049 1998 eng d00aBayesian identification of outliers in computerized adaptive testing0 aBayesian identification of outliers in computerized adaptive tes a910-9190 v933 aWe consider the problem of identifying examinees with aberrant response patterns in a computerized adaptive test (CAT). The vec-tor of responses yi of person i from the CAT comprise a multivariate response vector. Multivariate observations may be outlying in manydi erent directions and we characterize speci c directions as corre- sponding to outliers with different interpretations. We develop a class of outlier statistics to identify different types of outliers based on a con-trol chart type methodology. The outlier methodology is adaptable to general longitudinal discrete data structures. We consider several procedures to judge how extreme a particular outlier is. Data from the National Council Licensure EXamination (NCLEX) motivates our development and is used to illustrate the results.1 aBradlow, E T1 aWeiss, R E1 aCho, M uhttp://mail.iacat.org/content/bayesian-identification-outliers-computerized-adaptive-testing00325nas a2200121 4500008004100000245002500041210002500066260001400091100001100105700001800116700001400134856005500148 1998 eng d00aCAT item calibration0 aCAT item calibration aSan Diego1 aHsu, Y1 aThompson, T D1 aChen, W-H uhttp://mail.iacat.org/content/cat-item-calibration00522nas a2200109 4500008004100000245009000041210006900131260007000200100001500270700001300285856011400298 1998 eng d00aA comparative study of item exposure control methods in computerized adaptive testing0 acomparative study of item exposure control methods in computeriz aResearch Report Series 98-3, Iowa City: American College Testing.1 aChang, S-W1 aTwu, B-Y uhttp://mail.iacat.org/content/comparative-study-item-exposure-control-methods-computerized-adaptive-testing-000500nam a2200097 4500008004100000245009000041210006900131260007300200100001500273856011400288 1998 eng d00aA comparative study of item exposure control methods in computerized adaptive testing0 acomparative study of item exposure control methods in computeriz aUnpublished doctoral dissertation, University of Iowa , Iowa City IA1 aChang, S-W uhttp://mail.iacat.org/content/comparative-study-item-exposure-control-methods-computerized-adaptive-testing-100545nas a2200133 4500008004100000245012500041210006900166300001200235490000700247100001200254700001100266700001400277856012000291 1998 eng d00aA comparison of maximum likelihood estimation and expected a posteriori estimation in CAT using the partial credit model0 acomparison of maximum likelihood estimation and expected a poste a569-5950 v581 aChen, S1 aHou, L1 aDodd, B G uhttp://mail.iacat.org/content/comparison-maximum-likelihood-estimation-and-expected-posteriori-estimation-cat-using00471nas a2200109 4500008004100000245009400041210006900135100001200204700001300216700001900229856011300248 1998 eng d00aA comparison of two methods of controlling item exposure in computerized adaptive testing0 acomparison of two methods of controlling item exposure in comput1 aTang, L1 aJiang, H1 aChang, Hua-Hua uhttp://mail.iacat.org/content/comparison-two-methods-controlling-item-exposure-computerized-adaptive-testing00485nas a2200109 4500008004100000245007800041210006900119260004600188100001700234700001900251856010500270 1998 eng d00aDoes adaptive testing violate local independence? (Research Report 98-33)0 aDoes adaptive testing violate local independence Research Report aPrinceton NJ: Educational Testing Service1 aMislevy, R J1 aChang, Hua-Hua uhttp://mail.iacat.org/content/does-adaptive-testing-violate-local-independence-research-report-98-3300511nas a2200121 4500008004100000245010100041210006900142300001200211490000700223100001300230700001900243856012700262 1998 eng d00aItem selection in computerized adaptive testing: Should more discriminating items be used first?0 aItem selection in computerized adaptive testing Should more disc a249-2660 v381 aHau, K T1 aChang, Hua-Hua uhttp://mail.iacat.org/content/item-selection-computerized-adaptive-testing-should-more-discriminating-items-be-used-firs-000570nas a2200097 4500008004100000245020300041210006900244100001300313700001900326856012700345 1998 eng d00aItem selection in computerized adaptive testing: Should more discriminating items be used first? Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA0 aItem selection in computerized adaptive testing Should more disc1 aHau, K T1 aChang, Hua-Hua uhttp://mail.iacat.org/content/item-selection-computerized-adaptive-testing-should-more-discriminating-items-be-used-firs-101241nas a2200157 4500008004100000245006600041210006600107300001000173490000600183520071600189653003400905100001500939700001700954700001900971856009300990 1998 eng d00aMaintaining content validity in computerized adaptive testing0 aMaintaining content validity in computerized adaptive testing a29-410 v33 aThe authors empirically demonstrate some of the trade-offs which can occur when content balancing is imposed in computerized adaptive testing (CAT) forms or conversely, when it is ignored. The authors contend that the content validity of a CAT form can actually change across a score scale when content balancing is ignored. However they caution that, efficiency and score precision can be severely reduced by over specifying content restrictions in a CAT form. The results from 2 simulation studies are presented as a means of highlighting some of the trade-offs that could occur between content and statistical considerations in CAT form assembly. (PsycINFO Database Record (c) 2003 APA, all rights reserved).10acomputerized adaptive testing1 aLuecht, RM1 aChamplain, A1 aNungester, R J uhttp://mail.iacat.org/content/maintaining-content-validity-computerized-adaptive-testing00524nas a2200121 4500008004100000245012000041210006900161300001200230490000600242100001600248700001700264856012100281 1998 eng d00aSwedish Enlistment Battery: Construct validity and latent variable estimation of cognitive abilities by the CAT-SEB0 aSwedish Enlistment Battery Construct validity and latent variabl a107-1140 v61 aMardberg, B1 aCarlstedt, B uhttp://mail.iacat.org/content/swedish-enlistment-battery-construct-validity-and-latent-variable-estimation-cognitive00415nas a2200109 4500008004100000245003600041210003500077260009900112100001700211700001600228856006100244 1997 eng d00aCAST 5 for Windows users' guide0 aCAST 5 for Windows users guide aContract No. "MDA903-93-D-0032, DO 0054. Alexandria, VA: Human Resources Research Organization1 aMcBride, J R1 aCooper, R R uhttp://mail.iacat.org/content/cast-5-windows-users-guide00536nas a2200121 4500008004100000245004000041210003900081260017700120100001400297700001600311700001700327856007000344 1997 eng d00aCAT-ASVAB cost and benefit analyses0 aCATASVAB cost and benefit analyses aW. A. Sands, B. K. Waters, and J. R. McBride (Eds.), Computer adaptive testing: From inquiry to operation (pp. 227-236). Washington, DC: American Psychological Association.1 aWise, L L1 aCurran, L T1 aMcBride, J R uhttp://mail.iacat.org/content/cat-asvab-cost-and-benefit-analyses02916nas a2200133 4500008004100000245016300041210006900204300000800273490000700281520232300288653003402611100001402645856012302659 1997 eng d00aA comparison of maximum likelihood estimation and expected a posteriori estimation in computerized adaptive testing using the generalized partial credit model0 acomparison of maximum likelihood estimation and expected a poste a4530 v583 aA simulation study was conducted to investigate the application of expected a posteriori (EAP) trait estimation in computerized adaptive tests (CAT) based on the generalized partial credit model (Muraki, 1992), and to compare the performance of EAP with maximum likelihood trait estimation (MLE). The performance of EAP was evaluated under different conditions: the number of quadrature points (10, 20, and 30), and the type of prior distribution (normal, uniform, negatively skewed, and positively skewed). The relative performance of the MLE and EAP estimation methods were assessed under two distributional forms of the latent trait, one normal and the other negatively skewed. Also, both the known item parameters and estimated item parameters were employed in the simulation study. Descriptive statistics, correlations, scattergrams, accuracy indices, and audit trails were used to compare the different methods of trait estimation in CAT. The results showed that, regardless of the latent trait distribution, MLE and EAP with a normal prior, a uniform prior, or the prior that matches the latent trait distribution using either 20 or 30 quadrature points provided relatively accurate estimation in CAT based on the generalized partial credit model. However, EAP using only 10 quadrature points did not work well in the generalized partial credit CAT. Also, the study found that increasing the number of quadrature points from 20 to 30 did not increase the accuracy of EAP estimation. Therefore, it appears 20 or more quadrature points are sufficient for accurate EAP estimation. The results also showed that EAP with a negatively skewed prior and positively skewed prior performed poorly for the normal data set, and EAP with positively skewed prior did not provide accurate estimates for the negatively skewed data set. Furthermore, trait estimation in CAT using estimated item parameters produced results similar to those obtained using known item parameters. In general, when at least 20 quadrature points are used, EAP estimation with a normal prior, a uniform prior or the prior that matches the latent trait distribution appears to be a good alternative to MLE in the application of polytomous CAT based on the generalized partial credit model. (PsycINFO Database Record (c) 2003 APA, all rights reserved).10acomputerized adaptive testing1 aChen, S-K uhttp://mail.iacat.org/content/comparison-maximum-likelihood-estimation-and-expected-posteriori-estimation-computerized00403nas a2200109 4500008004100000245006200041210006200103260001500165100001400180700001500194856008400209 1997 eng d00aComputer assembly of tests so that content reigns supreme0 aComputer assembly of tests so that content reigns supreme aChicago IL1 aCase, S M1 aLuecht, RM uhttp://mail.iacat.org/content/computer-assembly-tests-so-content-reigns-supreme01800nas a2200169 4500008004100000245014100041210006900182300001200251490000700263520113700270653003401407100001401441700001301455700002101468700001401489856012701503 1997 eng d00aThe effect of population distribution and method of theta estimation on computerized adaptive testing (CAT) using the rating scale model0 aeffect of population distribution and method of theta estimation a422-4390 v573 aInvestigated the effect of population distribution on maximum likelihood estimation (MLE) and expected a posteriori estimation (EAP) in a simulation study of computerized adaptive testing (CAT) based on D. Andrich's (1978) rating scale model. Comparisons were made among MLE and EAP with a normal prior distribution and EAP with a uniform prior distribution within 2 data sets: one generated using a normal trait distribution and the other using a negatively skewed trait distribution. Descriptive statistics, correlations, scattergrams, and accuracy indices were used to compare the different methods of trait estimation. The EAP estimation with a normal prior or uniform prior yielded results similar to those obtained with MLE, even though the prior did not match the underlying trait distribution. An additional simulation study based on real data suggested that more work is needed to determine the optimal number of quadrature points for EAP in CAT based on the rating scale model. The choice between MLE and EAP for particular measurement situations is discussed. (PsycINFO Database Record (c) 2003 APA, all rights reserved).10acomputerized adaptive testing1 aChen, S-K1 aHou, L Y1 aFitzpatrick, S J1 aDodd, B G uhttp://mail.iacat.org/content/effect-population-distribution-and-method-theta-estimation-computerized-adaptive-testing-cat00597nas a2200145 4500008004100000245014200041210006900183300001200252490000700264100001200271700001100283700002100294700001200315856012400327 1997 eng d00aThe effect of population distribution and methods of theta estimation on computerized adaptive testing (CAT) using the rating scale model0 aeffect of population distribution and methods of theta estimatio a422-4390 v571 aChen, S1 aHou, L1 aFitzpatrick, S J1 aDodd, B uhttp://mail.iacat.org/content/effect-population-distribution-and-methods-theta-estimation-computerized-adaptive-testing02399nas a2200253 4500008004100000020002200041245012400063210006900187250001500256260000800271300001200279490000600291520152400297653001901821653003001840653002101870653003301891653002401924653001101948653002701959100001701986700001502003856012702018 1997 eng d a0962-9343 (Print)00aHealth status assessment for the twenty-first century: item response theory, item banking and computer adaptive testing0 aHealth status assessment for the twentyfirst century item respon a1997/08/01 cAug a595-6000 v63 aHealth status assessment is frequently used to evaluate the combined impact of human immunodeficiency virus (HIV) disease and its treatment on functioning and well-being from the patient's perspective. No single health status measure can efficiently cover the range of problems in functioning and well-being experienced across HIV disease stages. Item response theory (IRT), item banking and computer adaptive testing (CAT) provide a solution to measuring health-related quality of life (HRQoL) across different stages of HIV disease. IRT allows us to examine the response characteristics of individual items and the relationship between responses to individual items and the responses to each other item in a domain. With information on the response characteristics of a large number of items covering a HRQoL domain (e.g. physical function, and psychological well-being), and information on the interrelationships between all pairs of these items and the total scale, we can construct more efficient scales. Item banks consist of large sets of questions representing various levels of a HRQoL domain that can be used to develop brief, efficient scales for measuring the domain. CAT is the application of IRT and item banks to the tailored assessment of HRQoL domains specific to individual patients. Given the results of IRT analyses and computer-assisted test administration, more efficient and brief scales can be used to measure multiple domains of HRQoL for clinical trials and longitudinal observational studies.10a*Health Status10a*HIV Infections/diagnosis10a*Quality of Life10aDiagnosis, Computer-Assisted10aDisease Progression10aHumans10aPsychometrics/*methods1 aRevicki, D A1 aCella, D F uhttp://mail.iacat.org/content/health-status-assessment-twenty-first-century-item-response-theory-item-banking-and-computer00631nas a2200109 4500008004100000245011500041210006900156260013700225100001700362700001600379856012600395 1997 eng d00aModification of the Computerized Adaptive Screening Test (CAST) for use by recruiters in all military services0 aModification of the Computerized Adaptive Screening Test CAST fo aFinal Technical Report FR-WATSD-97-24, Contract No. MDA903-93-D-0032, DO 0054. Alexandria VA: Human Resources Research Organization.1 aMcBride, J R1 aCooper, R R uhttp://mail.iacat.org/content/modification-computerized-adaptive-screening-test-cast-use-recruiters-all-military-services00323nas a2200097 4500008004100000245004300041210004200084100001900126700001200145856006800157 1997 eng d00aMulti-stage CAT with stratified design0 aMultistage CAT with stratified design1 aChang, Hua-Hua1 aYing, Z uhttp://mail.iacat.org/content/multi-stage-cat-stratified-design00488nas a2200097 4500008004100000245012300041210006900164100001900233700001200252856012600264 1997 eng d00aNonlinear sequential designs for logistic item response theory models with applications to computerized adaptive tests0 aNonlinear sequential designs for logistic item response theory m1 aChang, Hua-Hua1 aYing, Z uhttp://mail.iacat.org/content/nonlinear-sequential-designs-logistic-item-response-theory-models-applications-computerized00433nam a2200097 4500008004100000245005800041210005800099260007600157100001700233856008500250 1997 eng d00aOptimization methods in computerized adaptive testing0 aOptimization methods in computerized adaptive testing aUnpublished doctoral dissertation, Rutgers University, New Brunswick NJ1 aCordova, M J uhttp://mail.iacat.org/content/optimization-methods-computerized-adaptive-testing00608nas a2200145 4500008004100000245012900041210006900170260001500239100002000254700002000274700001400294700001500308700001700323856012200340 1997 eng d00aRelationship of response latency to test design, examinee ability, and item difficulty in computer-based test administration0 aRelationship of response latency to test design examinee ability aChicago IL1 aSwanson, D., B.1 aFeatherman, C M1 aCase, A M1 aLuecht, RM1 aNungester, R uhttp://mail.iacat.org/content/relationship-response-latency-test-design-examinee-ability-and-item-difficulty-computer00548nas a2200121 4500008004100000245013100041210006900172260001500241100001400256700001500270700001700285856012400302 1997 eng d00aA simulation study of the use of the Mantel-Haenszel and logistic regression procedures for assessing DIF in a CAT environment0 asimulation study of the use of the MantelHaenszel and logistic r aChicago IL1 aRoss, L P1 aNandakumar1 aClauser, B E uhttp://mail.iacat.org/content/simulation-study-use-mantel-haenszel-and-logistic-regression-procedures-assessing-dif-cat00705nas a2200121 4500008004100000245007200041210006900113260025200182100001700434700001500451700001500466856010200481 1996 eng d00aAdaptive assessment using granularity hierarchies and Bayesian nets0 aAdaptive assessment using granularity hierarchies and Bayesian n aFrasson, C. and Gauthier, G. and Lesgold, A. (Eds.) Intelligent Tutoring Systems, Third International Conference, ITS'96, Montréal, Canada, June 1996 Proceedings. Lecture Notes in Computer Science 1086. Berlin Heidelberg: Springer-Verlag 569-577.1 aCollins, J A1 aGreer, J E1 aHuang, S X uhttp://mail.iacat.org/content/adaptive-assessment-using-granularity-hierarchies-and-bayesian-nets00374nam a2200097 4500008004100000245003800041210003800079260007900117100001700196856006300213 1996 eng d00aAdaptive testing with granularity0 aAdaptive testing with granularity aMasters thesis, University of Saskatchewan, Department of Computer Science1 aCollins, J A uhttp://mail.iacat.org/content/adaptive-testing-granularity00446nas a2200109 4500008004100000245007200041210006900113260002700182100001900209700001200228856009600240 1996 eng d00aBuilding a statistical foundation for computerized adaptive testing0 aBuilding a statistical foundation for computerized adaptive test aBanff, Alberta, Canada1 aChang, Hua-Hua1 aYing, Z uhttp://mail.iacat.org/content/building-statistical-foundation-computerized-adaptive-testing00585nas a2200133 4500008004100000245013900041210006900180260001600249100001100265700001200276700001700288700002100305856012500326 1996 eng d00aThe effects of methods of theta estimation, prior distribution, and number of quadrature points on CAT using the graded response model0 aeffects of methods of theta estimation prior distribution and nu aNew York NY1 aHou, L1 aChen, S1 aG., Dodd., B1 aFitzpatrick, S J uhttp://mail.iacat.org/content/effects-methods-theta-estimation-prior-distribution-and-number-quadrature-points-cat-using00440nas a2200121 4500008004500000245006700045210006500112300001200177490000700189100001600196700001200212856009400224 1996 Engldsh 00aA Global Information Approach to Computerized Adaptive Testing0 aGlobal Information Approach to Computerized Adaptive Testing a213-2290 v201 aChang, H -H1 aYing, Z uhttp://mail.iacat.org/content/global-information-approach-computerized-adaptive-testing-101734nas a2200145 4500008004100000020001400041245006700055210006500122300001200187490000700199520125900206100001901465700001201484856009201496 1996 eng d a0146-621600aA global information approach to computerized adaptive testing0 aglobal information approach to computerized adaptive testing a213-2290 v203 abased on Fisher information (or item information). At each stage, an item is selected to maximize the Fisher information at the currently estimated trait level (&thetas;). However, this application of Fisher information could be much less efficient than assumed if the estimators are not close to the true &thetas;, especially at early stages of an adaptive test when the test length (number of items) is too short to provide an accurate estimate for true &thetas;. It is argued here that selection procedures based on global information should be used, at least at early stages of a test when &thetas; estimates are not likely to be close to the true &thetas;. For this purpose, an item selection procedure based on average global information is proposed. Results from pilot simulation studies comparing the usual maximum item information item selection with the proposed global information approach are reported, indicating that the new method leads to improvement in terms of bias and mean squared error reduction under many circumstances. Index terms: computerized adaptive testing, Fisher information, global information, information surface, item information, item response theory, Kullback-Leibler information, local information, test information.1 aChang, Hua-Hua1 aYing, Z uhttp://mail.iacat.org/content/global-information-approach-computerized-adaptive-testing00437nas a2200097 4500008004100000245008600041210006900127260001600196100001900212856010800231 1996 eng d00aA model for score maximization within a computerized adaptive testing environment0 amodel for score maximization within a computerized adaptive test aNew York NY1 aChang, Hua-Hua uhttp://mail.iacat.org/content/model-score-maximization-within-computerized-adaptive-testing-environment00521nas a2200109 4500008004100000245009800041210006900139260004600208100001900254700001200273856012600285 1996 eng d00aRecursive maximum likelihood estimation, sequential design, and computerized adaptive testing0 aRecursive maximum likelihood estimation sequential design and co aPrinceton NJ: Educational Testing Service1 aChang, Hua-Hua1 aYing, Z uhttp://mail.iacat.org/content/recursive-maximum-likelihood-estimation-sequential-design-and-computerized-adaptive-testing00553nas a2200133 4500008004100000245011000041210006900151260001800220100001200238700001100250700002100261700001400282856012300296 1995 eng d00aThe effect of population distribution and methods of theta estimation on CAT using the rating scale model0 aeffect of population distribution and methods of theta estimatio aSan Francisco1 aChen, S1 aHou, L1 aFitzpatrick, S J1 aDodd, B G uhttp://mail.iacat.org/content/effect-population-distribution-and-methods-theta-estimation-cat-using-rating-scale-model00366nas a2200109 4500008004100000245004800041210004600089260001800135100001600153700001400169856007300183 1995 eng d00aEquating the CAT-ASVAB: Issues and approach0 aEquating the CATASVAB Issues and approach aSan Francisco1 aSegall, D O1 aCarter, G uhttp://mail.iacat.org/content/equating-cat-asvab-issues-and-approach00644nas a2200121 4500008004100000245013600041210006900177260010500246100001500351700001700366700001600383856012300399 1995 eng d00aAn evaluation of alternative concepts for administering the Armed Services Vocational Aptitude Battery to applicants for enlistment0 aevaluation of alternative concepts for administering the Armed S aDMDC Technical Report 95-013. Monterey, CA: Personnel Testing Division, Defense Manpower Data Center1 aHogan, P F1 aMcBride, J R1 aCurran, L T uhttp://mail.iacat.org/content/evaluation-alternative-concepts-administering-armed-services-vocational-aptitude-battery00405nas a2200097 4500008004100000245006700041210006500108260002100173100001900194856009400213 1995 eng d00aA global information approach to computerized adaptive testing0 aglobal information approach to computerized adaptive testing aSan Francisco CA1 aChang, Hua-Hua uhttp://mail.iacat.org/content/global-information-approach-computerized-adaptive-testing-000447nam a2200097 4500008004100000245007200041210006900113260006100182100001200243856009400255 1995 eng d00aItem equivalence from paper-and-pencil to computer adaptive testing0 aItem equivalence from paperandpencil to computer adaptive testin aUnpublished doctoral dissertation, University of Chicago1 aChae, S uhttp://mail.iacat.org/content/item-equivalence-paper-and-pencil-computer-adaptive-testing00519nas a2200109 4500008004100000245009900041210006900140260004200209100001200251700001900263856012700282 1995 eng d00aRecursive maximum likelihood estimation, sequential designs, and computerized adaptive testing0 aRecursive maximum likelihood estimation sequential designs and c aUniversity of Twente, the Netherlands1 aYing, Z1 aChang, Hua-Hua uhttp://mail.iacat.org/content/recursive-maximum-likelihood-estimation-sequential-designs-and-computerized-adaptive-testing00406nas a2200109 4500008004100000245006600041210006500107300001200172490001100184100001500195856008600210 1994 eng d00aComputer adaptive testing: A shift in the evaluation paradigm0 aComputer adaptive testing A shift in the evaluation paradigm a213-2240 v22 (3)1 aCarlson, R uhttp://mail.iacat.org/content/computer-adaptive-testing-shift-evaluation-paradigm00499nas a2200121 4500008004100000245010100041210006900142300001000211490000600221100001200227700001700239856012100256 1994 eng d00aComputerized-adaptive and self-adapted music-listening tests: Features and motivational benefits0 aComputerizedadaptive and selfadapted musiclistening tests Featur a25-510 v71 aVispoel1 aCoffman, D D uhttp://mail.iacat.org/content/computerized-adaptive-and-self-adapted-music-listening-tests-features-and-motivational00364nas a2200109 4500008004100000245004700041210004600088260001600134100001600150700001400166856007400180 1994 eng d00aEvaluation and implementation of CAT-ASVAB0 aEvaluation and implementation of CATASVAB aLos Angeles1 aCurran, L T1 aWise, L L uhttp://mail.iacat.org/content/evaluation-and-implementation-cat-asvab00577nas a2200121 4500008004100000245015100041210006900192300000900261490000700270653003400277100001700311856012700328 1993 eng d00aComputer adaptive testing: A comparison of four item selection strategies when used with the golden section search strategy for estimating ability0 aComputer adaptive testing A comparison of four item selection st a17720 v5410acomputerized adaptive testing1 aCarlson, R D uhttp://mail.iacat.org/content/computer-adaptive-testing-comparison-four-item-selection-strategies-when-used-golden-section00758nas a2200241 4500008004100000020002200041245006700063210006400130250001500194260000800209300001100217490000700228653001500235653002600250653003700276653002300313653001800336100002100354700001400375700002400389700001400413856008900427 1993 eng d a0744-6314 (Print)00aMoving in a new direction: Computerized adaptive testing (CAT)0 aMoving in a new direction Computerized adaptive testing CAT a1993/01/01 cJan a80, 820 v2410a*Computers10aAccreditation/methods10aEducational Measurement/*methods10aLicensure, Nursing10aUnited States1 aJones-Dickson, C1 aDorsey, D1 aCampbell-Warnock, J1 aFields, F uhttp://mail.iacat.org/content/moving-new-direction-computerized-adaptive-testing-cat00410nas a2200121 4500008004100000245005800041210005700099300001000156490000800166100001200174700001700186856008500203 1992 eng d00aComputerized adaptive testing of music-related skills0 aComputerized adaptive testing of musicrelated skills a29-490 v1121 aVispoel1 aCoffman, D D uhttp://mail.iacat.org/content/computerized-adaptive-testing-music-related-skills00517nas a2200109 4500008004100000245005600041210005200097260014600149100001700295700001600312856007900328 1992 eng d00aThe development of alternative operational concepts0 adevelopment of alternative operational concepts aProceedings of the 34th Annual Conference of the Military Testing Association. San Diego, CA: Navy Personnel Research and Development Center.1 aMcBride, J R1 aCurran, L T uhttp://mail.iacat.org/content/development-alternative-operational-concepts00500nam a2200109 4500008004100000245008800041210006900129260004600198100001800244700002000262856010800282 1992 eng d00aManual for the General Scholastic Aptitude Test (Senior) Computerized adaptive test0 aManual for the General Scholastic Aptitude Test Senior Computeri aPretoria: Human Sciences Research Council1 aVon Tonder, M1 aClaasswn, N C W uhttp://mail.iacat.org/content/manual-general-scholastic-aptitude-test-senior-computerized-adaptive-test00440nas a2200121 4500008004100000245006100041210006000102300001200162490000700174653003400181100001500215856008800230 1991 eng d00aInter-subtest branching in computerized adaptive testing0 aIntersubtest branching in computerized adaptive testing a140-1410 v5210acomputerized adaptive testing1 aChang, S-H uhttp://mail.iacat.org/content/inter-subtest-branching-computerized-adaptive-testing00453nas a2200121 4500008004100000245006800041210006700109260001700176100001700193700001500210700001500225856009100240 1990 eng d00aMusicCAT: An adaptive testing program to assess musical ability0 aMusicCAT An adaptive testing program to assess musical ability aSan Diego CA1 aVispoel, W P1 aCoffman, D1 aScriven, D uhttp://mail.iacat.org/content/musiccat-adaptive-testing-program-assess-musical-ability00548nas a2200145 4500008004500000245009400045210006900139300001200208490000700220100001500227700001500242700001700257700001400274856011400288 1989 Engldsh 00aAdaptive and Conventional Versions of the DAT: The First Complete Test Battery Comparison0 aAdaptive and Conventional Versions of the DAT The First Complete a363-3710 v131 aHenly, S J1 aKlebe, K J1 aMcBride, J R1 aCudeck, R uhttp://mail.iacat.org/content/adaptive-and-conventional-versions-dat-first-complete-test-battery-comparison-000542nas a2200145 4500008004100000245009400041210006900135300001200204490000700216100001500223700001500238700001700253700001400270856011200284 1989 eng d00aAdaptive and conventional versions of the DAT: The first complete test battery comparison0 aAdaptive and conventional versions of the DAT The first complete a363-3710 v131 aHenly, S J1 aKlebe, K J1 aMcBride, J R1 aCudeck, R uhttp://mail.iacat.org/content/adaptive-and-conventional-versions-dat-first-complete-test-battery-comparison00492nam a2200097 4500008004100000245009300041210006900134260006200203100001700265856011200282 1988 eng d00aApplication of appropriateness measurement to a problem in computerized adaptive testing0 aApplication of appropriateness measurement to a problem in compu aUnpublished doctoral dissertation, University of Illinois1 aCandell, G L uhttp://mail.iacat.org/content/application-appropriateness-measurement-problem-computerized-adaptive-testing00623nas a2200145 4500008004100000245011100041210006900152260005400221100001400275700001700289700001400306700001600320700001700336856012400353 1988 eng d00aRefinement of the Computerized Adaptive Screening Test (CAST) (Final Report, Contract No MDA203 06-C-0373)0 aRefinement of the Computerized Adaptive Screening Test CAST Fina aWashington, DC: American Institutes for Research.1 aWise, L L1 aMcHenry, J J1 aChia, W J1 aSzenas, P L1 aMcBride, J R uhttp://mail.iacat.org/content/refinement-computerized-adaptive-screening-test-cast-final-report-contract-no-mda203-06-c00474nas a2200121 4500008004100000245008200041210006900123260001300192100001700205700001500222700001200237856010300249 1987 eng d00aEquating the computerized adaptive edition of the Differential Aptitude Tests0 aEquating the computerized adaptive edition of the Differential A aNew York1 aMcBride, J R1 aCorpe, V A1 aWing, H uhttp://mail.iacat.org/content/equating-computerized-adaptive-edition-differential-aptitude-tests-001655nas a2200121 4500008004100000245007900041210006900120300001200189490000700201520121400208100001401422856009701436 1985 eng d00aA structural comparison of conventional and adaptive versions of the ASVAB0 astructural comparison of conventional and adaptive versions of t a305-3220 v203 aExamined several structural models of similarity between the Armed Services Vocational Aptitude Battery (ASVAB) and a battery of computerized adaptive tests designed to measure the same aptitudes. 12 plausible models were fitted to sample data in a double cross-validation design. 1,411 US Navy recruits completed 10 ASVAB subtests. A computerized adaptive test version of the ASVAB subtests was developed on item pools of approximately 200 items each. The items were pretested using applicants from military entrance processing stations across the US, resulting in a total calibration sample size of approximately 60,000 for the computerized adaptive tests. Three of the 12 models provided reasonable summaries of the data. One model with a multiplicative structure (M. W. Browne; see record 1984-24964-001) performed quite well. This model provides an estimate of the disattenuated method correlation between conventional testing and adaptive testing. In the present data, this correlation was estimated to be 0.97 and 0.98 in the 2 halves of the data. Results support computerized adaptive tests as replacements for conventional tests. (33 ref) (PsycINFO Database Record (c) 2004 APA, all rights reserved).1 aCudeck, R uhttp://mail.iacat.org/content/structural-comparison-conventional-and-adaptive-versions-asvab00353nas a2200121 4500008003900000245003600039210003600075300001200111490000700123100001800130700001700148856006600165 1984 d00aComputerized diagnostic testing0 aComputerized diagnostic testing a391-3970 v211 aMCArthur, D L1 aChoppin, B H uhttp://mail.iacat.org/content/computerized-diagnostic-testing00534nas a2200097 4500008004100000245010400041210006900145260008000214100001500294856012700309 1982 eng d00aComputerized adaptive testing system design: Preliminary design considerations (Tech. Report 82-52)0 aComputerized adaptive testing system design Preliminary design c aSan Diego CA: Navy Personnel Research and Development Center. (AD A118 495)1 aCroll, P R uhttp://mail.iacat.org/content/computerized-adaptive-testing-system-design-preliminary-design-considerations-tech-report-8200488nas a2200121 4500008004100000245006900041210006600110260005000176100002000226700001500246700001500261856009000276 1981 eng d00aA comparison of two methods of interactive testing Final report.0 acomparison of two methods of interactive testing Final report aNational Institute of Education Grant 79-10451 aNicewander, W A1 aChang, H S1 aDoody, E N uhttp://mail.iacat.org/content/comparison-two-methods-interactive-testing-final-report00516nam a2200097 4500008004100000245011700041210006900158260005100227100001800278856012200296 1981 eng d00aEffect of error in item parameter estimates on adaptive testing (Doctoral dissertation, University of Minnesota)0 aEffect of error in item parameter estimates on adaptive testing  aDissertation Abstracts International, 42, 06-B1 aCrichton, L I uhttp://mail.iacat.org/content/effect-error-item-parameter-estimates-adaptive-testing-doctoral-dissertation-university00617nas a2200133 4500008004100000245009600041210006900137260009700206100001400303700001600317700001800333700001400351856011800365 1980 eng d00aEffects of computerized adaptive testing on Black and White students (Research Report 79-2)0 aEffects of computerized adaptive testing on Black and White stud aMinneapolis: University of Minnesota, Department of Psychology, Psychometric Methods Program1 aPine, S M1 aChurch, A T1 aGialluca, K A1 aWeiss, DJ uhttp://mail.iacat.org/content/effects-computerized-adaptive-testing-black-and-white-students-research-report-79-200472nas a2200133 4500008004100000245007200041210006900113300001200182490000600194100001400200700001700214700001500231856009200246 1980 eng d00aImplied orders tailored testing: Simulation with the Stanford-Binet0 aImplied orders tailored testing Simulation with the StanfordBine a157-1630 v41 aCudeck, R1 aMcCormick, D1 aCliff, N A uhttp://mail.iacat.org/content/implied-orders-tailored-testing-simulation-stanford-binet00478nas a2200133 4500008004500000245007200045210006900117300001200186490000600198100001400204700001900218700001300237856009400250 1980 Engldsh 00aImplied Orders Tailored Testing: Simulation with the Stanford-Binet0 aImplied Orders Tailored Testing Simulation with the StanfordBine a157-1630 v41 aCudeck, R1 aMcCormick, D J1 aCliff, N uhttp://mail.iacat.org/content/implied-orders-tailored-testing-simulation-stanford-binet-000499nas a2200133 4500008004500000245008600045210006900131300001200200490000600212100001300218700001400231700001900245856010100264 1979 Engldsh 00aEvaluation of Implied Orders as a Basis for Tailored Testing with Simulation Data0 aEvaluation of Implied Orders as a Basis for Tailored Testing wit a495-5140 v31 aCliff, N1 aCudeck, R1 aMcCormick, D J uhttp://mail.iacat.org/content/evaluation-implied-orders-basis-tailored-testing-simulation-data-000467nas a2200121 4500008004100000245008600041210006900127300001200196490000600208100001500214700001700229856009900246 1979 eng d00aEvaluation of implied orders as a basis for tailored testing with simulation data0 aEvaluation of implied orders as a basis for tailored testing wit a495-5140 v31 aCliff, N A1 aMcCormick, D uhttp://mail.iacat.org/content/evaluation-implied-orders-basis-tailored-testing-simulation-data00482nas a2200133 4500008004500000245007700045210006900122300001000191490000600201100001400207700001700221700001300238856009700251 1979 Engldsh 00aMonte Carlo Evaluation of Implied Orders As a Basis for Tailored Testing0 aMonte Carlo Evaluation of Implied Orders As a Basis for Tailored a65-740 v31 aCudeck, R1 aMcCormick, D1 aCliff, N uhttp://mail.iacat.org/content/monte-carlo-evaluation-implied-orders-basis-tailored-testing-000480nas a2200133 4500008004100000245007700041210006900118300001000187490000600197100001400203700001900217700001500236856009500251 1979 eng d00aMonte carlo evaluation of implied orders as a basis for tailored testing0 aMonte carlo evaluation of implied orders as a basis for tailored a65-740 v31 aCudeck, R1 aMcCormick, D J1 aCliff, N A uhttp://mail.iacat.org/content/monte-carlo-evaluation-implied-orders-basis-tailored-testing00513nas a2200133 4500008004100000245009100041210006900132300001200201490000700213100001600220700001600236700001600252856011100268 1978 eng d00aCombining auditory and visual stimuli in the adaptive testing of speech discrimination0 aCombining auditory and visual stimuli in the adaptive testing of a115-1220 v431 aSteele, J A1 aBinnie, C A1 aCooper, W A uhttp://mail.iacat.org/content/combining-auditory-and-visual-stimuli-adaptive-testing-speech-discrimination00592nas a2200121 4500008004100000245010900041210006900150260008100219100001500300700001400315700001700329856012400346 1978 eng d00aEvaluations of implied orders as a basis for tailored testing using simulations (Technical Report No. 4)0 aEvaluations of implied orders as a basis for tailored testing us aLos Angeles CA: University of Southern California, Department of Psychology.1 aCliff, N A1 aCudeck, R1 aMcCormick, D uhttp://mail.iacat.org/content/evaluations-implied-orders-basis-tailored-testing-using-simulations-technical-report-no-400529nas a2200121 4500008004100000245007600041210006900117260008100186100001500267700001400282700001700296856009400313 1978 eng d00aImplied orders as a basis for tailored testing (Technical Report No. 6)0 aImplied orders as a basis for tailored testing Technical Report  aLos Angeles CA: University of Southern California, Department of Psychology.1 aCliff, N A1 aCudeck, R1 aMcCormick, D uhttp://mail.iacat.org/content/implied-orders-basis-tailored-testing-technical-report-no-600635nas a2200121 4500008004100000245007800041210006900119260018600188100001500374700001400389700001700403856009300420 1977 eng d00aAn empirical evaluation of implied orders as a basis for tailored testing0 aempirical evaluation of implied orders as a basis for tailored t aD. J. Weiss (Ed.), Proceedings of the 1977 Computerized Adaptive Testing Conference. Minneapolis MN: University of Minnesota, Department of Psychology, Psychometric Methods Program.1 aCliff, N A1 aCudeck, R1 aMcCormick, D uhttp://mail.iacat.org/content/empirical-evaluation-implied-orders-basis-tailored-testing00455nas a2200133 4500008004100000245006500041210006400106300001200170490000700182100001600189700001500205700001300220856008800233 1977 eng d00aTAILOR: A FORTRAN procedure for interactive tailored testing0 aTAILOR A FORTRAN procedure for interactive tailored testing a767-7690 v371 aCudeck, R A1 aCliff, N A1 aKehoe, J uhttp://mail.iacat.org/content/tailor-fortran-procedure-interactive-tailored-testing00465nas a2200121 4500008004100000245008000041210006900121300001200190490000700202100001700209700001500226856010200241 1977 eng d00aTAILOR-APL: An interactive computer program for individual tailored testing0 aTAILORAPL An interactive computer program for individual tailore a771-7740 v371 aMcCormick, D1 aCliff, N A uhttp://mail.iacat.org/content/tailor-apl-interactive-computer-program-individual-tailored-testing00399nas a2200097 4500008004100000245007100041210006900112300001200181100001500193856009300208 1977 eng d00aA theory of consistency ordering generalizable to tailored testing0 atheory of consistency ordering generalizable to tailored testing a375-3991 aCliff, N A uhttp://mail.iacat.org/content/theory-consistency-ordering-generalizable-tailored-testing00412nas a2200097 4500008004100000245007000041210006900111260002700180100001500207856009200222 1976 eng d00aElements of a basic test theory generalizable to tailored testing0 aElements of a basic test theory generalizable to tailored testin aUnpublished manuscript1 aCliff, N A uhttp://mail.iacat.org/content/elements-basic-test-theory-generalizable-tailored-testing00481nam a2200097 4500008004100000245004700041210004700088260015600135100001500291856007700306 1976 eng d00aIncomplete orders and computerized testing0 aIncomplete orders and computerized testing aIn C. K. Clark (Ed.), Proceedings of the First Conference on Computerized Adaptive Testing (pp. 18-23). Washington DC: U.S. Government Printing Office.1 aCliff, N A uhttp://mail.iacat.org/content/incomplete-orders-and-computerized-testing00588nas a2200133 4500008004100000245009400041210006900135260007200204100001600276700001500292700001800307700001900325856011000344 1976 eng d00aMonte carlo results from a computer program for tailored testing (Technical Report No. 2)0 aMonte carlo results from a computer program for tailored testing aLos Angeles CA: University of California, Department of Psychology.1 aCudeck, R A1 aCliff, N A1 aReynolds, T J1 aMcCormick, D J uhttp://mail.iacat.org/content/monte-carlo-results-computer-program-tailored-testing-technical-report-no-200440nam a2200097 4500008004100000245007300041210006900114260005100183100001500234856009300249 1976 eng d00aProceedings of the first conference on computerized adaptive testing0 aProceedings of the first conference on computerized adaptive tes aWashington DC: U.S. Government Printing Office1 aClark, C K uhttp://mail.iacat.org/content/proceedings-first-conference-computerized-adaptive-testing00647nas a2200097 4500008004100000245014800041210006900189260015300258100001400411856012400425 1976 eng d00aUsing computerized tests to add new dimensions to the measurement of abilities which are important for on-job performance: An exploratory study0 aUsing computerized tests to add new dimensions to the measuremen aC. K. Clark (Ed.), Proceedings of the First Conference on Computerized Adaptive Testing (pp. 64-74). Washington DC: U.S. Government Printing Office.1 aCory, C H uhttp://mail.iacat.org/content/using-computerized-tests-add-new-dimensions-measurement-abilities-which-are-important-job00491nas a2200097 4500008004100000245008200041210006900123260008100192100001500273856010500288 1975 eng d00aA basic test theory generalizable to tailored testing (Technical Report No 1)0 abasic test theory generalizable to tailored testing Technical Re aLos Angeles CA: University of Southern California, Department of Psychology.1 aCliff, N A uhttp://mail.iacat.org/content/basic-test-theory-generalizable-tailored-testing-technical-report-no-100446nas a2200109 4500008004100000245008400041210006900125300001200194490000700206100001500213856010800228 1975 eng d00aComplete orders from incomplete data: Interactive ordering and tailored testing0 aComplete orders from incomplete data Interactive ordering and ta a259-3020 v821 aCliff, N A uhttp://mail.iacat.org/content/complete-orders-incomplete-data-interactive-ordering-and-tailored-testing00429nas a2200109 4500008004100000245007200041210006900113260001100182100001500193700001400208856009700222 1975 eng d00aTailored testing: Maximizing validity and utility for job selection0 aTailored testing Maximizing validity and utility for job selecti aCanada1 aCroll, P R1 aUrry, V W uhttp://mail.iacat.org/content/tailored-testing-maximizing-validity-and-utility-job-selection00453nas a2200109 4500008004100000245007700041210006900118260003000187100001900217700001400236856009300250 1973 eng d00aThe potential use of tailored testing for allocation to army employments0 apotential use of tailored testing for allocation to army employm aLisbon, Portugalc06/19731 aKillcross, M C1 aCassie, A uhttp://mail.iacat.org/content/potential-use-tailored-testing-allocation-army-employments00567nas a2200181 4500008004100000245005100041210004900092300001100141490000700152653000900159653004900168653004100217653000900258100001300267700001400280700001600294856007500310 1972 eng d00aSequential testing for dichotomous decisions. 0 aSequential testing for dichotomous decisions a85-95.0 v3210aCCAT10aCLASSIFICATION Computerized Adaptive Testing10asequential probability ratio testing10aSPRT1 aLinn, RL1 aRock, D A1 aCleary, T A uhttp://mail.iacat.org/content/sequential-testing-dichotomous-decisions00623nas a2200121 4500008004100000245017800041210006900219260004700288100001300335700001400348700001600362856012300378 1970 eng d00aSequential testing for dichotomous decisions. College Entrance Examination Board Research and Development Report (RDR 69-70, No 3", and Educational Testing Service RB-70-31)0 aSequential testing for dichotomous decisions College Entrance Ex aPrinceton NJ: Educational Testing Service.1 aLinn, RL1 aRock, D A1 aCleary, T A uhttp://mail.iacat.org/content/sequential-testing-dichotomous-decisions-college-entrance-examination-board-research-and00449nas a2200121 4500008004100000245007300041210006900114300001200183490000700195100001300202700001600215856009600231 1969 eng d00aThe development and evaluation of several programmed testing methods0 adevelopment and evaluation of several programmed testing methods a129-1460 v291 aLinn, RL1 aCleary, T A uhttp://mail.iacat.org/content/development-and-evaluation-several-programmed-testing-methods00393nas a2200133 4500008004100000245004500041210004200086300001200128490000700140100001600147700001300163700001400176856006900190 1969 eng d00aAn exploratory study of programmed tests0 aexploratory study of programmed tests a345-3600 v281 aCleary, T A1 aLinn, RL1 aRock, D A uhttp://mail.iacat.org/content/exploratory-study-programmed-tests00538nas a2200121 4500008004100000245009800041210006900139260004600208100001300254700001400267700001600281856011900297 1968 eng d00aThe development and evaluation of several programmed testing methods (Research Bulletin 68-5)0 adevelopment and evaluation of several programmed testing methods aPrinceton NJ: Educational Testing Service1 aLinn, RL1 aRock, D A1 aCleary, T A uhttp://mail.iacat.org/content/development-and-evaluation-several-programmed-testing-methods-research-bulletin-68-500493nas a2200133 4500008004100000245008400041210006900125300001200194490000600206100001600212700001300228700001400241856010400255 1968 eng d00aReproduction of total test score through the use of sequential programmed tests0 aReproduction of total test score through the use of sequential p a183-1870 v51 aCleary, T A1 aLinn, RL1 aRock, D A uhttp://mail.iacat.org/content/reproduction-total-test-score-through-use-sequential-programmed-tests00437nas a2200097 4500008004100000245005200041210005200093260010200145100001800247856007400265 1966 eng d00aNew light on test strategy from decision theory0 aNew light on test strategy from decision theory aA. Anastasi (Ed.). Testing problems in perspective. Washington DC: American Council on Education.1 aCronbach, L J uhttp://mail.iacat.org/content/new-light-test-strategy-decision-theory00390nas a2200109 4500008004100000245006200041210005900103300001200162490000700174100001600181856008300197 1946 eng d00aAn application of sequential sampling to testing students0 aapplication of sequential sampling to testing students a547-5560 v411 aCowden, D J uhttp://mail.iacat.org/content/application-sequential-sampling-testing-students