The Aphasiology Archive
University Library System, University of Pittsburgh

Model Choice and Sample Size in Item Response Theory Analysis of Aphasia Tests

Hula, William D and Fergadiotis, Gerasimos and Martin, Nadine
Model Choice and Sample Size in Item Response Theory Analysis of Aphasia Tests. American Journal of Speech-Language Pathology, 21(Suppl.), May, 2012, pages 38-50.

This is the latest version of this eprint.


Purpose: The purpose of this study was to identify the most appropriate item response theory (IRT) measurement model for aphasia tests requiring 2-choice responses and to determine whether small samples are adequate for estimating such models.

Method: Pyramids and Palm Trees (Howard & Patterson, 1992) test data that had been collected from individuals with aphasia were analyzed, and the resulting item and person estimates were used to develop simulated test data for 3 sample size conditions. The simulated data were analyzed using a standard 1-parameter logistic (1-PL) model and 3 models that accounted for the influence of guessing: augmented 1-PL and 2-PL models and a 3-PL model. The model estimates obtained from the simulated data were compared to their known true values.

Results: With small and medium sample sizes, an augmented 1-PL model was the most accurate at recovering the known item and person parameters; however, no model performed well at any sample size. Follow-up simulations confirmed that the large influence of guessing and the extreme easiness of the items contributed substantially to the poor estimation of item difficulty and person ability.

Conclusion: Incorporating the assumption of guessing into IRT models improves parameter estimation accuracy, even for small samples. However, caution should be exercised in interpreting scores obtained from easy 2-choice tests, regardless of whether IRT modeling or percentage correct scoring is used.

EPrint Type:Journal (Paginated)
Keywords:aphasia, psychometrics, assessment
ID Code:2341
Conference:Clinical Aphasiology Conference: Clinical Aphasiology Conference (2011 : 41st : Fort Lauderdale, FL : May 31-June 4, 2011)
Publication:American Journal of Speech-Language Pathology
Alternative Locations:
DOI or Unique Handle:10.1044/1058-0360(2011/11-0090)
Additional Information:This article is based on a paper originally presented at the CAC20xx Conference. Permission to view the full-text articles is subject to the publisher’s access restrictions.

Available Versions of this Item