Model Choice and Sample Size in Item Response Theory Analysis of Aphasia Tests
Hula, William D and Fergadiotis, Gerasimos and Martin, Nadine
Model Choice and Sample Size in Item Response Theory Analysis of Aphasia Tests. American Journal of Speech-Language Pathology, 21(Suppl.), May, 2012, pages 38-50.
This is the latest version of this eprint.
Purpose: The purpose of this study was to identify the most appropriate item response theory (IRT) measurement model for aphasia tests requiring 2-choice responses and to determine whether small samples are adequate for estimating such models.
Method: Pyramids and Palm Trees (Howard & Patterson, 1992) test data that had been collected from individuals with aphasia were analyzed, and the resulting item and person estimates were used to develop simulated test data for 3 sample size conditions. The simulated data were analyzed using a standard 1-parameter logistic (1-PL) model and 3 models that accounted for the influence of guessing: augmented 1-PL and 2-PL models and a 3-PL model. The model estimates obtained from the simulated data were compared to their known true values.
Results: With small and medium sample sizes, an augmented 1-PL model was the most accurate at recovering the known item and person parameters; however, no model performed well at any sample size. Follow-up simulations confirmed that the large influence of guessing and the extreme easiness of the items contributed substantially to the poor estimation of item difficulty and person ability.
Conclusion: Incorporating the assumption of guessing into IRT models improves parameter estimation accuracy, even for small samples. However, caution should be exercised in interpreting scores obtained from easy 2-choice tests, regardless of whether IRT modeling or percentage correct scoring is used.
|EPrint Type:||Journal (Paginated)|
|Keywords:||aphasia, psychometrics, assessment|
|Conference:||Clinical Aphasiology Conference: Clinical Aphasiology Conference (2011 : 41st : Fort Lauderdale, FL : May 31-June 4, 2011)|
|Publication:||American Journal of Speech-Language Pathology|
|DOI or Unique Handle:||10.1044/1058-0360(2011/11-0090)|
|Additional Information:||This article is based on a paper originally presented at the CAC20xx Conference. Permission to view the full-text articles is subject to the publisher’s access restrictions.|
Available Versions of this Item
- The Effects of Sample Size and Guessing on Parameter Recovery in IRT Modeling of Aphasia Test Data (deposited 16 August 2011)
- Model Choice and Sample Size in Item Response Theory Analysis of Aphasia Tests (deposited 05 July 2012) [Currently Displayed]