Comparison of Confirmatory Factor Analysis (CFA) and Item Response Theory (IRT)-Based Likelihood Ratio in Detecting Differential Item Functioning at High Stakes Tests

Gerami Pour, Masoud; Falsafi Nejhad, Mohamad Reza; Delavar, Ali; Farrokhi, Nour Ali

Comparison of Confirmatory Factor Analysis (CFA) and Item Response Theory (IRT)-Based Likelihood Ratio in Detecting Differential Item Functioning at High Stakes Tests

Document Type : Research Paper

Authors

masoud gerami pour

mohamad reza falsafi nejhad

ali delavar

nour ali farrokhi

Abstract

Although numerous methods have been proposed for detecting biased items, but a few researches have empirically investigated the power and efficiency of these methods. The main goai of this research was to apply IRT-based likelihood ratio test and confirmatory factor analysis in detecting differential item functioning (DIF) in high stakes tests. Monte Carlo simulated methods were used to answer the research questions. Required data were simulated through WINGEN2 in the form of 100 tests with 30 items that were fitted to 2PL model. Distributions of item difficulties and discrimination powers of all the tests were normal. Responses of 1000 examinees were also simulated with normal ability distribution for each test. Estimation methods of marginal maximum likelihood and weighted least squares were used to detect type and magnitudes of DIF. Data analysis in consecutive replications showed that IRT based methods were superior to CFA methods in detecting DIF. This superiority was observed in ail DIF conditions (low, moderate and high). After all. differences between two methods were small at 1000 sample sizes. No differences were observed between two methods in detecting different types of DIF. Results of this study confirm the results of Meade and Lautenschlager (2004; 2006), but is in contrast with the results of Flowers et al (2002). Finally, Likelihood Ratio test is recommended if there is any limitation in applying methods of detecting DIF.

Keywords

differential item functioning

Likelihood Ratio Test. Confirmatory Factor Analysis. High Stakes Tests