Document Type : Research Paper

Authors

1 Assistant Professor, Department of Educational Science, Farhangian University, Tehran, Iran

2 Ph.D. Student in Measurement & Assessment, Allameh Tabataba'i University, Tehran, Iran

Abstract

 
The goal of this research is to explore the Likert scale questions using two distinct methods: Classical Test Theory and Item Response Theory. By comparing the results of these approaches, the study aims to address the question: "Do the outcomes from these two methodologies align, or do they contradict each other?" The research design followed a descriptive methodology and utilized secondary analysis techniques. The study population consisted of 977 junior high school students. After the data screening process, the final sample size for analyzing extraversion items was 783 students, 763 students for openness items, and 784 students for conscientiousness items. The research instruments were the three subscales of extraversion, openness, and conscientiousness from the Neo Personality Test. The statistical analysis yielded results indicating that a strong internal consistency among items enhanced the accuracy and validity of outcomes derived from the graded response model. However, when items exhibit low internal consistency, caution should be exercised, as the model may yield erroneous thresholds or discrimination coefficients (i.e., false negative or positive). Overall, combining multiple methods of statistical analysis can significantly contribute to more effective analysis and obtaining highly accurate results.

Keywords

تیلور، کاترین، اس. (1398). روایی و رواسازی. (ترجمه یونسی، جلیل). تهران: دانشگاه علامه طباطبائی. https://book.atu.ac.ir/
زمانپور، عنایت اله، یونسی، جلیل، رستگار آگاه، مصطفی، و مهرابی، مهسا. (1398). تحلیل داده‌های نگرش‌سنجی: تفاوت نظریه کلاسیک و سؤال ـ پاسخ. پژوهشهای ارتباطی. 26 (2)، 165-139. https://doi.org/10.22082/cr.2019.113772.1917
 
 
 
Baker, F. B., & Kim, S. H. (2004). Item response theory: Parameter estimation techniques. CRC press. https://doi.org/10.1201/9781482276725
Baker, F. B. (2001). The basics of item response theory. For full text: http://ericae.net/irt/baker
Courville, T. G. (2004). An empirical comparison of item response theory and classical test theory item/person statistics. Texas A&M University. https://www.proquest.com/openview/dc9dbe1471e8f2828066045a2396ea24/1?pq-origsite=gscholar&cbl=18750&diss=y
De Ayala, R. J. (2013). The theory and practice of item response theory. Guilford Publications. https://cehs.unl.edu/EdPsych/RJSite/de_Ayala_Appendices_2ndEd.pdf
Embretson, S. E., & Reise, P. E. (2000). Item response theory for psychologists. Psychology Press. https://doi.org/10.4324/9781410605269
Hambleton, R. K., & Jones, R. W. (1993). An NCME Instructional Module on Educational Measurement: issues and practice. Comparison of Classical Test Theory and Item Response Theory and Their Applications to Test Development, 12(3), 38-47. https://doi.org/10.1111/j.1745-3992.1993.tb00543.x
Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1985). Principles and applications of item response theory.
Hambleton, R. K., & Swaminathan, H. (2013). Item response theory: Principles and applications. Springer Science & Business Media. https://link.springer.com/book/10.1007/978-94-017-1988-9
Jeong, H. J., & Lee, W. C. (2016). Item response theory-based evaluation of psychometric properties of the safety attitudes questionnaire-Korean version (SAQ-K). Biometrics & Biostatistics International Journal, 3(5), 1-15. http://dx.doi.org/10.15406/bbij.2015.02.00020
Kang, S. M., & Waller, N. G. (2005). Moderated multiple regression, spurious interaction effects, and IRT. Applied Psychological Measurement, 29(2), 87-105. https://doi.org/10.1177/0146621604272737
Lord, F.M., Novick, M.R., & Birnbaum, A. (1968). Statistical theories of mental test scores. Addison-Wesley. https://psycnet.apa.org/record/1968-35040-000
Reeve, B. B., & Fayers, P. (2005). Applying item response theory modeling for evaluating questionnaire item and scale properties. Assessing quality of life in clinical trials: methods of practice, 2, 55-73. http://dx.doi.org/10.1093/oso/9780198527695.003.0005
Sorrel, M. A., Olea, J., Abad, F. J., de la Torre, J., Aguado, D., & Lievens, F. (2016). Validity and reliability of situational judgement test scores: A new approach based on cognitive diagnosis models. Organizational Research Methods, 19(3), 506-532. https://doi.org/10.1177/1094428116630065
Toland, M. D. (2014). Practical guide to conducting an item response theory analysis. The Journal of Early Adolescence, 34(1), 120-151. http://dx.doi.org/10.1177/0272431613511332
Taylor, C. S. (2013). Validity and validation. Oxford University Press, USA. [In Persian] https://doi.org/10.1093/acprof:osobl/9780199791040.001.0001
Zamanpour, E., Younesi, J., Rastegar Agah, M., & Mehrabi, M. (2019). Attitudinal Analysis: The Difference Between Classical Test Theory and Item Response Theory. Communication Research, 26(2), 139-165. [In Persian] https://doi.org/10.22082/cr.2019.113772.1917