Quarterly of Educational Measurement

Current Issue

By Issue

By Author

By Subject

Author Index

Keyword Index

About Journal

Aims and Scope

Publication Ethics

Indexing and Abstracting

Related Links

FAQ

Peer Review Process

Journal Metrics

News

Article Writing Guide

Files

Reviewers 2019

Reviewers 2020

Reviewers 2021

Reviewers 2022

Application of a Cognitive Diagnostic Assessment to Analysis English Comprehension's Items of Master's Degree

Document Type : Research Paper

Authors

1 faculty member of Deliberation and Measuring

2 student

Abstract

The purpose of this study was to determine the basic skills of reading and comprehension and analyze the test questions, and finally, strengths and weaknesses of subjects through cognitive diagnostic assessment models were fitted with retrofit analysis. The data examined in this study included responses from 3,000 participants in the graduate entrance examination for English language field, which were selected from among 16044 people randomly. At first, with the help of experts, a matrix Q, consisting of the relationship between 20 questions of reading test with 7 basic skills was formed and then, using the G-DINA model, last analysis was performed. Finally, using a posteriori expected method within the framework of the G-DINA model, probability of domination of every single person on each skill was calculated by CDM package skills in software R, and by determining the cut-off point 0.6 for dominance, investigated participants in each skill, divided into two dominant and non-dominant groups. The results showed that in each of the skills, most people have not reached to dominance, Although the situation in the SUM skill (Analyzing and evaluating the relative importance of the information contained in the text, by differentiating between the main idea and details) was better than other skills, and about half of those got dominant in the skill. The final result suggests that, in general, the participant's situation in the application of cognitive strategies to achieve the correct answer is unsatisfactory and these strategies must be included in the curriculum of reading skills and comprehension.

Keywords

References
افضلی، افشین. (1393). تدوین و ارزشیابی مدل تشخیصی شناختی (CDM) ریاضیات پایه اول دبیرستان با استفاده از روش سلسله مراتبی صفات (AHM). پایان‌نامه دکتری: دانشگاه علامه طباطبایی.
ثرندایک، رابرت ال. (1369). روانسنجی کاربردی. ترجمه حیدرعلی هومن. تهران: انتشارات دانشگاه تهران.
شریفی، حسن پاشا. (1390). اصول روان سنجی و روان آزمایی. تهران: رشد.
فرهادی، حسین. (1381). «نقدی بر آزمون­های کنکور کارشناسی ارشد رشته زبان انگلیسی» پژوهش­های زبان خارجی، 13، 79-106.
کبیری، مسعود (1393). کاربرد سنجش تشخیص به منظور تعیین مهارت های کسب شده علوم تجربی در دانش آموزان سال سوم راهنمایی ایران بر اساس داده های تیمز 2011. پایان نامه دکتری: دانشگاه تهران
محسن­پور، مریم (1393). طراحی و ساخت آزمون شناختی- تشخیصی سواد ریاضی کاربردی دانش آموزان سال اول دبیرستان و سنجش اثربخشی بسته آموزشی جبرانی مبتنی بر آن. پایان نامه دکتری: دانشگاه تهران.
مینایی، اصغر. (1391). مدل پردازی تشخیصی شناختی (CDM) سؤال های ریاضیات تیمز 2007 در دانش آموزان پایه هشتم ایران با استفاده از مدل یکپارچه با پارامتر پردازی مجدد (RUM) و مقایسه مهارت های ریاضی دانش آموزان دختر و پسر. پایان نامه دکتری: دانشگاه علامه طباطبایی.
مقدم، اعظم. (1394). کاربرد مدل­های تشخیصی شناختی به­منظور تعیین مهارت­های زیربنایی عملکرد داوطلبان در آزمون وروردی زبان انگلیسی عمومی دوره دکتری. پایان نامه دکتری: دانشگاه علامه طباطبایی.
Alderson, J. C. (2005). Assessing reading. Ernst Klett Sprachen.
Bernhardt, E. B. (1993). Reading development in a second language: Theoretical, empirical, & classroom perspectives. Ablex Publishing Corporation, 355 Chestnut St., Norwood, NJ (paperback: ISBN-0-89391-734-6; cloth: ISBN-0-89391-675-7).
Buck, G., Tatsuoka, K., & Kostin, I. (1997). The Subskills of Reading: Rule‐space Analysis of a Multiple‐choice Test of Second Language Reading Comprehension. Language Learning, 47(3), 423-466.
Davier, M. (2005). A general diagnostic model applied to language testing data. ETS Research Report Series, 2005(2).
Jang, E. E. (2009). Cognitive diagnostic assessment of L2 reading comprehension ability: Validity arguments for Fusion Model application to LanguEdge assessment. Language Testing, 26(1), 031-73.‏
Johnson, D. M., & Reynolds, F. A. (1940). A factor analysis of verbal ability. The Psychological Record, 4, 181.
Kasai, M. (1997). Application of the rule space model to the reading comprehension section of the test of English as a foreign language (TOEFL) (Doctoral dissertation, University of Illinois at Urbana-Champaign).
Koda, K. (2005). Insights into second language reading: A cross-linguistic approach. Cambridge University Press.‏
Kunina-Habenicht, O., Rupp, A. A., & Wilhelm, O. (2009). A practical illustration of multidimensional diagnostic skills profiling: Comparing results from confirmatory factor analysis and diagnostic classification models. Studies in Educational Evaluation, 35(2), 64-70.‏
Lee, Y. W., & Sawaki, Y. (2009). Application of three cognitive diagnosis models to ESL reading and listening assessments. Language Assessment Quarterly, 6(3), 239-263.
Leighton, J., & Gierl, M. (Eds.). (2007). Cognitive diagnostic assessment for education: Theory and applications. Cambridge University Press.‏
Li, H. (2011). A cognitive diagnostic analysis of the MELAB reading test. Spaan Fellow Working Papers in Second or Foreign Language Assessment, 9, 17-46.
Li, H., Hunter, C. V., & Lei, P. W. (2016). The selection of cognitive diagnostic models for a reading comprehension test. Language Testing, 33(3), 391-409.
Maydeu-Olivares, A. (2013). Goodness-of-fit assessment of item response theory models. Measurement: Interdisciplinary Research and Perspectives, 11 (3), 71-101.
Munby, J. (1981). Communicative syllabus design: A sociolinguistic model for designing the content of purpose-specific language programmes. Cambridge University Press.
Pettit, N. T., & Cockriel, I. W. (1974). A factor study of the literal reading comprehension test and the inferential reading comprehension test. Journal of Literacy Research, 6(1), 63-75.
Ravand, H., & Robitzsch, A. (2015). Cognitive Diagnostic Modeling Using R. Practical Assessment, Research & Evaluation, 20.
Scott, H. S. (1998). Cognitive diagnostic perspectives of a second language reading test (Doctoral dissertation, University of Illinois at Urbana-Champaign).
Snow, R. E., & Lohman, D. F. (1989). Implications of cognitive psychology for educational measurement. American Council on Education.‏
Svetina, D., Gorin, J. S., & Tatsuoka, K. K. (2011). Defining and comparing the reading comprehension construct: A cognitive-psychometric modeling approach. International Journal of Testing, 11(1), 1-23.
Toker, T. (2010). Cognitive Diagnostic Assessment of Timss-2007 Mathematics tems For 8th Graders in Turkey.‏
Vernon, P. E. (1962). The determinants of reading comprehension. Educational and Psychological Measurement.
Yi, Y. (2013). Implementing a cognitive diagnostic assessment in an institutional test: a new networking model in language testing and experiment with a new psychometric model and task type (Doctoral dissertation, University of Illinois at Urbana-Champaign).
Zhang, J. (2013). Relationships between missing responses and skill mastery profiles of cognitive diagnostic assessment (Doctoral dissertation, University of Toronto).
Quarterly of Educational Measurement
Volume 9, Issue 32
July 2018
Pages 17-40
Files
Share
How to cite
Statistics