فصلنامه اندازه گیری تربیتی

شماره جاری

بر اساس شماره‌های نشریه

بر اساس نویسندگان

بر اساس موضوعات

نمایه نویسندگان

نمایه کلیدواژه ها

درباره نشریه

اهداف و چشم انداز

اصول اخلاقی انتشار مقاله

بانک ها و نمایه نامه ها

پیوندهای مفید

پرسش‌های متداول

فرایند پذیرش مقالات

اطلاعات آماری نشریه

اخبار و اعلانات

راهنمای نگارش مقاله

فایل ها

داوران سال 1398

داوران سال 1399

داوران سال 1400

داوران سال 1401

کاربرد مدل طبقه پنهان چندبعدی مبتنی بر نظریه پاسخ سوال (LCMIRT) درمطالعه ابعاد شناختی و خوشه بندی سؤال های آزمون ریاضی: مطالعه موردی درس ریاضی کنکور سراسری ریاضی فیزیک

نوع مقاله : مقاله پژوهشی

نویسندگان

1 استادیار، تحقیقات آموزشی، دانشگاه خوارزمی، تهران، ایران

2 استادیار، تحقیقات آموزشی، دانشگاه خوارزمی، تهران، ایران.

3 کارشناسی ارشد، تحقیقات آموزشی، دانشگاه خوارزمی، تهران، ایران.

چکیده

مهارت ریاضی شامل حوزه های محتوایی و فرایندهای شناختی مختلف است که بیانگر پیچیدگی توانایی ریاضی و ویژگی های پنهان آن است. تاکنون مطالعه این پیچیدگی ها با روش های سنتی تحلیل داده ها یا به صورت ذهنی انجام شده است. بنابراین، پژوهش حاضر با استفاده از مدل چندبعدی طبقه پنهان پاسخ سوال (LCMIRT) به مطالعه ابعاد شناختی و متغیر های پنهان درس ریاضی آزمون سراسری ورود به دانشگاه پرداخت. در این راستا، داده های درس ریاضی آزمون سراسری سال های 87، 90 و 94 مورد مطالعه قرار گرفت. یافته ها نشان داد که آزمون ریاضی، به عنوان یکی از دروس سرنوشت ساز در آزمون سراسری ورود به دانشگاه، شامل مجموعه ای از ویژگی های شناختی چندبعدی است. نتایج روش تحلیل موازی وحدت یافته نشان داد که آزمون ها با مدل تک بعدی برازش ندارند و اضافه کردن ابعاد بیشتر، برازش مدل را به صورت معنی داری بهبود می بخشد. بعلاوه، حوزه های شناختی درک، حل مسئله و استدلال سه سازه اساسی در تبیین توانایی ریاضی شناسایی شدند که در خوشه بندی و تحلیل سؤالهای آزمون، اطلاعات دقیق تری از کیفیت سوال ها به دست می دهند. این ویژگی LCMIRT ، در مقایسه با سایر رویکردها، تعداد زیر خوشه های شناختی را افزایش می دهد. در نهایت توصیه می شود که در ساخت و تحلیل آزمون های روانی و تربیتی، مدل های LCMIRT مد نظر قرار گیرند تا آشکارسازی توانایی های شناختی پنهان آزمون ها از اعتبار بیشتری برخوردار باشد.

کلیدواژه‌ها

عنوان مقاله [English]

Application of Latent Class Multidimensional Item Response Theory (LCMIRT) on investigating cognitive dimensions and clustering math test items: A case study of the math test of national Konkur of physics and mathematics

نویسندگان [English]

  • Masoud Geramipour 1
  • Maryam Moghadasin 2
  • Reyhaneh Rezazadeh 3

1 Faculty member at faculty of psychology and education, Kharazmi university

2 Faculty member at faculty of psychology and education, Kharazmi university

3 MA of educational research, Kharazmi university

چکیده [English]

Math skills include different content and cognitive processes domains which indicate the complexity of math skill and its latent traits. Until now, Study of these complexities has been done through traditional data analysis or subjective methods. Therefore, present research studied cognitive dimensions and latent variables of the math test of national university entrance examination through the application of Latent Class Multidimensional Item Response Theory (LCMIRT). In doing so, the math test data of national examinations in 2008, 2011 and 2015 were studied. Results showed that the math test, as a high stakes test in the national university entrance examination, encompasses a set of multidimensional cognitive traits. Results of the unified parallel analysis method showed that the tests do not fit with the unidimensional model and adding additional dimensions can significantly improve model fit. Moreover, cognitive domains of comprehension, problem solving and reasoning were recognized as three essential constructs which account for math skills and give more detailed information about items quality in clustering and analysis of the test items. This property of LCMIRT, in comparison with other approaches, increases the number of cognitive sub-clusters. At last, it is recommended that LCMIRT models are considered in constructing and analyzing educational and psychological tests, in order to more validly detect latent cognitive skills of the tests.

کلیدواژه‌ها [English]

  • Cognitive Dimensions
  • Latent Class Multidimensional Item Response Theory (LCMIRT)
  • Test Items Clustering
  • Mathematics
  • High Stakes Tests
مراجع
Ackerman, T. (1989). Unidimensional IRT calibration of compensatory and
 noncompensatory multidimensional items. Applied Psychological Measurement, 13 (2), 113-127.
Ackerman, T. (1996). Graphical representation of multidmensional item response theory analyses. Applied Psychological Measurement, 20, 311–329.
Adams, R. J., Wilson, M., & Wang, W. C. (1997). The multidimensional random coefficients multinomial logit model. Applied psychological measurement, 21 (1), 1-23.
Anderson,L.W.,&Krathwohl,D.R. (2001).A Taxonomy for learning,teaching,and assessing: A Revision of Bloom's Taxonomy of educational objectives.NewYork:Longman.
Ansley, T. N., & Forsyth, R. A. (1985). An examination of the characteristics of unidimensional IRT parameter estimates derived from two-dimensional data. Applied Psychological Measurement, 9, 37-48.
Bacci, S., Pandolfi, S., & Pennoni, F. (2014). A comparison of some criteria for states selection in the latent Markov model for longitudinal data. Advances in Data Analysis and Classification, 8 (2), 125-145.
Bartolucci, F. (2007). A class of multidimensional IRT models for testing unidimensionality and clustering items. Psychometrika, 72 (2), 141-157.
Bolt, D. M., & Lall, V. F. (2003). Estimation of compensatory and noncompensatory multidimensional item response models using Markov chain Monte Carlo. Applied Psychological Measurement, 27 (6), 395-414.
Bradlow, E. T., Wainer, H., & Wang, X. (1999). A Bayesian random effects model for testlets. Psychometrika, 64 (2), 153-168.
Bragg, L. A., Vale, C., Herbert, S., Loong, E., Widjaja, W., Williams, G., & Mousley, J. (2013, January). Promoting awareness of reasoning in the primary mathematics classroom. In MAV 2013: Mathematics of the planet earth: Proceedings of the MAV 50th Annual Conference 2013 (pp. 23-30). Mathematical Association of Victoria.
Brooks, S., Gelman, A., Jones, G., & Meng, X. L. (Eds.). (2011). Handbook of Markov Chain Monte Carlo. CRC press.
Carroll, J. B. (1945). The effect of difficulty and chance success on correlations between items or between tests. Psychometrika, 10 (1), 1-19.
Chen, W. H., & Thissen, D. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22 (3), 265-289.
Christensen, K. B., Bjorner, J. B., Kreiner, S., & Petersen, J. H. (2002). Testing unidimensionality in polytomous Rasch models. Psychometrika, 67 (4), 563-574.
De Champlain, A. F. (1996). The effect of multidimensionality on IRT true-score equating for subgroups of examinees. Journal of Educational Measurement, 33, 181-201.
De Champlain, A., & Gessaroli, M. E. (1998). Assessing the dimensionality of item response matrices with small sample sizes and short test lengths. Applied Measurement in Education, 11 (3), 231-253
Drasgow, F., & Lissak, R. I. (1983). Modified parallel analysis: A procedure for examining the latent dimensionality of dichotomously scored item responses. Journal of Applied psychology, 68 (3), 363-373.
Dubravka Svetina & Roy Levy (2014) A Framework for Dimensionality Assessment for Multidimensional Item Response Models, Educational Assessment, 19:1, 35-57.
Frederiksen, N. (Ed.). (1990). Diagnostic Monitoring of Skill and Knowledge Acquisition. Psychology Press.
Fraser, C., & McDonald, R. P. (1988). NOHARM II: A FORTRAN program for fitting unidimensional and multidimensional normal ogive models of latent trait theory. The University of New England, Armidale, Australia. : University of New England, Centre for Behavioral Studies.
Geramipour, M., & Shahmirzadi, N. (2018). Application and Comparison of Multidimensional Latent Class Item Response Theory on Clustering Items in Comprehension Tests. The Journal of AsiaTEFL, 15 (2), 479-490.
Goodman, L. A. (1974). Exploratory latent structure analysis using both identifiable and unidentifiable models. Biometrika, 61 (2), 215-231.
Gu, Z. (2011). Maximizing the potential of multiple-choice items for Cognitive Diagnostic Assessment (Doctoral dissertation, University of Toronto).
Hambleton, R. K., & Rovinelli, R. J. (1986). Assessing the dimensionality of a set of test items. Applied psychological measurement, 10 (3), 287-302.
Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications (Vol. 7): Springer.
Hayton J. C., Allen D. G., and Scarpello V. (2004). Factor Retention Decisions in Exploratory Factor Analysis: A Tutorial on Parallel Analysis. Organizational Research Methods. 7 (2): 191–205
Heinen, T. (1996). Latent class and discrete latent trait models: Similarities and differences. Sage Publications, Inc.
Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30 (2), 179-185.
Hulin, C. L., Drasgow, F., & Parsons, C. K. (1983). Item response theory: Application to psychological measurement. Homewood, IL: Irwin.
Kim, H. R., & Stout, W. F. (1994). A new index for assessing the amount of multidimensionality and/or simple structure present in test data. In annual meeting of the American Educational Research Association, New Orleans.
Koretz, D. (2008). Measuring Up: What Educational Testing Really Tells Us. Cambridge, MA: Harvard University Press. pp. 7-14.
Langeheine, R., & Rost, J. (Eds.). (2013). Latent trait and latent class models. Springer Science & Business Media.
Lazarsfeld, P. F., & Henry, N. W. (1968). Latent structure analysis. Houghton Mifflin Co.
Lord, F. M. (1980). Applications of item response theory to practical testing problems:Mahwah, NJ: Erlbaum.
Lord, F. M., Novick, M. R., & Birnbaum, A. (1968). Statistical theories of mental test scores. (pp.359-382). Reading, MA: Addison-Wesley.
Masters, G. N. (1985). A comparison of latent trait and latent class analyses of Likert-type data. Psychometrika, 50 (1), 69-82.
Nandakumar, R. (1994). Assessing essential dimensionality of set of items. Journal of Educational Measurement,31:1.
Payne, DA (2003). Applied Educational Assessment (2nd ed.). US: Wads Worth.
Reckase, M. D. (1979). Unifactor latent trait models applied to multifactor tests: Results and implications. Journal of Educational and Behavioral Statistics, 4 (3), 207-230.
Reckase, M. (2009). Multidimensional item response theory (Vol. 150). New York: Springer.
Rost, J. (1988). Rating scale analysis with latent class models. Psychometrika, 53 (3), 327-348.
Rost, J. (1990). Rasch models in latent classes: An integration of two approaches to item analysis. Applied Psychological Measurement, 14 (3), 271-282.
Roznowski, M., Tucker, L. R., & Humphreys, L. G. (1991). Three approaches to determining the dimensionality of binary items. Applied Psychological Measurement, 15 (2), 109-127.
Santrock, J. W. (2004). Educational Psychology. New York: McGraw-Hill.
Shealy, R., & Stout, W. (1993). A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF. Psychometrika, 58 (2), 159-194.
Sireci, S. G., Thissen, D., & Wainer, H. (1991). On the reliability of testlet‐based tests. Journal of Educational Measurement, 28 (3), 237-247.
Slavin, R. E., & Davis, N. (2006). Educational psychology: Theory and practice.
Stout, W. F. (1987). A nonparametric approach for assessing latent trait unidimensionality. Psychometrika, 52 (4), 589-617.
Sympson, J. B. (1978). A model for testing with multidimensional items. Paper presented at the Proceedings of the 1977 Computerized Adaptive Testing Conference, Minneapolis: University of Minnesota, Department of Psychology, Psychometric Methods Program.
Takane, Y., & De Leeuw, J. (1987). On the relationship between item response theory and factor analysis of discretized variables. Psychometrika, 52 (3), 393-408.
Traub, R. E., & Lam, Y. R. (1985). Latent structure and item sampling models for testing. Annual review of psychology, 36 (1), 19-48.
Walker, C. M., & Beretvas, S. N. (2003). Comparing multidimensional and unidimensional proficiency classifications: Multidimensional IRT as a diagnostic aid. Journal of Educational Measurement, 40 (3), 255-275.
Way, W. D., Ansley, T. N., & Forsyth, R. A. (1988). The comparative effects of compensatory and noncompensatory two-dimensional data on unidimensional IRT estimates. Applied Psychological Measurement, 12 (3), 239-252.
Whitely, S. E. (1980). Multicomponent latent trait models for ability tests. Psychometrika, 45 (4), 479-494.
Wilson, D., Wood, R., & Gibbons, R. (1987). TESTFACT [Computer program]. Mooresville IN: Scientific Software.
Van der Linden, W. J., & Hambleton, R. K. (1996). Handbook of modern item response theory: Springer.
von Davier, A. A., & Wilson, C. (2008). Investigating the population sensitivity assumption of item response theory true-score equating across two subgroups of examinees and two test formats. Applied Psychological Measurement, 32 (1), 11-26.
Yen, W. M. (1993). Scaling performance assessments: Strategies for managing local item dependence. Journal of educational measurement, 30 (3), 187-214.
Zhang, J. (2004). Comparison of unidimensional and multidimensional approaches to IRT parameter estimation. ETS Research Report Series, 2004 (2), i-40.
Zwick, R. (1987). Assessing the dimensionality of NAEP reading data. Journal of Educational Measurement, 24 (4), 293-308.
فصلنامه اندازه گیری تربیتی
دوره 10، شماره 36
تیر 1398
صفحه 137-166
فایل ها
هم رسانی
ارجاع به این مقاله
آمار