Document Type : Research Paper
Abstract
According to many experts, entrance examination is the most important test in Iran and demographic characteristics such as gender, region (socio-economic status) and province (language) can affect the performance of participants’ responses to the test items. Controlling for the ability of examinees, if the answers to the test items are the function of demographic characteristics, the test items have Differential Item Functioning (DIF) toward them. The purpose of this investigation is to study Differential Item Functioning of the test items of Entrance Examinations of National Organization of Educational Testing across the demographic characteristics. The research sample consisted of all examinees of a test booklet which included some special exams of the experimental groups of Konkur from 2008 to 2011. Binary logisitic regression was used in order to analyzing DIF. After assuring unidimensionality through NOHARM approach, DIF analysis results indicated that the most number of detected DIF items were related to gender, region (socio-economic status) and province (language) variables respectively, but effect sizes were very small and negligible. Nevertheless, based on current research results, it is recommended that subject matter committees are formed to detect biased items based on final decisions of the committees and will be considered in designing test items in future.
Keywords
ایزانلو، ب.، و حبیبی، م.(۱۳۸۹). شناسایی کارکرد متفاوت سؤال بر اساس نظریه سؤال-پاسخ: کاربرد الگوی تک پارامتری با استفاده از نرم‌افزار بای لوگ-ام جی. فصلنامه روان‌شناسی کاربردی، (2) ۴، ۳۱ -۲۰.
خدایی، ابراهیم، (1388) الف. بررسی عوامل مؤثر بر قبولی در آزمون کارشناسی ارشد. فصلنامه پژوهش و برنامه‌ریزی در آموزش عالی، (54)، 34-19.
خدایی، ابراهیم (1388) ب. بررسی رابطه سرمایه اقتصادی و فرهنگی والدین دانش آموزان با احتمال قبولی آن‌ها در آزمون سراسری سال تحصیلی 1385. فصلنامه انجمن آموزش عالی ایران، (1)4، 84-65.
گرامی پور، مسعود (1393). ارزیابی توان آماری تحلیل رگرسیون لجستیک در آشکارسازی کنش افتراقی سؤال‌های آزمون. فصلنامه مطالعات اندازه‌گیری و ارزشیابی آموزشی، 4(8)، 211-187.
نوربخش، سید مرتضی (1389). نقش سرمایه‌های فرهنگی، اجتماعی و اقتصادی خانواده در موفقیت داوطلبان آزمون سراسری. فصلنامه برنامه‌ریزی رفاه و توسعه اجتماعی، 4،135-94.
همبلتون، رونالد ک؛ سؤامیناتان، اچ؛ راجرز، اچ. جین (1991). مبانی نظریه پرسش- پاسخ. ترجمه محمدرضا فلسفی‌نژاد (1389). تهران: دانشگاه علامه طباطبایی.
Abedalaziz, N. (2010). A gender-related differential item functioning of mathematics test items. The International Journal of Educational and Psychological Assessment, 5, 101- 116.
Amirian, S. M. R., Alavi, S. M., & Fidalgo, A. M. (2014). Detecting Gender DIF with an English Proficiency Test in EFL Context. Iranian Journal of Language Testing, 4(2).
Barati, H., & Ahmadi, A. R. (2010). Gender-based DIF across the Subject Area: A Study of the Iranian National University Entrance Exam. Journal of Teaching Language Skills, 2(3), 1-26.
Barnes, B. J., & Wells, C. S. (2009). Differential item functional analysis by gender and race of the national doctoral program survey. International Journal of Doctoral Studies, 4, 77-96.
Broer, M. (2005). Ensuring the fairness of GRE writing prompts: Assessing differential difficulty. ETS Research Report Series, 2005(1), i-41.
Byrne, B. M., & Stewart, S. M. (2006). The MACS approach to testing for multigroup invariance of a second-order structure: A walk through the process. Structural Equation Modeling, 13, 287-321.
Camilli, G., & Shepard, L. A. (1994). Methods for identifying biased test items. Thousand Oaks, CA: Sage Publications.
Cole, N.S., Zieky, M.J. (2001). The New Faces of Fairness. Journal of Educational Measurement, 38, no. 4: 369-382.
Dinis da Costa, P., & Araújo., L. (2012). Differential Item Functioning (DIF): What Functions Differently for Immigrant Students in PISA 2009 Reading Items?. Luxembourg: Publications Office of the European Union, Report EUR 25565 EN.
Gomez-Benito, J., & Navas-Ara, M. J. (2000). A comparison of χ2, RFA and IRT based procedures in the detection of DIF. Quality and Quantity, 34(1), 17–31.
Hambleton, R.K., Swaminathan, H. (1985). Item Response Theory: Principles and Applications. Kluwer–Nijhoff, Boston.
Harding, L. (2012). Accent, listening assessment and the potential for a shared-L1 advantage: A DIF perspective. Language Testing, 29(2), 163–180. DOI:10.1177/0265532211421161
Le, V. N. (1999). Identifying Differential Item Functioning on the NELS: 88 History Achievement Test. Center for the Study of Evaluation, National Center for Research on Evaluation, Standards, and Student Testing, Graduate School of Education & Information Studies, University of California, Los Angeles.
Lewis, Anne (April 2000). High-stakes testing: Trends and issues. Mid Continent Research for Education and Learning.
Mazor, K. M., Hambleton, R. K., & Clauser, B. E. (1998). Multidimensional DIF analyses: The effects of matching on unidimensional subtest scores. Applied Psychological Measurement. Vol 22(4), 357.367.
Doudeen, H., & Annabi, H. (2008). Sex-Related Differential Item Functioning (DIF) Analysis of TIMSS. Educational Sciences, 35, p697.
Millsap, R.E (2011). Statistical Approaches to Measurement Invariance. New York:NY, Routledge
Ogretmen, T. (2015). DIF analysis across genders for reading comprehension part of English language achievement exam as a foreign language. Educational Research and Reviews, 10(11), 1505-1513.
Owen, K. (1992b). Test-item bias: Methods, findings and recommendations. Pretoria: Human Sciences Research Council.
Popham,W.J. (2005). HIGH-STAKES TESTS: HARMFUL, PERMANENT, FIXABLE, American Educational Research Journal.6,p85.
Rezaee, A., & Shabani, E. (2009). Gender Differential Item Functioning Analysis of the University of Tehran English Proficiency Test. Pazhuhesh-e Zabanha-ye Khareji,56,89-108.
Rezaee, A., & Shabani, E. (2010). Gender differential item functioning analysis of the University of Tehran English Proficiency Test. Pazhuhesh-e Zabanha-ye Khareji, 56, 89, 108.
Shanmugam, S. K. S., & Lan, O. S. (2014). THE VALIDITY OF ADMINISTERING BILINGUAL MATHEMATICS TEST AMONG MALAYSIAN BILINGUAL STUDENTS USING DIFFERENTIAL ITEM FUNCTIONING (DIF). Asia Pacific Journal of Educators and Education, 29, 1-18.
Shuttleworth-Jordan, A.B. (1996). On not reinventing the wheel: A clinical perspective on culturally relevant test usage in South Africa. South African Journal of Psychology, 26 (2), 96- 102.
Shealy, R.T., Stout, W.F. (1993). A model based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF. Psychometrika 58, 197–239.
Song, X., Cheng, L., & Klinger, D. DIF investigations across groups of gender and academic background in a large-scale high-stakes language test. Language Testing and Assessment 4, 1, p97.
Swaminathan, H. & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361– 370.
Takala, S., & Kaftandjieva, F. (2000). Test fairness: A DIF analysis of an L2 vocabulary test. Language Testing, 17(3), 323-340.
Vanderberg, R. J. (2002). Toward a further understanding of and improvement in measurement invariance methods and procedures. Organizational Research Methods, 5, 139–158.
Van de Vijver, F. & Poortinga, Y.H. (1997). Towards an integrated analysis of bias in cross-cultural assessment. European Journal of Psychological Assessment, 13 (1), 29-37.
Van Zyl, E. & Visser, D. (1998). Differential item functioning in the figure classification test. Journal of Industrial Psychology, 24 (2), 25-33.
Young, J. W., Morgan, R., Rybinski, P., Steinberg, J., & Wang, Y. (2013). Assessing the Test Information Function and Differential Item Functioning for the TOEFL Junior® Standard Test. ETS Research Report Series, 2013(1), i-27
Zumbo, B. D. and Thomas, D. R. (1997). A measure of effect size for a model-based approachfor studying DIF. Prince George, Canada: University of Northern British Columbia, Edgeworth Laboratory for Quantitative Behavioral Science.
)