Reyhane Rahimi; Aso Mojtahedi
Abstract
The goal of this research is to explore the Likert scale questions using two distinct methods: Classical Test Theory and Item Response Theory. By comparing the results of these approaches, the study aims to address the question: "Do the outcomes from these two methodologies align, or do they ...
Read More
The goal of this research is to explore the Likert scale questions using two distinct methods: Classical Test Theory and Item Response Theory. By comparing the results of these approaches, the study aims to address the question: "Do the outcomes from these two methodologies align, or do they contradict each other?" The research design followed a descriptive methodology and utilized secondary analysis techniques. The study population consisted of 977 junior high school students. After the data screening process, the final sample size for analyzing extraversion items was 783 students, 763 students for openness items, and 784 students for conscientiousness items. The research instruments were the three subscales of extraversion, openness, and conscientiousness from the Neo Personality Test. The statistical analysis yielded results indicating that a strong internal consistency among items enhanced the accuracy and validity of outcomes derived from the graded response model. However, when items exhibit low internal consistency, caution should be exercised, as the model may yield erroneous thresholds or discrimination coefficients (i.e., false negative or positive). Overall, combining multiple methods of statistical analysis can significantly contribute to more effective analysis and obtaining highly accurate results.
Zahra Jahanbakhsh; Noorali Farrokhi; Jalil Younesi
Abstract
In order to increase the precision of measurement, survey studies apply scales for the evaluation of latent features. This study aimed to examine differential item functioning (DIF) of Academic Alienation scale items in students based on the Item Response theory (IRT) framework. DIF analysis is used ...
Read More
In order to increase the precision of measurement, survey studies apply scales for the evaluation of latent features. This study aimed to examine differential item functioning (DIF) of Academic Alienation scale items in students based on the Item Response theory (IRT) framework. DIF analysis is used to test the hypothesis if the item parameters for the different groups of identical ability are invariant. DIF analyses of gender and education duration were performed using a sample of 1100 Psychology and educational sciences students in the city of Tehran. The Academic Alienation items (Burbach, 1972) studied were 4-point Likert-type scales. The questionnaire had three subscales: isolation, meaningless, and powerlessness. Analyses were conducted using the graded item response model with likelihood ratio (IRT-LR) tests of DIF. DIF analyses were conducted with 4 anchor items and after Bonferroni adjustment only 3 items based on gender and 2 items according to education duration were detected with meaningful DIF. The explanation of DIF detection can be discussed based on the cultural context of universities. Sexual inequality, teachers’ attitudes, university management policies, and finally educational injustice may be the reasons for the sense of academic alienation in students.
Ali Baniasadi; Keyvan Salehi; Ebrahim Khodaie; Khosro Bagheri; Balal Izanloo
Abstract
The present study aimed to investigate the psychometric properties of fair classroom assessment Rubric based on Item-Response theory. For this purpose, a sample of 511 students of the University of Tehran was selected by the available sampling method and answered Rubric questions. At this stage, to determine ...
Read More
The present study aimed to investigate the psychometric properties of fair classroom assessment Rubric based on Item-Response theory. For this purpose, a sample of 511 students of the University of Tehran was selected by the available sampling method and answered Rubric questions. At this stage, to determine the application of unidimensional or multidimensional models, DETECT and parallel analysis methods were used. The results of both methods rejected the unidimensionality of the data and the results of the parallel analysis showed the extraction of three factors from the data. Also, the comparison of unidimensional and multidimensional model fit indices including log-likelihood, likelihood ratio, Root Mean Square Error of Approximation and comparison of Bayesian and Akaike information criteria confirmed the better fit of the multidimensional model for the data. Thus, due to the polytomous of the answers to the questions, the multidimensional graded response model was used to estimate the parameters of the questions. The reliability of each of the subscales of procedural fairness, nature of assessment and interactional fairness were 0.85, 0.69 and 0.63, respectively. Estimation of the discrimination parameters ranged from 1.048 to 5.802, which showed that all the questions performed well in the discrimination of the upper and lower levels of the fair classroom assessment, and after controlling the false discovery rate, the S-X2 statistic showed a good fit of all Rubric questions. In general, the results of this study show that the developed Rubric has appropriate psychometric properties to evaluate the quality of fairness in the classroom assessment.
mohammad ahmadi deh qutbuddini; ebrahim khodai; Valiollah Farzad; ali moghadam zadeh; masoud kabiri
Abstract
The present study has been done with the purpose of investigating the dimensionality and differential item functioning of the testlet-based test of Iran's PIRLS 2011. In order to analyze the dimensionality, graded response and bi-factor item-response theory models were used with full-information maximum ...
Read More
The present study has been done with the purpose of investigating the dimensionality and differential item functioning of the testlet-based test of Iran's PIRLS 2011. In order to analyze the dimensionality, graded response and bi-factor item-response theory models were used with full-information maximum likelihood estimation method and to analyze the differential item functioning multiple-group bi-factor model of Cai et al (2011) was applied. The results of the dimensionality investigation showed that the bi-factor model is better fitted to the data than the graded response model both in Iran's total sample and in boy and girl groups. The results of testlets effect variance showed that effects of second factors on Iranian students' performance in two testlet related to literal comprehension, has caused dimensionality in Iran's PIRLS testlets. The results showed that there was no significant difference in average students' performance of the boy and girl in general latent trait of reading comprehension, but the difference between the average reading proficiency of the boy and the girl in three literal and three informational testlet in favor of girls was significant. The result of differential items functioning based on the bifactor model showed that many items have an uniform and non-uniform differential item functioning, and boys in multiple-choice items and girls in constructed response items have better performances. In general, the results showed that in Iran's PIRLS 2011 testlets, the traits related to the two literal comprehension testlets were differently perceived between boy and girl students, and these two testlet had more local item dependence among girls than boys. Also, the results indicated a difference between the performance of Iranian boy and girl students in the mixed items format test of PIRLS.
