Somayeh Bahmanabadi; Mohammad reza Falsafinejad; noorali farrokhi; asghar minaei
Abstract
Identification of the consequences of violations of the assumptions of measurement models is a major concern in the field of psychometrics. The purpose of the present study was to investigating the role of test dimensionality violation in equating errors of IRT and classical theory models. Research method ...
Read More
Identification of the consequences of violations of the assumptions of measurement models is a major concern in the field of psychometrics. The purpose of the present study was to investigating the role of test dimensionality violation in equating errors of IRT and classical theory models. Research method was experimental and a 3 × 3 factorial design was used. The study population consisted of all the Mathematical and Technical Sciences Entrance Examiners in 2017 and 2018. The sample group consisted of 5000 examiners who were selected by random sampling. Mathematic test (55 items) was used for equating that three different types of data structure including one-dimensional, two- dimensional and three-dimensional data was generated from it. Data was equated with three equating methods including equipercentile method, true score and observed score equating method. Equating standard errors, bias and root mean square error were used to evaluate the effects of independent variables. To determine the error statistics, the equating results in all three datasets and the three equating methods were repeated 20 times in different samples. Data analysis showed that violation of dimensionality increases equating standard error, bias of equating results and the root mean square error. The bias of equating results in classical methods is more than the IRT methods. The effect of dimensionality violation on equating errors was not different between the classical theory and IRT models.
Hassan Moshtaghian Abarghouei; Mohammad Reza Flasafi Nejad; Ali Delavar; Noor Ali Farrokhi
Abstract
Identifying distractors as sources of Differential Item Functioning(DIF) in polyotomous items has great importance to designers and analysts. Although DIF is one of the common methods for examining the measurement invariance, It is accompanied by challenges and limitations, especially in multiple choice ...
Read More
Identifying distractors as sources of Differential Item Functioning(DIF) in polyotomous items has great importance to designers and analysts. Although DIF is one of the common methods for examining the measurement invariance, It is accompanied by challenges and limitations, especially in multiple choice items. The purpose of this study was to assess the performance of Nested logit Model(NLM) for detecting Differential Distractor Functioning(DDF) by using experimental (simulated data) and descriptive-analytical (real data) methods. Six items were simulated under different conditions of difficulty and slope, ability distribution, presence or absence of DIF/DDF, and DIF/DDF magnitude, with a sample size of 2000 and 50 replicates. The data of Math Entrance Exam (D-form,2018), with a random sample of 2000 men and women constituted the real data. Based on the results of the simulation analysis: The NLM revealed 88% of DIF and 97% of DDF, on average. the Type I error rates is very close to the theoretical expected values, although it showed some inflation in unequal distribution conditions. according to the findings, the detection rate was influenced by the item parameters(difficulty and slope) and the DIF or DDF levels. Based on real data analysis, 2 items represented both DIF(Large and Medium) and DDF (Partial to Moderate) simultaneously, whereas in the NRM approach, 11 items detected as DIF/DDF; so, as expected the approaches based on “divided by distractor” strategy, fewer items were detected as DIF/DDF. The NLM approach, while separating the DDF from the DIF test, allows for a clear evaluation of whether the distractor may be responsible for DIF. Since high-stakes tests have a special role in selection and DIF and DDF analyzes have a special place in determining the validity and measurement invariance of these exam items, it is recommended to screen the bias items, DIF/DDF comprehensive analyzes based on NLM be used.
meysam sadeghi; mohammadreza falsafinejad; ali delavar; noorali farrokhi; ehsan jamali
Abstract
The purpose of the present study was to identify the weight of each of the courses of academic record and scoring based on the psychometric approach of the IRT continuous model and approaches based on the viewpoint of the specialists of Topsis and AHP. The method of this study was a combination. In order ...
Read More
The purpose of the present study was to identify the weight of each of the courses of academic record and scoring based on the psychometric approach of the IRT continuous model and approaches based on the viewpoint of the specialists of Topsis and AHP. The method of this study was a combination. In order to conduct research in the first part, 11 items (courses) of academic background were taken from the expert group and weighed and prioritized using Tapis and AHP techniques. The final third year grades of secondary school students were received from the assessment organization and accepted for psychology and counseling in the humanities group. Theta tests were used to estimate the grades. In order to compare different weighting and grading models, the sample group was first ranked based on scores derived from all three approaches, and compared in terms of dispersion indices and the degree of difference between rankings. The results of the comparison of the three approaches indicated that the most variance is related to the IRT continuous modeling method. The results of Wilcoxon's statistical analysis to compare the average rankings showed that in the small sample size, the weighting method in all three methods produced a significant difference in the rankings compared to the same weight method, and therefore the weighting model in the ranking of the volunteers In small sample size affects. But there are no significant differences between the three weighting models in the candidate ranking. Also, the results showed that in the high sample size (2000 persons), the weighting method in all three methods compared to the same weights makes a significant difference in the rankings. IRT method scores are more reliable than expert-based approaches.However, the weighting method does not seem to have a significant effect on the reliability of the scores.
sara yadollahi; mohammadreza falsafinezhad; Ahmad Borjali; noorali farokhi
Abstract
Background: The ability to understand and being able to profoundly and deeply understand the within-person differences entail profile analysis which is considered to be a serious challenge in psychometrics area. Aim: This study was aimed at applying Profile Analysis via Multidimensional Scaling (PAMS) ...
Read More
Background: The ability to understand and being able to profoundly and deeply understand the within-person differences entail profile analysis which is considered to be a serious challenge in psychometrics area. Aim: This study was aimed at applying Profile Analysis via Multidimensional Scaling (PAMS) procedure in order to extract prototypical profiles underlying the Wechsler Intelligence Scale for Children- Fourth Edition (WISC-IV). Method: The design of the study is correlational and the population of the study consisted of all of the students studying at 6th distinct schools of Tehran. Using random cluster sampling method 200 students were selected. The sample were administered full subscales of WISC-IV. Results: Data analysis using the PAMS procedure resulted in extracting 3 cumulative profiles. Statistically meaningfulness of the subtests within profiles were determined and they were labeled according to their dimensionality and underlying meaning (processing speed vs. working memory, lexical knowledge vs. visual alertness and perceptual reasoning vs. verbal comprehension). Conclusions: using the PAMS procedure in interpreting WISC-IV results could be beneficiary. Relying on individual parameters further clinical and applied implications are discussed.
negar sharifi; mohammad falsafi; noorali farokhi; ehsan jamali
Abstract
Background: Test fairness is one of the main challenges in transition from paper- pencil towards computerized adaptive testing (CAT). Aim: This study was aimed at investigating differential item function (DIF), assessing intervening factors in clarifying DIF and suggesting the optimal method for DIF ...
Read More
Background: Test fairness is one of the main challenges in transition from paper- pencil towards computerized adaptive testing (CAT). Aim: This study was aimed at investigating differential item function (DIF), assessing intervening factors in clarifying DIF and suggesting the optimal method for DIF in computerized adaptive testing. Method: The empirical method was applied based on the nature of the study area. Data gathering procedure and manipulating the variables were done using simulation method. The responses of 1000 examinees (reference and focal group with equal 500 numbers) to item bank of 55 dichotomous items were simulated based on 3-parameter logistic model with 20 iterations. Fifteen items were manipulated in terms of DIF type and magnitude and test impact was evaluated based on mean difference of comparison groups. Computerized adaptive test with 30 items was administered via Firestar software package. Analysis was done by logistic regression (LR) and item response theory-likelihood ratio test (IRT-LRT) and the methods were compared based on their power and type I error rate. Results: Type I error rate of likelihood ratio test was less than logistic regression. The power of the methods was influenced by type, magnitude of DIF and test impact. Comparing with logistic regression, Item response theory-likelihood ratio test had more power in detecting uniform DIF for the impact and no-impact conditions and it showed more power by increasing the magnitude of DIF. The two methods showed no difference in assessing non-uniform DIF and both of them were poor. Conclusion: Given the power and type I error rate, likelihood ratio test is an optimal approach in detecting uniform DIF. However, assessing non-uniform DIF requires further investigation.
somayeh pourehsan; Mohammadreza Falsafinejad; Ali Delavar; Noorali Farokhi; Ahmad Borjali
Abstract
The main goal of this research was to apply the multi-level in determining the role of metacognitive components associated with drug abuse among adolescents of Kerman Province. The method used in this research was correlation type. In order to conduct this research 1000 school students of Kerman Province ...
Read More
The main goal of this research was to apply the multi-level in determining the role of metacognitive components associated with drug abuse among adolescents of Kerman Province. The method used in this research was correlation type. In order to conduct this research 1000 school students of Kerman Province have been selected by using multistage cluster sampling method. For data collection in this research, ZARGAR drug addiction preparation Questionnaire and Wels’s Metacognitive Questionnaire (MCQ30) have been used. The results of multi-level approach using One-way random effects Anova showed that the four components (positive beliefs about worries, negative beliefs about worries, low cognitive trust and cognitive self-awareness) explained about 49% of the variance of addiction tendency at school level. Moreover, the random intercept model explained 12% of the student-level variance of drug abuse tendency. Here, the significance of qi-square with the value of 37.983 at the significant level (P <0.0001) reflects the significant change of the null model and model 1. In response to the question that how much the relationship between the tendency to drug abuse and student-level components is similar among schools, a comprehensive model with both student-level and school-level variables (model 2) was used. In model 2, the results showed that at the school level, the mean of positive beliefs and cognitive self-awareness about drug abuse tendency have a significant relationship, but the mean of cognitive trust and negative beliefs has not a significant relationship with the tendency to drug abuse. At the students level, there is a significant relationship between cognitive trust and negative beliefs with drug abuse tendency (P <0.01). In total, it can be said that for variables related to samples with a nesting nature, it is necessary to use all levels in data analysis.
somayeh kahdouei; mohammadreza falsafinejad
Abstract
Background: Educational success and determining the effects of personal and environmental factors on the construct is among the most important concerns of educational systems that guarantee the success of the educational consulting procedure. Aim: This study was aimed at designing and evaluating a model ...
Read More
Background: Educational success and determining the effects of personal and environmental factors on the construct is among the most important concerns of educational systems that guarantee the success of the educational consulting procedure. Aim: This study was aimed at designing and evaluating a model for educational success including personal and environmental factors, determining their importance and comparing them in terms of the student’s majors. Method: The design of the study was ex post-facto and the population of the study consisted of all of the students studying at high schools of Yazd. Using random cluster sampling method 550 students were selected. The sample were administered Major Choice Success Questionnaire, Myers-Briggs Type Indicator, Strong Interest Inventory and Multiple Intelligence Test. Data Analysis was done using structural equation modelling. Results: The results indicated that all five factors (interests, personality, intelligence, educational performance and environmental factors) had meaningful, direct impact on educational success. Interest played the most important role and environmental factors were the least important component. Moreover, educational background, interest and multiple intelligence were the most important factors in empirical science and mathematics major. Conclusions: For counseling purposes the role of personal and environmental factors in each major should be considered. Keywords: causal model, educational counseling, educational success.
