Ebrahim Alizadeh
Abstract
The objective of this study is to investigate the sources of systematic errors associated with the scores obtained from interviews of job applicants and to estimate the generalizability and dependability coefficients within the framework of generalizability theory. Accordingly, the data on the job competency ...
Read More
The objective of this study is to investigate the sources of systematic errors associated with the scores obtained from interviews of job applicants and to estimate the generalizability and dependability coefficients within the framework of generalizability theory. Accordingly, the data on the job competency assessment scores of the applicants for the three positions of one the public universities were analyzed using a two-way, fully crossed design. Then, the regular sources of error, including the questions, interviewers and interaction of errors were determined and measured, with the help of which the generalizability and dependability coefficients of the scores were estimated. The findings suggest that generalizability and dependability coefficients of the assessment scores are not equal in all job interviews. Moreover, since the generalizability and dependability coefficients are less than expected, the mean assessment score of one job should not be the basis for decision-making for all the three job selections. Based on the results, it is suggested that, before the adoption of the interviewers' opinions about the merits of applicants, the generalizability and dependability coefficients of the scores be estimated using generalizability theory and if the coefficients are low, the job interview scores should not be the basis for decision-making.
Noor-Ali Farroukhi; laila bahrami
Abstract
Background: Recognizing multiple sources of measurement error and estimates each source separately, distinguishes between relative and absolute decisions, distinguishes between fixed and random facets and also the capability of dealing with different D study designs can be mentioned as the strength points ...
Read More
Background: Recognizing multiple sources of measurement error and estimates each source separately, distinguishes between relative and absolute decisions, distinguishes between fixed and random facets and also the capability of dealing with different D study designs can be mentioned as the strength points of generalizability theory which have no corresponding statuses in classical test theory. Generalizability theory is unknown for our researchers and there are rare researches in this area. Objective: The Purpose of this article was introduction of generalizability theory and presentation the practical applicability of this theory in assessing the reliability of measurements. Results: In addition to comparison between classical test theory and generalizability theory, conceptual framework of generalizability theory was explained easily. Also, in this article the process of design, analysis and interpretation of a measurement study in shape of an example with relevant calculations and equations explained in detail in 15 steps to guide researchers and test developers who aimed to assessing reliability. Conclusion: This article shows that utility of generalizability theory in reliability estimation especially in complicated measurement situations is more than classical test theory. Generalizability theory enables researchers to decrease errors in plan of measurement through optimization proceedings which will increase accuracy in generalization of results.
Ibrahim Alizadeh; Mohammad Hossein Mohebbi Noureddin-vand
Volume 4, Issue 14 , January 2014, , Pages 1-24
Abstract
Objectives: The major aim of this research was to introduce Generalizability Theory and its application to estimate the reliability of job analysis data.
Methods: Twenty employees of a state bank certified administrative specialist jobs were randomly selected from among qualified employees. After training, ...
Read More
Objectives: The major aim of this research was to introduce Generalizability Theory and its application to estimate the reliability of job analysis data.
Methods: Twenty employees of a state bank certified administrative specialist jobs were randomly selected from among qualified employees. After training, they were given job skills questionnaire to rate the importance of each of the skills for the job of administrative experts to determine.
Results: To estimate the reliability of job analysis data in the framework of Generalizability Theory, a five-step approach was introduced that can be used to estimate the reliability of job analysis data. The steps are: 1- Identify the object of measurement, 2- identify facets of measurement, 3- identify an appropriate measurement design, 4- estimate variance components and their interpretation and 5- identify the type of decision.
Conclusion: It became apparent that the generalizability theory is useful to estimate the reliability of job analysis. It provides the opportunity for job analysis in such a manner that in different situations such as estimating the contribution of each systematic error sources, such as career history, age, and place of work assessors and achieving the desired reliability, it can perform the required activities.
