Educational evaluation plays a pivotal role in improving the quality of educational programs. This study aimed to analyze the alignment of master's theses in educational sciences with the standards of Kirkpatrick's four-level model. The statistical population included all theses from four top Iranian universities (Tehran, Allameh Tabataba'i, Shahid Beheshti, and Ferdowsi Mashhad) between 2005 and 2023, from which 26 theses were selected through census sampling based on inclusion criteria.
The research instrument was a researcher-made checklist comprising 21 indicators across four levels (reaction, learning, behavior, and results), whose validity was confirmed by three experts and reliability was calculated with Cohen's kappa coefficient of 0.92. Findings revealed that the reaction level showed the best performance with 95.4% alignment, while this rate decreased to 54.6% for learning, 34.8% for behavior, and 13.2% for results. Major weaknesses included insufficient use of advanced assessment methods (15%), lack of long-term follow-up (8%), and neglect of organizational indicators measurement (77%).
The results demonstrate an urgent need for revising evaluation research methodologies, developing indigenous tools for higher-level assessment, and emphasizing longitudinal evaluations. These findings are significant for enhancing the quality of educational evaluation research and improving the effectiveness of educational programs.