Abstract:
Considering validity as a unitary concept, this study investigated the construct validity of the Iranian Ministry of Health Language Exam (MHLE). To meet this objective, we first conducted item analysis and reliability analysis, and verified KR20-if-item-deleted indices on the scores of 987 MHLE test takers before running factor analysis. Though the test was found to enjoy a high level of reliability, it suffered from 28 problematic items flagged through item analysis and KR20-if-item-deleted indices. Next, we ran factor analysis on the data, screened through item analysis, by implementing Horn’s parallel analysis and Velicer’s minimum average partial (MAP) tests. Parallel analysis resulted in overfactoring. The MAP test, however, produced results with two to seven factors. Though the 4-factor result of the MAP test seemed to be more logical at first glance, the overall results were rather disappointing. Nineteen items did not load significantly on any factor and a clear pattern of item loading was not found for many items. These findings can be viewed as evidence detracting from the validity of MHLE.
Machine summary:
To address these gaps, we conducted the present study to assess item analysis of the test and to perform an in-depth examina- tion of the reliability and construct validity of the test uses in light of appropri- ate statistical computational methods.
Factor analysis, as a multivariate technique (Alavi & Ghaemi, 2011; Field, 2009; In'nami & Koizumi, 2011; Khine, 2013; Ockey & Choi, 2015; Sawaki 2012; Schmitt, 2011), refers to "an analytic method for determining the number and nature of the variables that underlie larger numbers of variables or measures" (Kerlinger, 1979, p.
Salehi (2011), for instance, employed exploratory factor analysis to investigate construct validity of a reading comprehension section of University of Tehran English Proficiency Test (UTEPT).
According to Brooks and Johanson (2003), this program reports test analysis information, including raw scores, percentage scores, summary statistics, reliability, standard error of measurement, item difficulty, item discrimination, and distractor analyses.
In classical test theory, item analysis is considered as a source of evidence for construct validity (Van der Walt & Steyn, 2008), and due to the fact that factor analysis is based upon correlation matrices, determining problematic items using item analysis has been regarded as being a crucial pre‐ liminary step (O’Connor, 2000).
Descriptive Statistics of the Total Test and the Main Sections of the MHLE Sections No. of Items Mean Median Std. Deviation No. of Subjects / Listening Compre- hension Grammar Vocabulary Reading Compre- hension Total / Item Analysis Table A1 in the Appendix reveals the item difficulty, item discrimination, and adjusted point-biserial correlation indices.