Reliability and Validity Claims of a Foreign Language Proficiency Test

This analysis examined the reliability and validity claims of a modified version of the Spanish Modern Language Association Foreign Language Proficiency Test. The study used a computerized version that was administered to 277 test-takers. The study’s design was framed by the investigative steps proposed by Messick (1989) and Nitko and Brookhart (2007). An ex post facto approach was used, which mixed the confirmatory factor analysis and multiple regression analyses with descriptive interpretations. The quantitative analyses examined the impact of nationality, gender and average time spent on completing the tasks on the five subscales of this particular examination. The analyses provided mixed results related to the predictors’ ability to explain the subscale scores. The descriptive analyses revealed numerous validity flaws and gaps in the test construction. Such inadequacies overshadowed the positive aspects and eventually led to my questioning of the hypothesis that the judgments based on data from these particular examinations are reliable and valid. Recommendations for possible revisions were made, along with suggestions for future research avenues.