Reliability and Validity of Locally Developed English Proficiency Tests: A Case Study of the Department of English, Sulugh Campus, University of Benghazi, Libya

الملخص

This case study explores the reliability and validity of locally developed English proficiency tests administered in the Department of English, Sulugh Campus, University of Benghazi, Libya. The study adopted a convergent mixed-methods approach combining a quantitative analysis of test scores from two academic cohorts (2021/2022 and 2022/2023) with qualitative data from semi-structured interviews with five instructors, two focus groups with twenty undergraduate students, and ten non-participant classroom observations. Internal consistency, measured using Cronbach’s alpha, was .65 to.75 across cohorts (α =.72 and α =.68, respectively), reflecting moderate reliability, which is sufficient for group-level interpretation, but not high-stakes decisions about individual students. However, qualitative findings indicate considerable validity issues: around 60-70% of total marks were said to comprise discrete-point grammar and vocabulary items, and speaking was virtually excluded from formal assessment. Instructors and students agreed that the tests were narrowly focused on isolated linguistic knowledge, rather than on communicative competence, and were poorly aligned with descriptors of the Common European Framework of Reference for Languages (CEFR). The classroom observations also showed a widespread negative washback, since the instruction focused on test preparation rather than overall language development. The study concludes that the tests are sufficiently reliable, but their limited content and construct validity hinder their ability to measure real-world communicative ability. The study recommends re-designing test blueprints around authentic communicative tasks, providing structured assessment-literacy training for faculty, and embedding formative assessment to reduce negative washback and better serve Libyan EFL learners.