Validity Analysis of an Achievement Test

I. Introduction

Dose the test measure what it claims to measure？ If it does， it is valid. So， validity is the degree to which a test measures what it is supposed to measure， or can be used successfully for the purposes for which it is intended.

Achievement test refers to how much a learner has mastered the contents of a particular course. Clearly， the items in such a test should be on what has been taught. To judge the validity of an achievement test is just to see if it covers the requirements of candidates’ course， if it is relevant to teaching materials and contents. This sample test is a college English test as final examination for sophomores. I’ll analyze its validity from three aspects， say， face validity， content validity and concurrent validity.

II. Construction of this sample test

● Listening Comprehension 20%

● Reading Comprehension 40%

● Vocabulary and Structure 10%

● Cloze 10%

● Writing 20%

III. Face Validity

A test is considered to have face validity if it looks as if measures what it is supposed to measure， or it is the degree to which a test appears to measure the knowledge or abilities it claims to measure. For example， if dictation activity is used to measure reading ability， this test is judged as lacking face validity， even though there may be empirical evidence of a high correlation between these two skills.

This achievement test is to measure students’ mastery of what has been taught after a semester of learning the third volume of College English Course. We can say it possesses high face validity， after we inspect this test from its appearance， say types of items， which test candidates’ such basic skills as listening， reading， writing and some linguistic competence. On the other hand， test takers have performed on the test in a way that truly reflects their ability in terms of their scores and feedback.

IV. Content Validity

A test is said to have content validity if its content constitutes a representative sample of the language skills， structures， etc， with which it is meant to be concerned. It this test， students’ listening， reading and writing skills， language abilities and comprehensive competence are to be measured.

In Part I， Listening Comprehension， test constructor chose dialogues and short passages as the samples to test students’ skills to get key information from oral materials. We can see most of the topics are familiar to students， so the test can concentrate on their listening skills without test bias. In addition， these samples are all in oral style， avoiding those which are too formal. So， we can say the samples in this part are representative.

The same case is in Part II， Reading Comprehension， in which includes 4 passages. These passages talks about various themes， such as smoking， computer， human emotions and the relations between campus and society， in different styles which show characteristics of English for Academic Purpose. It is also clear that language difficulties and vocabulary are controlled in the range of this course book. Look at the following table：

From this table， we can say the samples in Reading Comprehension of this test are representative.

As for Part III， Vocabulary and Structure， it tests students’ knowledge about phrase meaning， usage， lexical collocation and grammatical structures， etc. The usage of prepositions is tested by 41， 46， 54， 61， 67， ellipsis by 45， verbs by 43， 70， nouns by 42， 48， 62， the usage of it by 49， etc. These representative samples measure well if students have mastered specific vocabulary and language structure and will gain positive feedback.

Part IV is Cloze which requires students’ comprehensive abilities to use language skills and structure， such as the ability to understand the main idea as soon as possible， to understand the relationships within sentences and the ability to employ grammatical and background knowledge. These abilities and skills are tested separately in this sample test. And the topic of this passage is very close with test takers， so it avoids the test bias——unfair content.

The last part is Passage Writing whose topic is very near to our daily life， even popular in daily life， so students’ schema can be stimulated sufficiently in their writing process. Then they can employ their language competence and other relevant skills to create as well as possible.

The above is the thorough inspection of this sample test. We then conclude that the samples in this test are representative enough for me to say it does have content validity.

In addition to the above conclusion， we say content validity refers to the extent to which the test adequately covers the syllabus area to be tested. It should reflect both the content and the balance of the teaching. That is to say， a test has content validity if it has tested what has been taught. Let’s have a look at the following table：

Then， we can see that what are tested can well match what has been taught and practiced.

IV. Concurrent Validity

Concurrent validity is concerned with relationship between what is measured by a test （usu. a newly developed test） and another existing criterion measure， which may be a well-established standardized test， or a set of judgment. If the two measures function similarly， they are considered to have concurrent validity. Concurrent validity is calculated by correlating the scores obtained by a single set of candidates the two measures and the degree of the agreement of the scores decides the concurrent validity of the test.

This sample test is for college students who are non-English majors. So， I compare it with CET-4， which is widely accepted to have high validity. Their types of test items are identical， say， Listening Comprehension， Reading Comprehension， Vocabulary and Structure， Cloze and Writing. And they follow the same criterion in designing. Take Reading Comprehension in these two tests for example； they both choose reading materials which are various in themes， styles， etc. On the other hand， the syllabus on which tests is based is the College English Syllabus which is the required range of CET-4. In detail， this sample test is much easier than CET-4 in terms of each item. So， the same candidates will get higher score in this test than in CET-4， but attribution of scores is nearly the same.

References

[1]Alderson， J. C. 1986. Innovations of Language Testing in Portal （ed）： 93～10

[2]Alderson， J. C. D.Wall. 1993. ‘Does Washback Exist？’ Applied Linguistics . 14：115～29

[3]成善德. 從語用學看近年來NMET 的構卷效度. 外語與外語教學（增刊），1999

[4]鄧衛東. 對高考英語題型改革的建議. 試題研究（高中英語），2002（369）

[5]黃大勇、楊炳均. 語言測試的反撥效應研究概述. 外語教學與研究，2002（7）