Validity of multiple—choice format in language testing

吳雪媚

【Abstract 】The aim of this paper is to discuss the validity MC format in language testing. It mainly includes an evaluation of MC format validity and some specific examples. This paper concludes by suggesting that although MC formats validity are still questionable， it is still quite applicable if we want to test the cognitive knowledge of a large number of candidates.

【Key words】MC format； language testing； validity

1. Introduction

Multiple -choice （MC） has been an important format for many language tests. However， it is still controversial concerning its effectiveness in testing language learners proficiency. To evaluate whether a language test instrument is effective or not， we have to take many factors into consideration. A very important factor is validity.

2. Evaluation

Validity is concerned with whether a test measures what it is intended to measure （Weir， 1990： 1）. In this paper， my main concerns are content validity and construct validity. Terminologically， construct validity indicates overall validity， which refers to the degree to which underlying traits can be inferred from scores on an assessment instrument （Cohen， 1994）. Under the umbrella of construct validity， just as its name implies， relates to the content of the test. It is determined by checking whether its content is representative of the kind of language skills we want to measure.

It is widely accepted that the MC format can only test recognition knowledge （Hughes， 2003）；it is less likely to test a candidates productive skills. If we intend to measure a students real ability to produce a second language， the MC format will possibly give us inaccurate information. In this sense， the MC format may lack content validity， due to the incompatibility of its test content and the test objectives. If the test is set out to measure a specific aspect of linguistic proficiency， a MC test may provide valid information. However， when we are trying to test general language proficiency， a MC test may not be desirable.

It is also problematic that some of the MC items can be answered without access to the source text. In the following， I will discuss the construct validity of the multiple-choices format in the TOEFL test. In this test， it is declared that some test-takers can answer the items without comprehending the source test （Freedle & Kostin， 1999）. The following example was given. It is said that if the test-takers only understand less than 30% of the minitalk， but heard words and phrases such as： registration， course enrollment form， stamp your form， pay for tuition， officially enrolled， they can be sure that this minitalk is about registration in school. They then can choose the correct answers from the following items without much effort.endprint

Q36. Who is the speaker？

（A） A new student.

（B） A physical education teacher.

（C） A professional photographer.

（D） A university administrator.

Q37. When would this talk be given？

（A） At the beginning of a semester.

（B） During the midsemester vacation.

（C） At final examination time.

（D） Just before a gymnastics event.

Q38. What must all students bring to the gymnasium tomorrow？

（A） Tickets.

（B） Stamps.

（C） New sports shoes.

（D） Course enrollment forms.

For Q36， because it is a minitalk about how to make registration， the speaker can only be a university administrator. For Q37， it is common sense that registration will only be made at the beginning of a semester. For Q39， one can also easily tell that students will only bring course enrollment forms to register， but not tickets， stamps and definitely not new sports shoes. From the above discussion， it seems quite reasonable to state that one can get high score without understanding of the source text. It this case， even if the candidates can make all the correct choices， we cannot tell how much the candidates understand the source text， which is against the will of the test makers. As thus， the construct validity of such tests is questionable.

3. Conclusion

In conclusion， we know that some aspects of its validity are still questionable. Nevertheless， different purposes of the tests should have different corresponding test instruments. MC format is still quite applicable if we want to test the cognitive knowledge of a large number of candidates. Of course， if we want to get a full picture of the candidates language ability， a combination of various test instruments should be used.

References：

[1]Cohen，A.D.，1994，Assessing language ability in the classroom （2nd edition），Heinle & Heinle Publishers.

[2]Freedle，R.&Kostin，I.，1999，Does the text matter in a multiple-choice test of comprehension？ The case for the construct validity of TOEFLs minitalks，Language Testing，16（1）2-32.

[3]Hughes，A.，2003，Testing for Language Teachers（second edition），Cambridge University Press.endprint