This paper reports the performance of 38 non-native senior university students of English on two achievement tests: a traditional content-based multiple choice item test (MCIT) and a schema-based cloze MCITmeasuring course objectives. The former was developed on the whole content of a teaching methodology textbook sampled stratifiedly and taught during an academic semester. The latter was constructed on some randomly selected passages of another teaching methodology textbook suggested for outside reading. The results indicated that schema-based cloze MCITs measuring objective-based achievement correlate significantly with traditional content-based MCITs and thus have empirical validity. They also enjoy superior construct validity in terms of their construction theory and measuring objectives rather than specific content. In contrast to the traditional content-based MCITs, schema-based cloze MCITs measuring course objectives, however, have lower reliability because they are developed on unseen passages and are heterogeneous in nature.

Keywords