STAGES OF TEST DEVELOPMENT

SUMMARY OF TOPIC 1: STAGES OF TEST DEVELOPMENT

By: Muhammad Yunus & Rojab Siti Rodliyah

1.      Stating the problem

Make it clear what one wants to know and for what purpose. Answer these questions:

(i)     What kind of test is it tobe? (achievement, proficiency, diagnostic, or placement?)

(ii)   What is its precise purpose?

(iii) What abilities are to be tested?

(iv) How detailed must the result be?

(v)   How accurate must the results be?

(vi) How important is backwash?

(vii)                       What constraints are set by unavailability of expertise, facilities, time (for costruction, administration and scoring?)

2.      Writing specifications for the test

This includes the information on content, test structure, timing, medium/channel, techniques to be used, critical levels of performance, and scoring procedures.

(i)     Content: operations, types of text, adressees of texts, length of texts, topics, readability, structural range, vocabulary range, dialect, accent, style, speed of processing

(ii)   Structure, timing, medium/channel and techniques: test structure, number of items, passages, medium/channel. Timing, techniques

(iii) Criteria levels of performancez; e.g, for oral interaction at level 2 Cambridege Certificate: accuracy, appropriacy, range, flexibility, size

(iv) Scoring procedures: hoe to achieve reliability and validity in scoring, what rating scale to be used, ho many people will rate, what if there is raters disagreement?

3.      Writing and moderating items

Include: sampling choices have to be made regarding the content to be covered), writing items (with the specification in mind), moderating items (scrutiny of proposed items by (ideally) at least two collegues

4.      Informal trialling of items on native speakers

Trialling the written testvitems to a number of native speakers (twenty or more)

5.      Trialling of the test on a group of non-native speakers similar to those for whom the test is intended

6.      Analysis of results of the trial: making of any necessary changes

Statistical : reveal the qualities such as reliability) of a test as a whole and of individiual items

Qualitative: discover misinterpretations, unanticipated but possibly ocrrect responses, and any otehr indicator of faulty items.

7.      Calibration of scales

Collecting samples of performance which cover the full range of the scales

8.      Validation

Essential for high stake tests

9.      Writing handbooks for testtakers, test users and staff

Content: the rationale, account of how the test was developed, description of the test, sampl items, advice on preparing for taking the test, explanation of how test scores are to be interpreted, training materials, details of test administration

10.  Training staff

All the satff who will be involved in the test process (interviewers, raters, scorers, computer operators, invigilators/proctors) should be trained.

Reference

Hughes, A. 2003. Testing for Language Teachers. Cambridge: Cambridge University Press.