THE EXAMINERS EXAMINED
[To the Editor of Tin SeEcraroa.] In,—An examination of An Examination of Examinations suggests that Mr. Verschoyle has not completely answered Mr. Mackenzie. It is true that, as Mr. Veischoyle says, the questions raised are " systematically discussed " in the pamphlet. Thus on page sz Sir Philip Hartog explains that the Oxford a13-y system of marking shows the relative merit among a group of candidates rather than any absolute standard of merit. One examiner will rarely give better .than
another will often use u. The important point is—do examiners give their best mark, whatever it is, to the best men, their average marksto average men, and so on among a group of candidates ? Sir Philip points all this out—but he soon forgets his caution. He proceeds on page 64 to turn Oxford History Final marks into grades on an absolute scale of merit- &c.—and converts the marks given in his mock Final Examination into this system: • By this inconsistency he exaggerates the divergence between the opinion of different examiners. Candidate eleven,. for example, had a miserable record in the third History Paper.
Eight examiners gave him their worst mark, one gave him his worst mark but two, the remaining examiner gave him his worst mark but four. There is close enough agreement for all practical purposes. But in Sir Philip's grades it appears that the examiners differed irresponsibly. There was a difference of ten grades between the worst mark which candidate eleven received—S—and the bes-f—/3. So the examiners differed by ten grades out of a possible twenty-four. Abolish examinations There was, in fact, considerable agreement on the best and worst men. Candidates sixteen and eighteen were placed among the best half-dozen by eight examiners out of ten. There was an extraordinary divergence of opinion on candidate nine. It is a pity that Sir Philip Hartog does not characterise the type of candidate on whom examiners differ. Nine was probably an original, interesting man to some examiners, a crank to others. The lesson to be drawn from Sir Philip's data is not perhaps that all examinations are unreliable, but that they are accurate and fair with one type of candidate, and very chancy with an original man, a clever idler, and other types.
The School Certificate History Examination offered extra- ordinary divergences of marking to Sir Philip Hartog, but then it was investigated in a curious way. A number of scripts were picked out which had been given the same mark in the actual examination and these were given to the new panel of examiners without comment. The examiners natur- ally thought that they were getting an ordinary unedited run of scripts, and, anxious to produce an ordinary run of marks, they unconsciously exaggerated such differences as the scripts contained, and so handed in marks ranging from sixteen to seventy-one.
Perhaps I have said enough to show that the statistician must not have the last word, but that the data are more fitted for the psychologist.—Yours faithfully,
Previous page
