AN EXAMINATION OF EXAMINATIONS
[To the- Editor of THE SPECTATOR.] • gm,---4 regret that, owing to pressure of work, I have -been unable to reply earlier to Mr. Gavin Bone's letter published, in The Spectator for February 14th, which raises two pointy.
Mr. Bone thinks that it would be fairer to compare the .
orders in which the History Honours candidates were placed by the various examiners rather than their actual marks. Our pamphlet gives the complete material necessary for this purpose. And in The Marks of Examiners Dr. Rhodes has calculated the 145 correlation coefficients measuring the agreements between the orders of the different couples examiners (10 for Paper I, for which there were five examiners, and 45 for each of the other three papers; for which there were ten examiners). The average correlation coefficient for Papers I, III and IV was approximately .45, and for Paper II .26. In some cases the coefficients were actually negative. These figures mean a low degree of consistency between the order of the different examiners. I think that we have given Mr. Bone all that he can want.
Mr. Bone also thinks that the use of numerical grades implies that all the examiners " meant the same thing when . they marked a script, say, 0." But how can we ever know that examiners mean the same thing " when they allot the same symbol to a script, whether the symbol . designates. a class or a grade ? Messrs. Ogden and Richards, in their well- known book, The Meaning of Meaning, have investigated the general problem of defming a " meaning." With that general problem I cannot deal here, but a practical illustration of the difficulties will be useful. It is well known that examiners may agree completely as to the class and grade to be allotted (say) to two candidates, .and disagree violently' about two other candidates at the same examination. Here is an actual example, in which no ambiguity arises as to class or grade.
Examiners L and M agree on paper pretty' eloSely about class limits expressed in terms of grades. I quote below the marks for Paper II given (both in literal and numerical grades) by these examiners to four candidates (see An 'Examination of Examinations, pages 65, 66 and 71) :
Examiner A. Examiner C.
p (111) ) p r. (II) (111 P 0 P-- (9) • • ,P (17) (18) .. p— (9) If A and C meant the same thing " when they allotted-$, a clear Second ClaSs, to Candidates Nos. 3 and 8, what -did-they mean when A gave a Second to Candidate No. 4 and-a First to Candidate No. 9. while C almost exactly reversed the two
verdicts ? •
We have been concerned to ascertain arid 'record the differences of marking between different- examiners. That is a first and important step. We desire that such discrepancies and the uncertainties arising therefrom should- be further investigated, and reduced as far as possible. That- is a matter for specialists in each subject, and for psychologists, and, if Mr. Bone and his felloW-psychologists•will•help in the-matter, examination-candidates and the-public will have reason -to, be
Candidate.
No. 3 .. No. 8 • No. 4 ..
.No. 9 . .
1 Plowden Buildings, Temple, London, E.e.4.
Previous page
