Have written about marking Bar Admission Course exams, at the time when such exams were not multiple choice. Have also marked professional qualification exams for those wishing to become Chartered Accountants. What was then called the School of Accountancy in Ontario, which was the screening mechanism to determine suitability to write the Uniform Final Examination or Uniform Final Evaluation, as it then was called.
Very different from Bar Admission Course marking, where the intent was to convert an expository answer to a letter grade, relative to a series of model answers. In School of Accountancy marking, everything was on a marking key, with specific points.
Realized I was getting uncomfortable marking to a detailed key, when encountering answers where the student had placed all the points favouring one direction and all the points favouring the opposite direction in the same answer. The student ended up with close to a perfect mark, even though from the answer, it was evident that the student had no idea which way to go. Turned out that this was one of the “write for success” strategies, based on knowledge that one was writing to a specific marking key, and there was no global penalty for demonstrating complete lack of logic. Also, if the answer was unique and thoughtful, but not on the key, the student failed. Started to go cloudy when realized I was to award a mark for the word “tents”, wherever it appeared in the answer, and however illogically it appeared.
The process seemed to work well in identifying the genuinely superior students (with the exception of the strategic student, writing in both directions). It also seemed to work well in identifying students who could demonstrate little, if any, knowledge of the subject. Process had a major problem in distinguishing students in the mid-range, particularly those who might have been evaluated much higher, if a global perspective and an educator’s judgement could influence the outcome.
Not so. The people marking these exams seemed to be significantly represented by semi-professional markers, travelling the cross-country circuit to mark professional exams, according to the marking keys. As one person said to me, “you don’t have to be a professional accountant to mark these exams”.
Wondering if there is some mid-range between the multiple choice computer scan sheets and the judgement brought to marking expository answers. Perhaps some machine that can evaluate point content, in relation to a marking key.
Did it once and never sought to do it again. “Tents” as the deal-breaker.