Advantages of second marking

Question

In the UK a portion, and in some cases all, work is "second marked" where an independent marker also marks the work. In cases where the 1st and 2nd marker disagree, a 3rd marker may be used. Finally, the entire work of each student over the course of his/her studies is evaluated by an exam board with (sometimes) 2 additional independent markers. These exam board markers tend to only consider cases that are on the border of different degree classifications.

From my understanding of statistics, having all of these different markers will regress marks towards the mean. As I am currently faced with the daunting task of 2nd marking a large stack of off topic papers, I am curious what are the advantages of second marking?

What understanding of statistics suggests it will regress to the mean of the population? See literature on reliability and classical test theory. If you consider the outcome scoring of the test to be a function of the true score plus random error, having multiple raters will provide an estimate of the true score, with more graders providing a more reliable estimate. — Andy W, Jun 17 '13 at 12:34
@DanielE.Shub You already know you're second marking the papers. What if you did not know that and think you're the first marker? — Nobody, Jun 17 '13 at 12:46
@AndyW presumably because the noise at the high and low ends is not zero mean. If markers will not/cannot give a mark higher than an A, A work will sometimes, because of noise be marked as A-, but never marked as A+ (since the A+ mark doesn't exist). Similarly, F work will sometimes be marked as D- but never as F- (again assuming F is the lowest mark). Meaning that with enough markers no one will get an A or an F, which seems like regression towards the mean to me. — StrongBad, Jun 17 '13 at 12:50
@scaaahu apart from marking fatigue, I don't think it matters since you often you split the stack in half with someone else and then swap after you have both "1st marked". — StrongBad, Jun 17 '13 at 12:54
@DanielE.Shub, that is reasonable, although presumes a negative correlation between the rating and the true score (which would violate the assumptions of classical test theory). It is not the same as regression to the mean, and like I said the literature on classical test theory would be of interest. — Andy W, Jun 17 '13 at 12:55

score 5 · Accepted Answer · answered Jun 17 '13 at 13:52

Double marking has many roles, but mostly it is to ensure accuracy and fairness. The main way of achieving this, and avoiding the statistical anomalies alluded to in the comments is to produce an effective marking scheme, so that academics with sufficient background can grade the exam and produce virtually the same grades. Easier said than done.

More details can be found on the Internet, for example, on Swansea University's website.

score 4 · Answer 2 · answered Jun 18 '13 at 10:43

Like the other answers, I don't think it is an issue of regression to the mean (in the sense that the mean is the mean of all the students in the module). It is an issue of finding the true quality of the student's work.

In my current university we do sampling in that one marker will check maybe 1/3 of another marker's work. The point here is clearly not to catch every mistake a marker might make but rather to check for signs of abuse of power. Because everyone marking knows that some of their marks will be checked by another is supposed to keep the original marker from giving inappropriately high or low grades to any students, since the marker does not know which of the marks will be checked (admins actually do the selection of the sample).

I've worked at other universities where all work is double marked and in this case my experience is similar to Peter Jansson's.

Again, as I see it, the purpose of double marking (whether full or sampling) is to simply make sure that the student is getting marked fairly. Unfair marks can happen with intent (bad marker) or on accident (marker interpreting an ambiguous question differently from the student).

score 3 · Answer 3 · answered Jun 17 '13 at 13:52

I will provide some little experience I have had. I am sure details will differ depending on how the system of two graders are set up.

In the system I experienced it is a custom to have the course responsible plus someone external (in my case even from a different country) do the grading. The grading was completed by having a discussion between the two graders about possible deviations. In the system grades are given as a number between one and six in steps of 0.1, so very detailed.

My experience was actually quite remarkable; it concerned a masters/PhD level course. We were most often within 0.2 of each other except in one case (answer) where one had given a 1 and the other a 6. In that case it turned out the question was ambiguous and could be interpreted in different ways. The grades were basically calculated as the average of both but only after we had discussed the problems/deviations. This is, for example, how we discovered the ambiguous question formulation.

From this, albeit miniscule, experience, I felt that the benefit of having two persons grading is that ambiguities in terms of questions and answers can be sorted out. It is also possible to discuss the apropriateness of the interpretations of answers given by students. The method also provides what I can call "legal certainty" since the grades will be based on two persons view rather than one. Of course the degree to which it is certian depends on the transparency of the process and to what extent the two gradings become official. The point in "my" case is that both graders have to agree so it is not signed by just one person.

As a grader I also appreciated the possibility to discuss the grading and the corrections jointly made were fair and made the process worth while. I would personally like ot see the system used more, but fear it will be difficult from a financial point of view in many universities (-y systems).

Advantages of second marking

3 Answers3

Linked