5

A number of questions (e. g., this one or that one) discuss the situations where the student is punished after invoking, in the exam context, knowledge that's outside the scope of the course. This leads to a more general question:

Suppose the student knows and understands perfectly all the material taught in the course, and perhaps some more. Is it fair to lower their grade for not knowing which material has been taught and how it was presented?

From the answers to the above questions, it seems that many feel that there's no problem with lowering the grade. I can think of two arguments in favour of that view:

  • in order to test knowledge, the exam questions have to include implicit context, notation, etc., that require familiarity with a particular course. More details on this type of reasoning can be found in this answer.
  • anyway, the grades are often not determined solely by what students know in the end; e. g., attending lectures, participating in the class discussions, etc., may be part of the grade. And if you have done so, surely you know what's been in the course.

Let me explain why the above arguments do not satisfy me. The first one concedes that it may be unfair, and then essentially admits a failure on the part of the professor to design a fair exam. But then, shouldn't one rather strive to make the questions as self-contained and unambiguous as possible? For the second one, we may ask, again, why it is fair to take anything but the end knowledge into account for the grade, and the answer will likely be that some version of "we use attendance etc. as a proxy for final knowledge" which, once again, brushes off the question as to how fair is this.

Is there a coherent argument that justifies that it is fair to lower the grade in the above circumstances?

Kostya_I
  • 4,071
  • 1
  • 14
  • 21
  • I really don't see how this question differs substantially from https://academia.stackexchange.com/questions/80898/should-a-student-be-penalized-for-using-a-theorem-outside-of-the-curriculum – Bryan Krause Aug 31 '18 at 15:49
  • 4
    There is only one way for the student to prove that they “know and understand perfectly all the material taught in the course”, and that is by giving answers to the exam questions that the instructor deems worthy of full points. Taking points off is not “penalizing”, it’s simply the instructor saying you haven’t convinced them that you know the material. And the reasons why an answer using out-of-scope knowledge can be unconvincing were discussed in detail in the questions you linked to. So I agree with Brian this is a duplicate question. – Dan Romik Aug 31 '18 at 16:01
  • @DanRomik, I think your first sentence is (extremely) false. There are plenty of ways. But if you rely overly on exams you give up most of those possibilities and introduce many problems. If we were speaking of oral exams, I'd be more inclined to agree with it. – Buffy Aug 31 '18 at 16:05
  • The question might be considered a duplicate if we reduce the context to only mathematics and similar things. But the issue may arise in other fields as well, so I disagree that it is a duplicate. – Buffy Aug 31 '18 at 16:07
  • @DanRomik, so, student didn't know the details of the course (although knew the material), and the exam question was so designed that this led him to giving an answer that didn't convince the professor. Fine. My question is: is this fair towards the student? Or is it a failure on the part of the professor to design a fair exam? If I understood you correctly, your answer is "yes, it is fair, no failure on the part of the professor". Why? What is the argument? – Kostya_I Aug 31 '18 at 16:21
  • 2
    @Buffy I don't think this question takes it outside mathematics - the question itself really just seems like a rant against the answers on that other thread, and itself specifically references that thread and one other that are focused on math. I find it difficult to think of a situation outside of math or a math-related discipline where this would come up in the same way. – Bryan Krause Aug 31 '18 at 16:24
  • @Kostya_I yes, it’s fair. The student signed up to take the course and then have his knowledge assessed in a particular way within that framework. Just like he is expected to show up for the exam at a particular time and place and knows that he will be “penalized” for not doing that even if he knows the material better than the professor, so too is he expected to know various other facts about the course that are relevant to the task of demonstrating his knowledge satisfactorily within the framework of the course, even if they are mathematically not strictly necessary to master the material. – Dan Romik Aug 31 '18 at 16:55
  • 2
    @Buffy to clarify, that sentence about “There is only one way...” refers specifically to the context of a university course. In other contexts, of course the sentence is extremely false. And I’m not going to argue the merits and fairness of exams as an assessment tool. For the purposes of my above comment, I assumed exams as a fact of life and was only addressing what is fair given that that’s how courses are graded. – Dan Romik Aug 31 '18 at 16:58

1 Answers1

6

I hesitate to answer this, as my views might be taken as just a rant, but I have a lot of experience - doctorate in mathematics and 40 years teaching mostly CS. Moreover, I've thought and written a lot about student learning.

My most basic question about exams is whether they tell us what we think they tell us and I think the answer is no. It depends on the exam, of course, but too many exam questions can be misleading and it is difficult to come up with good ones unless you revise and refine old ones, taking into account an analysis of old results. In one of the other questions cited by the OP, it took several suggestions by other academics to come up with a "fair" version of a question that didn't seem to be a minefield when first written, but turned out to be.

Other issues with exams (in fields like mine), is that too often an exam is simply too simple a vehicle to determine what a student has deeply learned at an operational level. Too many exams only test what a student has managed to remember from yesterday's cram session, as the high-risk nature of the process pushed them to actual destructive behavior. Can you make an exam for which cramming isn't going to be of any use. Yes, you can, but it is very difficult. It is even harder to convince them not to cram, forcing certain things into short term memory and perhaps obscuring more fundamental things.

Another issue, especially with the predecessor questions, is that when a student taking the exam (maths) gets an idea into their head it is very difficult to get it out, even when they realize that it is the wrong solution and isn't going anywhere. The first idea dominates the thought. Given enough time (not available in most exams) they might get it right, but the pressure itself is a mind-killer. This advantages some kinds of students, but not necessarily the most able or the ones with the best grasp. It is very complicated.

I have two suggestions, the first of which doesn't scale. That is oral exams in place of written ones. Now the questioner has a chance to interject if the student has made a wrong turn and can evaluate the depth of learning directly. The student has a chance to inquire about things unstated. But (personal experience here) the student also has a chance to explain why a first attempt is wrong, which may reveal a lot more about their learning than even a correct answer would.

But, that doesn't scale well. My major advisor also occasionally taught elementary courses (say Calculus) to moderately large groups and gave only oral exams (about 5 minutes each) and announced to the students at the end of the five minutes what their grade was. But even in a few minutes he could ascertain in broad terms how well the student knew the material just from their approach.

The other means, which scales better, is to rely less on exams altogether. Toward the end of my teaching I let exams count for no more than 30% of the grade. The rest being written work of various kinds, including group work. This lowered the pressure. Students could demonstrate what they had learned by demonstrating what they could do not what they could remember from yesterday. There was little need to cram and they were all promised a question that they would find very hard and that required interpretation, not memory.

Heavy reliance on exams scales well as exams are easier to grade than projects are to evaluate and comment on. But they also advantage a certain kind of person and disadvantage others. This is especially true, and I think unavoidable, if the exam is timed and important to the grade. People freeze. People prepare ineffectively. People have other commitments that get in the way of effective exam study.

However, there is one potentially positive aspect to an exam. Students do need to review what they have learned. That is an important aspect of changing the brain to solidify learning and turn it from simple rote learning into something that can be actively used. Lots of reinforcement of the learning. So, an exam that asks them to review what they have learned isn't a bad thing as long as it isn't so high pressure that it forces bad and unproductive behavior before or during the exam. In particular, essay questions, that ask for interpretations and consequences of things learned can be useful. I wouldn't want my courses to use only exams to help with this review and refinement, but it can be one aspect. I also sometimes used take-home exams with few rules forbidding things other than actual collaboration.

Most things, especially mathematics and CS, aren't actually produced in an exam-like atmosphere. I hope you will think about that.

Buffy
  • 363,966
  • 84
  • 956
  • 1,406
  • So, given that you state that you don't believe exams tell us what we really want, it begs the question do you still use exams in any form? – Solar Mike Aug 31 '18 at 16:39
  • @SolarMike, oops. I'll make an edit. – Buffy Aug 31 '18 at 16:40
  • 1
    Can you make an exam for which cramming isn't going to be of any use. Yes, you can, but it is very difficult. — On the other hand, we do difficult things all the time. Arguably we are hired as faculty precisely because we have expertise in doing difficult things. – JeffE Aug 31 '18 at 18:43
  • 1
    @JeffE, are you perfect? Do you ever neglect to cover all the ways a student can misinterpret? Or even to discover all those ways? Don't make light of the issue, please. – Buffy Aug 31 '18 at 18:58
  • I'm not making light of the issue; I agree with you. Even after decades of practice and polish, students still find unexpected ways to misinterpret my exam questions. But as we repeatedly tell our students, our inevitable imperfection is not a reason not to try, or even a particularly good argument to try something else (which will also be imperfect, of course). As with any other skill, we can (and therefore should) get pretty good with practice and feedback. – JeffE Aug 31 '18 at 19:37
  • @JeffE. I think too few people are willing to revise old questions in light of previous evidence. Of course, we hope they will learn from making mistakes in phrasing, but most will want new questions to prevent other kinds of student bad behavior on exams (another strike against high value exams). Also, I think too few are willing to simply cancel a question when it turns out to be misleading as it is a fraught process. If you don't use competitive (curve) grading it is easier, but not without pitfalls. Anything that seems to dis/advantage one student in relation to another is suspect. – Buffy Aug 31 '18 at 19:45
  • I've had no qualms about dropping a question if I realize that I made a mistake. If a question is open to multiple interpretations because of how it's written, I accept anything in the realm of plausible interpretations (I'm in engineering, so some things are reasonable to assume and some things just aren't). But I've also realized I don't want students to memorize a lot of stuff, so now I give them access to equation sheets that have everything they should need to know for the exams well in advance. . . . – aeismail Sep 09 '18 at 23:33
  • That said, there are a few things that every engineer in my field should be able to write down from memory, such as the ideal gas constant or the heat capacity of water, so I make sure to use those in class so they "sink in." But I also de-emphasize the numerical part of the problems, so it's still not essential to know the values to do "well," but you might need it to get 100% of the points on a problem. (Why ask them to do anything numerical? Because things like unit handling and reporting of significant figures are still important for my students to be proficient in.) – aeismail Sep 09 '18 at 23:37