49

In the (not so distant) past I have received the student evaluations for my course. The evaluation was mediocre, something that surprised me since I thought I had good connection with the students coming to the class (about a third of the total number of enrolled students). These evaluations raised some eyebrows among the relevant committee members.

By accident, chatting recently with a student that really liked and performed very well on the course, I realized that the student did not receive invitation to evaluate the course. I thought it might be an isolated "bug" of the system but I casually mentioned it to a member of the relevant committee. He dug out the system and actually sent me a list of all students that had been invited.

To my shock, 38% of the enrolled students were not invited! What is even more weird is that this missing part is highly non-random and included many of the best students (that got 7+) and only one of the students that got 9+ was invited. The "informal" explanation was that because of some mistake (I do not know if it was the system's or some person's) the students who follow a research track (naturally, the good students) were left out and nobody was actually aware of that. They didn't seem to care even after I pointed it out.

To make it more concrete, out of the 75 or so students, the evaluations included the answer of 26 (non-randomly selected) students.

How can I deal (internally) with this situation appropriately? I have in my record a very mediocre evaluation but it turns out the system made a big mistake leaving out many (good) students. How can I explain this to possible applications where they ask about teaching statement and evaluations?

The main problem is that my department takes evaluations extremely seriously, and my mediocre evaluations had some (admittedly mild, but still...) impact on my development and assessment in the past and might have in the future as well (to some extent, my tenure-track offer depends on that).

Can the department use these evaluations to decide if they will offer me assistant professorship based on the situation explained above? If they indeed use them, is there anything I can do?

Added: The protocol is to invite every student in the evaluation. Moreover, the evaluations should be collected before the grades of the exam are communicated to the students. In the case described above, besides the fact that a particular student group did not receive invitations, these were sent after the students have received their grades. I do not claim that this was done on purpose. But, besides being extremely unprofessional, it seems that no one cares about these facts and they still want to use the mentioned, obviously bugged in my opinion, evaluations.

PsySp
  • 7,901
  • 6
  • 34
  • 58
  • @StephanBranczyk They did not. But even if they did, how can these evaluations be used? How can anyone make any meaningful conclusion based on a small and non-random sample? – PsySp Dec 14 '17 at 08:24
  • @StephanBranczyk Northern Europe. No third party websites. Only internally circulated evaluations. – PsySp Dec 14 '17 at 08:27
  • @StephanBranczyk I am not aware of anything like that. I am not sure I understand how this could help me even if such site existed.. – PsySp Dec 14 '17 at 09:28
  • @StephanBranczyk Yeah but the point is: can (should?) they use these obviously glitched evaluations to decide if i'm suitable enough for them? – PsySp Dec 14 '17 at 13:08
  • 1
    No, they shouldn't. But you've already made that argument and for whatever reason, they didn't accept it. Either there is a conspiracy afoot, or they're just apathetic and don't want to make extra work for themselves. Either way, you need to make your argument another way, because it's not going well for you. If nothing else works, at least send them a registered mail telling them of your findings and asking them to include your findings in your personnel file to ensure that your side of the story becomes part of your official HR record. – Stephan Branczyk Dec 14 '17 at 14:02
  • @StephanBranczyk No, it doesn't work well for me and I do not understand the reason. Anyway, I will try to make it more official as you suggest because it seems they base their decision on a sample of 26 answers out of 74 (50% of the students they asked actually replied) – PsySp Dec 14 '17 at 14:11

1 Answers1

55

If you can prove that 38% of the students where not even invited, you should go to the relevant committee and challenge the evaluation. Demand that they either invite the rest of the students as well (if it is still possible to reach them and the evaluation is not too long ago) or that they declare the evaluation null and void because it was not done properly.

J Fabian Meier
  • 13,166
  • 4
  • 37
  • 62
  • 3
    I think it is normal to ask only bad grade students for evaluation – SSimon Dec 05 '17 at 15:00
  • 55
    @SSimon I never heard of such a practise. All my evaluations where done by all students (or at least all students that were not sick that day). – J Fabian Meier Dec 05 '17 at 15:03
  • 33
    @SSimon how can you ask only bad grade students when the invitation is sent before they receive the grades? – PsySp Dec 05 '17 at 15:08
  • 11
    I thought the comment by @SSimon was intended to be humorous. – Dave L Renfro Dec 05 '17 at 15:21
  • 1
    Thanks. How do I explain to other potential applications for academic positions that demand to see proofs of teaching achievements, including evaluations? – PsySp Dec 05 '17 at 16:09
  • 3
    @PsySp If you can succeed in getting the evaluation voided, you can honestly reply in those requests that due to a technical error no evaluation results are available. Of course you can provide them with alternate proofs of achievements (and if they really hammer on it you can allow them to check this with the uni, but this may be tricky if someone misunderstands the situation). – Dennis Jaheruddin Dec 05 '17 at 16:46
  • 7
    In arguing to invalidate the results, it is important that the omitted students were not a random sample but had academic characteristics in common. For a reasonably large population, a 62% random sample would give usable results. – Patricia Shanahan Dec 05 '17 at 17:28
  • 20
    @PatriciaShanahan Not only a particular student population was left out, but the invitations were sent after they got their grades in the exam (something that should not happen for obvious reasons) – PsySp Dec 05 '17 at 18:55
  • @PsySp: instructor could make an informed guess based on their homework, class participation, questions asked, quizzes and/or midterms. Or even guess based on other track record – smci Dec 05 '17 at 19:48
  • @smci But is not the instructor that invites students for evaluations! It is some other person, unrelated to the class/courses. – PsySp Dec 05 '17 at 19:50
  • @PsySp: whoever is doing the inviting could guess, even if only based on historical GPA – smci Dec 05 '17 at 19:51
  • 3
    @SSimon It is not normal at all. What is normal here that the students' overall grades are recorded with their evaluation (everything is done electronically) so that you know if good students happen to criticize a teacher for his teaching. (Because obviously, you don't care that much about overly bad students.) – yo' Dec 05 '17 at 22:04
  • @J.FabianMeier IDK, we have online student system that doesnt need ivitation. But by invitation only are invited bad students – SSimon Dec 06 '17 at 05:09
  • @PsySp by invitation. – SSimon Dec 06 '17 at 05:09
  • @PsySp sorry I just some one more comment that you replied to ptriciasjanahan, actually it is normal to invite bad grade students, I think you committee is suspicious of your teaching, why is this done? it is so rare to meet this cases, but I think something is going on on your institution – SSimon Dec 06 '17 at 05:12
  • 3
    @SSimon I don't understand your speculations. Maybe your university classifies students for the evaluations, but I am sure that most universities just invite every student for the evaluations. – J Fabian Meier Dec 06 '17 at 08:23
  • @J.FabianMeier You are completely right. I never heard that only a subset of students should be invited. I updated my question to highlight the simple protocol in my university. – PsySp Dec 06 '17 at 10:12
  • 1
    @J.FabianMeier no mine dont invite, everyone can access evaluation system it is a system of SIS, but they invite people who failed or score bad to ask them what was wrong with their stufy – SSimon Dec 06 '17 at 17:30