45

I'm teaching an Introduction to Computer Science Course, where we mainly do C programming. I've been assigned a lab for teaching the course, but there are more students than available computers. Also, I have no control over those PCs there, and almost every student has admin rights to them. It is also common that students bring their laptop to the classroom and connect to campus Wi-Fi, which I don't have control at all.

Exams are small algorithms to solve, and initially I was planning - as do other teachers in the course - to allow using a computer (from the lab or their own) for the students to verify the correctness of their algorithms. However, in the current state of things I think is pretty easy for students to cheat: They could use Facebook, e-mail, or any Internet tool to exchange source code or information.

Are there software tools that can help to allow use of a computer but still prevent unauthorized collaboration?

ff524
  • 108,934
  • 49
  • 421
  • 474
Guillermo Guardastagno
  • 739
  • 1
  • 5
  • 10
  • 6
    +1 Same problem exists (typically) in Computational Physics lab components. Looking forward to reading the answers. – 299792458 Sep 21 '14 at 05:10
  • 3
    Sorry but this question seems to be off-topic to me for two reasons. 1. Are there Software Tools that can help to use a Computer and avoid plagiarism? If you are looking for softwares to prevent your students from cheating; you may ask your question on SoftwareRecommendations.SE. 2. Your question is not specifically about academia, what about a company manager who wants to test it's employees via a computer-based exam? – enthu Sep 21 '14 at 08:36
  • 4
    This question appears to be off-topic because it is a shopping request for software. Try [softwarerecs.se] - biut see their meta first, for guidelines on how to ask, there. – 410 gone Sep 21 '14 at 09:07
  • This question might help you since it is about plagiarism software although it is not for programming. – earthling Sep 21 '14 at 11:01
  • 4
    @Enthusiastic per this meta post, the community does not consider software questions as inherently off topic, if they are otherwise OK questions (focused, not too broad) – ff524 Sep 21 '14 at 15:37
  • 2
    If your employer doesn't supply you with the correct facilities, that is not your fault and you cannot be held accountable for any consequences of not having the correct facilities (assuming you have requested them). Ultimately all you can do is be as vigilant as you can and regularly ask students what their code is doing to ensure that they know what it does and they are confident in what it is doing. – Pharap Sep 21 '14 at 16:07
  • 1
    Exams are small algorithms to solve, and initially I was planning... to allow using a computer... for the students to verify the correctness of their algorithms. — Why? Shouldn't students be able to figure on their own, before actually implementing them, whether their algorithms are correct? – JeffE Sep 21 '14 at 17:55
  • 15
    Strongly disagree with the close votes. This is a question about computer science pedagogy, not just about software recommendations. Pedagogical questions are firmly on topic. – JeffE Sep 21 '14 at 18:02
  • 2
    Written exam. Seriously. You are teaching computer science (or so you say) not computer literacy. – Raphael Sep 21 '14 at 18:15
  • 1
    If they use their own computer, certainly you should't ever consider forcing them to install software there... – o0'. Sep 21 '14 at 22:30
  • 2
    Would you please do me a favor? Anybody you find 'cheating,' send them my way. In the real world of programming we call that 'problem solving' and I can't find enough of it. – Chad Robinson Sep 22 '14 at 12:06
  • 1
    I'm not allowed to answer directly for some reason; nevertheless I am allowed to comment. This problem has no solution. Commercial non-solutions do nasty things to the machine in question while the cheaters have simply learned to use another computer or mobile phone and type in the answer. – Joshua Sep 22 '14 at 15:45
  • I know some teachers that use some program called lockeddown browser..I don't really know if it would work with the students computers though. If there were enough computers in the lab you could also disconnect the ethernet cable... hope it helps –  Sep 21 '14 at 19:59
  • You may want to google for 'PRAKTOMAT' to see what is possible. Though I believe this project is related to Java it may help you with your evaluation. – applicate Sep 22 '14 at 11:20
  • Sometimes a psychological approach works best. You should tell your student you will "MOSS" their answers to find similar code: http://theory.stanford.edu/~aiken/moss/ (like someone below pointed out) This will deter collaboration. – unknownprotocol Sep 23 '14 at 07:46
  • One important point: Exams are usually time-critical, at least in my courses we always were running on edge to finish all questions in the given time. So you don't need anything special to make it impossible for them to cheat, just make it a little hard and time-consuming, and most people won't bother and rather inverst their precious time into problem-solving! – Falco Sep 23 '14 at 08:16
  • 1
    'almost every student has admin rights to them', there is nothing in software you can do then. You can solve this at a network level with appropriate firewalls, but that won't help you if student are using their own device and connecting to a network you don't control, then as far as technical solutions go, you have no real options. Admin rights == do what you want with the computer. – Frames Catherine White Sep 23 '14 at 11:01

9 Answers9

38

At my university in the UK we have two kinds of assessment for programming courses. Pass/fail 'practicals' where students are given programming tasks, and are explicitly allowed to discuss them with each other and search online. When the student is ready to submit their solution a demonstrator will look it over and ask questions etc to make sure the student understands why they wrote the code they wrote. Students don't get a grade for this that counts towards their overall mark for the year, but they are required to 'pass' the practicals in order to pass the year.

Then there are written exams with no computer access. There are questions along the lines of 'write some code that will solve this problem', and also more conceptual 'why is this the case? When would you use that?' questions. For the 'write code' bits, the markers will deduct very few points for simple syntax errors and things like that. The question is not so much 'can the student write a program that will compile on the first try?' as 'can the student come up with a good algorithm which solves the problem and give a reasonable expression of it in code?'. Note that these courses (at my uni) are never for specific languages; they are courses like 'functional programming' or 'object oriented programming' or 'machine learning', which will each use a particular language, but we care more that the student understands the concepts than learns the intricacies of the course's language.

This also encourages students to learn ways of designing and writing provably correct programs: if they can reason about programs on a deeper level than intuition, then writing programs on paper is a better test of this than allowing them to experiment and test until the output 'looks right' for the inputs they've given it.

This seems like a good way to avoid cheating on grades that matter and encourage solid understanding of the important course material. That said, many students hate having to write code on paper without access to S.O. or a compiler!

jfhc
  • 516
  • 3
  • 4
  • 23
    Having graduated from a UK University where we had pass/fail practicals and hand-written exams I have to disagree with your last paragraph. I hated it not because I didn't have access to the Internet, I hated it because a piece of paper is not a good IDE. It often took me 3-4 cycles of "write some code -> see a deficiency fixable by a line in the middle -> rewrite entire exercise, because there is no space". This is trivial to do even in Notepad, yet so frustrating and time consuming on paper. However I cannot imagine the proctors ever agreeing to an exam with any kind of electronics. – Ordous Sep 22 '14 at 11:27
  • Also common in the United States, and even in technical portions of job interviews. 2 of my last 4 had me either write on paper or whiteboard how I would do certain actions in a language, and the last basically asked me questions specific to the language. It doesn't have to compile, but it should make sense! – Compass Sep 22 '14 at 13:28
  • 8
    Regarding "why is this the case" questions: I always hate it when my professor asks a question like "why does this code compile?" because my response is always "because there's no error!" :) If you're going to ask "Why does ___ do ___?" sort of questions, please ask them in a form that only has one specific answer. – apnorton Sep 22 '14 at 15:38
  • My uni in NZ was a lot like this as well, down to the classes all using whatever language they felt like, except without any of the 'reasoning' component or any direct interaction with the people doing the marking. I remember not liking to write code on paper at the time. After being in the real world for a while though, I've found that university exams are about the only place where you'd ever want a programmer to work alone without internet access. Not the best preparation for reality. – Magus Sep 22 '14 at 16:14
  • 2
    +1, this may be an intro CS class, but that just means it's the best time to train the students that CS isn't actually about computers! – Brian S Sep 22 '14 at 19:37
  • 3
    @Magus: but then, exams aren't intended to prepare students for reality even if the course they're examining was. The ways programmers are actually assessed in reality probably aren't suitable for undergraduate assessment, although it would be fun to try. Put 8 students in a room with a 200k loc code base and a sheaf of user stories, and tell them they have 2 weeks to close as many outstanding feature requests as possible to the satisfaction of the QA team ;-) – Steve Jessop Sep 23 '14 at 08:17
32

Consider two things that are (almost) unique to computer science:

  • researching programming methods on the internet is standard practice, with stackexchange.com being the #1 hit for many searches. Requiring someone to memorize the details is not industry practice. I would not consider accepting a programming job anywhere that said "no internet access". And I hit the function references 10 times a day to verify things like function_name vs. functionName or replace(needle, haystack) / replace(haystack, needle) headaches.

  • It is quite likely that 10 students will return nearly-identical results for simple algorithms. There just aren't too many ways to answer "write a function to average the values in an array". Lets see: loop, sum, count. vars i, s, c. If you taught standard code formatting it's quite possible you might get 8/10 absolutely identical responses yet zero cheating. This is a common problem for cheat/plagiarism processors that are not tweaked for CS.

Good luck, and do come back with whatever solution worked out for you.

peter
  • 971
  • 6
  • 5
  • 25
    "It is quite likely that 10 students will return nearly-identical results for simple algorithms." Don't underestimate students' imagination ;-) – Massimo Ortolano Sep 21 '14 at 08:17
  • 3
    "Requiring someone to memorize the details is not industry practice." Hmmm ... requiring student to know the syntax and idiom of the language *is* standard practice. Likewise for know the identifying features and most common applications of data structures and algorithms studied in an introductory class. – dmckee --- ex-moderator kitten Sep 21 '14 at 17:13
  • 2
    With a tough time limit, there is not enough time to google everything. – Per Alexandersson Sep 21 '14 at 17:16
  • 2
    @dmckee - idioms, sure. But syntax? I google syntax every day, and I'm considered one of the best programmers, if not the best, in my company. – Davor Sep 21 '14 at 17:24
  • 5
    He's teaching c. I don't use it regularly and the only thing I need to look up is the precedence of the less commonly used operators. I agree that you might need to look up syntax in a big language, but c is so small you might call it "cute". Well, I also look up function pointer syntax if (FSM forbid) I have to go that route. – dmckee --- ex-moderator kitten Sep 21 '14 at 17:33
  • @MassimoOrtolano, as someone who has graded the work of freshman CS students, peter is very much on the ball. Grading the assignments was easy for me to do quickly, as almost every one was the same. "Yup, this one is the same as the last, 100%!" Obviously not every student had identical code character-for-character, but the similarities were striking over the course of the 3 years I did the job. I only had to launch one academic misconduct investigation during the whole period, and it turned out to be a simple mistake with a communal printer. – Brian S Sep 22 '14 at 19:44
  • @BrianS: Actually your comment indirectly contains several interesting points that would be worth discussing, which are also country dependent, but this would lead the discussion too far. All what I can say here is that at my very first exam as a teacher, before grading the papers, I prepared a table listing what I thought would be conceivable errors, with the corresponding marks to be subtracted; today, some 15 years later, it is plain that I've always failed in predicting possible errors and that students' imagination is by far more developed than mine. PS: I'm in a field different from CS. – Massimo Ortolano Sep 22 '14 at 20:56
  • @peter: "I would not consider accepting a programming job anywhere that said "no internet access"" -- although there are some quite respectable jobs where internet access is such a significant effort that you have to stop doing it for trivial reasons. For example, an AV company might have a lab with no internet access, full stop. If you want to look something up, you go out through security and use a different machine. Or it might be somewhat more relaxed, with two different machines that at least are in the same room. – Steve Jessop Sep 23 '14 at 08:24
  • @SteveJessop I would have no objection to a security policy that said "this box is for coding, this other box is for the internet." Works for me, works for them, shows they have a good sense of balance. – peter Oct 28 '14 at 12:36
14

In one of my programming subjects, we were allowed the class notes, our own solved exercises, and internet; but no communication with other students (email, facebook...). There was nothing really blocking them, just the lecturer wandering around.

Really, it is not that difficult to check: a student should be most of the time working on the IDE, and only sporadically browsing. If anyone seems to browse too much, you can do a closer inspection. GMail or Facebook will blatantly shine on the screen.

You can of course be more sneaky, like using Lynx to enter your mail. But really, if you can do that, you can probably write a simple algorithm without cheating.

Davidmh
  • 21,215
  • 5
  • 60
  • 105
  • 3
    It is very easy to exchange code between the students or even an outside tutor. All a student has to do is save a file into drop box, google drive ... that is shared. No teacher walking around the room will ever notice. – Matthias Aug 07 '15 at 22:38
10

Though you may not count this as a software tool: A custom Linux live distribution, which is directly booted from a removable medium, and contains just the necessary development tools (no browser, maybe even remove the ethernet/wifi drivers) would do it; just make them boot that for the exam.

(If anyone is able – within the time limit and with the tools provided – to write the drivers and a browser he not only gets a passing grade but a job offer.)

Wrzlprmft
  • 61,194
  • 18
  • 189
  • 288
Bloke Down The Pub
  • 101
  • 2
  • 2
    Yes, a custom Linux counts as a software tool. Or tools, if you want to go deeper. Also, if you want to go deeper, put the Linux on a rewritable media, and preset the IDE to write to that medium. Then the students can surrender the media at the end of the assignment if desired. Also disable mounting of the hard drives. – Mindwin Remember Monica Sep 22 '14 at 20:27
  • 1
    Our administrator just locked down the installed linux distribution, disabling non-local network routes (and this was in a networking course, so we needed access to specific servers to test our work). Of course, students didn't have admin rights, that sounds like a very bad idea... – fmark Sep 23 '14 at 05:13
7

The solution employed by my professor is to allow the textbook in the classroom (or any other sheet of paper), but no internet access; not even computer access. The whole exam was on paper.

The exam itself was to develop some algorithms, or maybe creating some classes; stuff for which the textbook is useless , but it still gives you the opportunity to look up names and basic concepts.

As a student, I think there is nothing wrong with this method. Of course others may disagree.

Ant
  • 810
  • 7
  • 18
  • 6
    +1 I was about to suggest paper. I think making students do some computing exams on paper is a good teaching method. Aside from preventing cheating it helps to challenge the modern world's over-reliance on the internet (not saying I'm any better of course). It's also an excellent way to make sure students have been taking the knowledge in; students can't expect to be using the internet every 5 minutes if they get a real programming job - they'll have to know by heart. – Pharap Sep 21 '14 at 15:56
  • 2
    This. They won't learn real-world programming at university, anyway, so why bother? Teach them the concepts and ignore syntax "errors". – Raphael Sep 21 '14 at 18:16
  • Let us continue this discussion in chat. – Raphael Sep 22 '14 at 10:56
5

Law schools have been solving this problem for many years by making students buy software for exams that locks their systems down and only allows them to use the exam-taking software during the examination time. However, there have been problems. Also, law school exams are generally essays or other non-functional questions, whereas a programming exam could also require a compiler. I'm not aware of a version of this that exists to allow a text-editor, terminal, and compiler.

Bill Barth
  • 48,733
  • 6
  • 112
  • 194
  • 5
    The students have to pay for software that performs a function one would expect from a virus? I think I'd rather write my essay on the table in blood. – Pharap Sep 21 '14 at 16:01
  • 1
    @Pharap, it does no such thing. I don't like the practice, but it's a pretty effective anti-cheating device that allows students to type out an essay on a device that they are familiar with. It's reasonably efficient and makes grading easier than going through hand-written exams written in haste. If you have a better solution, you stand to make some money. – Bill Barth Sep 21 '14 at 16:08
  • 1
    I'd say the ability to lock down a system something that would be somewhat beneficial to a virus. If it's merely a full screen word processor/office suite that won't quit until a certain amount of time has elapsed, that's not so bad. Personally I still think there's no substitute for making students use computers provided (with said computers having strict rights management applied). – Pharap Sep 21 '14 at 16:18
  • 3
    @Pharap, the hallmarks of a virus is that it's invocation isn't voluntary, and that there is a malicious purpose. Being able to execute is also beneficial to a virus, but not all executables are viruses. Simply not being able to quit isn't enough to prevent cheating; the rest of the machine needs to be locked out as well. Some allow for pen and paper exams to be done instead if you don't the software, but most students these days work more effectively at their own keyboard. Having enough labs full of computers for law school exams and not much else is a prohibitive cost for many law schools. – Bill Barth Sep 21 '14 at 16:28
  • 1
    Sure, the blackboard CMS provides a locked-down browser (used for taking quizzes and exams) which has a similar purpose: only the browser can be accessed. – dmckee --- ex-moderator kitten Sep 21 '14 at 17:16
3

Are there Software Tools that can help to use a Computer and avoid plagiarism?

I can't answer the above question and I'd be surprised by the effectiveness of such a tool. So, I suggest another solution: do the exams on paper, scan the papers at the end of the exams, send the copies back to the students and give them a couple of days to debug their programs. The differences between the paper version and the working one can thus be taken into account to decide the final mark.

Massimo Ortolano
  • 55,428
  • 19
  • 166
  • 207
  • 1
    Please don't. We are already producing plenty of "programmers" who can't program for the life of them. Let them use literature during the exam, they can use it in the workplace anyway... – Davor Sep 21 '14 at 17:27
  • @Davor I haven't written anything about not allowing students to use literature during the exam. Indeed, they won't be able to use online literature, but they could surely bring their own books to the exam. – Massimo Ortolano Sep 21 '14 at 17:48
  • 4
    We are already producing plenty of "programmers" who can't program for the life of them. Let them use literature during the exam, they can use it in the workplace anyway. — Oh, the irony. – JeffE Sep 21 '14 at 17:58
  • 2
    If the design goal of a computer science course is to produce craftsmen (programmers) then something is seriously amiss. – Raphael Sep 21 '14 at 18:17
  • @Raphael As with a lot of disciplines CS has programs have several goals. One, of course, is to advance the state of the art but as a side effect they turn out a corps of people who will find work as programmers but (hopefully) bring a deeper theoretical knowledge than their less specifically educated peers. A great deal of technical work can be done just fine with people who have very task focused education and training, but it is often useful to have a few deep thinkers hanging around the place. – dmckee --- ex-moderator kitten Sep 21 '14 at 19:28
  • @dmckee While I agree with your last sentence, the implication "computer scientists sometimes work as programmers, therefore we have to teach them programming" is a fallacy. Do you teach mechanical engineers how to weld or electrical engineers how to solder? (I don't think we should discuss this further here as it takes us away from the topic of th equestion.) – Raphael Sep 21 '14 at 19:57
  • @Davor Your assumptions (and conclusions) are wrong. Wannabe programmers have no business being at a scientifically oriented university. Why go knocking at ivory tower and ask for a carpenter? Have programmers teach future programmers (it's called "apprenticeship") and both your and my problem are solved. It's funny that you'd assume that researchers could do the job. – Raphael Sep 23 '14 at 11:59
  • @Raphael - the problem is, universities are not scientifically oriented, just individual arrogant professors who find it beneath them to teach "craftsmanship". And your suggestion that software engineers be taught like plumbers tells me just how much you have no idea what you're talking about. Again, just stop, please. – Davor Sep 23 '14 at 13:06
  • @Davor Maybe you need some intercultural experience. Your claim is definitely wrong here in Germany. We even have our own track of higher ed for applied sciences, our universities are oriented towards research. But agreed, it seems not worth my time to "discuss" with you since you can't tell the differente between programmer, software engineer and computer scientists. – Raphael Sep 23 '14 at 13:18
  • The design goal of a CS education should not be to teach programming, just as the design goal of an English Lit. education should not be to teach spelling. Those are merely the natural way in which to express the actual content of the education. And to remain on-topic, would you ban a Lit. student from using the spellchecker in Word?! – MSalters Sep 23 '14 at 13:47
2

Well, I haven't checked whether there are actual software tools for that, but I'll tell you what I used to do. The number of approaches to a correct solution are quite limited; if it's an exam-type problem, you should expect to receive many almost-identical, or even actually-identical, replies - as @Peter points out. But

the ingenuity of error is infinite, and the creativity of the misguided is boundless...

So mistakes + copying = easy to catch. If you want to mechanize, check for the same or similar parsing/compilation errors, or identical choices of identifiers of programs which fail to compile or run as expected.

Community
  • 1
einpoklum
  • 39,047
  • 6
  • 75
  • 192
  • The issue seems to be unauthorized collaboration (i.e. students can help each other by sharing information), not necessarily students handing in identical code. – ff524 Sep 22 '14 at 08:41
  • Its difficult if not impossible to detect (or prevent) students from chatting in the computer lab. Student who understand the concepts will hand in similar but working code (normal), student who do not will hand in similar broken code, which should raise a red flag. – sevensevens Feb 03 '15 at 15:36
2

Two such programs are MOSS and JPlag:

http://theory.stanford.edu/~aiken/moss/

Moss (for a Measure Of Software Similarity) is an automatic system for determining the similarity of programs. To date, the main application of Moss has been in detecting plagiarism in programming classes. Since its development in 1994, Moss has been very effective in this role. The algorithm behind moss is a significant improvement over other cheating detection algorithms (at least, over those known to us).

http://jplag.ipd.kit.edu/

JPlag is a system that finds similarities among multiple sets of source code files. This way it can detect software plagiarism. JPlag does not merely compare bytes of text, but is aware of programming language syntax and program structure and hence is robust against many kinds of attempts to disguise similarities between plagiarized files. JPlag currently supports Java, C#, C, C++, Scheme and natural language text.

Tools similar to these (not sure exactly which) were used at both the schools I've been to. Students were caught and penalized even after they had renamed variables and changed order of functions in the file.

hojusaram
  • 135
  • 3
  • 2
    The issue seems to be unauthorized collaboration (i.e. students can help each other by sharing information), not necessarily students handing in identical code. – ff524 Sep 22 '14 at 08:41
  • 1
    Solutions to any reasonably scoped exam problem are bound to be similar. Syntactical similarity is therefore a bad indicator for cheating. – Raphael Sep 22 '14 at 11:00