I ran across a blog post this morning that makes some excellent points about timed exams. Here are three points from Jon Dron’s blog post What exams have taught me:
- that slow, steady, careful work is not worth the hassle — a bit of cramming (typically one-three days seemed to work for me) in a mad rush just before the event works much more effectively and saves a lot of time
- the corollary — adrenalin is necessary to achieve anything worth achieving
- that the most important things in life generally take around three hours to complete
As Marshal McLuhan said, the medium is the message. That is, the context of a message may speak louder than its content. Still, I’d like to defend timed exams in a limited context. You need to have quick recall of some facts. There are some skills you need to practice to the point that they are second nature. Not because these things are ultimately important but so you don’t have to think about them and can move on to other things.
Joel Spolsky gave an example along these lines in his recent podcast. He said that Serge Lang once began a calculus class with an algebra quiz, one expression to simplify. Thirty seconds into the quiz, it made everyone stop and turn in their work. At the end of the year, he compared the final grades to the grades on his algebra quiz. The students who got A’s in freshman calculus were almost exactly the same as those who were able to simply the algebra expression quickly. (The story begins around 8:12 in the audio file. It’s also on the transcript wiki.)
There are a couple ways to interpret this anecdote. One is that Lang’s exams measured quick reaction time and that students who were able to do algebra quickly were also able to do calculus quickly and thus succeed on Lang’s exams. There may be some truth to that. But I think more fundamentally, those who had mastered algebra were able to pay attention to the new material. Because algebra was second nature to these students, they could think about calculus.
I agree that typical hour-long exams are artificial and create some perverse incentives. I see a place for leisurely evaluation: take-home exams, projects, portfolios, etc. But I also see a place for timed evaluation, even quiz show-like rapid recall, though such evaluation need not factor into assigning grades. I think Jon Dron’s criticism is that timed exams are usually not created deliberately. I don’t think he would necessarily find fault with someone explicitly identifying a list of fundamental skills and explaining that these need to be performed quickly. I believe his criticism is that everything is evaluated in a rush by default.
Thanks to Daniel Lemire for pointing out Jon Dron’s post. Read Daniel’s commentary here.
Great post John.
I would argue though that you have to watch out of causality arguments. You say “if you are good, you will pass the exams”. Fine. Eventually, if you work on enough Quantum Mechanics, you will find the Quantum Mechanics exams easy. But the reverse causality does not hold… you can do well on exams without mastery of the material.
I can prove that I got A+ in all Quantum Mechanics classes I took. And I took them all. I could also prove that I am incompetent at all but elementary Quantum Mechanics. I passed tests where I was asked to give the energy levels of the helium atom. I have no idea how to do this, I never knew… but I was able to fake it during a test.
There is also a deeper question, beyond basic competency… and that’s “what kind of work ethics do you want to encourage?”. Take-home exams encourage students to take the long view, it discourages cramming. Also, to succeed in life, you must take the long view… work over the long term. Cramming will only buy short-term success.
We are both people who work over the long term (our long-standing blogs are proof enough)… but the average undergraduate student wants quick rewards. That’s how we train them: work 2 days, get a good grade, go on. That’s not how life works.
My mentor at MIT, the late Gian-Carlo Rota, assigned grueling take-home problem sets (with open problems for extra credit!) but nonetheless used timed exams. Those exams were deliberately easy–they were intended as basic competency tests for the material. I see merit to his approach: there is basic material that you need to have handy in order to do advanced work. Much to his credit, he did not use timed exams to test people’s creativity under time pressure.
ps. Cross-posting this at Daniel Lemire’s blog, since I’m not sure where the discussion will evolve.
First Disclaimer: As an instructor, I prefer in-class exams because they are easier to grade. This isn’t laziness, it’s pragmatism.
Second disclaimer: What kind of work ethics do you want to encourage? Take-home exams encourage cheating, and that is always a challenge to deal with.
Given unlimited time and no worries of cheating, I’d give take-home exams exclusively. But for the time being, I prefer in class exams.
The problems with in class exams that you’ve outlined become a big problem at the graduate level. In class exams do not prepare students for performing research and writing a thesis. Many grad students are horribly shocked when they realized that they can’t cram and churn out a thesis. I have a list of advice that I give to new grad students to prepare them for this unsettling piece of reality.
I’d agree with you John that there are occasions that exams make perfect sense and actually offer some pedagogical benefits. There are plenty of skills where performance under stressful conditions is precisely what is required (e.g. music, drama, face-to-face teaching and even things like programming or lab work in some instances) so we have to provide opportunities to discover, in safety, how to perform. I also take the point that expertise and mastery are often typified by the ability to do things as though they were second nature (though might object that, despite our aspirations, our expectations of students are not always that they should be experts or masters, at least not right away). However, except where the ability to perform on demand is the intended outcome, we generally judge ability in the real world on the outputs, not the process, and therefore it is still at least worth reflecting on alternative approaches to enabling that.
As an aside, I’d also argue that lectures to large groups are sometimes a good idea, that there are probably occasions that punishment can be an effective pedagogy, that there is a place for simply dumping lecture notes on an LMS and it is not always wrong to read out lecture notes while averting your eyes from the audience. It is a wonderfully diverse world in education, where the expected norm is to find a myriad of different contexts, different teachers, different students, different needs. The problems only arise when we adopt processes unreflectively out of habit or laziness rather than things like the real needs of learners or the constraints of the subject domain and context.
@Laura – I guess the big point behind my initial post is that if a system makes it pragmatic to do in-class exams then we need to tear down the system.
The issue of cheating arises because we have reversed the priorities of assessment and accreditation, encouraging a focus on inauthentic and generally useless goals that not only fail to reflect real life but actually encourage a way of thinking and learning that is positively dangerous, manifestly unfair and prone to abuse. Most of us in the educational system and its surrounding ecosystem (including students, and I too have played my part) are responsible for perpetuating this and the way we use exams is as much a symptom as a cause. Stopping that particular infection won’t cure the system of all its ills, but we have to start somewhere!