Sunday, June 18, 2017

The Evaluation Conundrum


How representative a class do you give when you’re being observed by a dean?

I’m guessing that for many people, it’s sort of like how they drive when there’s a cop in the rear-view.  It’s not faking, exactly, but there’s an element of wanting to put on a good show.  

In the case of the cop, that’s harmless enough, and probably inevitable.  In the case of teaching, though, that’s kind of a problem.  The only people who see the truth of long-term performance are students, and they often have agendas -- conscious or unconscious -- of their own.

That’s why I was so taken with this piece in IHE last week.  Alison Cook-Sather, a professor at Bryn Mawr and Haverford, used grant funding to develop a system in which students who aren’t taking a given class are hired to sit in on it and provide constructive, non-evaluative feedback to the professor on a regular basis.  

It’s a nifty idea for several reasons.  For one, it separates feedback-for-improvement from feedback-for-accountability.  They really need to be separate, and in practice, that can only work when they’re done by separate people.  Having students provide feedback that never makes it to the dean can help the faculty improve before a problem becomes a performance issue.  

Even better, the constructive feedback is from a student’s-eye view, and not based on a single day.  As hard as a peer or a dean might try to emulate a student’s view, it’s hard not to bring experience and training to what you see.  That can be a good thing, but it’s not the same thing.  Many years ago I observed a professor that most students considered a hateful jerk, and came away wondering why; he had a sly sense of humor that I really enjoyed.  They may have sensed that they weren’t getting a joke, or there may have been something else.  A student observer might have had a better shot at ferreting out the real issue.  To this day, I don’t know what it was.

I also like that it employs students in a way that treats their student status as an asset.  It suggests that they have something useful to say, and offers a venue in which they can say it.  

Too many colleges have some version of feedback-for-accountability, but no formal mechanism for feedback-for-improvement.  Instead, they try to get a single system to serve both purposes.  In trying to serve two purposes, it does justice to neither.  If I’m teaching, and I suspect that a shaky performance on observation day will have negative professional consequences for me, I’m likely to stick to the hits on observation day.  That’s individually rational, but collectively destructive.  I’m unlikely to break the cycle myself, though, because if I offer a warts-and-all performance and nobody else does, I look like a problem.  But splitting the roles between two people allows for both functions to be served fully.

I’ve long held that we could and should try something similar in the classroom with grading.  If we separate teaching from grading -- as in, swapping papers -- then the psychological dynamic between professor and student becomes simpler.  Instead of the fraught and complicated “I’m helping you, but I’m also judging you,” it becomes “I’m helping you; that guy behind the curtain is judging you.  It’s you and me against him.”  Athletics often work like that, with either the clock or the other team serving as the de facto grader.  

At Brookdale we’ve established a cadre of faculty who are “on call” to provide non-evaluative observations and feedback for other faculty.  Uptake so far has been modest, but I remain convinced that there’s merit in the model.  The observers are sworn to secrecy about who they observe and what they see; all I ask at the end of the semester is a total number.  Anecdotally, the observers report that the post-class conversations are often quite good, since there’s nothing stopping them from telling the truth.  And from an administrative perspective, I’d much rather have faculty performing well than performing badly for a whole bunch of reasons.  If the occasional tune-up helps someone who’s struggling to get back on track, everybody wins.

Kudos to Prof. Cook-Sather for finding a way to take that model and expand it to include students.  It’s a terrific idea, and the concept -- if not the funding -- is easy to replicate.  I tip my cap.