Bias

It is going to be a busy winter at work, in large part because I have a couple of big service responsibilities. One of them is the Most Dreaded Service Commitment: the admissions committee. We have changed up the way we evaluate portfolios again, and it is a slow and cumbersome process. I have 47 to go, friends, and they will consume a lot of hours over the next few weeks.

One of our video prompts asks students to reflect on a class they've taken; it offers them a couple of different options to consider. One of the trickiest parts of the whole process for me is evaluating the students who say that they loved one of my classes. It's happened a couple of times now, that students will say, "Dr. Gladly's class was especially valuable for me because..." -- with a list of features that made it challenging but rewarding. 

My instinct is to say "You get a 10!!" (I do not award 10s, because it's a 1-3 scale.) I keep interrogating that instinct, and I'm not entirely sure I'm satisfied with the answers. 

It would be more clear-cut if they were attempting to flatter me, but (a) they're talking about the class, not the instructor, and (b) they have no idea if I'm even on the committee. The committee's composition changes every year and we keep it quiet -- precisely so we don't have students attempting to butter up committee members. And it's not as if every student who mentions my class gets automatic high marks from me. It's really gratifying, though, to hear students say, "Dr. Gladly asked us to do this thing that I found difficult at first, but I stuck with it and I learned so much!"

Part of the rationale for this prompt is that we want to hear students talk about the process of learning. We hope to avoid the situation where students come into grad school having coasted through their undergrad program on strong memorization skills, and then struggle to apply memorized knowledge in complex clinical cases. When I create assignments for upper-level undergrad classes, there's always a specific purpose. So maybe it's entirely reasonable to take it as a good sign when students say, "The assignment was hard, but I see the purpose clearly and the purpose is worthwhile!"

It's messy stuff, though. Rater bias can play a role in who gets into grad school, which is a decision with significant repercussions for students. Even small things, like how many successive portfolios I have evaluated, can influence the ratings I give. They put me on this committee because I have more than ten years of experience teaching grad students here; I have seen a lot of successful (and less successful) students go through the program. It's still messy, though. There's no way to make it anything but messy.