Grading (again)

by Mike Gleicher on February 1, 2017

OK. Let me try this again, since questions are coming in. And this time I see another complication which makes this confusing: canvas requires us to enter grades in a numerical scale – even though we don’t use a numerical scale in class.

So, for small assignments that are on time, the “grades you can get” are:

  • didn’t turn anything in
  • no check – (turned something in, but not complete)
  • check

However, if you did a really good job, I (or whoever grades it) wants to be able to say “great job! keep up the good work.” In first grade, my teacher used to put little star stickers on things (not always gold, but you get the point).

But, we don’t have assignments on papers, so I can’t put star stickers on. We are using Canvas, and it makes me assign numbers, … I made up an arbitrary scale that maps these words to numbers. It’s rather arbitrary.

So, when I want to tell you “great job! keep up the work!” what I have to type into canvas is “5” – it’s not nearly as satisfying as getting that little star sticker. Worse – Canvas insists on these scales being quantitative/rational (we’ll actually discuss this in class when we get to data abstraction). So Canvas may tell you 70%, even though I was trying to give you a star!

An aside, I was going to make the divisor 37, so you’d be getting 135%, which probably feels a lot better than 71% even though you know its the same score (there’s psychology literature on that), but Canvas didn’t want to let me do that. now that we’re becoming more experienced with Canvas, we’ll try again.

And the problem is even worse… In that same number (that canvas is going to treat at a rational scale), I need to encode a bunch of other information: was the assignment late, was the assignment something to flag for another person to look at, was the assignment a particularly good example that we might want to use it in class, … So, there’s this combinatorial explosion of possible grades that means I need to make the scale “big” (have lots of values so I can code everything) – so Canvas makes the divisor big, so the percentage score looks low …

And actually, it gets even worse… Canvas rewards me for using the same scale for all assignments. So a simple assignment where it’s hard for a student to excel needs to use the same scale. Something where we’re just checking “yes you did it” it’s hard for us to say “wow, that’s amazing” (since there’s no opportunity for you to do something amazing, and/or no fair way for us to assess amazing from just excellent). So those numbers we reserve for “give this student a gold star and make a note to show it off” won’t get used on these assignments.

So again, if Canvas tells you something like “71%” or “50” remember that I am trying to put a little gold star sticker on your paper and say “great job! keep up the good work.” If I put a sticker on your computer screen it might make a mess.

This is actually a good point for visualization: we’re learning about scale type mismatch, conveying non-quantitative data, the emotional connection to data, … I’d say we can use it for a redesign exercise, but sadly, we’ll mainly learn the lesson of “sometimes you’re stuck with what you’re stuck with by practical concerns.”

Now, after that, you can read what the codes we assign mean… in the Grading post.

Previous post:

Next post: