Understanding Rubrics

A Couple of Quick Definitions

  • Reviewer - the person that is doing the Review.
  • Subject - the person or thing being Review.
  • Rubric - the instructions that help the Reviewer judge the Subject.
  • Review - the evaluation of a Subject structured by a Rubric (and performed by a Reviewer)

What is a Rubric?

A Rubric is a clear set of standards and examples that allow a non-expert to accurately judge a Subject. A Rubric encapsulates an expert's knowledge of how to make a judgement so that the non-expert knows what to look for. Over time, of course, the non-expert internalizes the rules and becomes an expert judge themselves.

##Why Rubrics are Good Things

Rubrics are a great way to do evaluation for two reasons:

  1. As just described, they make it possible for a non-expert to assess something the same way an expert would. As a result, evaluations become more consistent. Additionally, when marking with a Rubric, the non-expert won't miss things they would otherwise not even look for.
  2. When a Subject (e.g. a student) receives a mark of 58%, what does that really mean? There's nothing in a score of 58% that indicates what they did correctly and what they still don't understand. Scoring with a Rubric produces much more feedback for the subject, and they like that.

##Creating Good Rubrics is Hard Work

Creating a good Rubric is difficult for two reasons:

  1. It requires expert knowledge on the part of the author, yet...
  2. Many authors have built their expert knowledge up over a period of years. Consequently, they "just know" whether something is good or bad. They recognize a good hockey player, a good teacher or a quality car. They are so adept at evaluating subjects in their realm of expertise that they find it frustratingly difficult to write down exactly how they do it.

Defining a Rubric – Part One (Elements and Domains)

Thinking is always the hard part of any job, and it's no exception when creating a Rubric. The first step is to figure out what aspects of the subject you're going to evaluate - and you don't need a computer for that first step.

In this example, we will consider a Rubric for evaluating a salesperson. Whether or not a salesperson is doing a good job depends on many factors, such as:

  • attire
  • hygiene
  • use of appropriate language
  • product knowledge
  • attitude
  • work ethic
  • fills in forms so orders actually get shipped
  • who the competitors are
  • competitor advantages and how to sell against them
  • disadvantages of the competition
  • etc.

As you can see, there are a lot of different items that need to be looked at and we've only listed a few! Rubrix calls each of these things an Element. If there are a lot of Elements, the list becomes long and difficult to manage. For that reason, Rubrix lets you group related Elements into Domains. Our example list would have 3 Domains:

  • Customer Interaction, which in turn depends on these Elements:
    • attire
    • hygiene
    • use of appropriate language
    • product knowledge
    • etc.
  • Working with Others, which depends on these Elements:
    • attitude
    • work ethic
    • fills in forms so orders actually get shipped
    • etc.
  • Knowledge of Competition, which depends on these Elements:
    • who the competitors are
    • competitor advantages and how to sell against them
    • disadvantages of the competition

All of the Elements within a Domain should be related. If they are, then a numeric score can be associated with performance on each Element and those scores can be added up to give a score for the Domain.

But if Elements are not related, then they should be in different Domains. After all, how well you write essays has very little to do with how well you throw a football in P.E. or can solve equations in Mathematics. English, P.E. and Mathematics would be Domains. Throwing a football would be one of the Elements within the P.E. Domain.

Defining a Rubric – Part Two (Degrees)

Once the Elements and Domains have been determined, you have the general structure of the Rubric. Next, you have to define the Degrees, which are levels of performance.

With Rubrix, you can have as many Degrees as you like for any Element. The most common situation is to define 4 degrees: unsatisfactory, satisfactory, good and excellent. Of course, you might only have 2 Degrees (e.g. Pass, Fail) or 5 Degrees (unacceptable, poor, so-so, good and excellent) or any quantity you choose. Also, you can have a different number of Degrees for each Element if you like.

It is usually a good idea to have an even number of Degrees to avoid the "average" score in the middle; it forces you to ask yourself whether the Subject is slightly above or slightly below average.

Let's say that you have decided on 6 Degrees for the Hygiene Element. Your next task is to write descriptive text for each one. Here's an example:

  • Unacceptable - salesperson is indistinguishable from a long-dead zombie
  • Bad - deals well with customers that have colds or who stand at least 5 feet away
  • Below Average - salesperson is deficient in at least one area: properly brushed teeth, clean hair, clean fingernails, clean clothing
  • Acceptable - salesperson hygiene is acceptable
  • Good - grooming is generally very good and there is no basis for complaint
  • Excellent - salesperson attracts customers like a flower attracts bees.

Defining a Rubric – Part Three (Scoring)

Not all Rubrics include scoring. If a Rubric is designed to be part of a Subject's summative evaluation (e.g. their yearly review), scoring may be appropriate. However, if it is designed as an informal review intended to help the subject improve, text and examples plus personalized notes may be all that is needed.

But if you want to use Score, here's how:

  1. Create an Element With Points – this option is in a drop-down menu in the dialog that appears when you create an Element. Choosing for each Elements means that you can have a variety of Elements with or without points in one Domain. Only those that are set to displaywith points will count toward the final Grade for that Domain.
  2. Define Degrees and Points Values – for example, from Unacceptable=0 through Excellent=5
  3. Create a Marking Table – you will create a Marking Table for each Domain. Consider this a lookup table. If Excellent=5 (see above) and you have 4 Elements, the maximum score is 20. So in your Marking Table you might make Excellent=18, Average=10 and so on. The Scores you define in the Marking Table are the minimum required to get the corresponding Grade.

Remember: a Marking Table only refers a single Domain at this time.