Descriptors

Why Include Levels of Performance?

Analytic Versus Holistic Rubrics

How Many Levels of Performance Should I Include in my Rubric?

 

Rubric:  A scoring scale used to assess student performance along a task-specific set of criteria

Authentic assessments typically are criterion-referenced measures.  That is, a student's aptitude on a task is determined by matching the student's performance against a set of criteria to determine the degree to which the student's performance meets the criteria for the task.  To measure student performance against a pre-determined set of criteria, a rubric, or scoring scale, is typically created which contains the essential criteria for the task and appropriate levels of performance for each criterion.  For example, the following rubric (scoring scale) covers the research portion of a project:

Research Rubric

As in the above example, a rubric is comprised of two components:  criteria and levels of performance.  Each rubric has at least two criteria and at least two levels of performance.  The criteria, characteristics of good performance on a task, are listed in the left-hand column in the rubric above (number of sources, historical accuracy, organization and bibliography). Actually, as is common in rubrics, the author has used shorthand for each criterion to make it fit easily into the table. The full criteria are statements of performance such as "include a sufficient number of sources" and "project contains few historical inaccuracies."

  For each criterion, the evaluator applying the rubric can determine to what degree the student has met the criterion, i.e., the level of performance. In the above rubric, there are three levels of performance for each criterion. For example, the project can contain lots of historical inaccuracies, few inaccuracies or no inaccuracies.

Finally, the rubric above contains a mechanism for assigning a score to each project. (Assessments and their accompanying rubrics can be used for purposes other than evaluation and, thus, do not have to have points or grades attached to them.) In the second-to-left column a weight is assigned each criterion. Students can receive 1, 2 or 3 points for "number of sources." But historical accuracy, more important in this teacher's mind, is weighted three times (x3) as heavily. So, students can receive 3, 6 or 9 points (i.e., 1, 2 or 3 times 3) for the level of accuracy in their projects.

Descriptors

The above rubric includes another common, but not a necessary, component of rubrics -- descriptors. Descriptors spell out what is expected of students at each level of performance for each criterion. In the above example, "lots of historical inaccuracies," "can tell with difficulty where information came from" and "all relevant information is included" are descriptors. A descriptor tells students more precisely what performance looks like at each level and how their work may be distinguished from the work of others for each criterion. Similarly, the descriptors help the teacher more precisely and consistently distinguish between student work.

Many rubrics do not contain descriptors, just the criteria and labels for the different levels of performance. For example, imagine we strip the rubric above of its descriptors and put in labels for each level instead. Here is how it would look:

It is not easy to write good descriptors for each level and each criterion. So, when you first construct and use a rubric you might not include descriptors. That is okay. You might just include the criteria and some type of labels for the levels of performance as in the table above. Once you have used the rubric and identified student work that fits into each level it will become easier to articulate what you mean by "good" or "excellent." Thus, you might add or expand upon descriptors the next time you use the rubric.

Why Include Levels of Performance?

As mentioned in Step 3, it is very useful for the students and the teacher if the criteria are identified and communicated prior to completion of the task. Students know what is expected of them and teachers know what to look for in student performance. Similarly, students better understand what good (or bad) performance on a task looks like if levels of performance are identified, particularly if descriptors for each level are included.

In addition to better communicating teacher expectations, levels of performance permit the teacher to more consistently and objectively distinguish between good and bad performance, or between superior, mediocre and poor performance, when evaluating student work.

Furthermore, identifying specific levels of student performance allows the teacher to provide more detailed feedback to students. The teacher and the students can more clearly recognize areas that need improvement.

Analytic Versus Holistic Rubrics

For a particular task you assign students, do you want to be able to assess how well the students perform on each criterion, or do you want to get a more global picture of the students' performance on the entire task? The answer to that question is likely to determine the type of rubric you choose to create or use: Analytic or holistic.

Analytic rubric

Most rubrics, like the Research rubric above, are analytic rubrics. An analytic rubric articulates levels of performance for each criterion so the teacher can assess student performance on each criterion. Using the Research rubric, a teacher could assess whether a student has done a poor, good or excellent job of "organization" and distinguish that from how well the student did on "historical accuracy."

In contrast, a holistic rubric does not list separate levels of performance for each criterion. Instead, a holistic rubric assigns a level of performance by assessing performance across multiple criteria as a whole. For example, the analytic research rubric above can be turned into a holistic rubric:

In the analytic version of this rubric, 1, 2 or 3 points is awarded for the number of sources the student included. In contrast, number of sources is considered along with historical accuracy and the other criteria in the use of a holistic rubric to arrive at a more global (or holistic) impression of the student work. Another example of a holistic rubric is the "Holistic Critical Thinking Scoring Rubric" (in PDF) developed by Facione & Facione.

Analytic rubrics are more common because teachers typically want to assess each criterion separately, particularly for assignments that involve a larger number of criteria. It becomes more and more difficult to assign a level of performance in a holistic rubric as the number of criteria increases. For example, what level would you assign a student on the holistic research rubric above if the student included 12 sources, had lots of inaccuracies, did not make it clear from which source information came, and whose bibliography contained most relevant information? As student performance increasingly varies across criteria it becomes more difficult to assign an appropriate holistic category to the performance. Additionally, an analytic rubric better handles weighting of criteria. How would you treat "historical accuracy" as more important a criterion in the holistic rubric? It is not easy. But the analytic rubric handles it well by using a simple multiplier for each criterion.

When to choose a holistic rubric

So, when might you use a holistic rubric? Holistic rubrics tend to be used when a quick or gross judgment needs to be made. If the assessment is a minor one, such as a brief homework assignment, it may be sufficient to apply a holistic judgment (e.g., check, check-plus, or no-check) to quickly review student work. But holistic rubrics can also be employed for more substantial assignments. On some tasks it is not easy to evaluate performance on one criterion independently of performance on a different criterion. For example, many writing rubrics (see example) are holistic because it is not always easy to disentangle clarity from organization or content from presentation. So, some educators believe a holistic or global assessment of student performance better captures student ability on certain tasks. (Alternatively, if two criteria are nearly inseparable, the combination of the two can be treated as a single criterion in an analytic rubric.)

How Many Levels of Performance Should I Include in my Rubric?

There is no specific number of levels a rubric should or should not possess. It will vary depending on the task and your needs. A rubric can have as few as two levels of performance (e.g., a checklist) or as many as ... well, as many as you decide is appropriate. (Some do not consider a checklist a rubric because it only has two levels -- a criterion was met or it wasn't. But because a checklist does contain criteria and at least two levels of performance, I include it under the category of rubrics.) Also, it is not true that there must be an even number or an odd number of levels. Again, that will depend on the situation.

To further consider how many levels of performance should be included in a rubric, I will separately address analytic and holistic rubrics.

Analytic rubrics

Generally, it is better to start with a smaller number of levels of performance for a criterion and then expand if necessary. Making distinctions in student performance across two or three broad categories is difficult enough. As the number of levels increases, and those judgments become finer and finer, the likelihood of error increases.

Thus, start small. For example, in an oral presentation rubric, amount of eye contact might be an important criterion. Performance on that criterion could be judged along three levels of performance: never, sometimes, always.

Although these three levels may not capture all the variation in student performance on the criterion, it may be sufficient discrimination for your purposes. Or, at the least, it is a place to start. Upon applying the three levels of performance, you might discover that you can effectively group your students' performance in these three categories. Furthermore, you might discover that the labels of never, sometimes and always sufficiently communicates to your students the degree to which they can improve on making eye contact.

On the other hand, after applying the rubric you might discover that you cannot effectively discriminate among student performance with just three levels of performance. Perhaps, in your view, many students fall in between never and sometimes, or between sometimes and always, and neither label accurately captures their performance. So, at this point, you may decide to expand the number of levels of performance to include never, rarely, sometimes, usually and always.

There is no "right" answer as to how many levels of performance there should be for a criterion in an analytic rubric; that will depend on the nature of the task assigned, the criteria being evaluated, the students involved and your purposes and preferences. For example, another teacher might decide to leave off the "always" level in the above rubric because "usually" is as much as normally can be expected or even wanted in some instances. Thus, the "makes eye contact" portion of the rubric for that teacher might be

So, I recommend that you begin with a small number of levels of performance for each criterion, apply the rubric one or more times, and then re-examine the number of levels that best serve your needs. I believe starting small and expanding if necessary is preferable to starting with a larger number of levels and shrinking the number because rubrics with fewer levels of performance are normally

• easier and quicker to administer
• easier to explain to students (and others)
• easier to expand than larger rubrics are to shrink

The fact that rubrics can be modified and can reasonably vary from teacher to teacher again illustrates that rubrics are flexible tools to be shaped to your purposes. To read more about the decisions involved in developing a rubric, see the chapter entitled, "Step 4: Create the Rubric."

Holistic rubrics

Much of the advice offered above for analytic rubrics applies to holistic rubrics as well. Start with a small number of categories, particularly since holistic rubrics often are used for quick judgments on smaller tasks such as homework assignments. For example, you might limit your broad judgments to

• satisfactory
• unsatisfactory
• not attempted

or

• check-plus
• check
• no check

or even just

• satisfactory (check)
• unsatisfactory (no check)

Of course, to aid students in understanding what you mean by "satisfactory" or "unsatisfactory" you would want to include descriptors explaining what satisfactory performance on the task looks like.

Even with more elaborate holistic rubrics for more complex tasks I recommend that you begin with a small number of levels of performance. Once you have applied the rubric you can better judge if you need to expand the levels to more effectively capture and communicate variation in student performance.

To read more about the decisions involved in developing rubrics, see

Step 4: Create a Rubric

Copyright 2014, Jon Mueller. Professor of Psychology, North Central College, Naperville, IL. Comments, questions or suggestions about this website should be sent to the author, Jon Mueller, at jfmueller@noctrl.edu.