Rubric for Assessment of CSE Capstone Design Projects (for use by capstone course instructors)

Rubric for Assessment of CSE Capstone Design Projects (for use by capstone course instructors) (pdf)

Background: This rubric is intended to help assess key aspects of the outcomes that the capstone design courses contribute to. The instructor of each section of each CSE capstone design course should complete one of these rubrics for each project team in the class and give a copy of the completed rubric to Neelam at the end of the semester; note that it is one rubric per team, not per student. Please do not modify this rubric in any manner. An ideal time to complete this rubric, for any given team, would be during or immediately following the final (in-class) presentation/demo of the project by the team.

The rubric includes seven dimensions. Each dimension is assigned a score of 1 through 4, these values representing increasing degrees of achievement as described below. The instructor should assign, in the rightmost column, a value between 1 and 4 --fractional values okay-- for each dimension. Additional comments may be noted at the bottom.

Course number, semester, instructor name:  ______________________________________________________
Title of capstone project being evaluated:     ______________________________________________________
Students in the team:  ________________________________________________________________________________________________________________________________

   1 2 3 4 Points
assigned

Problem formulation Unclear formulation;
Relation to original requirements not mentioned, nor changes in scope. Mostly clear but relation to original requirements and/or rationale for changes in scope not clear. Satisfactory formulation;
Relation to client's original requirements, changes in scope and rationale thereof mostly clear with some gaps. Excellent problem formulation;
Relation to client's original requirements and changes in the scope, if any, explained and justified.

Design approach Poor design;
No exploration of alternative approaches;
No attention to effective use of resources. Some attention to alternative design approaches but not a careful analysis of their advantages/ disadvantages;
Team picked an approach based on superficial comparisons. Careful consideration of alternative design approaches and their resource requirements;
Not all trade-offs fully analyzed. Thorough consideration and evaluation of a good set of design approaches;
Careful analysis of resource requirements of each and the resulting trade-offs;
Where appropriate, client's input sought before making final choice.

Implementation (including resource considerations, testing approach, adherence to standards, etc.) If implementation is incomplete, assess based on current state. Not even basic consideration of memory and other resource requirements;
System is very buggy.
No systematic testing, nor use of standard approaches/ processes such as agile. Limited amount of attention to memory and other resource usage;
Team has followed a standard (agile/ waterfall/ ...) process but not consistently.
Team has put some effort into systematic testing but some bugs remain. Careful attention to memory and other resource usage and how system might scale with increased demand for services;
The team adopted and mostly followed a standard process in its work;
The team used a systematic approach to testing and the system seems bug-free. Meticulous attention to resource usage and to user interface factors;
Has ensured that system can evolve to deal with increased demand for services.
Team has consistently followed a standard process in its work;
Adopted a suitable testing approach, followed it systematically, and thoroughly tested the system.
Client involved at all appropriate points.

Other factors such as use of professional tools, security considerations, ethical issues. Little attention paid to factors beyond minimal functional requirements;
No systematic use of professional tools;
Ethical issues related to system and impact on society not considered. Some use of common tools seen in earlier courses;
Modest effort to ensure basic reliability and security properties;
Mostly ignored ethical issues and potential impact on society of systems of this kind. Good use of professional tools going beyond ones previously seen;
System designed to be reliable/ secure under normal operation and under stress;
Some consideration of impact of system on society including potential harm system may cause in some situations. Excellent use of professional tools and systems, identified by careful research;
Detailed analysis of security holes with implementation designed to deal with ones that can be reasonably handled and documentation of rest;
Analysis of ethical issues related to system and its impact on society including implications of ACM/IEEE Code as it applies to the system, in consultation with client.

Effectiveness as a project team Dysfunctional team;
Members blamed each other for problems in project;
Team spirit completely lacking. Team functioned at minimal level of effectiveness;
Members concentrated on distinct parts of system without concern for impact on other members' work.
In presentations, individual members did not make any attempt to help other members address audience questions. Generally effective team;
Members interested in presenting a positive picture of the team's work;
Members helped each other during team presentations.
Team members had a general idea of other members' work. Very effective team;
Team members went out of the way to describe how each member contributed to various aspects of project.
Team worked as a cohesive unit during presentations, with members seamlessly handing over the conversation from one to another to answer questions, etc.

Effectiveness of written communication Documentation consisted of little more than (poorly commented) system code;
Hardly any mention of system's scope, design rationale, implementation choices, etc. Documentation mostly effective at conveying main aspects of project including scope and design/ implementation choices (but not the rationale behind the choices);
Skimpy user manual;
Information future teams may need to evolve system lacking. Team's documentation clearly presented all important aspects of project: original scope, changes made, implementation choices, processes used etc.
Test scripts and important parts of code explained;
Lessons learned were summarized;
Well-written user manual. Excellent documentation;
Project's original scope, design choices, relevant code details, processes and tools used, and test scripts all described in a structured and integrated manner;
Information to enable future designers to evolve system included;
Well-designed user manual provided all necessary information;
Illustrations, graphics, and layout executed to excellent effect.

Effectiveness of oral communication Presentations not effective;
Failed to present information about some essential aspects of project;
Team members ineffective in responding to even simple questions. Presentations adequate at conveying main ideas behind project including design choices, etc., but not engaging or inspiring.
Team responded appropriately to specific questions about specific aspects of project but some responses were unclear. Presentations were well done and presented all important aspects of project;
Team explained rationale behind its choices and summarized important lessons learned;
Responses to questions were reasonable although some went into too much technical detail, compromising their effectiveness. Team's presentations were polished, informative and engaging.
In answering questions, the team provided the right level and type of detail for questions ranging from implementation detail to test methodology to future evolution of project.

Comments:

	1	2	3	4	Points assigned
Problem formulation	Unclear formulation; Relation to original requirements not mentioned, nor changes in scope.	Mostly clear but relation to original requirements and/or rationale for changes in scope not clear.	Satisfactory formulation; Relation to client's original requirements, changes in scope and rationale thereof mostly clear with some gaps.	Excellent problem formulation; Relation to client's original requirements and changes in the scope, if any, explained and justified.
Design approach	Poor design; No exploration of alternative approaches; No attention to effective use of resources.	Some attention to alternative design approaches but not a careful analysis of their advantages/ disadvantages; Team picked an approach based on superficial comparisons.	Careful consideration of alternative design approaches and their resource requirements; Not all trade-offs fully analyzed.	Thorough consideration and evaluation of a good set of design approaches; Careful analysis of resource requirements of each and the resulting trade-offs; Where appropriate, client's input sought before making final choice.
Implementation (including resource considerations, testing approach, adherence to standards, etc.) If implementation is incomplete, assess based on current state.	Not even basic consideration of memory and other resource requirements; System is very buggy. No systematic testing, nor use of standard approaches/ processes such as agile.	Limited amount of attention to memory and other resource usage; Team has followed a standard (agile/ waterfall/ ...) process but not consistently. Team has put some effort into systematic testing but some bugs remain.	Careful attention to memory and other resource usage and how system might scale with increased demand for services; The team adopted and mostly followed a standard process in its work; The team used a systematic approach to testing and the system seems bug-free.	Meticulous attention to resource usage and to user interface factors; Has ensured that system can evolve to deal with increased demand for services. Team has consistently followed a standard process in its work; Adopted a suitable testing approach, followed it systematically, and thoroughly tested the system. Client involved at all appropriate points.
Other factors such as use of professional tools, security considerations, ethical issues.	Little attention paid to factors beyond minimal functional requirements; No systematic use of professional tools; Ethical issues related to system and impact on society not considered.	Some use of common tools seen in earlier courses; Modest effort to ensure basic reliability and security properties; Mostly ignored ethical issues and potential impact on society of systems of this kind.	Good use of professional tools going beyond ones previously seen; System designed to be reliable/ secure under normal operation and under stress; Some consideration of impact of system on society including potential harm system may cause in some situations.	Excellent use of professional tools and systems, identified by careful research; Detailed analysis of security holes with implementation designed to deal with ones that can be reasonably handled and documentation of rest; Analysis of ethical issues related to system and its impact on society including implications of ACM/IEEE Code as it applies to the system, in consultation with client.
Effectiveness as a project team	Dysfunctional team; Members blamed each other for problems in project; Team spirit completely lacking.	Team functioned at minimal level of effectiveness; Members concentrated on distinct parts of system without concern for impact on other members' work. In presentations, individual members did not make any attempt to help other members address audience questions.	Generally effective team; Members interested in presenting a positive picture of the team's work; Members helped each other during team presentations. Team members had a general idea of other members' work.	Very effective team; Team members went out of the way to describe how each member contributed to various aspects of project. Team worked as a cohesive unit during presentations, with members seamlessly handing over the conversation from one to another to answer questions, etc.
Effectiveness of written communication	Documentation consisted of little more than (poorly commented) system code; Hardly any mention of system's scope, design rationale, implementation choices, etc.	Documentation mostly effective at conveying main aspects of project including scope and design/ implementation choices (but not the rationale behind the choices); Skimpy user manual; Information future teams may need to evolve system lacking.	Team's documentation clearly presented all important aspects of project: original scope, changes made, implementation choices, processes used etc. Test scripts and important parts of code explained; Lessons learned were summarized; Well-written user manual.	Excellent documentation; Project's original scope, design choices, relevant code details, processes and tools used, and test scripts all described in a structured and integrated manner; Information to enable future designers to evolve system included; Well-designed user manual provided all necessary information; Illustrations, graphics, and layout executed to excellent effect.
Effectiveness of oral communication	Presentations not effective; Failed to present information about some essential aspects of project; Team members ineffective in responding to even simple questions.	Presentations adequate at conveying main ideas behind project including design choices, etc., but not engaging or inspiring. Team responded appropriately to specific questions about specific aspects of project but some responses were unclear.	Presentations were well done and presented all important aspects of project; Team explained rationale behind its choices and summarized important lessons learned; Responses to questions were reasonable although some went into too much technical detail, compromising their effectiveness.	Team's presentations were polished, informative and engaging. In answering questions, the team provided the right level and type of detail for questions ranging from implementation detail to test methodology to future evolution of project.
Comments: