This is **old**; go here for current results.

Assessment results and evaluation of "professional" & "societal" outcomes

Improvements: Lecture in 601 on importance of professional appearance/manner during presentations; Discussion in 682 on speaking style (avoid fillers such as umm../ahh.. etc.); Development of rubric to evaluate how effectively students analyzed project risks (758); possible rubric for technology-teams' activities.
Introduction: We have classified the program outcomes for the BS-CSE program into three groups. The first group consists of outcomes dealing with technical skills; the second consists of outcomes dealing with "professional skills" (also occasionally called "soft skills"), i.e., team-skills, communication skills, and life-long learning ability (respectively, outcomes d, g, and i). The third consists of the two outcomes dealing with societal issues, i.e., an understanding of professional, ethical, social, etc. issues (outcome f); and a knowledge of contemporary issues (outcome j). The outcomes in the second and third groups are achieved via a range of activities in CSE 601, the required course on social, professional, and ethical issues in computing; and the various capstone design courses. These activities are assessed using a number of rubrics. Full details of the activities are available here, especially Section 2.2 onward of that page.

While the rubrics have been effective as assessment tools and help individual instructors assess students in their respective courses, the issue of evaluating the assessment results, identifying potential problems and possible solutions and, most importantly, enabling instructors to learn from each others' experiences has been challenging. After discussions in the Undergraduate Studies Committee in Autumn 2010, an approach, the results of which are documented in the current page, was set up to help addres this. The approach will be modified as we experiment with it.

Process: Twice each year (typically at the start of Winter and Spring quarters for now; once in each semester after we switch to semesters) the Undergraduate Studies Committee will organize a session where instructors from the most recent offerings of 3 or 4 of these courses will be asked to summarize the results they obtained using the rubrics. The numerical summary results will be added to the table below. The discussion in the meeting, for each course, will focus on two or three specific dimensions/component skills (from the total of seventeen dimensions) in the table below. A summary of this discussion will be included in the evaluation section below.

Summary results of these assessments appear in the table below. Each of the outcomes in these two groups has a number of component skills/dimensions which also, as may be expected, are the components in the rubrics used to evaluate the corresponding activities. hese components/dimensions appear in the first column of the table below. The remaining columns correspond to the various courses. For each course, the class average achievement during the most recent offering of the course in the current/past year, using the scales defined in the various rubrics, for each component skill is listed.

Evaluation of the assessment results: The instructors for each course will be asked to talk specifically about those two or three components in which students in the course seemed to either excel or seemed to have noticeable problems achieving a reasonable level of ability. In the former case, the hope is that other instructors will be able to copy whatever seemed to work in this course; in the latter case, that the others at the session will be able to offer ideas on how the problems may be addressed the next time the course is offered. It is also expected that the discussion may also lead to ideas for revisions in the rubrics. A summary of the discussion will be added to the evaluation portion of this page that follows the table of summary assessment results.

Summary Results

Some outcomes in the two groups are closely related so they are grouped together. Links to the corresponding rubrics are also included. Some rubrics are included more than once because the dimensions in those rubrics correpond to distinct outcomes.

Outcome dimension CSE 601 CSE 682 CSE 731 CSE 758 CSE 762 CSE 772 CSE 786
 
Outcome d. An ability to function on multi-disciplinary teams
Related Rubrics: Teamwork Rubric; Team Presentation Rubric
Contribution to the team work/ project Not applicable Wi '10: Sp '10: Au '10: Au '10: Wi '10: Sp '10:
Taking responsibility Not appl. Wi '10: Sp '10: Au '10: Au '10: Wi '10: Sp '10:
Valuing other team members Not appl. Wi '10: Sp '10: Au '10: Au '10: Wi '10: Sp '10:
Contribution as a team member during team presentations Not appl. Wi '10: Sp '10: Au '10: Au '10: Wi '10: Sp '10:
Outcome f. An understanding of professional, ethical, legal, security and social issues and responsibilities;
Outcome h. An ability to analyze the local and global impact of computing on individuals, organizations, and society;
Outcome j. A knowledge of contemporary issues;
Related Rubrics: Professional, Social, and Contemporary Issues Rubric.
Awareness of global effects Wi '10: Wi '10: Sp '10: Au '10: Au '10: Wi '10: Sp '10:
Understanding of economic factors Wi '10: Wi '10: Sp '10: Au '10: Au '10: Wi '10: Sp '10:
Awareness of implications to society at large Sp '10: Wi '10: Sp '10: Au '10: Au '10: Wi '10: Sp '10:
Awareness of (other) contemporary issues (political, cultural, ...) wi '10: Wi '10: Sp '10: Au '10: Au '10: Wi '10: Sp '10:
Understanding of ethical and professional issues Wi '10: Wi '10: Sp '10: Au '10: Au '10: Wi '10: Sp '10:
g. an ability to communicate effectively with a range of audiences;
Related Rubrics: Individual Pres. Rubric; Team Pres. Rubric; Rubric for evaluation of paper on Professional, Social, and Contemporary Issues; Rubric for evaluation of paper on tool/product/service
Organization of Oral Presentation Wi '10: 4 Wi '10: Sp '10: Au '10: Au '10: Wi '10: Sp '10:
Mechanics of presentation Wi '10: 3.86 Wi '10: Sp '10: Au '10: Au '10: Wi '10: Sp '10:
Delivery of presentation Wi '10: 3.57 Wi '10: Sp '10: Au '10: Au '10: Wi '10: Sp '10:
Relating to audience Wi '10: 3.93 Wi '10: Sp '10: Au '10: Au '10: Wi '10: Sp '10:
Presentation of ideas, organization of paper Wi '10: Wi '10: Sp '10: Au '10: Au '10: Wi '10: Sp '10:
Style (of paper) Wi '10: Wi '10: Sp '10: Au '10: Au '10: Wi '10: Sp '10:
i. a recognition of the need for, and an ability to engage in life-long learning and continuing professional development;
Related Rubric: Rubric for evaluation of paper on tool/product/service
Researching/ gathering information Not appl. Wi '10: Sp '10: Au '10: Au '10: Wi '10: Sp '10:
Analysis/ evaluation Not appl. Wi '10: Sp '10: Au '10: Au '10: Wi '10: Sp '10:

Evaluation

Note: Possible evaluation issues should correspond to specific dimensions listed above. For each course (offering), we should have two or three specific observations (from the instructor) each related to a specific dimension, followed by ideas on what we might change. The dimensions chosen should be ones that the students in the course seemed to have particular difficulties with in achieving a reasonable level of ability or ones in which they seemed to excel, preferably one from each. This might be followed by one or more observations unrelated to any of the dimensions (such as that we need a new dimension in one of the rubrics). This section will contain a summary of these observations so that faculty can learn from each other and use the ideas in their own courses.

  1. (Jan. '11) Outcome g: Based on Kitty Reeves' 601-experience with oral presentations in CSE 601 in Sp '09 and Wi '10, some useful changes:
  2. (Dec. '10) f, h: We need to evaluate projects in the capstone design courses against outcome (f) and possibly (h) but we don't do so now.
    Possible improvement: Perhaps create a rubric for this? Or, alternately and more directly, extend the team presentation rubric used in those courses to include dimensions related to these aspects?