Direct Assessment of BS-CSE Program/Criterion 3 Outcomes

See also assessment model and our rubrics.

1. Background

EAC/ABET now requires programs to have direct assessment of program outcomes, in addition to any indirect assessments they may be using. Direct assessment of outcomes means evaluating the degree to which various program outcomes are achieved by students by the time of their graduation based on actual performance of students on specific tasks (such as answering specific technical questions, their work on specific projects, etc.) related to the various outcomes, rather than on the basis of opinion surveys and the like.

Also, recently, we revised our program outcomes so that they are essentially identical to the EAC Criterion 3 outcomes with the exception that at the end of outcome (3.k), we used the phrase "practice as a CSE professional" in place of the phrase "engineering practice". For details, please see, the objectives and outcomes page of the on-line brochure.

Our program outcomes were revised in Feb. 2009, following changes in the CAC Criteria. The changes were the introduction of outcomes (l), (m), and (n), and some changes in the wording of outcomes (a) and (c). The net effect of these changes was relatively minor since what they did was to elaborate our previous set of outcomes by explicitly incorporating computing into the outcomes rather than, as did the previous set of outcomes, refer to generic engineering. Our existing assessment processes did not need any major revision following this revision of the (statement of) program outcomes. But we are still in the process of updating various documents to reflect the revised outcomes.

Our program outcomes are:

  1. an ability to apply knowledge of computing, mathematics including discrete mathematics as well as probability and statistics, science, and engineering;
  2. an ability to design and conduct experiments, as well as to analyze and interpret data;
  3. an ability to design, implement, and evaluate a software or a software/hardware system, component, or process to meet desired needs within realistic constraints such as memory, runtime efficiency, as well as appropriate constraints related to economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability considerations;
  4. an ability to function on multi-disciplinary teams;
  5. an ability to identify, formulate, and solve engineering problems;
  6. an understanding of professional, ethical, legal, security and social issues and responsibilities;
  7. an ability to communicate effectively with a range of audiences;
  8. an ability to analyze the local and global impact of computing on individuals, organizations, and society;
  9. a recognition of the need for, and an ability to engage in life-long learning and continuing professional development;
  10. a knowledge of contemporary issues;
  11. an ability to use the techniques, skills, and modern engineering tools necessary for practice as a CSE professional;
  12. an ability to analyze a problem, and identify and define the computing requirements appropriate to its solution;
  13. an ability to apply mathematical foundations, algorithmic principles, and computer science theory in the modeling and design of computer-based systems in a way that demonstrates comprehension of the tradeoffs involved in design choices;
  14. an ability to apply design and development principles in the construction of software systems of varying complexity.

These outcomes may be classified into three groups:

Outcome (f), in fact, falls under both the "technical skills" group as well as the "societal issues" group. It falls under the former group because of the importance of the ACM Code and the technical factors underlying the code; and the understanding that students must have of the Code and these factors in order to achieve this outcome. It falls under the latter group because of the general ethical considerations that students must understand in order to achieve this outcome. Hence this achievement of this outcome is evaluated as part of both groups.

2. Our approach

Given the distinct nature of the outcomes in the three groups, we have developed three distinct approaches, each carefully tailored to evaluating the degree of student achievement of the outcomes in the corresponding group.

2.1 Group 1 outcomes (technical skills): To evaluate the degree of achievement of outcomes in the first group, we have instituted an exit test that all BS-CSE majors will take prior to graduation. When a BS-CSE major applies for graduation, generally three quarters before the expected date of graduation, he or she will be asked to sign up to take the test. The test will be offered once each quarter, typically in the third week of the quarter. The questions on this test are designed to help assess the degree to which each outcome in the first group is achieved by BS-CSE students as they near completion of the program. Full details of this test, including a sample test, are available. Of course, in addition to this test, in the courses that help students achieve these outcomes, faculty use a variety of student activities and related assessments (such as midterm and final exams, programming projects, etc.) to assess the extent to which the course is achieving its intended outcomes and use the results of the assessments to effect improvements in the courses. Summaries of those assessments are included in the respective course group reports.

2.2 Group 2 outcomes (soft skills): The degree of achievement of the outcomes in the second group which deal with effective oral and written communication, teamworking, and lifelong learning, clearly cannot be tested in the same manner as the outcomes in the first group. Instead, the degree of achievement of these outcomes have to be assessed based on performance of students in suitable activities that require them to exercise the corresponding skills. Each of our capstone courses has been designed to incorporate, in an essential manner, such activities. We have designed specific rubrics that each of the capstone course instructors uses to evaluate specific student activities directly related to each of these specific outcomes, in their respective courses. Details of our approach to evaluating this group of outcomes, including these rubrics, are available.

2.3 Group 3 outcomes (societal issues): The degree of achievement of the outcomes in the third group which deal with "broad education" and "contemporary issues", also cannot be easily tested in the exit test. Hence, these outcomes are evaluated in CSE 601, the one credit required course on social and ethical issues in computing that students typically take immediately prior to the capstone course. Students in this course are required to write a paper on a suitable topic such as privacy, copyright, first amendment rights, etc., that is directly related to these outcomes. A suitable rubric has been designed that is used in CSE 601 to evaluate these papers in an appropriate manner. Given the nature of this student activity, it also contributes to further developing students' written communication skills (outcome (g)) and the rubric evaluates this aspect as well. Details of our approach to evaluating this group of outcomes, including this rubric, are available.

3. Process

The Undergraduate Studies Committee is responsible for coordinating these assessments and for discussing the results on a regular basis (typically once a year). Ideas for program improvement, based on the results of these direct assessments of each of the Criterion 3 outcomes, following discussions in UGSC are suggested to appropriate faculty for possible action. The results of the assessments and the improvements based on the results of the assessment will be documented.

4. Results of assessments and program improvements based on the results