CSE Undergraduate Studies Committee
Agendas and Minutes of Meetings (2008-'09)

Autumn:	(Meetings on Fridays at 12:30-1:30 pm in DL 298.) Oct. 10; Oct. 17; Nov. 7; Nov. 21; Dec. 5;
Winter:	(Meetings on Wednesdays at 9:30-10:30 am in DL 698.) Feb. 4; Feb. 18; March 4; April 8;
Spring:	(Meetings on Wednesdays at 3:30-4:30 pm in DL 298.) Apr. 22; May 6; May 27;

Agenda for meeting of Oct. 10, '08

1. Agenda for the year

Minutes of the meeting

1. Agenda for the year: The idea was to discuss the agenda items for the year but we got into details of some of the items.
   • CSE 421: This course is now a permanent course and students in both BS-CIS and BS-CSE majors can use it as a tech elective. The question here is when to make this a required course for both programs and a prerequisite for CSE 560 (as was the original idea of the course). The general consensus was that for now we should leave it as an elective course given the long approval process we would have to go through to make it a required course and given the apparently soon-to-come conversion to a semester system. If we have to go through such a conversion, it is going to severely impact every course and we will have to make many, many changes. So we might as well wait a while longer to see how that plays out. If the conversion doesn't happen, we can take up this issue at a later date. Paul Sivilotti who is the coordinator for 421, in email discussions after the meeting, agreed with this approach. At the same time, CSE 321 instructors will be requested to urge the students in their classes to take CSE 421 and to take it before CSE 560.
   • Enrollment management: The number of students being admitted to the CIS and CSE majors has been steadily increasing over the last two years or so. The four-quarter total admits is now around 200. Although this number is not huge by historical standards, since imposing a GPA requirement for admission to the major has a long lead time (it will apply only to freshmen entering OSU in the next fall after the faculty approves it), we need to consider taking appropriate action now. We will discuss this at the next UGSC meeting.
   • BS-MS program: Many of our majors seem unaware of the possibility of completing a joint BS-MS program (in which upto 15 credit hours counts toward both degrees). While there are some difficulties, the main one being that the student will be considered a graduate student during the time he or she is meeting the joint requirements and would have to pay the higher tuition, there are some important benefits as well, one being the substantially higher starting salaries that MS graduates can expect. It is not clear that there are any ways of addressing the higher tuition problem; in any case, the program at least needs to be publicized better among our undergrad majors and we will take steps toward this.
   • Accreditation: The next accreditation evaluation of the BS-CSE program is three years away (Au '11) but we need to make sure that our (assessment) processes meet the current requirements and that we follow the processes in a timely manner. There have been some changes in the CAC Criteria which may require us to revise our processes. We will discuss this in a near-future meeting.

The meeting was adjourned at 1:30 pm.

Next meeting: To be announced.

Agenda for meeting of 10/17/08

1. Agenda for the year (contd.)
2. Enrollment management
3. BS-CSE program outcomes

Minutes of the meeting

1. Enrollment management: We have not yet got some information --about the number of MS and PhD grads in recent years-- needed to go through the calculations related to enrollment management. Moreover, it may make sense to wait for one more quarter since, as long as we approve any new GPA requirement for admission to the major by the middle of Winter quarter, the requirement will apply to students admitted to OSU in Au '09; and waiting would give us one more quarter's data with regard to trends. So we will take up this topic early next quarter.

2. Agenda items: Most of the items have to do with making sure that we are going through all the activities dictated by the EAC and CAC accreditation requirements for the BS-CSE program. These have to do with such things as looking at recent alumni survey results, the results with regard to various activities introduced in the capstone courses following the last accreditation evaluation etc.

3. Advising workshops for new/recent faculty: One of the activities we had introduced some years ago, partly in order to satisfy accreditation requirements, was an annual advising workshop for recently hired faculty. The workshop has not been organized during the last couple of years. Given that we have several (relatively) new faculty in the dept., it would make sense to organize them again. Neelam will work with Peg on this.

4. BS-CSE Program outcomes: CAC has recently revised its accreditation criteria. The new criteria include a specific set of outcomes that all CS programs are required to include (either explicitly or implicitly) among the set of outcomes for their graduates. Full details of the new CAC criteria are available on-line at the ABET web site (www.abet.org). Our current set of outcomes (available on-line at Undergrad Program/BSCSE section of the dept.'s homepage) do not quite meet this CAC requirement. Hence, we need to change our outcomes appropriately; note that our current curriculum does ensure that gradutes will achieve all the CAC-specified outcomes; it is just that our published set of outcomes for the program, originally designed based on the EAC-specified outcomes, do not explicitly include some of the new CAC outcomes. Neelam presented a proposed revision to the outcomes to address this. There was an extensive discussion. Based on this discussion, a further revision has been prepared:

   Students in the BS-CSE program will attain:

   1. an ability to apply knowledge of computing, mathematics including discrete mathematics as well as probability and statistics, science, and engineering;
   2. an ability to design and conduct experiments, as well as to analyze and interpret data;
   3. an ability to design, implement, and evaluate a software or a software/hardware system, component, or process to meet desired needs within realistic constraints such as memory, runtime efficiency, as well as appropriate constraints related to economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability considerations;
   4. an ability to function effectively on multi-disciplinary teams;
   5. an ability to identify, formulate, and solve engineering problems;
   6. an understanding of professional, ethical, legal, security and social issues and responsibilities;
   7. an ability to communicate effectively with a range of audiences;
   8. an ability to analyze the local and global impact of computing on individuals, organizations, and society;
   9. a recognition of the need for, and an ability to engage in life-long learning and continuing professional development;
   10. a knowledge of contemporary issues;
   11. an ability to use the techniques, skills, and modern tools necessary for practice as a CSE professional.
   12. an ability to analyze a problem, and identify and define the computing requirements appropriate to its solution;
   13. an ability to apply mathematical foundations, algorithmic principles, and computer science theory in the modeling and design of computer-based systems in a way that demonstrates comprehension of the tradeoffs involved in design choices;
   14. an ability to apply design and development principles in the construction of software systems of varying complexity.

   We will discuss this further in the next meeting.

The meeting was adjourned at 1:20 pm.

Next meeting: TBA

Agenda for meeting of 11/07/'08

1. POCAT results
2. Course credit for AP exam
   (Our current policy: A student gets credit for CSE 201 if he or she has a score of 4 or 5 in the CS-A AP test; credit for CSE 201 with a score of 3, 4, or 5 in CS-AB test; and credit CSE 201, 214 with a score of 4 or 5 in CS-AB.
   The Board of Regents has apparently mandated that students must get course credit for a score of 3 in the AP test. So the proposal is to change the first item in the policy to say "credit for CSE 201 with a score of 3, 4, or 5 in the CS-A test"; the rest will stay the same.)
3. ABET Criteria
4. Proposed EEIC courses (multidisciplinary capstone courses) in the BS-CSE curriculum.

Minutes of the meeting

1. EEIC courses: The (newly formed) Engineering Education Innovation Center (EEIC) has developed a series of 3 courses, Eng 658, 659.01, 659.02, each for three credits, as a multidisciplinary capstone sequence for all engineering majors. 658, to be offered in fall quarters, is a lecture/concepts course which discusses a range of topics such as team working, ethical issues, project management, etc., related to working in teams on such projects. 659.01 and .02, to be offered in winter and spring quarters respectively, will form the capstone project. According to EEIC, the projects will typically come from industry partnerships and have a multidisciplinary nature. EEIC has asked for support of its proposal from each program and asked each program to consider whether these courses would count, for student who take the courses, as replacement for any other courses in the program, and whether the hours would count as technical elective hours in the respective program and, if so, to what extent.

   Although the courses seem reasonable, the committee had two main concerns. First, if we were to allow students to count all nine hours of these courses toward their tech electives, that would have a major (negative) impact on the number of CSE courses that these students would take. Second, it is not clear how much computing would be involved in these courses (beyond using a standard packages such as CAD/CAM packages). Based on the description in the materials that EEIC sent us, 658 contains almost no CSE content; and there is no guarantee that the individual projects in 659.01/.02 will have any specified CSE content. Thus if BS-CSE students were allowed to count this sequence toward their capstone course requirement, we may have problems with accreditation of the program. One possibility would be if a multidisciplinary engineering minor were to be developed with this sequence as part of the minor, and other appropriate courses (such as the design course that was proposed a while ago by EEIC(?)), we could allow our students to take this minor in which case the students would be able to count several of the hours from the minor toward their tech electives. (In such a case, the students will still be required to take one of our designated capstone design courses to meet their capstone requirement.)

   Based on these considerations, the committee decided that we will support the proposal but postpone the decision about allowing BS-CSE students to count all or part of these 9 credit hours as part of their program to a later date, perhaps after the courses have been offered once or twice. We will also suggest, to EEIC, the idea of developing a multidisciplinary engineering minor.

2. Credit for AP exam: Our current policy is that a student gets credit for CSE 201 if he or she has a score of 4 or 5 in the CS-A AP test; credit for CSE 201 with a score of 3, 4, or 5 in CS-AB test; and credit CSE 201, 214 with a score of 4 or 5 in CS-AB. The Board of Regents has apparently mandated that students must get course credit for a score of 3 in the AP test. So we considered a proposal to revise the first item in the current policy to say "credit for CSE 201 with a score of 3, 4, or 5 in the CS-A test"; the rest of the policy will stay the same.

   After a brief discussion, the proposal was approved. Neelam will send a message to the faculty mailing list asking for approval by the faculty.

3. POCAT results: The results of this quarter's POCAT were discussed. Some interesting (and somewhat worrisome) points were noted:
   • Student performance in the question related to relocatable code dropped compared to previous quarters.
   • Student performance in the question related to pre-conditions of methods dropped quite a bit compared to previous quarters.
   • Student performance in the question related to search trees dropped quite a bit compared to previous quarters.
   • Student performance in the new question related to database concepts was relatively poor.
   • Student performance in the remaining questions was comparable to previous quarters.
   There was no obvious explanation for the first item in the list. For the second item, one hypothesis was that several of the students who took POCAT this time might have taken the new CSE 421 course; and that one of the messages in this course --be generous in writing assertions describing the conditions expected to be satisfied at various points in your code-- may be contrary to the message in the 221-sequence about which part of the code is responsible for ensuring pre-conditons of methods. This difference is not so much because of different philosophies being adopted by instructors of the 221-sequence and 421, as because Java, which is the one used in 421 (as against RESOLVE-C++ used in the 221-sequence) makes it hard to abide strictly by the approach advocated in the 221-sequence. In any case, this suggests an interesting potential topic of discussion in 421.

   There was no obvious explanation for the reduced performance in the other questions. For the database-concepts question, part of the problem may be the relatively complex phrasing of the question. We will see if this can be addressed in future tests or if a completely different question on the topic can be introduced.

   Returning to the question about method pre-conditions and the hypothesis above that performance in this question may be related to whether the student did or did not take CSE 421, there is currently no way to verify or refute this hypothesis. This is becasue the way POCAT is currently set up, there is no way to know anything about the record of a student --such as which courses he/she took or his/her grade in a particular course-- who provided a certain answer to a certain question on the POCAT. This was done deliberately since we wanted to protect student anonymity. While this is clearly an important goal, given POCAT's purposes and the usefulness of being able to confirm/refute hypotheses such as this one, it may be worth looking into whether it would be possible to obtain some information about the students (such as which courses a student with a given POCAT code took and what grades he/she obtained in them) while still protecting the identity of the students. We will consider this further in future discussions.

4. ABET Criteria: We didn't have time to get to this topic.

The meeting was adjourned at 1:25 pm.

Next meeting: ??

Agenda for meeting of 11/21/'08

1. POCAT results (contd.)
2. Results of (BS-CSE) alumni survey
3. ABET Criteria (and related)

Minutes of the meeting

1. POCAT results: We continued discussion of the results of the Au '08 POCAT. David was concerned about the performance in the 660-based question (on interrupts). During this quarter as well as in Spring '08, student performance was much lower than expected by faculty. Two possible explanations were offered. First, the wording of this question may be less than ideal; it asks students to consider a number of possible sets of operations and then asks, which of the sets should be allowed only in the kernel mode; the idea being that if a set contained even one operation allowed in user mode, it was the wrong set. Some of the committee members felt this was a convulted way of asking, for each of a given list of operations, which should be allowed in user mode and which should be reserved for kernel mode. The 670-based questions, some of the committee felt, were even more involved. The point of all this is that, to the extent these concerns are valid, the POCAT results are not giving us a true picture of students' understanding of the material in question.

   Matt brought up another important point; i.e., that the ability of a student to answer some of the POCAT questions may depend, to a considerable extent, on which section of the particular course the student attended since different instructors tend to focus on different aspects of the course. While some variation in course material is to be expected and is appropriate, the intent of the questions on the POCAT are that they will deal with essential ideas. If the average performance of students on such questions is a function of which particular person they had as instructor for their particular section of the course, either the POCAT questions are not of the right kind or there is too much variation among the different sections or a combination of the two.

   Currently, given the way POCAT is administered, there is no way to determine if student performance is indeed a function of who the particular student's instructor for that course was. The reason for this was to preserve students' anonymity. There was some discussion of whether we could get some additional information while still protecting student anonymity. For example, if the test were on-line, a student taking the test could be asked to log into his/her CSE account, and the system could automatically obtain (from the university database) information about when the student took each of the required courses, who the instructors for the corresponding sections were, possibly what grades the student got in those courses, and include a suitably "anonymized" summary of this information in the "results summary". Such an approach would also eliminate the need for someone to enter the student answers into the system by hand each quarter, as is done currently. Moreover, once the student logs in, he or she can also complete the exit survey at the same time.

   Even more important, having the test on-line would make it possible to provide students with immediate feedback on their performance once they submit their answers; the system could display, for each question, what the right answer was, why it was right and, if the student's choice was incorrect, why that choice was incorrect. Thus the test would not only help assess the program but also serve to improve students' understanding of the material. Plus, of course, it will help answer such questions as whether student performance on specific questions depends on which section of the course the student took. We will discuss this further. A key question, if we want to make such a change, is protecting students' anonymity.

   The POCAT discussion occupied the entire hour; the other topics will be considered in future meetings.

The meeting was adjourned at 1:25 pm.

Next meeting: Dec. 5.

Agenda for meeting of 12/5/'08

1. Results of BS-CSE alumni survey
2. ABET Criteria (and related)

Minutes of the meeting

1. BS-CSE alumni survey: The results of the alumni survey conducted in Spring '08 are now available:
   • Background information
   • Program/ABET outcomes
   • Program objectives
   • Additional comments
   These results are similar to previous years' results. In the "Program objectives" survey we had asked respondents for their reaction to the proposal to rewrite our objectives to be consistent with the new ABET terminology (that "program objectives" should describe the expected accomplishments of graduates several years after graduation). The general reaction was positive, although there were also some negative comments; see the "additional comments" link above. We will discuss this further in UGSC, then present it during the annual Undergrad Forum in Winter quarter before asking for faculty approval.

The meeting was adjourned at 1:25 pm.

Next meeting: Winter quarter.

Agenda for meeting of 2/4/09

1. Annual Forum
2. BS-CSE program objectives/outcomes

Minutes of the meeting

1. Forum: Plans for the forum were discussed. The most convenient date seems to be Thursday, Feb. 5 at 5:30. So that is when we will hold it. The agenda will be similar to that for previous forums. Bruce, Paolo, and Peg will attend. Bettina Bair, Rajiv Ramnath, and Paul Sivilotti have also agreed to attend (Rajiv had to since cancel). We will try to get some local alums to attend as well.
2. BS-CSE program objectives/outcomes: One of the items for forum is to get student reaction to the proposed revisions to the BS-CSE program objectives and outcomes. The revised objectives and outcomes are available here.

The meeting was adjourned at 10:30 am.

Next meeting: ??

Agenda for meeting of 02/18/09

1. Report on Annual Forum
2. BS-CSE program objectives/outcomes
3. POCAT Results

Minutes of the meeting

1. Report on Annual Forum: Neelam reported on the student forum that was held on 2/5. The report on the forum is available on-line.
   The forum was attended by 14 students, including two representatives of student organizations (Jamie Colley of ACM-W and Aaron Joseph of Opensource Group). Faculty and staff present were Bettina Bair, Bruce, Neelam, Paolo, Peg, and Paul Sivilotti (who had to join late). Michael Compton and Shaun Rowland from the SOC Lab also attended. Two alumni, Jason Kirschenbaum and Derek Bronish (both current grad students in the dept.) also attended.
   Students had very positive reports about DL 172. One suggestion was to provide additional power outlets; Michael promised to look into this.
   The ACM-W and the Opensource group are both very active. Students who are not members are urged to check them out. Links to their web sites are in the forum report.
   The job market for computing majors seems to be holding steady despite the poor economy. Recent CSE and CIS graduates have reported job offers in the mid-fifties. Students who have not yet registered with the Engineering Career Services (link in the forum report) are urged to do so. Both CSE and CIS majors are eligible to use the services of the ECS office.
   Students are urged to read emails they receive from the the dept. Bruce and Peg will on trying to minimize the number of messages sent out on the undergrad mailing lists.
   Neelam presented proposed changes in the BS-CSE program objectives and outcomes (link in the forum report). These changes are necessitated by changes in accreditation criteria and their interpretation. There was general consensus that the changes are reasonable. These changes will be presented to faculty for its approval.
   Courses: There was some discussion of the RESOLVE/C++ sequence. There was a suggestion that the sequence should move to Java (with another student responding that the sequence is not about any language but about software engineering). There was some discussion of 421 with students reporting very favorable reactions to it. The question of making 421 a required course and the problems in doing so (increase in the number of required hours) was discussed; one suggestion was to replace CSE 541 the required course on numerical analysis) with 421. The Undergrad Studies Comm. will discuss this. Recent new courses were mentioned: CSE 786 (Game design and development; capstone design course); 459.24 (C#); 694X (Applied Information Sec. project); and 794Q (Cryptography). None of the students at the forum had taken any of these courses, so there was not much discussion.
   Scholarship applications are due March 3; interested should talk to someone in the Advising Office.
   The forum started at 5:30 pm and continued till about 7:20 pm with most students staying on till the end. Thanks to everyone who attended.

2. BS-CSE program objectives/outcomes: Following the consensus among students in favor of the proposed changes in the BS-CSE program objectives and outcomes, the last step is to ask for faculty approval. Neelam will send out a message to the faculty mailing list and ask for this. If Neelam receives any questions or concerns, the changes may have to be discussed at a faculty meeting. Otherwise, the changes will be considered approved and will replace the current set of objectives and outcomes.

3. POCAT Results: There was no time to discuss the POCAT results. Postponed to next meeting.

The meeting was adjourned at 10:30 am.

Next meeting: ??

Agenda for meeting of 03/04/'09

1. POCAT results
2. Possible revisions to POCAT
3. CSE 421 in the BS-CSE and BS-CIS curricula

Minutes of the meeting

1. CSE 421: The original idea behind developing CSE 421 was that it would become a required course and a prerequisite for CSE 560. However, given the looming change to semesters, it doesn't seem worth the trouble we would have go through to add the course to the list of required courses. As an alternative, it had been proposed that students be given the option of taking CSE 421 in place of CSE 541; students would also have the option of taking both courses (in which case one will be used as a tech elective). However, some faculty members objected to this, arguing that 541 contains important material and students should not be allowed to replace it with 421; the point was also made that we already have a three-quarter sequence (CSE 221, 222, 321) focused on software engineering and that should be adequate. Given these objections, and given the pending change to semesters, we decided not to try now to add 421 to the list of required courses (even as an alternate for a course such as 541).

2. POCAT results: The results of the Winter POCAT were similar to results of earlier tests. Performance on two of the questions on the test were cause for concern. The first had to do with CSE 560, the second with CSE 680. It turns out that the 560 question was supposed to have been revised since the version used was ambiguous; however, due to an oversight, the revised version was not used. This will be corrected for the Spring POCAT and we expect better results. The 680-related question has to do with the (worst case) running time to perform an insert operation in a binary search tree. Student performance, as in previous quarters, on this question was poor. For the Spring POCAT, we will revise the language of the question so students are reminded that these trees may not necessarily be balanced. Student performance in other questions was reasonable and matched faculty expectations.

3. Possible changes to POCAT: A recent trend in ABET circles is that (engineering) programs should be required/expected to define performance criteria (PC) corresponding to each program outcome and that assessment be performed of these PCs rather than of the outcomes directly. A PC, unlike a program outcome which is fairly general, identifies a relatively specific item of knowledge or skill and specifies an expected level of achievement with respect to that item. Given this, the intended learning outcomes (ILOs) that are listed in our course syllabi, can be considered as PCs since each ILO corresponds to a specific item of knowledge or skill at one of three levels of performance, mastery, familiarity, or exposure. Thus we could take the ILOs from the syllabi of the high-level required courses (that POCAT is based on), map them to our program outcomes, defining the ILOs corresponding that maps to each outcome as the corresponding PCs. Next we would have to map each POCAT question to one or, in some rare cases, more of these PCs; further, we would have to design one or more questions corresponding to each of the PCs. We cannot include all these questions in each POCAT since there will be far too many; instead, we will have to rotate through them so that each PC is assessed regularly, say, once every three or four POCATs.

   The major work involved in doing this would be to come up with suitable POCAT questions corresponding to each ILO of each high-level required course. Neelam and Nasko have come up with such questions for CSE 655; while it is not a trivial effort, it is a one-time effort and will help ensure that we meet ABET expectations.

   The committee agreed that we should make this change and that the questions proposed for CSE 655 serve as a good model. The main task now is to work with (some of) the faculty involved with the other high-level required courses to come up with similar sets of questions.

The meeting was adjourned at 10:30.

Next meeting: ??

Agenda for meeting of 04/8/'09

1. Common "soft outcomes" in capstone course syllabi and CSE 601 (see: http://www.cse.ohio-state.edu/~neelam/ugsc/minutes/softOutcomes.html).
2. Assessment/evaluation of soft outcomes
3. Change to semesters (very preliminary discussion)

Minutes of the meeting

The meeting was adjourned at ???

Next meeting: ??

1. POCAT: The results of the Spring POCAT are available on-line (along with results of earlier POCATs. The two CSE 655-related questions followed the "new model" of POCAT. That is, the questions were based on specific learning outcomes in the course syllabus. The performance of students in one of these questions matched faculty expectations (70% of the students got the right answer); it was below expectations (48% vs. the expected 70%) in the other question. The first question related to the outcome, "master analyzing data abstraction-related issues". The second related to the outcome, "master using syntax-related concepts including context-free grammars, parse trees, and recursive-descent parsing, printing, execution and code generation".
2. topic 2: ...
3. topic 3: ...

The meeting was adjourned at ???

Next meeting: ??