CSE Undergraduate Studies Committee
Minutes of Meetings (2017-'18)

Spring:	Feb 26; Feb 12;
Autumn:	Nov 9; Sept 28; Sept 14; Sept 7;

• GE Comm. final report (Feb 2018)

Feb. 26

1. POCAT results, Au '17 (the tests may be accessed via the link in the second para of that page).

2. Capstone poster session results, Au '17

Feb. 12

1. Probation policy:
   Each program in CoE has a "Special Action Probation policy" (SAP policy) regarding the circumstances under which a major or a pre-major may be placed on probation or dismissed from the program or college, etc. We will consider a change in our SAP policy to ensure that students who are not making progress in the CSE curriculum don't hang around indefinitely. If we approve the change, it will have be approved by the faculty before being sent to CoE's Academic Standards and Progress (ASAP) committee for its approval before it becomes official.
   Action item: Discuss and approve the revised SAP policy.

2. Status of ABET evaluation:
   We are still awaiting the "draft statement" from ABET. We have been told that the statement should not have any surprises; i.e., we can expect it to list the same three shortcomings that the team listed at the end of its visit: insufficient strength in the Advising Office; failure to enforce prereqs for a number of courses; not meeting the requirement that the program ensure that the each student's program include coverage of programming language concepts. We have made progress on each of these items:
   Advising Office: We got approval to hire two additional full-time advisors. We have advertised the positions and are in the process of interviewing candidates.
   Prerequisites: We have started enforcing prerequisites. We have also set up an online system to allow coourse coordinators to approve exceptions for the rare cases where a student, although does not have credit for a particular prereq course, has, via job experience or the like, is able to demonstrate that he/she has the required knowledge and skills to succeed in the course. The coordinator will be required to document how the student demonstrated this. We expect such exceptions will be extremely rare and clearly justifiable. (The online system may be accessed via the CSE portal.)
   Prog. lang. concepts: All levels of the university have approved the change we discussed previously in the BS-CSE and BS-CIS programs: require all students to take CSE 3341 with CSE 3321 becoming a tech elective, rather than CSE 3321/3341 being a core-choice pair. This requirement will apply to students entering OSU in Au '18 or later.
   Once we receive ABET's draft statement, we have to respond, in 30 days, with details of the changes we have made to address the shorcomings; we can continue to provide updates to ABET until early May or so. For example, if we hire one new advisor by the time our response is due and are able to hire the next advisor a month later, we will be able to send that information as an update.
   There is one other complication: Currently, the SWS (Software Eng.) option requires students to take one of the following courses: CSE 3341, 5234, 5235, 5236. Since, after Au '18, all students will be required to take 3341, it has to be deleted from this list. Paul, Mike Bond, Nasko, and Neelam discussed whether we should consider replacing it with another course (such as 3321) but nothing seemed reasonable given the intent of this option. We did, however, think it would be useful to recommend 3901 as the junior project course for this option.
   Action item: Discuss and approve deletion of 3341 from the list of four courses that students in the SWS option have to choose one from.

3. POCAT results, Au '17 (the tests may be accessed via the link in the second para of that page).

4. Capstone poster session results, Au '17

1. Probation policy: Nikki described the existing "special action probation (SAP)" policy applicable to BS-CSE majors and the proposed changes. The primary change is to introduce conditions applicable to pre-majors, especially related to the problem of "failure to make progress". We have a number of pre-majors who remain pre-majors while taking few or no CSE courses; and/or who are so far behind in their GPAs compared to what is (currently) required for being admitted to the major that they have no realistic chance of getting into the major but continue to stay on as pre-majors semester after semester. After a brief discussion, the committee approved the change.

   Neelam will ask for (email) approval by the regular faculty of the proposed changes. (A marked-up document showing the changes is available here.)

2. Status of ABET evaluation:
   a) Nikki mentioned that we have received applications for the new advisor positions from a number of qualified candidates. We are in the process of interviewing several of them and expect to make offers in the next few days. The hope is that the new hires will be able to begin by late March; temporary office space is being worked on and will be on the third floor of Dreese. Permanent space will require considerable renovation to the the current Advising Office and is planned for Summer.
   b) We discussed the new prereq enforcement activities. One question that Paolo brought up had to do with CSE 2221 (Software I). Currently, the course requires the student to be concurrently enrolled in Math 1151 (Calculus I) although calculus is not required for the course; it is a question of math maturity, more than anything else. The best way to address such cases is to change the prereqs for the course (rather than completing a waiver that says, "calculus not really required for the course" because then the question becomes, "so why is it listed as a prereq?"). In the case of Software I, it would seem replacing the corequisite of Math 1151 with a prerequisite of "Math placement level L" would serve the purpose. We will work with the Curriculum Committee on this (and other similar changes that might be appropriate).
   c) Programming language concepts: As noted in the agenda, CSE 3341 will become a required course for students entering OSU starting in Au '18 and beyond, with 3321 becoming a tech elective. We discussed the question of how to revise the SWS (Software Eng.) specialization in view of this change and decided we will replace 3341 in that option with 3321. So the requirement for students following the SWS option, once 3341 becomes a required course, will be to take one of 3321, 5234, 5235, 5236. Nikki will work with the people responsible for updating the DARS system to program this into the system.

3. Being out of time, the meeting was adjourned at this point.

Next meeting: ??

Nov. 9, '17

1. Results of ABET evaluation, follow-up actions
2. CSE 2501, Phil 1338

1. Results of ABET evaluation and follow-up: Neelam summarized the results:
   • Shortcoming with respect to advising/monitoring-student-progress criterion because of insufficient number of advisors in the Advising Office;
   • Shortcoming with respect to advising/monitoring-student-progress criterion because we don't enforce all prerequisites.
   • Shortcoming with respect to the (CAC)curriculum criterion because we don't include (sufficient) coverage of programming language concepts in the core curriculum; and CSE 3341 which covers this material is a core-choice course, so students who take CSE 3321 in place of 3341 do not meet this requirement.
   With respect to the first item, we have requested the college for additional advisors (as well as office space) and we hope for some action soon. With respect to the second item, we have started enforcing prerequisites. We are also working on an online mechanism to enable faculty to waive prereqs in cases where it is appropriate, generally only in cases where the student clearly has knowledge of the relevant material; and to document such waivers in an archive.
   With respect to the third item, after considerable discussion via e-mail and during a meeting of the tenure-track and clinical-track faculty on Oct. 25, a motion to require all BS-CSE and BS-CIS majors to take CSE 3341 rather than either 3341 or 3321, was approved unanimously. CSE 3321 will become a technical elective course and students who wish to do so, may take that course as part of their tech elective hours. But, once the change is in effect, students will be required to take 3341, independently of whether or not they have taken 3321. The proposals have been sent to CCAA and to the Curriculum Committee for NMS (Natural and Math Sciences of Arts & Sciences) for their approval; following that, they will be sent to CAA for its approval. Once CAA approves the proposals, the new requirement will apply to all students entering OSU after the date of approval. [Note: As of Oct. 15, both CCAA and the NMS Curr. Comm. have approved the proposals.]
2. CSE 2501, Phil 1338: Currently, CSE and CIS majors are required to take either CSE 2501 or Phil 1338. 1338 is a 4-cr hr course, unlike 2501 which is a 1-cr hr. course; the additional 3 hours is counted towards the student's GE requirements. 1338 is relatively new and has been reasonably well received by our students.
   The Philosophy Dept. requested us to consider requiring 1338 of all our majors. There was an extended discussion about the pros and cons of this. On the one hand, it would slightly reduce the teaching load for us. On the other hand, there are some advantages to having a 1 cr hr. course to help with teaching assignments since it makes it possible to reach "full-time load" status for lecturers more easily in certain situations. Perhaps more importantly, as Viral pointed out, requiring all students to take 1338 as one of their GE courses seems unreasonable since some other courses (e.g., some sociology courses) that fall in the same GE category as 1338 might be more appealing to some students. Plus, of course, it reduces students' fleibility in scheduling courses. Also, CS minors need to take 2501 and that cannot be replaced with Phil 1338. Given all these factors, we decided that we will not go down this path.

The meeting was adjourned at 11:10 am.

Next meeting: ??

Sept. 28, '17

1. Plans for upcoming ABET evaluation
2. Revised rubric for use at the capstone poster session
3. Results of Exit Survey: Importance and level of achievement of program outcomes
4. Results of Exit Survey: Free-form questions

1. Plans for upcoming ABET evaluation: We did not spend time discussing this ... but they are coming along.

2. Capstone poster session rubric: (current rubric; revised version)
   We looked at the proposed changes in the rubric and they seemed reasonable. One important change is the introduction of a dimension, "Final product", which is intended to allow the person completing the rubric to provide an assessment of the quality of the system produced by the capstone team. This may be particularly relevant for courses such as 5912 (games capstone) where users' reaction to the actual final system is a critical part of the success metric. One change we decided on was to replace, in the revised rubric, the lowest rating for each dimension of "Unacceptable Performance" to "Unacceptable" to make it parallel the other ratings (such as "Minimally Acceptable"). (The rubric at the link above has been updated to include this change.)

3. Results of Exit Survey: Importance and level of achievement of program outcomes: There were no real surprises here, the results being similar to those of previous years (which can be seen in the same page). There was some discussion of how to interpret the "importance" rating vs. the "achievement" rating for the various outcomes. We agreed that the most sensible approach was to focus on the particular outcomes for which the apparent difference between the two is high, especially the ones for which importance is high (compared to the importance for other outcomes) and level of achievement is low (compared to the achievement for other outcomes). One outcome that sort of fit this category was outcome (c) [and the related (k)]: "Students in the BS-CSE program will attain an ability to design, implement, and evaluate a software or a software/hardware system, component, or process to meet desired needs within realistic constraints such as memory, runtime efficiency, ..." We do have a number of courses that require students to work on small and large projects, from Software I, II; Systems I, II; through several of the core-choice courses, the junior project course, the capstone design course, and a number of the technical electives. At the same time, conceptual foundations are, of course, critical as well; so it is a balancing act between these two aspects. In one respect, students (as well as employers!) don't always recognize that it is the foundation provided by the courses that focus on conceptual issues that enable students to do well in the project-focused courses; thereby leading to the argument that the program has too many courses on conceptual issues and too few that are project-focused. In any case, we will continue to tweak the courses in appropriate ways to account for both considerations.
   Two other outcomes that may also fall in this category (high importance, relatively lower achievement) are the ones related to communication and team skills. This issue is often reported by many CSE (as well as other engineering) programs. Here too, we will continue to look for opportunities to tweak the curriculum to enable students to improve their communication and team skills; but it is also worth noting that the results, e.g., reported from our assessment of the capstone poster sessions concerning the level of achievement of these skills displayed by students at the sessions have been reasonably high.

4. Results of Exit Survey: Free-form questions: Here again there were no major surprises. As in the past, students were highly appreciative of the quality of advising provided by the CSE Advising Office and of the high quality of teaching of several of our faculty. There were also a few negative comments about the quality of teaching, especially, by senior tenure-track faculty. Turning to the course offerings, students seemed to feel, on the one hand, that our project courses are very effective and, on the other, that we need more of them. There were some specific comments with respect to Software I, II (SW I, II). On the one hand, some students expressed very positive views about the courses and how they contributed to, as one student put it, "learning the proper techniques to design a maintainable system utilizing known data structures". At the same time, there were some negative comments as well about these courses; perhaps the most common one related to their use of non-standard components (as against standard Java library facilities).
   Much of the discussion in the committee focused on SW I, II. Viral (who took the course a couple of years ago) argued that, contrary to the claims of the some of the respondents in the survey that these courses, because of their use of non-standard components and practices, result in major problems for students in their internships etc., in his experience, these courses, especially the projects in SW II, were directly responsible for his landing an internship in Amazon. Viral also noted that he did find the first several weeks of SW I to be rather slow and repetitive.
   With respect to this last point, Paul and Paolo noted that this is a bit unavoidable because of the very wide variation in the backgrounds of the people entering SW I. With respect to the comments concerning non-standard components being used, Paul and Paolo noted that this is deliberate decision since some of the standard components have serious design flaws and a key point of the SW I/II sequence is the focus on well-designed components; moreover, the last part of SW II does have students go through several programming assignments using standard Java components. They also noted that they will continue to look for ways to increase the number of these activities to the extent possible while ensuring that the key goals of the sequence are not compromised (as would happen, for example, if the sequence were replaced by a "standard" CS I/II sequence).
   There was also a suggestion that it might be useful to send out a brief survey to students (perhaps juniors, early seniors) asking for their opinions about specific aspects of SW I, II, including feedback on how the sequence helped (or did not help) with their obtaining or performance in an internship. We will explore this further.

Sept. 14, '17

1. Plans for upcoming ABET evaluation
2. Rubric results for capstone poster session from Spring '17
3. Rubric results for CSE 2501 from Spring '17

1. Preparations for ABET site visit: Neelam provided a summary of the preparations that are being made, especially for 10/16, the day when the evaluators will be meeting with faculty, staff, and students. Our program will have three people assigned to it, a CAC team chair, a CAC program evaluator (PEV), and an EAC PEV. The CAC team chair will spend most of his time in meetings at the college and university-level people, the exceptions being a meeting with Xiaodong in the morning and the student meeting in the afternoon. The CAC PEV and EAC PEV will attend those meetings and will also meet one-on-one with a number of faculty; Neelam will be in touch with the faculty who are on the schedule. In addition, the CAC and EAC PEVs will meet with Nikki and also tour the (departmental) computing facilities. But all this is tentative and subject to change based on what the team decides it wants to do ...

2. CSE 2501 rubric results: The recently developed rubric for CSE 2501/Phil 1338 is at here. Each year, results from the use of the rubric in one section of each course is planned to be discussed in a UGSC meeting to identify weaknesses in the program, including in the rubric, and come up with possible improvements. [Note that it is up to the individual instructor to decide whether the use of the rubric is appropriate for the particular section that he or she is teaching. Many instructors are likely to use the rubric since it is designed to give them a good understanding of their students' performance in the class. But based on our past experience in trying to discuss results from all sections of a course we decided it was sufficient for UGSC to discuss the results from one section each year since there is no reason to expect that the results will differ substantially from one section to another. This is our intent with respect to the rubrics for the capstone courses as well.]
   During the Spring '17 semester, Michelle Mallon reported the rubric results from her section of CSE 2501. The results (anonymized to remove student identification) are available here. This was a discussion of the results from that section. Michelle was not able to attend the meeting since she had a class at the time of the meeting.
   A number of points were noted. The scores in the dimensions related to the oral presentation seemed mostly satisfactory (with only a few students performing poorly). By contrast, the scores in the dimensions related to the "research paper" showed substantial variation with several fairly low scores in the dimensions related to awareness of contemporary issues and awareness of implications to society at large. In a way, this is not surprising since many of our students tend to be focused on technical matters and somewhat oblivious of societal issues.
   Is there a way to address this? A main problem, of course, is that 2501 is a one-credit hour course so there is hardly room for the instructor to provide much instruction, especially given that each student has to make presentations; plus, the specific computing-related issues naturally take precedence over general discussions of societal issues. Of course, students are supposed to develop understanding of societal issues in their general education courses but since different students will take different GE courses (and at different times in their programs), there is no way to rely on that.
   There may, however, be an alternative. The Philosophy Dept. which has worked with us to develop the 4-credit hour Phil 1338 course, is eager to have us require all our students to take that course rather than have it as just an alternative to CSE 2501 (and a 3-cr hr GE course). In other words, the idea would be to replace CSE 2501 in the curriculum with Phil 1338 (and reduce the other GE requirements by 3 cr hrs so that the change would be credit hour-neutral). Nikki noted that feedback from our students who have taken Phil 1338 has been quite positive. We will keep this option in mind.

   A different question had to do with the fractional scores (such as 3.52) that some students received on some dimensions. One would expect, in using a rubric, that each student would receive a score such as 3 or 4 or possibly 3.5 but something like 3.52 seems rather strange. After the meeting, Neelam contacted Michelle by email and she clarified that she uses her own scoring method to score the papers and, for the purposes of the rubric, converted those scores to a 4-point scale; hence the fractional values. We will have to discuss whether this is a suitable approach or whether it would be better to use the rubric results from a section in which the instructor uses the rubric directly.

   There was another question, this one concerning the adequacy of the rubric, with respect to assessing the papers. While it addresses the ethical and professional issues as well as societal implications, a common problem that many students, including our majors, face when presenting a point of view is their failure to know the difference between stating an opinion on the one hand versus presenting a cogent, logical argument on the other. The dimension, "Presentation of ideas and organization of the paper", addresses this but to a very limited extent and in an unclear manner, focusing primarily on the information presented in the paper. Al Cline mentioned that, when he teaches 2501, he gives students clear, written guidance on the paper, including the importance of presenting a logical argument, grounded in relevant ethical theories. We will work on revising the rubric to include a dimension that addresses this.
   (Since then, Neelam has worked on revising the rubric; a draft of this revised version is available here. The changes are all in the "Presentation of ideas" dimension. Comments are welcome.)

3. Capstone poster session rubric results: The rubric for the capstone poster session is here. The results from the poster session at the end of Sp '17 are here.

   There were a number of comments. Overall, the capstone project teams' performance seems satisfactory. The results in the two dimensions related to team effectiveness seem very good, even better than we might have expected, given the common refrain that our students do not seem to be effective team players; on the other hand, these scores are based on performance in the final poster session and, as was mentioned, teams tend to put their best foot forward in such settings. A different point that was noted had to do with the scores in the dimension related to "other factors". The scores for some of the teams, the ones in 5914 (in which each team works on a project that uses the Watson system to solve some suitable problem), were rather low. This seemed to be because these teams, given the nature of the course and the project they were expected to work on, did not consider the possibility of using various professional tools. Thus the problem here seems related to the rubric dimension, rather than to the performance of the teams. We will explore revising the rubric to address this. For example, it may be appropriate to have an alternative dimension that would be suitable for the Watson-based course that addresses similar outcomes as the existing dimension. This may be also relevant for CSE 5913, the capstone course on animation that Huamin plans to teach in the spring since all student teams in that course are expected to use a specific tool, Maya, in their projects. [Note: After the minutes were posted, Roger Crawfis who teaches 5912 regularly (and who could not attend the UGSC meeting), noted the following: "I find the teamwork of our students not just satisfactory, but exceptional. Yes, there may occasionally be a dysfunctional team, but I find this rare and frankly, more often in other disciplines. I fear this is a historical statement that is perhaps constantly being perpetuated. Not sure if CSE 5912 is a weird exception to this or I somehow get them to cooperate, but I doubt it. With the 390x and perhaps other factors, let's take the time to praise our students and take pride that the evaluators noticed this."

   Another important point that was noted was that having each of the people who completes the evaluation to evaluate 8-10 projects each may be a bad idea since this doesn't allow the evaluator to spend sufficient time with each project to do a sufficiently careful evaluation. On the other hand, the idea of having only capstone course instructors perform these evaluations given that they have an in-depth understanding of the nature of these courses, seems reasonable. A possible compromise would seem to be to do a sampling of the posters rather than try to get each poster evaluated. In other words, each evaluator would be asked to evaluate, say, 3 posters which means that we will have evaluations of fifteen to twenty posters (rather than all forty or so posters at the Spring session). We will make a decision on this before the fall poster session (in December).

The meeting was adjourned at 11:10

Next meeting: ??

Sept. 7, '17

1. Plans for the year
2. Waitlist procedures for CSE courses
3. Possible changes in ECE 2020
4. Rubric results for capstone poster session from Spring '17
5. Rubric results for CSE 2501 from Spring '17

1. Waitlist procedures:
   Recently, we changed the waitlist procedures for admitting students to closed courses. With the new procedure, automatic enrollment of waitlisted students into the respective courses, e.g., when someone who is enrolled in the course drops the course, is turned off several days before the start of the semester. This is intended to allow faculty to appropriately prioritize students waitlisted for enrollment in the course.

   Although the new procedures seem to have worked as far as enrolling the right students into the classes, the advisors in the Advising Office have had to put in a lot of effort to get it done. This is especially problematic towards the deadline of the second Friday of the semester because many faculty have sent in their enrollment permissions on that day, some times late on that day, putting considerable pressure on the advisors. To avoid this as well as to address other problems that might come up, Rafe will revise the message he sends out to include more precise instructions for faculty to follow. All faculty are urged to follow the instructions carefully so students, especially CSE/CIS majors, are able to enroll in the courses they need.

2. ECE 2020, 2060:
   BS-CSE majors are required to take ECE 2020 (analog circuits), and 2060 (digital logic); BS-0CIS majors are required to take ECE 2060. ECE majors are required to take both courses.
   The ECE faculty have noticed that BS-CSE students do not perform as well as ECE students in 2020; there doesn't seem to be any particular difference in performance between the two groups in ECE 2060. An importance reason for the difference with respect to 2020 may be that ECE students are required to take Physics 1251 one of whose major topics is electricity and magnetism and CSE majors are not required to take this course. Phys 1251 is not officially a pre-requisite for ECE 2020 but the standard bingo-sheet for ECE lists Phys 1251 in a semester preceding 2020; hence many ECE students are likely to have completed the course before taking 2020. Informal feedback from CSE students suggests that general lack of interest, among CSE majors, in hardware-related topics may also have something to do with this difference. It should also be noted that it is not that CSE majors are failing the course or anything like that; it is just that they are not doing as well as ECE students.

   Gagan Agrawal and Neelam met some of the ECE faculty about this. There seemed to be various ways to address this but the most popular one seemed to be to have ECE offer two versions of the course, one for CSE majors and the other for ECE majors with latter including Phys 1251 as a pre-req and the former tweaked to be not only more accessible to CSE majors but also more interesting (by possibly including such things as the use of software packages to analyze circuits). The flip side is that there may be some concern among some ECE faculty about the impact on their ability to staff the two distinct versions of the course; at the same time, given the 200+ students CSE students (and a similar number of ECE students) who take the course each year, this should, hopefully, be manageable. We will try to get this option implemented.

   There was no time to discuss any of the other items in the agenda. They will be taken up in a later meeting.

The meeting was adjourned at 11:10.

Next meeting: 9/14/'17