Artificial Intelligence Course Group Report (2009)

Course no.	Title	Credit Hours	Reqd (R)/ Elective (E)
CSE 612*	Introduction to Cognitive Science	3	E
CSE 630	Survey of Artificial Intelligence I: Basic Techniques	3	E
CSE 634	Computer Vision for Human Computer Interaction	3	E
CSE 730	Survey of Artificial Intelligence II: Advanced Topics	3	E
CSE 731	Knowledge-Based Systems (capstone)	4	E
CSE 732	Computational Linguistics	4	E
CSE 733	Foundations of Spoken Language Processing	3	E
CSE 735	Machine Learning and Statistical Pattern Recognition	3	E
CSE 737*	Proseminar in Cognitive Science	2	E
CSE 739*	Knowledge-Based Systems in Engineering	3	E
CSE 779**	Introduction to Neural Networks	3	E
* : Cross-listed course taught primarily in another department ** : Cross-listed course taught primarily in CSE

1. Summary

Artificial Intelligence (AI) is a major area of computer science, broadly concerned with how to make computers "smart". It is one of the focus areas of the Department.

The field of AI continues to evolve at a rapid pace, fueled by the ancient dream of the mankind to make a machine that behaves like a man, as well as by the research and development taking place in industry at companies such as Microsoft, IBM, AT&T, Google, GE, just to name a few. In addition, AI has become indispensable in a variety of application areas because of advances in computer vision, speech and language processing, inference engines, knowledge-based systems, and robotics. AI has also become a main bridge between computer science and many other disciplines, such as other engineering disciplines, cognitive science, linguistics, psychology, neuroscience, math/physics/statistics, speech and hearing sciences, and medicine.

We have evolved the AI curriculum continuously to keep pace with the development of the field, the interests of students, and the demands of employers. We currently have nine regular courses in AI, many of which are cross-listed with other departments, reflecting the interdisciplinary nature of the field. The 630/730 sequence is the core introduction to AI. The AI faculty also offer research seminars that cover recent developments in computer vision, neural networks, speech processing, knowledge systems, and cognitive science. Developing new courses on these topics is a priority of the AI faculty.

While the AI curriculum is completely elective for undergraduates, the AI curriculum is compatible with many of the CSE and ABET objectives. On the application side, AI is an area in which students can exercise the principles of software design, databases, algorithms, data structures and programming languages which they have learned in the foundation courses. The AI courses rely heavily on the application of various principles from mathematics, physics, statistics, neuroscience, and psychology. Students who study AI in the Department receive a good grounding in the area, are usually ready for graduate studies in AI and cognitive science, and are very employable. AI also draws from an interdisciplinary background, and several of our courses offer opportunities for working with interdisciplinary teams.

2. Detailed Analysis

The CSE department regularly offers the following AI courses: CSE 630, CSE 634, CSE 730, CSE 731, CSE 732, CSE 733, CSE 735, CSE 779; CSE 739 is regularly offered as a cross-listed course by a non-CSE faculty member. It has also offered two courses in cognitive science: CSE 612 and CSE 737 on a regular basis, most recently taught by psychology faculty.

Section 2.1 describes the individual courses in the group, including their relations to the other courses in the group as well as to the rest of the curriculum. Section 2.2 contains an evaluation of each course to see how well it meets its intended learning outcomes (LOs). Section 2.3 considers the relation between the LOs of the various courses and the (proposed new) BS-CSE program outcomes (listed in Section 2.3). Section 2.4 summarizes the main changes we have made in the courses since the previous report. Section 2.5 discusses some continuing concerns and our plans to address them.

2.1 Summary of the courses

CSE 612: Introduction to Cognitive Science provides an introduction to the interdisciplinary study of the nature of human thought, and psychological, philosophical, linguistic, and artificial intelligence approaches to knowledge representation. The course is cross-listed with the Departments of Linguistics, Philosophy, and Psychology.

CSE 630: Survey of Artificial Intelligence I: Basic Techniques is a survey of the basic concepts and techniques of problem solving paradigms and knowledge representation schemes in Artificial Intelligence. This is the entry-level AI course, and is a prerequisite for many subsequent AI courses.

CSE 634: Computer Vision for Human Computer Interaction is a course in computer vision algorithms for use in human-computer interactive systems. Topics include image formation, image features, segmentation, shape analysis, object tracking, motion calculation, and applications.

CSE 730: Survey of Artificial Intelligence II: Advanced Topics is a survey of advanced concepts, techniques, and applications of artificial intelligence, including perception, learning, reasoning under uncertainty, natural language processing, and speech recognition. This course continues the introduction of artificial intelligence in CSE 630 for undergraduates. It is often taken as a first course in AI for entering graduate students.

CSE 731: Knowledge-based Systems is a survey of theory and practice of expert systems and knowledge-based systems, and makes use of current knowledge-based systems software tools. This is a BSCSE capstone design course.

CSE 732: Computational Linguistics is an exploration of the computational processing of natural language; syntactic, semantic, and pragmatic processing techniques are applied to understanding and generating written and spoken language. This course has been recently redesigned and has been offered twice in its new format.

CSE 733: Foundations of Spoken Language Processing is a new course since the last report that covers automatic speech recognition and text-to-speech technologies. Labs focus on small system-building projects of speech processing technologies.

CSE 735: Machine Learning and Statistical Pattern Recognition is a survey of pattern recognition and machine learning techniques; this course has been resurrected after a long hiatus and has been offered twice since the last report (2005).

CSE 737: Proseminar in Cognitive Science is an in-depth examination of the interdisciplinary field of cognitive science, and it emphasizes fundamental issues of each discipline and provides illustrations of representative research being conducted at OSU. This is the second course in cognitive science, subsequent to CSE 612. The course is cross-listed with ISE, Linguistics, Philosophy, Psychology, and Speech and Hearing Science.

CSE 739: Knowledge-Based Systems in Engineering is an application of knowledge-based system principles to engineering problems, including practical knowledge engineering, techniques for problem assessment, and implementation. This course is cross-listed with Chemical, Mechanical, and Nuclear Engineering Departments.

CSE 779: Introduction to Neural Networks is a survey of fundamental methods and techniques of the field of neural networks including single- and multi-layer perceptrons; radial-basis function networks; support vector machines; recurrent networks; supervised and unsupervised learning; application to pattern classification and function approximation problems. This course is cross-listed with Electrical Engineering.

2.2 Evaluation of courses

Note: The syllabus of each course lists a set of intended learning outcomes (LOs). Each LO specifies an item of knowledge and/or skill at one of three levels of performance, mastery, familiarity, or exposure. Mastery means the student should be able to apply the knowledge or skill even in a new context, and even when not specifically instructed to do so; familiarity means the student will be able to apply it even in a new context, when instructed to do so; and exposure means the student will have heard the term and/or seen it used, but may not be able to discuss or use it effectively without further instruction. We have only evaluated courses for which CSE is the primary instructional unit.

CSE 630: The LOs of this course include:

Master basic search techniques for problem-solving, including systematic blind search, heuristically-guided search, and optimal search.
Master use of logic and proof as a basis for knowledge representation and automated reasoning.
Be familiar with agent-based approaches to problem formulation.
Be familiar with game tree search methods and the requirements for expert-level game play.
Be exposed to problems in common sense reasoning and language understanding.
Be exposed to machine learning techniques and the kinds of problem they solve.

In preparing this report, faculty who have recently taught this course reported that students have been mostly achieving the learning outcomes listed above, as evidenced both by faculty impression and exam scores. The faculty did felt that students were not achieving the "mastery" level for "Master use of logic and proof as a basis for knowledge representation and automated reasoning;" there is a population of students who do not do well on this section of the course. We propose to change the level to "be familar with;" the students often are able to use logic-like formalisms in subsequent activities, such as planning algorithms.

Similarly, most versions of the course do not cover language understanding, so the fifth learning outcome should be "Be exposed to problems in common sense reasoning." We will discuss these changes at the review meeting, but the updates are to reflect how we currently teach the course.

In short, the new proposed LOs are:

Master basic search techniques for problem-solving, including systematic blind search, heuristically-guided search, and optimal search.
Be familiar with logic and proof as a basis for knowledge representation and automated reasoning.
Be familiar with agent-based approaches to problem formulation.
Be familiar with game tree search methods and the requirements for expert-level game play.
Be exposed to problems in common sense reasoning.
Be exposed to machine learning techniques and the kinds of problem they solve.

Since CSE 630 is a popular elective, one CSE 630 related question has been added to the POCAT; the POCAT evaluation suggests that this question is being answered correctly at an adequate rate for those students who have taken 630. We may design an additional question addressing logic for the POCAT in conjunction with CSE 670 faculty.

CSE 634: The LOs for this course include

Master fundamental computer vision algorithms.
Be familiar with Matlab programming environment.
Be exposed to original research and applications in computer vision.

The learning outcomes are all currently appropriate and are achieved by most students. These outcomes are serviced by covering essential algorithms in which the students have weekly programming assignments related to implementing and testing the algorithms; all programming assignments (weekly) are required to be in Matlab; and recent applications and ideas are discussed in class. The final project is used to expose the students to their own original ideas (the students have to come up with a project idea - a list of projects is not given) and a means to implement their idea in an application.

CSE 730: The LOs for this course include:

Master advanced AI concepts, theories, and terminology.
Master computational techniques in typical AI subareas.
Master knowledge representation and reasoning methods in AI.
Be exposed to current research topics in AI.

All of these LOs are currently correct and being achieved by most students. Students must demonstrate mastery of several different AI techniques, including Bayesian network modeling, probability theory, Expectation-Maximization algorithm and machine learning algorithms, demonstrating these techniques via homeworks, exams, and a final team project. The final LO is covered by a series of paired lectures -- introductory material by the professor and a graduate student guest lecturer -- covering the AI research areas in the department.

CSE 731: The LOs for this course are:

Master task-level analysis and problem solving methods for classification problems.
Master task-level analysis and problem solving methods for configuration (design) problems.
Be familiar with the analysis and methods of diagnosis problems.
Be familiar with methods for representing and reasoning with uncertain knowledge.
Be familiar with task-level architectures.
Be familiar with at least one modern programming tool for knowledge system development.

As part of the assessment of capstone courses before the most recent ABET evaluation, we redesigned the course to add the degree-wide capstone requirements of teamworking (which was already present) and assessment of communication skills. However, we did not update the LOs. Given the similarity of the first two LOs, we propose the new set of LOs to be:

LO1: Master task-level analysis and problem solving methods for both classification and configuration (design) problems.
LO2: Be familiar with the analysis and methods of diagnosis problems.
LO3: Be familiar with methods for representing and reasoning with uncertain knowledge.
LO4: Be familiar with task-level architectures.
LO5: Be familiar with at least one modern programming tool for knowledge system development.
LO6: Be familiar with issues of teamwork, project scheduling, individual and group time management.
LO7: Be familiar with techniques for effective oral and written communication.

Because we only have access to the current offering of CSE 731, we did an analysis of each learning outcome with respect to the current class.

LO1: In lecture we covered various knowledge representation structures such as semantic nets, frames, and property lists for not only encoding knowledge but to design the structure of the knowledge and perform the task of classification. All of the groups are providing solutions to the classification problem directly through hierarchical designs. Input data is abstracted first and matched with candidate solutions that are refined to one or more final outcomes. This is being implemented specifically in the CLIPS programming language in all groups. The result is a knowledge base of rules that cover a problem domain where data is extracted from expert in the area.

LO2: We have studied in lecture particular KBS and their diagnostic abilities including MYCIN, PROSPECTOR, and MOLE. All the projects will have this component in their solutions. Some groups will try to verify their systems with their domain expert(s) in order to gain feedback as to how the diagnosis problem is being solved in their implementation.

LO3: We covered material particularly about certainty factors, the mathematics behind them, and how to use CLIPS to implement them. In lecture, I presented examples of how these are used in particular KBS as examples.

LO4: As you've probably been able to gather above, we have covered particular examples of KBS, most notably MYCIN. We have covered the design of KBS as a whole.

LO5: We covered the CLIPS language in detail during lecture. Every project is required to use CLIPS to implement their solution. Some groups are using JAVA or a particular GUI package that interfaces with CLIPS for input and output to the system.

LO6: Each group is required to document all progress achieved throughout the project. The group must log all group meetings including time, attendance at the meeting, goals for the meeting, what was accomplished, and future work before the next meeting on webpage to act as a "web journal". The web pages contain a description of the project, details of group meetings, progress events, milestones, and even prototypes of the system. I have helped groups to formulate an appropriate scope for their project and prioritize their goals and tasks. Groups understand the timeline on which they are working with. Every group must report on progress every week (see below). I can see from the group progress reports that groups are adequately formulating schedules and partitioning work between group members. I am helping a couple of groups that are experiencing difficulty due to group members that are not participating enough or have dropped out. All groups consist of 4 students except for a group with 5 students and one with 3 students. The group with 3 students are struggling to work together as a team to be able to provide something for the final project and final report.

LO7: Each group is required to present the results of their project in a final project presentation, which lasts for ~40 - 45 minutes. All groups members must participate in the final projet presentation. The group is required to submit a final report of at least 10 pages. This is a formal report that is targeted to a general audience. As mentioned above, all groups are required to maintain a web page that serves as an up-to-date "web journal" of the groups progress, tasks, accomplishments, milestones, and participation of each member. In addition, all group members are required to give an ~10 - 15 min. individual presentation of the group's progress for that week. Shortly after groups were put together, each class has included these individual presentations. During an individual presentation, the entire class is given an evaluation sheet to fill out during the presentation to comment on the content of the presentation and the effectiveness of the communication of the speaker. These evaluation sheets are given to the presenter as feedback of their performance.

In general, it seems that the current course structure is allowing students to achieve the learning outcomes, so no changes are recommended.

CSE 732: LOs for this course include:

Master some current techniques for language processing.
Master the fundamentals of symbolic and statistical learning in language processing tasks.
Be familiar with building interoperable components for a small language processing agent.
Be familiar with the impact of context on language processing.

The faculty member who has most recently taught a full quarter of CSE 732 has left the university, so we can only offer a partial assessment of LOs in terms of the course that is currently being taught. In the current version, the first two learning outcomes are being achieved by the students, as evidenced by homework assignments which require implementing language processing algorithms for part-of-speech taggers and chart parsers. The third outcome (familiarity with interoperable components) is also being achieved, although not in an agent-based paradigm so we may in future slightly adjust that outcome. The final LO (impact of context on language processing) is being implemented differently by the different instructors (the current instructor is teaching in terms of application context, whereas the previous instructor taught in terms of semantic/pragmatic context), but both types of instruction exercise the need for context-dependent thinking about problems, thus achieving the goal of the LO.

CSE 733: LOs for this course include:

Master fundamental concepts in automatic speech recognition, such as "hidden Markov models", acoustic modeling, language modeling.
Master fundamental concepts in text-to-speech synthesis, such as concatenative synthesis and text analysis.
Be familiar with a finite state framework integrating all of speech processing.
Be familiar with methods of constructing speech recognition and synthesis systems.
Be exposed to current speech processing research.
Be exposed to toolkits for both speech recognition and speech synthesis.

Students are achieving the first two outcomes; as a necessary (but not sufficent) condition of receiving an A in the class, they must demonstrate mastery of ASR and speech synthesis concepts by achieving 90% on an online final exam. Multiple attempts are allowed; students are often motivated to try to achieve 100% even after passing the 90% threshold. The average student required roughly 1 re-take of the exam to reach 90%.

The second two outcomes are demonstrated by the lab assignements; most students do well in constructing speech recognition and synthesis system components using a pedagogically novel finite state model framework. The final two outcomes are achieved by both lectures on research and toolkits, as well as requiring a final project that builds a small working system.

CSE 735: LOs for this course include:

Master basic techniques of machine learning, including linear methods, prototype-based methods, and kernel methods.
Master the statistical framework of machine learning and basic concepts, such as Bayes optimal classifier.
Be familiar with theoretical analysis of complexity and other properties of statistical learning techniques.
Be exposed to the spectrum of methods for classification, regression and clustering, including boosting, spectral clustering, and other methods.

This course was recently revamped to be updated with a new set of LOs. The current assessment of the instructor is that all LOs are being achieved by most students. The assessment is based on a set of homeworks designed to test the mastery of the material and a final project, which gives students and opportunity to apply skills learned in class to a real-world problem. achieved by most students.

CSE 779: LOs for this course include:

Master basic neural network methods
Be familiar with solving problems using neural network techniques
Be familiar with enough background about neural networks to take other specialty courses on neural networks

Students in recent offerings of 779 have achieved the current set of objectives, which is evidenced by

Students achieve reasonable grades. In the last several offerings, the students achieve the grade of B on average.
Part of the class requirements are two programming assignments. With effort, a vast majority of students have done the lab assignments well.
Anecdotal evidence suggests that what the students have learned in the class enable them to use neural network techniques to solve problems in their own research. For example, one student designed a new classifier based on RBF nets covered in the class, and another one used SVM for recognizing spoken digits in a research project.

2.3 Relation to BS-CSE program outcomes (POs) and evaluation

As noted earlier, the BS-CSE POs are currently being revised. The (proposed) new POs are:

an ability to apply knowledge of computing, mathematics including discrete mathematics as well as probability and statistics, science, and engineering;
an ability to design and conduct experiments, as well as to analyze and interpret data;
an ability to design, implement, and evaluate a software or a software/hardware system, component, or process to meet desired needs within realistic constraints such as memory, runtime efficiency, as well as appropriate constraints related to economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability considerations;
an ability to function effectively on multi-disciplinary teams;
an ability to identify, formulate, and solve engineering problems;
an understanding of professional, ethical, legal, security and social issues and responsibilities;
an ability to communicate effectively with a range of audiences;
an ability to analyze the local and global impact of computing on individuals, organizations, and society;
a recognition of the need for, and an ability to engage in life-long learning and continuing professional development;
a knowledge of contemporary issues;
an ability to use the techniques, skills, and modern tools necessary for practice as a CSE professional.
an ability to analyze a problem, and identify and define the computing requirements appropriate to its solution;
an ability to apply mathematical foundations, algorithmic principles, and computer science theory in the modeling and design of computer-based systems in a way that demonstrates comprehension of the tradeoffs involved in design choices;
an ability to apply design and development principles in the construction of software systems of varying complexity.

Although every course, including technical electives such as CSE 630 or 730, that a student takes contributes to his or her achieving one or more of the POs listed above, in order to focus on the program as a whole, rather than consider every possible combination of electives an individual student could take, following the pattern of other CGRs we consider only the contributions that the required courses make toward achieving the POs. No course in the AI Course group is required; however, since CSE 731 is an elective capstone course we do provide an assessment of the relationship of the LOs to the POs.

Below, for CSE 731, we first list the new proposed LOs for the course (copied from Section 2.2). Next, a table summarizes the contribution of the course to the program outcomes (POs). The table is organized as follows. There are as many rows in the table as the number of LOs for the course. The first column of each row specifies the number of the corresponding LO. The remaining columns specify the contribution of the particular LO to the fourteen program outcomes, (a) through (n), respectively. In specifying this contribution, we use "XXX" to signify strong contribution; "XX" to signify moderate contribution; "X" to signify minimal contribution; if a given LO makes no contribution to a particular outcome, the cell is empty. Following the table is a brief summary, based on the discussion in 2.2, of how well the recent offerings of the course achieved each LO as indicated by both the course content and student performance in the course; and results from POCAT, if any, related to the various LOs of the course. A brief summary of the proposed changes in the course (content or activities or LOs) should conclude the summary for this course.

CSE 731: The learning outcomes of this course are:

LO1: Master task-level analysis and problem solving methods for classification and configuration (design) problems.
LO2: Be familiar with the analysis and methods of diagnosis problems.
LO3: Be familiar with methods for representing and reasoning with uncertain knowledge.
LO4: Be familiar with task-level architectures.
LO5: Be familiar with at least one modern programming tool for knowledge system development.
LO6: Be familiar with issues of teamwork, project scheduling, individual and group time management.
LO7: Be familiar with techniques for effective oral and written communication.

Learning
Outcome

Relation to BS-CSE Program Outcomes

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

(j)

(k)

(l)

(m)

(n)

LO1

XXX

LO2

XXX

LO3

XXX

LO4

XXX

LO5

XXX

LO6

XXX

LO7

XXX

Summary: By and large, 731 does a good job as a capstone course bringing together the goals of the program. The only mismatch between LOs and POs here is (f) an understanding of professional, ethical, legal, security and social issues and responsibilities, and (i) a recognition of the need for, and an ability to engage in life-long learning and continuing professional development. This is in part because of the switch to a new instructor this year, who redeveloped the course from scratch. (f) is covered to some degree by the prerequisite CSE 601, and previous offerings of the course had a section on how to responsibly collect expert data. We should also consider adding a requirement to report on expert system software tools, similar to reports in other capstones, to strengthen outcome (i). These will be recommended changes for the next offering of the course. We will also work with undergraduate studies to update the LOs so that they better match the POs in categories (f) and (i).

2.4 Major changes since the previous report

As part of the ABET evaluation, CSE 731 was reviewed as a capstone course in the Undergrad Studies committee. The major change to this course was to emphasize teamwork and communication skills as part of the course; we are also proposing to change the learning outcomes to reflect that change.

CSE 733 was made into a permanent course and is now offered every other year; it alternates with CSE 732, which was switched to an alternating year format after enrollment decreases.

CSE 735 was revived with the addition of a faculty member in that area and had its name changed from "Methods in Pattern Recognition" to "Machine Learning and Statistical Pattern Recognition". We suspect that the last previous offering was in 1980. The course was redesigned to follow a modern textbook and is currently offered every other year, although if demand allows we would like to offer it every year.

CSE 776, Hardware/Software Interface Design Project, was removed from the registrar's database during this timeperiod and will no longer be offered.

2.5 Planned changes

Basically, the AI curriculum has reached a relatively stable point from the viewpoint of classes. We do note a few upcoming issues:

While not a "change" per se, it has been noted that the demand for CSE 630 continues to rise (closing out two sections in SP09); the department will offer 5-6 sections in 2009-2010. While this is an excellent validation of that course, it does cause short-term problems as there is temporarily not enough AI faculty teaching time to cover the increased load. We will continue to work with the Associate Chair to find creative solutions for this problem.

CSE 731 continues to be taught by adjunct faculty or lecturers, because of the availabilty of excellent lecturers on the topic, the aforementioned lack of faculty time, and the fact that none of our faculty are currently expert in that area. The latter fact suggests that we will probably want to reconsider the topic of the AI capstone course in the future, possibly when we move to semesters. Until then, we plan to offer this course in a similar vein to the current offering, with the additions to promote ethics and life-long learning outlined in Section 2.3. Learning outcomes will be updated to reflect ethical/societal issues and life-long learning in conjunction with direction from the Undergrad Studies committee.

Clearly, the largest change forthcoming will be the conversion to semesters; as with most groups at this time, the AI group has not yet begun discussions about what needs to be done for that area.

3. Conclusions

AI is a popular application area for the CSE program and serves as a testbed for the core programming concepts and techniques taught in the spine of the curriculum. AI is also an intrinsically interdisciplinary area. The AI courses, as they currently stand, are doing well; students are generally satisfied with the courses. We are placing students in with top-notch employers and graduate programs in the field and there is strong faculty commitment to the courses.

Course.	Coordinator	Recent Instructors
CSE 612	Petrov	Jaymiung, Petrov
CSE 630	Davis	Byron, Brew, Davis, Fosler-Lussier, Shareef
CSE 634	Davis	Davis
CSE 730	Fosler-Lussier	Fosler-Lussier
CSE 731	Fosler-Lussier	Mikkilineni, Shareef
CSE 732	Brew	Byron, Brew
CSE 733	Fosler-Lussier	Fosler-Lussier
CSE 735	Belkin	Belkin
CSE 737	Wang	Todd
CSE 739	Adeli	Adeli
CSE 779	Wang	Krishnamurthy, Wang

People involved in preparing report: Eric Fosler-Lussier, Misha Belkin, Chris Brew, Jim Davis, Naeem Shareef, Leon Wang.

Date of report: May 2009

Eric Fosler-Lussier
(Draft: 18 May 2009; Final: 19 May 2009).