Artificial Intelligence Course Group Report: Transition to Semesters (2011)

1. Background

The field of AI continues to evolve at a rapid pace, fueled by the ancient dream of the mankind to make a machine that behaves like a man, as well as by the research and development taking place in industry at companies such as Microsoft, IBM, AT&T, Google, GE, just to name a few. In addition, AI has become indispensable in a variety of application areas because of advances in computer vision, speech and language processing, inference engines, knowledge-based systems, and robotics. AI has also become a main bridge between computer science and many other disciplines, such as other engineering disciplines, cognitive science, linguistics, psychology, neuroscience, math/physics/statistics, speech and hearing sciences, and medicine.

We have evolved the AI curriculum continuously to keep pace with the development of the field, the interests of students, and the demands of employers. Under quarters, we currently have nine regular courses in AI, many of which are cross-listed with other departments, reflecting the interdisciplinary nature of the field. The 630/730 sequence is the core introduction to AI. The AI faculty also offer research seminars that cover recent developments in computer vision, neural networks, speech processing, knowledge systems, and cognitive science. Developing new courses on these topics is a priority of the AI faculty.

Under quarters, the AI curriculum was completely elective for undergraduates. In the new semester curricula for all three of our degrees, the Introduction to AI course (now CSE 3521) is an option for the Application part of the CSE Core. The AI curriculum is compatible with many of the CSE and ABET objectives. On the application side, AI is an area in which students can exercise the principles of software design, databases, algorithms, data structures and programming languages which they have learned in the foundation courses. The AI courses rely heavily on the application of various principles from mathematics, physics, statistics, neuroscience, and psychology. Students who study AI in the Department receive a good grounding in the area, are usually ready for graduate studies in AI and cognitive science, and are very employable. AI also draws from an interdisciplinary background, and several of our courses offer opportunities for working with interdisciplinary teams.

2. Current set of courses

This section repeats our relatively recent assessment of the AI courses (May 2009); the AI faculty met as a group to discuss the disposition of each course under quarters.

Section 2.1 describes the individual courses in the group, including their relations to the other courses in the group as well as to the rest of the curriculum. Section 2.2 contains the 2009 evaluation of each course to see how well it meets its intended learning outcomes (LOs). Section 2.3 considers the relation between the LOs of the various courses and the (proposed new) BS-CSE program outcomes (listed in Section 2.3). Section 2.4 summarizes the main changes we have made in the courses since the 2005 report. Section 2.5 discusses some continuing concerns and our plans to address them.

2.1 Summary of the courses

CSE 630: Survey of Artificial Intelligence I: Basic Techniques  is a survey of the basic concepts and techniques of problem solving paradigms and knowledge representation schemes in Artificial Intelligence. This is the entry-level AI course, and is a prerequisite for many subsequent AI courses.

CSE 634: Computer Vision for Human Computer Interaction  is a course in computer vision algorithms for use in human-computer interactive systems. Topics include image formation, image features, segmentation, shape analysis, object tracking, motion calculation, and applications.

CSE 730: Survey of Artificial Intelligence II: Advanced Topics is a survey of advanced concepts, techniques, and applications of artificial intelligence, including perception, learning, reasoning under uncertainty, natural language processing, and speech recognition. This course continues the introduction of artificial intelligence in CSE 630 for undergraduates. It is often taken as a first course in AI for entering graduate students.

CSE 731: Knowledge-based Systems is a survey of theory and practice of expert systems and knowledge-based systems, and makes use of current knowledge-based systems software tools. This is a BSCSE capstone design course.

CSE 732: Computational Linguistics is an exploration of the computational processing of natural language; syntactic, semantic, and pragmatic processing techniques are applied to understanding and generating written and spoken language. This course has been recently redesigned and has been offered twice in its new format.

CSE 733: Foundations of Spoken Language Processing is a new course since the last report that covers automatic speech recognition and text-to-speech technologies. Labs focus on small system-building projects of speech processing technologies.

CSE 735: Machine Learning and Statistical Pattern Recognition is a survey of pattern recognition and machine learning techniques; this course has been resurrected after a long hiatus and has been offered twice since the last report (2005).

CSE 737: Proseminar in Cognitive Science is an in-depth examination of the interdisciplinary field of cognitive science, and it emphasizes fundamental issues of each discipline and provides illustrations of representative research being conducted at OSU. This is the second course in cognitive science, subsequent to CSE 612. The course is cross-listed with ISE, Linguistics, Philosophy, Psychology, and Speech and Hearing Science.

CSE 739: Knowledge-Based Systems in Engineering is an application of knowledge-based system principles to engineering problems, including practical knowledge engineering, techniques for problem assessment, and implementation. This course is cross-listed with Chemical, Mechanical, and Nuclear Engineering Departments.

CSE 779: Introduction to Neural Networks is a survey of fundamental methods and techniques of the field of neural networks including single- and multi-layer perceptrons; radial-basis function networks; support vector machines; recurrent networks; supervised and unsupervised learning; application to pattern classification and function approximation problems. This course is cross-listed with Electrical Engineering.

2.2 Evaluation of courses (from AI CGR 2009)

CSE 630: The LOs of this course include:

• Master basic search techniques for problem-solving, including systematic blind search, heuristically-guided search, and optimal search.
• Master use of logic and proof as a basis for knowledge representation and automated reasoning.
• Be familiar with agent-based approaches to problem formulation.
• Be familiar with game tree search methods and the requirements for expert-level game play.
• Be exposed to problems in common sense reasoning and language understanding.
• Be exposed to machine learning techniques and the kinds of problem they solve.

Similarly, most versions of the course do not cover language understanding, so the fifth learning outcome should be "Be exposed to problems in common sense reasoning." We will discuss these changes at the review meeting, but the updates are to reflect how we currently teach the course.

In short, the new proposed LOs are:

• Master basic search techniques for problem-solving, including systematic blind search, heuristically-guided search, and optimal search.
• Be familiar with logic and proof as a basis for knowledge representation and automated reasoning.
• Be familiar with agent-based approaches to problem formulation.
• Be familiar with game tree search methods and the requirements for expert-level game play.
• Be exposed to problems in common sense reasoning.
• Be exposed to machine learning techniques and the kinds of problem they solve.

Since CSE 630 is a popular elective, one CSE 630 related question has been added to the POCAT; the POCAT evaluation suggests that this question is being answered correctly at an adequate rate for those students who have taken 630. We may design an additional question addressing logic for the POCAT in conjunction with CSE 670 faculty.

CSE 634: The LOs for this course include

• Master fundamental computer vision algorithms.
• Be familiar with Matlab programming environment.
• Be exposed to original research and applications in computer vision.

CSE 730: The LOs for this course include:

• Master advanced AI concepts, theories, and terminology.
• Master computational techniques in typical AI subareas.
• Master knowledge representation and reasoning methods in AI.
• Be exposed to current research topics in AI.

CSE 731: The LOs for this course are:

• Master task-level analysis and problem solving methods for classification problems.
• Master task-level analysis and problem solving methods for configuration (design) problems.
• Be familiar with the analysis and methods of diagnosis problems.
• Be familiar with methods for representing and reasoning with uncertain knowledge.
• Be familiar with task-level architectures.
• Be familiar with at least one modern programming tool for knowledge system development.

• LO1: Master task-level analysis and problem solving methods for both classification and configuration (design) problems.
• LO2: Be familiar with the analysis and methods of diagnosis problems.
• LO3: Be familiar with methods for representing and reasoning with uncertain knowledge.
• LO4: Be familiar with task-level architectures.
• LO5: Be familiar with at least one modern programming tool for knowledge system development.
• LO6: Be familiar with issues of teamwork, project scheduling, individual and group time management.
• LO7: Be familiar with techniques for effective oral and written communication.

LO1: In lecture we covered various knowledge representation structures such as semantic nets, frames, and property lists for not only encoding knowledge but to design the structure of the knowledge and perform the task of classification. All of the groups are providing solutions to the classification problem directly through hierarchical designs. Input data is abstracted first and matched with candidate solutions that are refined to one or more final outcomes. This is being implemented specifically in the CLIPS programming language in all groups. The result is a knowledge base of rules that cover a problem domain where data is extracted from expert in the area.

LO2: We have studied in lecture particular KBS and their diagnostic abilities including MYCIN, PROSPECTOR, and MOLE. All the projects will have this component in their solutions. Some groups will try to verify their systems with their domain expert(s) in order to gain feedback as to how the diagnosis problem is being solved in their implementation.

LO3: We covered material particularly about certainty factors, the mathematics behind them, and how to use CLIPS to implement them. In lecture, I presented examples of how these are used in particular KBS as examples.

LO4: As you've probably been able to gather above, we have covered particular examples of KBS, most notably MYCIN. We have covered the design of KBS as a whole.

LO5: We covered the CLIPS language in detail during lecture. Every project is required to use CLIPS to implement their solution. Some groups are using JAVA or a particular GUI package that interfaces with CLIPS for input and output to the system.

LO6: Each group is required to document all progress achieved throughout the project. The group must log all group meetings including time, attendance at the meeting, goals for the meeting, what was accomplished, and future work before the next meeting on webpage to act as a "web journal". The web pages contain a description of the project, details of group meetings, progress events, milestones, and even prototypes of the system. I have helped groups to formulate an appropriate scope for their project and prioritize their goals and tasks. Groups understand the timeline on which they are working with. Every group must report on progress every week (see below). I can see from the group progress reports that groups are adequately formulating schedules and partitioning work between group members. I am helping a couple of groups that are experiencing difficulty due to group members that are not participating enough or have dropped out. All groups consist of 4 students except for a group with 5 students and one with 3 students. The group with 3 students are struggling to work together as a team to be able to provide something for the final project and final report.

LO7: Each group is required to present the results of their project in a final project presentation, which lasts for ~40 - 45 minutes. All groups members must participate in the final project presentation. The group is required to submit a final report of at least 10 pages. This is a formal report that is targeted to a general audience. As mentioned above, all groups are required to maintain a web page that serves as an up-to-date "web journal" of the groups progress, tasks, accomplishments, milestones, and participation of each member. In addition, all group members are required to give an ~10 - 15 min. individual presentation of the group's progress for that week. Shortly after groups were put together, each class has included these individual presentations. During an individual presentation, the entire class is given an evaluation sheet to fill out during the presentation to comment on the content of the presentation and the effectiveness of the communication of the speaker. These evaluation sheets are given to the presenter as feedback of their performance.

In general, it seems that the current course structure is allowing students to achieve the learning outcomes, so no changes are recommended.

CSE 732: LOs for this course include:

• Master some current techniques for language processing.
• Master the fundamentals of symbolic and statistical learning in language processing tasks.
• Be familiar with building interoperable components for a small language processing agent.
• Be familiar with the impact of context on language processing.

CSE 733: LOs for this course include:

• Master fundamental concepts in automatic speech recognition, such as "hidden Markov models", acoustic modeling, language modeling.
• Master fundamental concepts in text-to-speech synthesis, such as concatenative synthesis and text analysis.
• Be familiar with a finite state framework integrating all of speech processing.
• Be familiar with methods of constructing speech recognition and synthesis systems.
• Be exposed to current speech processing research.
• Be exposed to toolkits for both speech recognition and speech synthesis.

The second two outcomes are demonstrated by the lab assignments; most students do well in constructing speech recognition and synthesis system components using a pedagogically novel finite state model framework. The final two outcomes are achieved by both lectures on research and toolkits, as well as requiring a final project that builds a small working system.

CSE 735: LOs for this course include:

• Master basic techniques of machine learning, including linear methods, prototype-based methods, and kernel methods.
• Master the statistical framework of machine learning and basic concepts, such as Bayes optimal classifier.
• Be familiar with theoretical analysis of complexity and other properties of statistical learning techniques.
• Be exposed to the spectrum of methods for classification, regression and clustering, including boosting, spectral clustering, and other methods.

CSE 779: LOs for this course include:

• Master basic neural network methods
• Be familiar with solving problems using neural network techniques
• Be familiar with enough background about neural networks to take other specialty courses on neural networks

• Students achieve reasonable grades. In the last several offerings, the students achieve the grade of B on average.
• Part of the class requirements are two programming assignments. With effort, a vast majority of students have done the lab assignments well.
• Anecdotal evidence suggests that what the students have learned in the class enable them to use neural network techniques to solve problems in their own research. For example, one student designed a new classifier based on RBF nets covered in the class, and another one used SVM for recognizing spoken digits in a research project.

2.3 Relation to BS-CSE program outcomes (POs) and evaluation (AI CGR 2009)

1. an ability to apply knowledge of computing, mathematics including discrete mathematics as well as probability and statistics, science, and engineering;
2. an ability to design and conduct experiments, as well as to analyze and interpret data;
3. an ability to design, implement, and evaluate a software or a software/hardware system, component, or process to meet desired needs within realistic constraints such as memory, runtime efficiency, as well as appropriate constraints related to economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability considerations;
4. an ability to function effectively on multi-disciplinary teams;
5. an ability to identify, formulate, and solve engineering problems;
6. an understanding of professional, ethical, legal, security and social issues and responsibilities;
7. an ability to communicate effectively with a range of audiences;
8. an ability to analyze the local and global impact of computing on individuals, organizations, and society;
9. a recognition of the need for, and an ability to engage in life-long learning and continuing professional development;
10. a knowledge of contemporary issues;
11. an ability to use the techniques, skills, and modern tools necessary for practice as a CSE professional.
12. an ability to analyze a problem, and identify and define the computing requirements appropriate to its solution;
13. an ability to apply mathematical foundations, algorithmic principles, and computer science theory in the modeling and design of computer-based systems in a way that demonstrates comprehension of the tradeoffs involved in design choices;
14. an ability to apply design and development principles in the construction of software systems of varying complexity.

Below, for CSE 731, we first list the new proposed LOs for the course (copied from Section 2.2). Next, a table summarizes the contribution of the course to the program outcomes (POs). The table is organized as follows. There are as many rows in the table as the number of LOs for the course. The first column of each row specifies the number of the corresponding LO. The remaining columns specify the contribution of the particular LO to the fourteen program outcomes, (a) through (n), respectively. In specifying this contribution, we use "XXX" to signify strong contribution; "XX" to signify moderate contribution; "X" to signify minimal contribution; if a given LO makes no contribution to a particular outcome, the cell is empty. Following the table is a brief summary, based on the discussion in 2.2, of how well the recent offerings of the course achieved each LO as indicated by both the course content and student performance in the course; and results from POCAT, if any, related to the various LOs of the course. A brief summary of the proposed changes in the course (content or activities or LOs) should conclude the summary for this course.

CSE 731: The learning outcomes of this course are:

• LO1: Master task-level analysis and problem solving methods for classification and configuration (design) problems.
• LO2: Be familiar with the analysis and methods of diagnosis problems.
• LO3: Be familiar with methods for representing and reasoning with uncertain knowledge.
• LO4: Be familiar with task-level architectures.
• LO5: Be familiar with at least one modern programming tool for knowledge system development.
• LO6: Be familiar with issues of teamwork, project scheduling, individual and group time management.
• LO7: Be familiar with techniques for effective oral and written communication.

Summary: By and large, 731 does a good job as a capstone course bringing together the goals of the program. The only mismatch between LOs and POs here is (f) an understanding of professional, ethical, legal, security and social issues and responsibilities, and (i) a recognition of the need for, and an ability to engage in life-long learning and continuing professional development. This is in part because of the switch to a new instructor this year, who redeveloped the course from scratch. (f) is covered to some degree by the prerequisite CSE 601, and previous offerings of the course had a section on how to responsibly collect expert data. We should also consider adding a requirement to report on expert system software tools, similar to reports in other capstones, to strengthen outcome (i). These will be recommended changes for the next offering of the course. We will also work with undergraduate studies to update the LOs so that they better match the POs in categories (f) and (i).

3. Major changes since previous report, plans for semester transition

CSE 630 will be replaced with two courses; CSE 3521/5521 and CSE 4521; the former course will be part of the undergrad core as an option in the Applications area. The course will have stronger CSE prerequisites (CSE 2331). The quarter version (CSE 630) has also been an important component of curricula outside the department (e.g. Undergraduate/Graduate Cognitive Science minor). Some non-CS students have found the prerequisites daunting; those that enroll sometimes find it difficult to keep up with the required CS background (as evidenced both by non-major and major comments to instructors, both direct and in course evaluations). The new non-major course (4521) will allow us to serve these populations while focusing the content for 3521 on majors. In the expansion to semesters, we will be adding more information on Machine Learning, which is popular with students and has many useful current real-life examples to draw from.

CSE 634 will be converted to CSE 5524.

CSE 730 will be converted to CSE 5521; the additional material will expand coverage of statistical inference, machine learning, and research in the department.

CSE 731 will be converted to CSE 5914; additional time for project development will be allotted under semesters. With the refinement of achievement levels in LOs, and after assessing current and desired LO levels through class assessments, a revised set of learning outcomes were set:

• Master task-level analysis and problem solving methods for both configuration (design) and classification problems.
• Be competent with methods for representing and reasoning with uncertain knowledge.
• Be familiar with the analysis and methods of diagnosis problems.
• Master synthesizing and applying prior knowledge to designing and implementing solutions to open-ended computational problems while considering multiple realistic constraints.
• Be competent in evaluating design alternatives.
• Be competent with software design and development practices and standards.
• Be familiar with researching and evaluating computing tools and practices for solving given problems.
• Be competent with deadline driven projects in a team setting.
• Be competent with issues of project management, such as teamwork, project scheduling, individual and group time management.
• Be competent with presenting work to a group of peers.
• Be familiar with presenting work to a range of audiences.
• Be competent with techniques for effective written communication for a range of purposes (user guides, design documentation, storyboards etc.)
• Be familiar with analyzing professional issues, including ethical, legal and security issues, related to computing projects.

CSE 732 and 733 will be merged into one course, CSE 5525 (Foundations of Spoken Language Processing). This course will be offered every year rather than the alternating years that 732/733 were offered. Course material will be selected from both parent courses, with overlapping material in techniques (e.g., Hidden Markov Models) shown in multiple guises.

CSE 735 will be converted to CSE 5523.

CSE 739 will be effectively eliminated from the CSE curriculum under quarters. This course was taught regularly as a cross-listed course in other departments, but the AI faculty feel that this type of material might be better taught as part of the new CSE 4521.

CSE 779 will be converted to CSE 5526. Previously this course was co-listed with ECE, but under semesters it will be taught as a CSE-only course.

The two Cognitive Science courses (cross listed in many departments, primarily taught in Psychology) will continue to be cross listed under semesters. CSE 612 is tentatively CSE 5531, and CSE 737 will become CSE 5891.

4. Additional information

In the 2009 AI CGR, we noted that the demand for CSE 630 had risen; this trend has continued into 2011, where 6 sections have been close to full or closed out in the last year. Furthermore, an additional section of CSE 730 needed to be added starting in Winter 2010 because of capacity concerns; CSE 730 now fulfills part of the Applications Core requirement as part of the MS program.

This additional demand has meant that faculty demand is higher than ever: in part because of one of the AI faculty leaving, in 2011-2012 we have been covering 630 with senior lecturers and senior grad students, and one section of CSE 730 has been covered by a senior lecturer. This problem will ameliorate slightly when a new junior hire comes on board in Winter 2012 (with full teaching in 2012-13), but we will need to continue to work with the Associate Chair to find creative solutions for this problem.

We noted in the 2009 report that the CSE 731 capstone course continues to be taught by lecturers rather than tenure-track faculty. We have been closely monitoring this course, and have been pleased with the now regular senior lecturer's development of the course. The course utilizes the timebox model implemented in CSE 786 (Game Design), and the reviews have been positive for the class. Thus, in our semester discussion, we decided to retain this class under semesters with the ability to do deeper projects. We will re-assess the direction of this course regularly (possibly shifting the topic slightly while retaining it as an AI capstone), especially as personnel change. CSE 731 continues to be taught by adjunct faculty or lecturers, because of the availability of excellent lecturers on the topic, the aforementioned lack of faculty time, and the fact that none of our faculty are currently expert in that area. The latter fact suggests that we will probably want to reconsider the topic of the AI capstone course in the future, possibly when we move to semesters. Until then, we plan to offer this course in a similar vein to the current offering, with the additions to promote ethics and life-long learning outlined in Section 2.3. Learning outcomes will be updated to reflect ethical/societal issues and life-long learning in conjunction with direction from the Undergrad Studies committee.

5. Conclusions

People involved in preparing report: Eric Fosler-Lussier, with contributions on current draft from Jim Davis, based on previous report by Eric Fosler-Lussier, Misha Belkin, Chris Brew, Jim Davis, Naeem Shareef, Leon Wang.

Date of report: October 2011.