CSE logo College of Engineering Slate Lab Logo CSE Website



  • Articulatory ASR
  • Clinical Narrative NLP
  • NLP for social media

Articulatory feature-based models for ASR

(joint work with Karen Livescu, TTI-C)

Conversational speech is characterized by large amounts of pronunciation variability and disfluencies that continue to present challenges for speech recognition systems. To specifically address this kind of variability in conversational speech, we have been working on articulatory feature-based models as an alternative framework to the HMM-based approaches that are widely used in ASR. The main idea behind the articulatory feature-based models is to account for the observed variation by hypothesizing it to be the result of asynchrony between the various articulators. Within this domain we have been exploring the use of both directed (dynamic Bayesian networks) as well as undirected graphical models (conditional random fields).

We are also modeling pronunciation variability using context dependent articulatory feature-based model. The model is an extension of previous work based on dynamic Bayesian networks (DBNs) that allow for easy factorization of a state into multiple variables representing the articulatory features. We use the more ubiquitous finite state machines (FSMs) to represent these articulatory feature-based models in order to integrate them easily with a conventional recognizer. The key conceptual challenge is to design models that are computationally viable while incorporating the additional constraints involving contextual dependencies.

Students: Rohit Prabhavalkar, Preethi Jyothi

Understanding Clinical Narratives

(joint with Dr. Albert Lai, OSU BMI)

Our work aims to build an information extraction and fusion pipeline leveraging semantic and temporal information found in clinical narratives such as discharge summaries, progress notes and radiology reports. A patient could be involved in multiple clinical narratives, which may contain overlapping, semantically related or inconsistently described events. Clinical narratives also exhibit a unique medical sub-language with characteristics such as semantic categorization of words, co-occurrence of patterns and constraints, domain specific terminology, incomplete phrases and omission of information. We consider a machine learning approach to temporally ordering medical events and integrating structured and unstructured clinical information. Our corpus consists of Chronic Lymphocytic Leukemia patient narratives collected over the last 10 years at The Ohio State University Medical Center. We start by learning pair-wise ordering between all pairs of events in a narrative. Following this, we infer a global ordering by modeling the problem as a temporal constraint graph, thus generating a longitudinal health record. The ability to use a semi-automated approach to enrolling patients in clinical studies would be helpful in accelerating clinical research. However, this semi-automated approach depends on the availability of a temporally coherent longitudinal health record of sufficient quality and consistency, which we generate as part of this research.

Student: Preethi Raghavan

NLP for social media

Lexical graphs based on word distributional similarity have been used to induce a sentiment lexicon for classifying sentiment of online reviews in previous work. Although this method works well for general web documents, it is not directly applicable to real-time short messages (e.g. tweets) in social media; lexical graph built on top of these messages is expected to be less reliable due to their highly irregular spellings and nonstandard use of language. In order to learn a social-media-specific sentiment lexicon, we extend the existing method to a large unlabeled Twitter corpus by allowing deletions of (incorrect) word similarity relations on the lexical graph. We are exploring a global optimization method for inferring the polarity of words while correcting edges from the graph.

Student: Yanzhang He