In many situations, the most convenient way for humans to interact with intelligent systems is via language. To arrive at natural models of interaction, computer systems need to be equipped with the means to efficiently process language and extract meaning from it. However, computing the meaning of utterances is a complex task, which involves various linguistic levels, e.g. the lexical meaning of individual words, the semantic argument structure of clauses and sentences (who did what to whom?), the discourse context (how does a sentence relate to its neighbouring sentences?), and finally the situational context (who is speaking/listening? what does the speaker want to achieve through making the utterance?). Context plays a crucial role here: sentential context influences the meaning of words, discourse context influences the meaning of sentences, and situational context influences the meaning of a discourse. In our research group, we aim to develop intelligent models of language meaning which take context information into account. We envisage that such context-aware models will perform better than approaches which deal with different linguistic phenomena in isolation. More specifically, we work on combining lexical semantics and discourse processing, for instance as applied to word sense disambiguation and semantic parsing.

A second guiding principle of our work is a focus on unsupervised or semi-supervised models, i.e., models which require no — or only a small set of — manually-labelled training data. Manual data annotation is extremely time-consuming and thus costly, particularly for semantic and discourse phenomena. Consequently, there is a severe shortage of annotated data for tasks dealing with (deep) language meaning. Moreover, if annotated data are available, they are available only for some domains and a small minority of languages. Models trained on these data are typically highly specialised, and their performance drops if they are applied to other data. In our group, we focus on developing technology that uses and combines information from various (contextual) sources in order to reach acceptable performance levels even without large amounts of manually-labelled data. We also experiment with semi-automatic annotation schemes, e.g., using automatic pre-annotation or active learning.

The research group Computational Modelling of Discourse and Semantics was established in September 2008. The group's research focus is on the development of statistical and machine learning models for computing the meaning of language. In addition to the group leader, Caroline Sporleder, the group currently has two other members: Linlin Li joined in December 2008 as a PhD student, and Alexis Palmer started in May 2009 as a postdoc. Also, from October 2009 to March 2010, Cécile Grivaz joined the group as a visiting PhD student from Geneva University, Switzerland.

Cohesion Graph of a "Barbecue-Situation"

Distinguishing Literal and Non-Literal Word Senses

Idioms and other non-literal expressions are frequent in language, and detecting and processing them is an important task in natural language processing. So far, most work in this area has focused on context-independent classification of expressions as "idiomatic" or "literal". However, many expressions, such as "break the ice", can be used in either a literal or a non-literal sense and must be disambiguated in context. In our group, we created a novel framework for idiom detection that can disambiguate between the two different senses in a given context. We exploit the fact that literal usages typically exhibit cohesive links with the context, while non-literal ones do not. To compute the cohesive structure of a text, we developed a graph-based model. By combining this model with a more sophisticated second-stage classifier which takes into account a wider range of linguistic cues, it is for the first time possible to reach performance levels rivaling those of supervised systems without requiring manually labelled training data. Moreover, our models are also able to deal with truly creative language, i.e., new metaphors or creative variations of idioms which are beyond the reach of other systems. In our current work, we address more general word sense disambiguation tasks by developing robust statistical methods based on topic models.

Deep Semantic Parsing

A second research area involves the computation of semantic argument structures, i.e., who did what to whom. Being able to compute semantic argument structures (semantic parsing) is important for a number of language processing tasks, such as information extraction, recognising entailment, or machine translation. While automatic methods for deep semantic parsing exist, these currently do not achieve acceptable performance levels. One problem is the limited amount of annotated data available, combined with the fact that deep semantic parsing often requires world knowledge and inferencing. To address the shortage of annotated data, we developed one of the earliest methods for semi-supervised semantic parsing. The method exploits similarities between potential arguments in a semantic vector space. In ongoing work, we study what role coverage gaps play when analysing complete texts rather than individual sentences, and we are developing models that exploit redundancies in a discourse context to overcome coverage problems in full-text analyses.

Robust Text Mining for Cultural Heritage Data

On the applied side, we are working on robust text mining technology for domains that are not typical target domains of natural language processing, in particular in the area of cultural heritage. Cultural heritage institutes possess large amounts of textual data, such as digitised manuscripts, field books, or databases describing the objects in a collection. Most data sources are un- or semi-structured and contain relatively large amounts of noise, e.g., due to OCR errors introduced in the digitisation process. Due to domain and genre differences, off-the-shelf language technology, which is typically modelled on news wire texts, does not perform very well on cultural heritage data. Moreover, few or no manually-annotated data sets are available for this domain. Consequently, text mining technology for cultural heritage data has to make use of semi- or unsupervised techniques and be very robust in the presence of noise. We developed a number of unsupervised methods for cleaning, enriching and linking cultural heritage documents. Our methods are based on the idea of incorporating as much existing information as possible. For example, cultural heritage databases often provide insights about how a domain is structured, and this information can be exploited to automatically detect errors  and extract information from free text data (e.g., field books).

Collaborations within the Cluster of Excellence

• With Michael Backes' group, we have collaborated on a study on acoustic side-channel attacks on printers, which resulted in a publication at an international conference (19th USENIX Security Symposium, 2010).
• With Jan Baumbach, we are working on a joint project proposal on biomedical text mining. In particular, we will develop knowledge-enriched models in which the text mining process is continuously informed by domain knowledge and new empirical findings.
• With Ivan Titov, we work on sophisticated semantic parsing models, which incorporate both linguistic insights and state-of-the-art statistical technique. We plan a jointly supervised PhD project and are currently working on a concrete research plan.
• With Matthew Crocker's group, we work on developing psycholinguistically plausible models for processing figurative language (joint work with Berry Claus, now at HU Berlin, and Pirita Pyykkönen).
• With Michael Kipp, we are investigating the relationship between gesture and discourse, initially in a jointly supervised master's thesis project.

External Collaborations

• Together with Roser Morante (University of Antwerp, Belgium), Collin Baker (ICSI, Berkeley, USA) and Martha Palmer (University of Colorado at Boulder, USA) we investigate discourse effects on local semantic argument structures. In particular, we jointly organised a shared task for SemEval 2010 on the detection of referents for null instantiations.
• We have a long-standing collaboration with Antal van den Bosch's group at Tilburg University (The Netherlands), with whom we continue to develop text mining technology for cultural heritage data.
• With Cécile Grivaz (University of Geneva, Switzerland), we are collaborating on discourse parsing, particularly on the joint identification of temporal and causal relations in texts.