RA1: Text and Speech Processing

Vision and Research Strategy

The technology-centric environment in which citizens of the Information Society live and work comprises information in numerous different modalities and will eventually offer the user a variety of natural modes of interaction. As outlined in the original proposal, natural language will play a central role in such settings. The explosive growth of text-based documents on the Internet is increasingly accompanied by large volumes of speech data. Equally, language is the most natural and expressive device for human communication: It allows people to encode and decode complex content with minimal effort and at an appropriate level of specificity. In language production and understanding, people make extensive, efficient and adaptive use of contextual information of various kinds and modalities, apparently without any conscious effort.

We observe a severe discrepancy between the enormous potential lying in contextualized natural language use on the one hand, and the way in which natural language is utilized in today's computer systems on the other: commercial search engines support little more than isolated bag-of-words queries, and dialogue interfaces mimic traditional interaction through buttons and menus with pre-defined command words. To enable the applications envisioned in the Excellence Cluster, we aim for environments that allow the user to interact, combining language with a range of other modalities, while ensuring appropriate, efficient and robust understanding.

To achieve such an advanced level of natural language interaction, we are developing robust, efficient, and reliable methods for mapping between (written and spoken) language and content. Central to this research is the semantic integration of text and speech with context: meaning representations are integrated into structured units that connect information across sentence, utterance and even document boundaries; speech understanding techniques combine and interleave with diverse communicative modalities; linguistic expressions are anchored in the virtual and physical environment. Advances in these areas will enable the construction of interactive environments which maximize the accessibility of rich and varied information using multimodal communication technologies.

Research Topics and Achievements

Responsible Investigators and Personal Development