RA1: Text and Speech Processing

Coordinators

Prof. Dr. Manfred Pinkal - UdS-LST, UdS-CS

Prof. Dr. Matthew Crocker - UdS-LST, UdS-CS

Prof. Dr. Hans Uszkoreit - UdS-CS, DFKI

Vision and Research Strategy

Language is the most natural and expressive medium for human communication and interaction. The richness of language derives from its grounding in our knowledge of the world and the immediate linguistic and non-linguistic context. In Research Area 1, we have focused on structurally informed models of distributional semantics, using unsupervised and minimally supervised approaches that combine linguistic knowledge with linguistic information. We substantially extended our work by going beyond intra-sentential context in two directions. On the one hand, we created text-level models of discourse relations and script knowledge, and applied these models to improve text-level comprehension. On the other hand, we have advanced cross-modal methods that deeply integrate language processing with both extra-linguistic knowledge and visual information, and lead to more naturalistic interaction with dialog systems and avatars. Furthermore, our methodological contributions regarding neurophysiological and pupillometric measures have enabled us to investigate situated human-computer interaction in greater detail. Our results have significantly contributed to improvements in several types of natural language processing tasks: the offline extraction of complex knowledge from text to feed knowledge bases; the online semantic interpretation (disambiguation and composition) required for deep text understanding, enrichment of text documents, and deep question-answering; the understanding of pictures and visual scenes; and the integration of speech and vision modalities to enrich spoken-language understanding and generation technologies in virtual interactive environments.

Research Topics and Achievements

Structured Distributional Models of Semantics

Our efforts in the second funding phase have focused on methods that include a rich representation of linguistic structure (including semantic frames and roles) and make use of prior knowledge to more effectively use large amounts of linguistic data. We integrated distributional structured vector space models with an incremental parsing model, and developed the first incremental semantic role labeler, as well as, more recently, a deep neural network for selectional preference prediction. We explored non-parametric Bayesian models in shallow semantic parsing tasks, and achieved competitive results both on a question-answering task and on direct evaluation against human annotation. For languages for which role labeling annotation is not available, we also showed that competitive results on a semantic role labeling task can be achieved by using a cross-lingual model transfer approach. We used semantic role information for the task of phrase-level paraphrase identification, and successfully applied the resulting model to short-answer scoring as an educational application.

In a collaboration with RA2 (Bernt Schiele’s group), we grounded lexical semantics in visual information extracted from videos, thus obtaining a finer-grained distributional analysis of event and action verbs. Our distributional semantic models can also be used for cognitive modelling tasks, and we have shown that semantic surprisal is predictive of word durations in spontaneous speech; we are currently extending this work to improve the naturalness of synthesized speech.

Lexical Semantics

In collaboration with RA5, we have made major advances in disam- biguating the senses of words or multi-word phrases, with a focus on phrases that denote se- mantic relations. This includes the development of large resources for relational paraphrases and their alignment with other resources. New methods for word sense disambiguation, with applications in named entity recognition and classification, have been developed. We have devised probabilistic graphical models that allow joint inference on several NLP tasks in an integrated manner, enhancing the overall output quality: co-reference resolution and named entity disambiguation, as well as named entity recognition and named entity disambiguation. Additionally, we worked on structured distributional models for thematic fit tasks (i.e., distributional spaces where words are represented as a function of their role in the sentence), and showed that an unsupervised word sense disambiguation step significantly improves over the state of the art on this task.

Combining machine-learning and linguistic processing for information extraction

On the topic of schema-based relation extraction, we moved from a minimally supervised learn- ing approach to truly web-based learning via distant supervision. A new method was de- veloped for learning dependency structures belonging to instances of the target relations. Like other research efforts in the area, we took semantic seed instances from existing knowledge repositories, and extracted tens of thousands of Freebase facts for a rather com- plete harvest of learned patterns. We cover n-ary relations, learning patterns for instances containing between two and n arguments. Instead of training a classifier, we learn discrete dependency rules represented in a recursive rule formalism, and apply a range of semantic filters to the learned rules. One of these filters checks if there is at least one content word in the learned pattern that is semantically connected to the target relation, using BabelNet and lexical disambiguation via Babelfy. We furthermore successfully increased effi- ciency of our information extraction pipeline in a collaboration with Berlin Big Data Center (BBDC), and integrated our pipeline with a real-time analytics system. A current focus of our research in this area lies on extracting information from indirect mentions of relations (textual entailment, presupposition, metonymy, metaphor) or relation arguments outside the sentence.

Discourse-level Processing

Script knowledge, i.e., common-sense knowledge describing patterns of event sequences for typical activities, is relevant for the prediction of discourse relations as well as for other discourse-level tasks like coreference resolution and temporal ordering classification. We explored different models of script knowledge based on existing corpora, a hierarchical Bayesian model as well as a neural network model, the latter ones substantially outperforming the state of the art on the temporal ordering task. As a basis for high-accuracy extraction of script knowledge, we crowdsourced a new, much larger corpus containing generic descriptions of script information plus partial event-level alignments between the scripts (the DeScript corpus), as a basis for semi-supervised script extraction. We also created a corpus of simple narrative texts annotated with detailed script information (the InScript corpus). We have achieved good results on the task of automatic participant prediction. In a collaboration with RA2, we used script data for the improvement of action recognition in video understanding, as well as for the generation of text from videos. We also developed a distributional approach to discourse relation processing, improving the state of the art on discourse relation parsing by employing a neural network model.

Unlike the assumptions in standard models of script processing, only some verb instances de- note events, or describe specific situations at all. We developed models for the classification of aspectual classes along different dimensions (stative/eventive, generic/specific/habitual). We were able to outperform the state of the art, and could show that a part of these distinc- tions is dependent on the wider discourse context, using sequence labeling techniques (CRF).

Language-centered Multimodal Interaction

In order to approach our long-term goal of flexible adaptive open-domain multimodal dialogue systems driven by adequate cognitive modeling of human behavior, we have undertaken research and development efforts in hu- man computer interaction using speech, gaze and a shared situated frame of reference.

Flexible management strategies can be learned from dialogue data; we therefore collected interactive multimodal data in domains of various complexities, e.g. multi-player games, tutor- ing dialogues for training metacognitive skills, political debate and multi-issue bargaining dialogues. We annotated a total of about 30 hours from more than 100 speakers, using multiple sensors, e.g. microphones and Kinect, according to the ISO 24617-2 dialogue act annotation scheme (issued under our co-authorship). The resulting corpora have been used for automatic dialogue act recognition. Current research building on this resource is concentrated on cognitive aspects of dialogue modeling and management. We successfully integrated ACT-R-based cognitive task models into our multi-thread dialogue management architecture.

We have also investigated the interaction of spoken and visual modalities in human cognition. Specifically, we have examined the benefit of visual and linguistic contexts for word learning, complementing our previous work on computational modeling. Eye-tracking evidence reveals the extremely rapid deployment of both immediate context and cross-situational learning mechanisms.

In order to improve virtual reality interactions with avatars, we investigated whether the interaction can be improved by enabling the computer system to react to the eye gaze patterns of the human user. We found that the natural-language generation system that provided eye-movement-based feedback outperformed baseline systems that did not have access to eye gaze information. This suggests that NLG systems can, in principle, exploit human eye movements and react quickly enough in order to encourage correct, or prevent wrong, reference resolution and any resulting actions in the virtual environment. Based on this success, IRGs Staudte and Heloir have now set up an avatar that receives information about listener gaze and can follow the gaze of the human speaker. This exciting novel setup allows for truly interactive and adaptive behavior which enables us to further explore how an artificial agent should exploit and react to human gaze and speech.

IRGs Steiner and Wuhrer have collaborated on improving realtime naturalistic speech anima- tion for an open-source game engine. They performed unsupervised segmentation of vocal tract MRI and intra-oral 3D scan data to create multilinear shape space models of the tongue and oral cavity, which are then deformed using data from electromagnetic articulogra- phy. Furthermore, IRG Steiner extended work on facial-expression-based affective speech translation.

A further important task in situated interaction using speech is speaker localization. Re- searchers from Klakow’s group have proposed a novel probabilistic framework for localizing multiple speakers with a microphone array. This approach increases localization precision, especially when few microphones are available.

Behavioral, neurophysiological and pupillometric measures of situated interaction

A continuing focus of RA1 has been the experimental investigation of how spoken language processing interacts with visual context. To better understand the cognitive mechanisms underlying scene-utterance integration, we have begun to use neurophysiological methods (EEG/ERP). The focus so far has been on determining the neurophysiological indices as- sociated with the comprehension of referring expressions in visual contexts. In a study comparing over-specified, minimally specified, and mismatching expressions, for example, we found a clear gradient response of the N400 component (over-specified being preferred). In addition, underspecified expressions (in which the visual referent is not uniquely picked out) result in an early positive-going potential which is being further investigated in ongoing research.

In order to improve the naturalness of interaction with speech-enabled virtual agents, ongo- ing research is examining both (a) how the neurophysiological (ERP) correlates of referent comprehension are modulated by (artificial) speaker gaze in visual contexts, and (b) the ex- tent to which traditional ERP correlates of spoken language comprehension are modulated by synthesized speech.

In some types of situated tasks (e.g., interacting with a virtual agent or dialog system while driving), the use of EEG or eye movements during reading is, however, not possible (due to the driving tasks and resulting movement artifacts). We have therefore also investigated the suitability of a novel pupillometric measure, the Index of Cognitive Activity (ICA). The ICA is a reliable indicator of linguistic processing difficulty, and is at the same time a highly dynamic measure which allows us to separate effects of the driving task from effects of language processing. This method has also delivered positive results for measurning effects of processing difficulty in visual world settings.

Collaborations

Internal Collaborations

M. Pinkal’s group jointly with B. Schiele’s group (RA2) investigated the use of visual information for semantic modeling and the use of linguistic information for video understanding, and developed a model for generating text from videos. H. Uszkoreit’s group collaborated with G. Weikum’s group (RA5): YAGO was used for seed construction. The features for disambiguation in the AIDA entity detection tool were adopted for the combined lexical disambiguation and entity-linking system MOOD. IRG Demberg collaborated with DFKI’s automotive IUI group (Christian Müller) on the design and testing of a highly controlled driving simulation task, and with Jörg Hoffmann’s group (CS department) on optimizing search in natural language generation tasks.

There were also numerous collaborations among IRGs across different research areas. IRG Steiner collaborated with IRG Wuhrer on improving real-time naturalistic speech animation and with IRG Demberg on using information-theoretic measures for text-to-speech synthesis. IRG Staudte collaborated with IRG Heloir on creating a virtual agent that can communicate using gaze, and with IRG Bulling on the role of listener gaze on instruction givers, involving a large mobile-eyetracking user study. IRGs Staudte and Demberg collaborated on the appli- cation of the index of cognitive activity (ICA) as a measure of human cognitive load during language processing. IRGs Steiner and Demberg are collaborating on integrating semantic surprisal into high-level features for text-to-speech synthesis.

External Collaborations

H. Uszkoreit’s group collaborated with Volker Markl and his group in the integration of natural language understanding into the Flink big-data analytics plat- form, with Roberto Navigli at La Sapienza Rome in semantic filtering and lexical disambigua- tion, with Volker Markl and his group in the integration of natural language understanding into the Flink big-data analytics platform, and with SAP Innovation Center Potsdam and SAP Boston in medical concept recognition and disambiguation. M. Pinkal’s group collaborated with Anette Frank (University of Heidelberg) and Nianwen Xue (Brandeis University) on aspectual classification. M. Crocker collaborated with Pia Knoeferle (HU Berlin) on examining ERP correlates of visual context with spoken language comprehension.

D. Klakow’s group cooperated extensively with Alex Waibel, David Yarowsky, Rich Stern, Alexander Rudnicky and other researchers from CMU and Johns Hopkins University on speech recognition for underresourced languages like Zulu, Tamil, Telugu etc. In the context of ROCKIT, a roadmap for conversational interaction technologies was developed for the EU. This was done in cooperation with Steve Renals from Edinburgh University and Herve Bourlard from IDIAP. David Traum, Niels Taatgen and Harry Bunt are core collaborators in Metalogue, a project on metacognitive dialogue systems. V. Demberg’s group has several ac- tive external collaborations in the area of discourse relation processing, with Hannah Rohde and Bonnie Webber (Edinburgh University), with Ted Sanders (Utrecht University), as well as with Sandrine Zufferey (Fribourg) and Rajakrishnan Rajkumar (IIT Delhi). Further exter- nal collaborations of the group include Frank Keller (Edinburgh University) on incremental syntactic and semantic processing. I. Steiner collaborated with Carson-Berndsen at the University College Dublin on facial expressions based affective speech translation, and with Universities of Edinburgh and Trier on speech production data for vocal tract analysis. M. Staudte has an ongoing collaboration with Gerry Altmann (University of Connecticut) on the function of eye movements for retrieving object information from memory, and with Thies Pfeiffer from CITEC, Bielefeld University, on connecting a 3D model of a scene and mobile eye tracking with a natural language generation system for instruction generation.

We continue to maintain intensive collaborations with all former IRG leaders in Language and Speech Processing. In particular, there are ongoing collaborations of Alexander Koller with Crocker and Staudte on the generation of referring expressions, and with Demberg on incremental parsing; of Ivan Titov with Demberg and Pinkal on language models informed with script knowledge; and between Titov and Caroline Sporleder on a joint modeling of sentiment and discourse.

Awards and Honors

Publications

Marcus Rohrbach and Anna Rohrbach and Michaela Regneri and Sikandar Amin and Mykhaylo Andriluka and Manfred Pinkal and Bernt Schiele Recognizing Fine-Grained and Composite Activities Using Hand-Centric Features and Script Data In: Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA, 2016
Hewer, Alexander and Wuhrer, Stefanie and Steiner, Ingmar and Richmond, Korin Tongue mesh extraction from 3D MRI data of the human vocal tract In: Perspectives in Shape Analysis, 2016
Konstantina Garoufi and Maria Staudte and Alexander Koller and Matthew W. Crocker Exploiting Listener Gaze to Improve Situated Communication in Dynamic Virtual Environments In: Proceedings of the 39th Annual Meeting of the Cognitive Science Society, CogSci 2017, London, UK, 16-29 July 2017, 2016
Demberg, Vera and Sayeed, Asad The Frequency of Rapid Pupil Dilations as a Measure of Linguistic Processing Difficulty. In: PLoS ONE, 2015
Judith Köhne and Matthew W. Crocker The Interplay of Cross-Situational Word Learning and Sentence-Level Constraints In: Cognitive Science, 2015
Asad B. Sayeed and Stefan Fischer and Vera Demberg Vector-space calculation of semantic surprisal for predicting word pronunciation duration In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers, 2015
Hong Li and Sebastian Krause and Feiyu Xu and Andrea Moro and Hans Uszkoreit and Roberto Navigli Improvement of n-ary Relation Extraction by Adding Lexical Semantics to Distant-Supervision Rule Learning In: ICAART 2015 - Proceedings of the International Conference on Agents and Artificial Intelligence, Volume 2, Lisbon, Portugal, 10-12 January, 2015., 2015
Sourav Dutta and Gerhard Weikum Cross-Document Co-Reference Resolution using Sample-Based Clustering with Knowledge Enrichment In: Proceedings of the 25th International Conference on World Wide Web, WWW 2016, Montreal, Canada, April 11 - 15, 2016, 2015
Annemarie Friedrich and Manfred Pinkal Discourse-sensitive Automatic Identification of Generic Expressions In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers, 2015
Schweitzer, Katrin and Walsh, Michael and Calhoun, Sasha and Schütze, Hinrich and Möbius, Bernd and Schweitzer, Antje and Dogil, Grzegorz Exploring the relationship between intonation and the lexicon: Evidence for lexicalised storage of intonation In: Speech Communication, 2015
Lasarcyk, Eva and Drenhaus, Heiner and Möbius, Bernd Experimente zur Wahrnehmung gezielt degradierter synthetischer Sprache In: Elektronische Sprachsignalverarbeitung 2015, Tagungsband der 26. Konferenz (Eichstätt), 2015
Zimmerer, Frank and Andreeva, Bistra and Jügler, Jeanin and Möbius, Bernd Comparison of pitch profiles of German and French speakers speaking French and German In: Proceedings of the 18th International Congress of Phonetic Sciences (Glasgow), 2015
Jügler, Jeanin and Möbius, Bernd Auditory feedback methods to improve the pronunciation of stops by German learners of French In: Proceedings of the 18th International Congress of Phonetic Sciences (Glasgow), 2015
Jouvet, Denis and Bonneau, Anne and Trouvain, Jürgen and Zimmerer, Frank and Laprie, Yves and Möbius, Bernd Analysis of phone confusion matrices in a manually annotated French-German learner corpus In: Proceedings of the Workshop on Speech and Language Technology for Education (SLaTE 2015, Leipzig), 2015
Andreeva, Bistra and Möbius, Bernd and Demenko, Grazyna and Zimmerer, Frank and Jügler, Jeanin Linguistic measures of pitch range in Slavic and Germanic languages In: Proceedings of Interspeech 2015 (Dresden), 2015
Jügler, Jeanin and Zimmerer, Frank and Möbius, Bernd and Draxler, Christoph The effect of high variability training on the perception and production of French stops by German native speakers In: Proceedings of Interspeech 2015 (Dresden), 2015
Michael Breuß and Alfred M. Bruckstein and Petros Maragos and Stefanie Wuhrer New Perspectives in Shape Analysis (Dagstuhl Seminar 14072) In: Proceedings of the Twenty-Fifth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2014, Portland, Oregon, USA, January 5-7, 2014, 2014
Staudte, Maria and Crocker, Matthew W and Heloir, Alexis and Kipp, Michael The influence of speaker gaze on listener comprehension: Contrasting visual versus intentional accounts In: Cognition, 2014
Lea Frermann and Ivan Titov and Manfred Pinkal A Hierarchical Bayesian Model for Unsupervised Induction of Script Knowledge In: Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, {EACL} 2014, April 26-30, 2014, Gothenburg, Sweden, 2014
Éva Székely and Ingmar Steiner and Zeeshan Ahmed and Julie Carson-Berndsen Facial Expression-based Affective Speech Translation In: Journal on Multimodal User Interfaces, 2014
Maria Staudte, Matthew W. Crocker, Alexis Heloir, Michael Kipp The influence of speaker gaze on listener comprehension: Contrasting visual versus intentional accounts In: Cognition, 2014
Trouvain, Jürgen and Möbius, Bernd Individuelle Ausprägung von Atmungspausen in der Mutter- und in der Fremdsprache als Anzeichen kognitiver Belastung In: Elektronische Sprachsignalverarbeitung 2014, Tagungsband der 25. Konferenz (Dresden), 2014
Volha Petukhova and Harry Bunt Incremental Recognition and Prediction of Dialogue Acts In: Computing Meaning, 2014
Michaela Regneri and Marcus Rohrbach and Dominikus Wetzel and Stefan Thater and Bernt Schiele and Manfred Pinkal Grounding Action Descriptions in Videos In: TACL, 2013
Mikhail Kozhevnikov and Ivan Titov Cross-lingual Transfer of Semantic Role Labeling Models In: ACL (1), 2013
Marcus Rohrbach and Wei Qiu and Ivan Titov and Stefan Thater and Manfred Pinkal and Bernt Schiele Translating Video Content to Natural Language Descriptions In: ICCV, 2013
Andrea Moro and Hong Li and Sebastian Krause and Feiyu Xu and Roberto Navigli and Hans Uszkoreit Semantic Rule Filtering for Web-Scale Relation Extraction In: International Semantic Web Conference (1), 2013
Youssef Oualil and Mathew Magimai.-Doss and Friedrich Faubel and Dietrich Klakow A probabilistic framework for multiple speaker localization. In: Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference, 2013
Alishahi, Afra and Fazly, Afsaneh and Koehne, Judith and Crocker, Matthew W Sentence-based attentional mechanisms in word learning: Evidence from a computational model In: Frontiers in Psychology, 2012
Sebastian Krause and Hong Li and Hans Uszkoreit and Feiyu Xu Large-Scale Learning of Relation-Extraction Rules with Distant Supervision from the Web In: International Semantic Web Conference (1), 2012
Maria Staudte and Alexander Koller and Konstantina Garoufi and Matthew W. Crocker Using listener gaze to augment speech generation in a virtual 3D environment In: Proceedings of the 34th Annual Conference of the Cognitive Science Society, 2012
Alexander Koller and Konstantina Garoufi and Maria Staudte and Matthew W. Crocker Enhancing referential success by tracking hearer gaze In: Proceedings of the 13th Annual SIGdial Meeting on Discourse and Dialogue, 2012
Alishahi, Afra and Fazly, Afsaneh and Koehne, Judith and Crocker, Matthew Sentence-Based Attentional Mechanisms in Word Learning: Evidence from a Computational Model In: Frontiers in Psychology, 2012
Ndapandula Nakashole and Gerhard Weikum and Fabian M. Suchanek PATTY: A Taxonomy of Relational Patterns with Semantic Types In: EMNLP-CoNLL, 2012
Michaela Regneri and Alexander Koller and Josef Ruppenhofer and Manfred Pinkal Learning Script Participants from Unlabeled Data In: Proceedings of RANLP, 2011
Maria Staudte and Matthew W. Crocker and Alexis Heloir and Michael Kipp Speaker gaze affects utterance comprehension beyond visual attention shifting In: Proceedings of the 33rd Annual Conference of the Cognitive Science Society, 2011
Maria Staudte and Matthew W. Crocker Investigating Joint Attention Mechanisms through Spoken Human-Robot Interaction In: Cognition, 2011
Demberg, V. and Sayeed, A. Lingistic cognitive load: implications for automotive UIs In: Workshop on Cognitive Load and In-Vehicle Human-Machine Interaction at AutomotiveUI, 2011
Antske Fokkens Metagrammar engineering: Towards systematic exploration of implemented grammars In: ACL, 2011
Hans Uszkoreit Learning Relation Extraction Grammars with Minimal Human Intervention: Strategy, Results, Insights and Plans In: CICLing (2), 2011
Michaela Regneri and Alexander Koller and Manfred Pinkal Learning Script Knowledge with Web Experiments In: Proceedings of ACL, 2010
Michael Backes and Markus Dürmuth and Sebastian Gerling and Manfred Pinkal and Caroline Sporleder Acoustic Side-Channel Attacks on Printers In: USENIX Security Symposium, 2010
Lizhen Qu and Georgiana Ifrim and Gerhard Weikum The Bag-of-Opinions Method for Review Rating Prediction from Sparse Text Patterns In: COLING, 2010
Yi Zhang 0003 and Valia Kordoni Discriminant Ranking for Efficient Treebanking In: COLING (Posters), 2010
Maria Staudte and Matthew W. Crocker When Robot Gaze Helps Human Listeners: Attentional versus Intentional Account In: Proceedings of the 32nd Annual Conference of the Cognitive Science Society, 2010
M. Staudte Joint Attention in Spoken Human-Robot Interaction In: 2010
R. Wang and Y. Zhang MARS: A Specialized {RTE} System for Parser Evaluation In: Proceedings of the SemEval-2010 Evaluation Exercises on Semantic Evaluation, 2010
Alishahi, Afra and Fazly, Afsaneh Integrating Syntactic Knowledge into a Model of Cross-situational Word Learning In: Proceedings of the 32nd Annual Conference of the Cognitive Science Society, Portland, Oregon., 2010
Bart Cramer and Yi Zhang Constraining robust constructions for broad-coverage parsing with precision grammars In: COLING, 2010
Crocker, Matthew W. and Knoeferle, Pia and Mayberry, Marshall R. Situated Sentence Comprehension: The Coordinated Interplay Account and a Neurobehavioral Model. In: Brain and Language, 2010
Alexander Koller and Stefan Thater Computing Weakest Readings In: ACL, 2010
Michael Wiegand and Dietrich Klakow Convolution Kernels for Opinion Holder Extraction In: HLT-NAACL, 2010
Köhne, J. and Matthew Crocker Sentence Processing Mechanisms Influence Cross-Situational Word Learning In: Proceedings of the 32nd Annual Conference of the Cognitive Science Society, 2010
Yi Zhang 0003 and Timothy Baldwin and Valia Kordoni and David Martinez and Jeremy Nicholson Chart Mining-based Lexical Acquisition with Precision Grammars In: HLT-NAACL, 2010
Feiyu Xu and Hans Uszkoreit and Sebastian Krause and Hong Li Boosting Relation Extraction with Limited Closed-World Knowledge In: COLING (Posters), 2010
Aljoscha Burchardt and Marco Pennacchiotti and Stefan Thater and Manfred Pinkal Assessing the impact of frame semantics on textual entailment In: Natural Language Engineering, 2009
Maria Staudte and Matthew W. Crocker Visual Attention in Spoken Human-Robot Interaction In: Proceedings of the 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI'09), 2009
G. Dinu and S. Thater and M. Pinkal Ranking Paraphrases in Context In: Proceedings of the ACL Workshop TextInfer: Applied Textual Inference, 2009
Maria Staudte and Matthew W. Crocker Visual attention in spoken human-robot interaction In: HRI, 2009
Marshall R. Mayberry and Matthew W. Crocker and Pia Knoeferle Learning to Attend: A Connectionist Model of Situated Language Comprehension In: Cognitive Science, 2009
Georgiana Dinu and Rui Wang 0005 Inference Rules and their Application to Recognizing Textual Entailment In: EACL, 2009
Hagen Fürstenau and Mirella Lapata Semi-Supervised Semantic Role Labeling In: EACL, 2009
Ken'ichi Kumatani and John W. McDonough and Barbara Rauch and Dietrich Klakow and Philip N. Garner and Weifeng Li Beamforming With a Maximum Negentropy Criterion In: IEEE Transactions on Audio, Speech & Language Processing, 2009
Wölfel, Matthias and McDonough, John Distant Speech Recognition In: 2009
Zhang, Yi and Wang, Rui Classifying textual relatedness with predicate-argument structures In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2009
ARAI, M. and KELLER, F. and DEMBERG, V. Prediction of modifiers during sentence processing In: Technical report of IEICE. Thought and language , 2009
Yi Zhang and Rui Wang Cross-Domain Dependency Parsing Using a Deep Linguistic Grammar In: ACL/AFNLP, 2009
Regneri, Michaela and Egg, Markus and Koller, Alexander Efficient Processing of Underspecified Discourse Representations In: Proceedings of ACL-08: HLT, Short Papers, 2008
Maria Staudte and Matthew W. Crocker The utility of gaze in human-robot interaction In: Proceedings of "Metrics for HRI", Workshop at the 3rd ACM/IEEE Conference on Human-Robot Interaction (HRI'08), 2008
Georgiana Ifrim and Gökhan H. Bakir and Gerhard Weikum Fast logistic regression for text categorization with variable-length n-grams In: KDD, 2008
Knoeferle, P. and Boukje Habets and Matthew Crocker and Thomas Münte Visual scenes trigger immediate syntactic reanalysis: evidence from ERPs during situated spoken comprehension In: Cerebral Cortex, 2008
Zhang, Y. and R. Wang and H. Uszkoreit Hybrid Learning of Dependency Structures from Heterogeneous Linguistic Resources In: Proceedings of the 12th Conference on Computational Natural Language Learning, 2008
Demberg, V. and Keller, F. Data from eye-tracking corpora as evidence for theories of syntactic processing complexity In: Cognition, 2008
Demberg, V. and Keller, F. A psycholinguistically motivated version of TAG In: Proceedings of the 9th International Workshop on Tree Adjoining Grammars and Related Formalisms. Tübingen, 2008
Feiyu Xu and Hans Uszkoreit and Hong Li A Seed-driven Bottom-up Machine Learning Framework for Extracting Relations of Various Complexity In: ACL, 2007

Cluster of Excellence Multimodal Computing and Interaction

News

A webcam is enough to produce a real-time 3D model of a moving hand

Cebit 2018: Digital assistant teaches runners healthier running style

Partners