Andy Lücking (LLF)
Aspects of Multimodal Communication
Natural language interaction is multimodal. The thesis focuses on four aspects of multimodal communication: quantification in dialogue, pointing gestures, head shakes, and a cognitive construal of dialogue semantics in general.
A representational theory of count noun quantification is developed. A QNP now involves three sets, a refset, a compset and a maxset, the union of the former two sets. These sets provide antecedents for pronominal anaphora. A novel explanation for the highly restricted phenomenon of compset anaphora is given in terms of horror vacui: compset anaphora is only allowed if the QNP antecedent involves an empty refset. Quantification is expressed NP-internally, that is, without reference to a scope set. This denotational underpinning leads to a significant reduction of the logical space of quantifiers.
Based on previous experimental evidence, a spatial semantics for pointing gestures is developed, replacing directly referential models. It is claimed that demonstration acts like pointing gestures provide locational restrictions which indicate where the addressee should look for the referent according to the descriptive information of the accompanying noun phrase. Furthermore, it is shown It is shown that the components introduced on linguistic grounds correspond to pathways identified in psycho- and neuro-scientific studies.
Based on previous work on the verbal negative particle 'No', different uses of the head shake are identified. Following work on laughter, emotionally evaluative uses of head shaking are formally analysed: Here, the trigger of a head shake can root either in positively or a negatively aroused disbelief (with regard to an event, for instance). The intertwining of emotional and more classical descriptive contents has been construed as ‘noetic head shake’.
A cognitive stance underlies the individual aspects of multimodal communication above. A cognitive stance is also inherent in dynamic semantic theories. Following the cognitive route, KoS is introduced as a representational semantic framework and linked to findings from cognitive science. By this means evidence is collected that dialogue semantics and cognitive science by and large can reasonably be said to share a common empirical domain.