Language and Vision. Using visual information in real-world language situations
Author
Summary, in English
This dissertation is about the role of visual context information in communication. All accounts of language processing and communication agree that visual context is beneficial for understanding. Despite this, it is not clear through which mechanisms visual context works and whether these mechanisms also work in communication outside the lab setting. This dissertation aims to provide answers to these concerns and this is achieved through three different paths:
1. Testing known mechanisms under more realistic conditions.
2. Exploring new mechanisms for how visual context may inform communication
3. Resolve methodological issues that prevents experiments from being carried out under more realistic conditions.
This dissertation reports the work of five different papers. Paper I provided concise answers to the problem of temporal imprecision in eye-trackers with low sampling frequencies. Paper II investigated the habit of looking at visual referents as they are mentioned and found that this habit works even in very demanding situations with complex speech and complex scenes. Paper III suggested a new measure for evaluating communicative success in unrestricted conversations. This paper also found that visual context information allow speakers to understand each other better. Paper IV found that visual context information, which use is often made mandatory in language experiments, is used even if it is completely optional to the communication task. Paper V found that images of high emotional content is perceived as more helpful in a communicative task. This paper also found that participants do not gravitate toward a similar use of emotional language, which is predicted by a current theory.
1. Testing known mechanisms under more realistic conditions.
2. Exploring new mechanisms for how visual context may inform communication
3. Resolve methodological issues that prevents experiments from being carried out under more realistic conditions.
This dissertation reports the work of five different papers. Paper I provided concise answers to the problem of temporal imprecision in eye-trackers with low sampling frequencies. Paper II investigated the habit of looking at visual referents as they are mentioned and found that this habit works even in very demanding situations with complex speech and complex scenes. Paper III suggested a new measure for evaluating communicative success in unrestricted conversations. This paper also found that visual context information allow speakers to understand each other better. Paper IV found that visual context information, which use is often made mandatory in language experiments, is used even if it is completely optional to the communication task. Paper V found that images of high emotional content is perceived as more helpful in a communicative task. This paper also found that participants do not gravitate toward a similar use of emotional language, which is predicted by a current theory.
Publishing year
2012
Language
English
Publication/Series
Lund University Cognitive Studies
Volume
152
Document type
Dissertation
Topic
- Philosophy
Keywords
- language
- commmunication
- visual context
- visual world
- pragmatics
- ecological validity
Status
Published
Project
- Thinking in Time: Cognition, Communication and Learning
Research group
- Lund University Cognitive Science (LUCS)
Supervisor
- Kenneth Holmqvist
- Jana Holsanova
- Sven Strömqvist
ISBN/ISSN/Other
- ISSN: 1101-8453
- ISBN: 978-91-7473-292-4
Defence date
20 April 2012
Defence time
13:15
Defence place
Sal 104, Kungshuset, Lundagård, Lund
Opponent
- Zenzi Griffin (Professor)