Research Esther de Boe

Abstract

Simultaneous interpreting (SI) is a complex cognitive activity comprising concurrent tasks: language comprehension, production, and monitoring. It is carried out in an increasingly technologized environment, in which information is presented through multiple channels, i.e., multimodally. Incoming verbal and nonverbal information from the speaker is frequently complemented by supports such as a slide presentation (with or without captioning) or computer-assisted interpreting (CAI) tools (e.g., terminology software). Yet, the equation between the facilitating effect of these supports in terms of performance and the added cognitive load on information processing lacks empirical investigation. We attempt to fill this gap by investigating the effect of multimodality on interpreters' performance, visual attention, cognitive load, stress, and user experience. To this end, we will conduct two experimental, within-subject studies involving 12 professional interpreters. Study 1 investigates one support: participants will perform their task in 4 conditions, that is, (1) SI (no support - baseline); (2) SI + slides; (3) SI + CAI-tool; (4) SI + intralingual captioning. Study 2 investigates 4 combinations of supports: (1) SI + slides (baseline); (2) SI + slides + CAI; (3) SI + slides + intralingual captioning; (4) SI + slides + interlingual captioning. During the tasks, speech rate, a major problem trigger in SI, is manipulated to measure its effect in relation to the support type. The project combines objective and subjective data collection and analysis such as mobile eye tracking, wristbands, and interviews, to generate profound fundamental knowledge on multimodality and cognition, and benefit interpreter training and practice.

Researcher(s)

• Promoter: Robert Isabelle
• Co-promoter: de Boe Esther

Research team(s)

• Translation, Interpreting and Intercultural Studies (TricS)

Project type(s)

• Research Project

Abstract

Simultaneous Interpreting (SI) is a real-time oral translation of a speaker's oral discourse by an interpreter who usually works from a sound-proof cabin in a conference room. SI requires complex information processing, since the interpreter has to carry out several tasks in parallel (i.e., listening, comprehending and producing speech). Interpreters are often provided with a written version of the oral speech to be delivered by the speaker beforehand, an additional information source that can be used for task preparation, but also during the delivery of the oral speech. The latter practice is referred to as "SI with text" or "SIMTEXT". This involves multimodal cognitive processing since the interpreter now receives information through different channels, simultaneous auditory and visual. Although intuitively, the availability of the text appears to be useful support for achieving the interpreting task, previous experiments with SIMTEXT have indicated that multimodal input may in fact complicate cognitive processing and, as such, undermine the potential facilitating effect. However, the evidence supporting this hypothesis is inconclusive, since only few studies have addressed the matter, and their methodological design often lacks ecological validity due to artificial experimental conditions, the use of less accurate tools, subjective measurements or the exclusive focus on only one input variable. We therefore propose to explore the effect of multimodal input on cognitive processing in SIMTEXT by designing experiments with professional simultaneous interpreters who will carry out one SI and one SIMTEXT task in near to real-life conditions, that is, in an interpreting booth, wearing eye tracking glasses. By manipulating a combination of input variables related to content, form and delivery of the oral speeches in both experimental conditions (SI and SIMTEXT), we will investigate the facilitating effect of text in SI. The study has a mixed-method design. One the one hand, quantitative data are collected by means of mobile eye tracking glasses to measure the effect of the use of text on cognitive processing and on the allocation of visual attention. On the other hand, qualitative data will be collected by means of retrospective online questionnaires aimed to elicit the participants' perceived cognitive load. Apart from studying the effect of the use of text on information processing, we will also look into its effect on the interpreting product in the form of recordings from the interpreting tasks, which will be submitted to a qualitative analysis. By using a combination of input variables and complementary methods, we intend to investigate the cognitive processing of multimodal information sources in SI from a much broader perspective than has so far been the case in studies on this topic. As such, we intend to generate new insights into the study of multimodality and cognition, that will also be invaluable for SI practice and training.

Researcher(s)

• Promoter: Robert Isabelle
• Co-promoter: de Boe Esther

Research team(s)

• Translation, Interpreting and Intercultural Studies (TricS)

Project type(s)

• Research Project