Entry | Didactic Audiovisual Translation

SPA Traducción audiovisual didáctica

Introduction

Didactic audiovisual translation (DAT) is an innovative field of study that has emerged within the domains of translation studies, applied linguistics, and education, and has offered promising results when it comes to implementing research-led language teaching and learning. The uses and applications of audiovisual translation (AVT) in language education date back to the 1980s, when some scholars (Price 1983 and Vanderplank 1988, among others) discussed the benefits observed when using subtitled materials to improve foreign-language skills in the language classroom. From that moment onwards, there has been a growing scholarly interest in the active application of AVT to language education, that is, in DAT, starting with the seminal work of Williams & Thorne (2000), and much has been published in the last two decades (see a recent revision of the literature in Talaván, Lertola & Fernández-Costales 2024). In the last years in particular, by combining the latest research with classroom practice, DAT studies have strived to provide scholars, teachers, and learners with an ever-growing body of research (see Ávila-Cabrera & Corral Esteban (2021), Bolaños-García-Escribano & Ogea-Pozo (2023), Ibáñez Moreno & Vermeulen (2021), Nicora (2022) or Rodríguez-Arancón (2023), for recent representative samples).

International and national institutions in Europe have recognized the potential advantages of DAT by funding several research-led projects. At international level, the first project which created software specifically designed for didactic subtitling was LeViS-Learning via Subtitling (2006-2008) (Sokoli, Zabalbeascoa & Fountana 2011). Following this international collaboration, ClipFlair-Foreign language learning through interactive revoicing and captioning of clips (2011-2014) provided teachers and learners the opportunity to carry out subtitling and revoicing DAT tasks in a good number of languages within the same platform (Sokoli 2015).


LvS (Learning via Subtitling), the first subtitling platform specifically designed for language learning. [Source: Sokoli 2006]	Sample of a ClipFlair activity. [Source: Baños & Sokoli 2015]	TRADILEX, a platform for carrying out DAT-based Lesson Plans. [Source]

At national level, the PluriTAV-Audiovisual Translation as a Tool for the Development of Multilingual Competence in the Classroom -Project (2017-2019) proposed DAT sequences, designed for face-to-face learning, to foster plurilingual competence (Reverter Oliver & Cerezo Merchán 2021). More recently, the TRADILEX-Audiovisual Translation as a Didactic Resource in Foreign Language Education -Project (2020-2023) developed a methodological proposal for DAT and designed a user-friendly platform, with more than 60 LPs of diverse DAT modes for B1-B2 English as a Foreign Language learners (other LPs on different proficiency levels and languages will be available in the near future), suitable for online and face-to-face learning (Fernández-Costales, Talaván & Tinedo-Rodríguez 2023).

Research findings have already established that employing DAT can have an impact on language education (Talaván, Lertola & Fernández-Costales 2024), among other things because the use of DAT activities allows for a holistic enhancement of language competence in general, as well as mediation, production, and reception skills in an integrated manner.

DAT also presents some limitations: although previous translation experience is not required, it is advisable to introduce learners to simple DAT tasks first, before asking them to produce the complete subtitling or revoicing of an excerpt from scratch. Also, teachers should receive some training, especially at technical level. Lastly, one important issue is the lack of ready-to-use materials, which is being gradually solved by the creation of online language platforms such as ClipFlair or TRADILEX.

Most AVT modes can be applied as didactic resources in the language education context, and many of them have already been put into practice by teachers and researchers over the last twenty years (Talaván, Lertola & Fernández-Costales 2024). Two of them, subtitling and dubbing, have received closer attention, since they are the best known and most widely used AVT modes, and so they are more easily accessible and familiar for teachers and learners alike. In the following sections and subsections, the main DAT modes will be presented, ranging from the most used and familiar ones to the modes that have received less attention and still need further research and practice to be thoroughly described and categorized in pedagogical terms: subtitling, dubbing, audio description (AD), subtitles for the deaf and hard of hearing (SDH), voice-over, and free commentary.

Didactic subtitliing

Didactic subtitling (DS) is understood as the addition of subtitles to pre-selected video extracts as a language learning task. It should be distinguished from the use of subtitles as a support, since in the latter case they are not applied as an active didactic resource, but as an aid for comprehension. However, the two concepts are not totally separated, since the use of subtitles as a support is often included in DS tasks in the viewing phase, that is, the video to be subtitled is often pre-visualized with subtitles as a support (normally in a different language to the one required in the subtitling task).

DS offers a great flexibility of application, since it can take multiple forms depending on the type and direction of the translation. Each type and direction can be more suitable for certain contexts and groups of students, as well as to enhance some communicative skills more intensely than others. The main types are: intralingual (L2-L2), interlingual (direct from L2 to L1 or reverse from L1 to L2), and creative (that implies the ‘re-creation’ of the original audiovisual text by the students, where they produce a different version of the dialogues in the form of subtitles, and it can be intralingual or interlingual—direct or reverse).

Subtitling sample in the TRADILEX platform. [Source]

To subtitle, due to time and space constraints, learners should fully understand the original text and transfer the core message, making use of reduction and condensation strategies (including omission) that will effectively train their mediation skills, as described in the CEFR companion volume (Europe 2018). In the case of interlingual subtitling when there is a transfer from one language to another, apart from the development of mediation skills, the main advantages are those related to translation as a pedagogical tool in language learning, related to the contrastive analysis between L1 and L2. Furthermore, the multimodal nature of the subtitling task can foster vocabulary acquisition and intercultural awareness.

Subtitling in all its forms has been the most studied DAT mode in language education, ever since scholars foresaw the potential of moving from exploiting intralingual or interlingual subtitles as a support to the creation of subtitles by language learners. The first monograph on DS (Talaván 2013) provided the readers with all the practical information needed to apply subtitling to the classroom. The second related monograph (Lertola 2019) revised the main research lines and topics studied and researched as regards DS (as well as other DAT modes) for the first two decades of the 21^st century. As described and analyzed in detail in the most recent book on DAT to date (Talaván, Lertola & Fernández-Costales 2024), all foreign and second language skills can be enhanced thanks to DS, both productive (speaking and writing) and receptive skills (listening and reading), as well as mediation. Depending on the approach of the task and the type of subtitling (as well as the language combination), one skill or communicative activity may be enhanced more intensely than the others.

Didactic subtitling for the deaf and hard of hearing

Didactic subtitling for the deaf and hard of hearing (or didactic SDH) can be defined as a pedagogical task where students get involved in a media accessibility practice by producing SDH for a pre-selected video extract. That is, students create SDH with the aim of making a short video extract accessible for someone with hearing impairments or deaf. The video selected for didactic SDH should have elements such as sounds, music, mood changes, and other paralinguistic features; for example, cartoons are typically good options for didactic SDH, given the music, sounds and changing moods they tend to portray.

Just as DS, didactic SDH offers a great flexibility of application, since it can take multiple forms depending on the type and direction of the translation. Each type and direction can be more suitable for certain contexts and groups of students, as well as to enhance some communicative skills more intensely than others. The main types of didactic SDH coincide with those for DS: intralingual (L2-L2), interlingual (direct from L2 to L1 or reverse from L1 to L2), and creative (that implies the ‘re-creation’ of the original audiovisual text by the students, where they produce a different version of the dialogues and the paralinguistic elements in the SDH, and it can be intralingual or interlingual—direct or reverse).

SDH sample [Source]

Even if there are obvious similarities between the two practices, the main difference between DS and didactic SDH relies on the accessibility dimension: (1) the description of sounds, mood, music and other paralinguistic features needed to produce SDH leads learners to develop creativity more intensely, as well as to focus more specifically on vocabulary; (2) media accessibility awareness can be developed by the mere fact of introducing didactic SDH in the educational context.

It should be noted that there is always an extra effort in didactic SDH on the part of learners, no matter their L2 level, since they need to perform an intersemiotic translation (in addition to the standard translation): they have to look for accurate and appropriate ways to describe a specific sound, mood, tone, music, etc. Therefore, in a sequence where more than one DAT mode is used, DS should precede didactic SDH, thus presenting the DAT tasks in a scaffolded manner whenever possible.

Recent research (see Bolaños-García-Escribano & Ogea-Pozo 2023 and Tinedo-Rodríguez & Frumuselu 2023) has brought to the fore the potential benefits of SDH in language education. This didactic accessibility mode can enhance integrated communicative skills, just as DS, with the difference of the extra work on vocabulary enhancement no matter the type or direction; it can help learners acquire new words and expressions and retain them for future use more efficiently, provided the searching process (for the most accurate and concise way of expressing a particular sound, mood, music style, etc.) they go through, the intersemiotic nature of the mediation in this case, and the effective incorporation of the corresponding term in their subtitles.

Didactic dubbing

Didactic dubbing (DD) can be described as the replacement of the original spoken text of pre-selected video excerpts with a pre-scripted oral dialogue made by language learners as a pedagogical task. The new audio track should be synchronized with the lip movements of the characters and, to record it, learners should previously prepare the dubbing script and then read it as naturally as possible.

DD can also take multiple forms depending on the type and the direction of the translation. Each type and direction can be more suitable for certain contexts and groups of students, as well as to enhance some communicative skills more intensely than others, similarly to the case of DS; DD can be: intralingual (L2-L2), interlingual (only reverse, L1-L2), and creative (the ‘re-creation’ of the original by the students, where they produce a different version of the dialogues in the form of either intralingual dubbing or interlingual reverse dubbing).

Screenshot from a student’s dubbing sample. [Source]

Regardless of the type, the most challenging aspect of DD is lip synch. Leaners should fit their oral dialogues into the characters’ mouth movements. All language skills can be enhanced thanks to DD, namely speaking, writing, listening, and reading, as well as mediation. Depending on the type of dubbing and the language combination, one skill may be fostered more intensely than the others. Didactic intralingual dubbing requires leaners to dub in the L2, thus producing a verbatim reproduction of the original dialogues. First, they should understand the original to write the script, then they should practice speaking while reading and recording the dubbing script. Furthermore, DD might engage learners from a more playful perspective as they are supposed to lend their voices to the actors and optionally include some dramatization. Didactic interlingual dubbing asks learners to convey the oral message from L1 to L2, known as reverse, which is the only possible interlingual combination for DD as the other option would eliminate the target language from the learners’ final output. Creative DD involves the reformulation of the original spoken text by the learners, either intralingual or interlingual (reverse), to produce a particular effect on the audience (usually humoristic); besides fostering integrated language skills, the incorporation of both creativity and humor into the DAT task enhances motivation.

DD is the second most studied DAT mode after subtitling. Several empirical studies conducted to date furnish evidence supporting its efficiency (both intralingual and interlingual reverse DD) in language education (see Danan 2010 or Sánchez-Requena 2020 as representative examples). Empirical research displays that DD can enhance oral production skills, especially in terms of intonation, pronunciation, and fluency. More recently, creative didactic dubbing has received extra attention and calls for further research. Remarkably, experimental studies suggest that creative DD can prove especially motivating for language learners in diverse educational contexts, namely primary and tertiary education, including languages for specific purposes (Ávila-Cabrera & Corral-Esteban 2021; Fernández-Costales 2021).

Didactic voice-over

Didactic voice-over (DVO) is the addition of a pre-scripted oral dialogue to pre-selected video excerpts, in which the original spoken text is still audible at a lower volume. The new audio track is recorded by language learners with a slight delay of 1-2 seconds after the characters start speaking and no lip synch is required. As in the case of DD, before recording, learners should prepare the voice-over script in advance and then deliver it when they record it with as much naturalness as possible.

DVO offers similar possibilities for application to DD. Once again, each type and direction can be more suitable for certain contexts and groups of students, as well as to enhance some communicative skills more intensely than others; DVO can be: intralingual (L2-L2), interlingual (direct from L2 to L1 or reverse from L1-L2), and creative (the ‘re-creation' of the original by the students, where they produce a different version of the dialogues in the form of either intralingual or interlingual –reverse or direct—voice-over).

Short film based on Virginia Woolf’s “A Room of One’s Own” developed for didactic voice-over tasks. [Source]

Just as other DAT modes, DVO also allows for enhancing integrated language skills and mediation. Didactic intralingual voice-over requires learners to listen to the L2 audio of a pre-selected video excerpt, write a preliminary script and revoice it in the L2. A distinctive element of voice-over is isochrony (i.e., the new audio should enter 1-2 seconds after the original dialogue starts and if possible, end 1-2 seconds before the original ends). Because of this, voice-over implies a reduction and/or rephrasing of the original text, by means of condensation and reformulation strategies including omission of avoidable elements when applicable, as it happens with subtitling. Therefore, even if the transfer is intralingual, the new voice-over script will differ from the original. Contrary to DD, didactic interlingual voice-over can be either reverse (L1-L2) or direct (L2-L1), since direct interlingual voice-over from L2 to L1 could also be beneficial for language learning provided that the L2 is still audible in the background during the recording and in the final video produced by the learner.

Empirical research on DVO is still limited but interest in this DAT mode is steadily increasing. Pioneering studies and proposals show its benefits on the development of speaking skills, especially pronunciation (Talaván & Rodríguez-Arancón 2018). DVO might prove especially beneficial in the development of integrated language skills due to its comprehensive process of listening to the original soundtrack, writing the script (either paraphrasing or translating), and reading it out loud to create a new audio track. Furthermore, it is a flexible learning task that could be easily employed in synergy with Content and Language Integrated Learning (CLIL), English for Specific Purposes (ESP), and more specifically with English for Social Purposes of Cooperation (ESoP) (Tinedo-Rodríguez 2022).

Didactic audio description

Didactic audio description (DAD) requires language learners to transfer the visual information of an audiovisual product into words during the silent intervals of a pre-selected video excerpt. Learners should carefully identify and provide additional information about the individuals appearing in the video as well as the events unfolding, and the location where actions occur. Provided that AD is originally an accessible mode, learners are tasked with describing only what visually impaired people would not perceive.

DAD sample. [Source]

It should be noted that, unlike other DAT modes, the transfer of DAD is intersemiotic (i.e., from non- verbal to L2). However, for pedagogical purposes, intersemiotic plus intralingual as well as interlingual reverse are also envisaged for DAD, since the pre-selected video excerpt can contain L2 or L1 dialogues. These should not be translated but their understanding is of paramount importance for preparing the AD. In the educational context, creative DAD is also encouraged, since learners can translate images into words in a creative manner in order to produce a new, usually humorous, effect on the audience.

DAD is a challenging task for language learners, as it demands a high level of precision in detailing visual elements within time constraints. This media accessibility mode can contribute to the enhancement of integrated language skills and has proven beneficial for vocabulary acquisition and grammar development (see Calduch & Talaván 2018 or Ibáñez & Vermeulen 2021 for representative examples). Like didactic SDH, DAD might also contribute to raise awareness among language learners about accessibility issues (Ogea-Pozo 2022).

Didactic free commentary

Didactic free commentary (DFC) asks learners to transfer and adapt the original dialogue (if present) and deliver the visual information in an L2, by incorporating additions or clarifications of intercultural elements. As in the case of DAD, learners should previously prepare the script in L2 and then record it while synchronizing it with the visuals.

As with DAD, the quintessential type of transfer is intersemiotic, plus intralingual and interlingual reverse, since learners should translate the images into words. In particular, in DFC, learners not only transfer or translate into L2 the L2 or the L1 original spoken text, but they are also expected to enrich the final audio track with relevant information about intercultural aspects. In all cases, the main aim is not to be especially faithful to the images and the original soundtrack but to convey the core message by adding more relevant information and omitting unnecessary details. Overall, DFC provides learners with a greater degree of freedom compared to other DAT modes, as they can focus on certain elements at their own discretion. Creative DFC is another possible task in which learners can create a completely new script inspired from the visual clues, ideally for humorous purposes.

Free commentary is a relatively less frequent AVT mode and is primarily featured in documentaries and children’s programs. Only recently, researchers and teachers have started showing interest in applying this mode to language education due to its flexible nature: the primary goal in DFC is to convey the core message rather than strictly adhering to the source text and learners can focus on inter(cultural) elements either freely or in a guided way. These inherent features allow for employing DFC at all educational levels including infant and primary education (Lertola 2021).

Research potential

DAT has traditionally been associated with language education and translator training. As seen in the literature, scholars have always emphasized the benefits of using DAT, arguing that, when used effectively, revoicing and subtitling activities contribute to the development of both productive and receptive communicative skills, as well as vocabulary acquisition, intercultural awareness, and motivation. Although the existing research has provided substantial evidence of the beneficial effects of DAT in language education and translator training, there is still room for further research, and there are several areas that have not been sufficiently examined just yet.

To begin with, there is an urgent need for more long-term mixed-method research designs that combine quantitative and qualitative data, since such studies would contribute to ensuring DAT methodologies are robust and empirically sustainable. Also, combining literature with film and media in the language classroom has great potential in relation to the field of transmedia studies. Similarly, the application of DAT to minority languages seems to be very promising and so requires further research. Although there are studies on DAT and fluency, phonetics and intonation continue to be oral production elements that deserve more attention. A similar argument could apply to less studied DAT modes, such as videogame localization or respeaking, that are still under-explored. Other areas that have merely been mentioned but remain unresearched to date are the application of DAT to visually or hearing-impaired language learners, the possibilities of DAT adaptation to vertical videos produced by the learners themselves via social media video sharing, or applications to other less related areas, such as psychology or healthcare, where DAT may be used in therapy (see Dulcinea project).

Furthermore, DAT can be used in primary and secondary education, but there is not much research in the potential of subtitling and revoicing as teaching resources in English language learning for children (Fernández-Costales 2021; Nicora 2022); further attention should probably be given to the application of DAT to bilingual education contexts, especially in terms of Content and Language Integrated Learning (CLIL) in primary and secondary education schools, as well as through English-Medium Instruction (EMI) programs in higher education. Training language instructors how to design and implement tasks is of utmost importance, provided that when integrating video materials in language education contexts, many teachers may not be familiar with AVT. Hence, DAT teacher training, one of the most needed research avenues within the area at the moment (Lertola & Talaván 2022; Sánchez-Requena, Igareda & Bobadilla-Pérez 2022), should explain instructors the foundations of the various AVT modes and also provide technical guidance and assistance on the design of pedagogically sound DAT tasks.

Exploring a young yet well-established discipline such as DAT becomes particularly useful due to its interdisciplinary nature, which presents a fertile ground for further investigation. The convergence of insights from various fields not only expands the scope of research but also holds considerable pedagogical potential in the realm of language education. Delving deeper into DAT becomes crucial for assisting language learners in developing comprehensive language proficiency that may enable them to perform effective communication in real-world situations.