Subtitles and captions in ELT video
Jono Ryan
Being late to the party, I’ve only recently learned the distinction between video subtitles and video captions. It's a distinction that often gets conflated, with many people (and Netflix) using subtitles to refer to both, but it's one that did get me thinking about how the different ways they are utilised in language teaching.
Captioning is the term describing the use of on-screen text to represent dialogue in a way that what you hear and what you read are both in English, for instance. Subtitling, in the narrower sense, is a translation of what is being spoken. Recently, for instance, I saw the wonderful Senegalese film ‘The little girl who sold The Sun’ (La Petite Vendeuse de Soleil), where I read English subtitles, but the dialogue was spoken in Wolof and French.
Both captioning and subtitling have useful and distinctive roles in second language learning, and not always in the most obvious ways. There has been some research comparing the different impacts of the two. Unsurprisingly, subtitles (providing an L1 translation) lead to better comprehension of a video (Pujadas & Muñoz, 2020), but I suspect there’s much more to investigate there in terms of overall effects on language learning.
To give one example, I recall watching an English language film in a small art house cinema in Mexico, with Spanish subtitles (i.e. the opposite scenario from what is usually researched, which is L2 audio, L1 subtitles). On that occasion, I was very struck by the Spanish translation of a particular English expression. I can no longer remember what the expression was, but at the time it was a particularly vivid learning experience. Here was a conversational expression that I felt I desperately needed but it had never come up in any formal classroom learning or self-study materials. Film dialogue provided it for me, and subtitles were the key.
The role of captioning is perhaps more interesting. Some recent studies of English language learning through video have shown that more vocabulary is acquired when the English audio is accompanied by English captions. This also shouldn't come as a surprise. For one thing, in a language with a very complex spelling system such as English, very often the spelling of a word has to be learned as an additional step in the process. Probably more important is that the visual support of captions helps viewers to decode the stream of speech. Relatedly, there is evidence that captions can help learners of syllable-timed languages (such as Spanish) to acquire the stress-timing of English pronunciation (Mora & Cerviño-Povedano, 2019).
On this note, one of initiatives that Chasing Time English is discussing is the use of captioning that is highlights the rhythm of speech. This would be similar to the cueing systems used to represent the rhythm of song lyrics in karaoke and elsewhere, where a moving dot provides a prompt for timing. A variation on such a system could provide effective support for listeners in recognizing word boundaries, illustrating the way sounds blend into each other, and especially the de-stressing of function words (e.g. prepositions and articles).
For teachers looking for a way in which they can use subtitling or captioning in the class, I'd recommend reading Alonso-Pérez (2019). This chapter describes a class project for undergraduate students of Spanish, in which students were tasked with creating the subtitles for a collection of short films that then screened as part of a festival of Spanish films. The subtitling was the main task, but students also wrote a film synopsis and information about the director and actors, and then introduced their film to the audience. With an inspired choice of real-life task, the students’ motivation and responsiveness – which was already high – climbed even higher.
References
Alonso-Pérez, R. (2019). Enhancing student motivation in foreign language learning through film subtitling projects. In C. Herrero & I. Vanderschelden (Eds.), Using film and media in the language classroom: Reflections on research-led teaching (pp. 108-126). Multilingual Matters.
Mora, J. C., & Cerviño-Povedano, E. (2019). The effects of bimodal L2 input on the processing of function words by Spanish EFL learners: An eye-tracking study. In C. Herrero & I. Vanderschelden (Eds.), Using film and media in the language classroom: Reflections on research-led teaching (pp. 76-91). Multilingual Matters.
Pujadas, G., & Muñoz, C. (2020). Examining adolescent EFL learners’ tv viewing comprehension through captions and subtitles. Studies in Second Language Acquisition, 42(3), 551-575. https://doi.org/10.1017/S0272263120000042