The effect of bimodal input on implicit and explicit memory: An investigation into the benefits of within-language subtitling (2002)

Stephen A Birds
John N Williams
Research Centre for English and Applied Linguistics
Cambridge, UK

Birds, S. A., & J. N. Williams. (2002). The effect of bimodal input on implicit and explicit memory: An investigation into the benefits of within-language subtitling. Applied Psycholinguistics.

ABSTRACT
Two experiments examined the effect of single modality (sound or text) and bimodal (sound andtext) presentation on word learning, both as measured by improvements in spoken wordrecognition efficiency (long lag repetition priming) and recognition memory. Native and advancednon-native speakers of English were tested. In Experiment 1 auditory lexical decisions on familiarwords were equally primed by prior bimodal and sound-only presentation, whereas there wereno priming effects for nonwords. Experiment 2 employed a rhyme judgement task on nonwords.Repetition priming of auditory rhyme judgement decisions was now obtained, and this wasgreater in the bimodal than the sound-only condition. In both experiments prior bimodalpresentation improved recognition memory for spoken words and nonwords compared to singlemodality presentation. We conclude that simultaneous text presentation can aid novel wordlearning under certain conditions, both as assessed by explicit and implicit memory tests.


Allport, D.A., & Funnell, E. (1981). Components of the mental lexicon. Philosophical Transactions of the Royal Society of London B295, 397-410.

Balota, D.A., & Chumbley, J.I. (1984). Are lexical decisions a good measure of lexical access? The role of word frequency in the neglected decision stage. Journal of Experimental Psychology: Human Perception and Performance, 10, 340–357.

Bird, S.A., & Jacobs, G.M. (1999). An examination of the keyword method: How effective is it for native speakers of Chinese learning English? Asian Journal of English Language Teaching, 9, 75–97.

Borras, I., & Lafayette, R.C. (1994). Effects of multimedia courseware subtitling on the speaking performance of college students of French. The Modern Language Journal, 78, 61–75.

Burnage, G. (1997). Integrating digital video into language learning software. University of Cambridge Faculty of Modern and Medieval Languages Internet Website.

Chung, J. (1996). The effects of using advance organizers and captions to introduce video in the foreign language classroom. TESL Canada Journal, 14(1), 61–65.

Coltheart, M. (1978). Lexical access in simple reading tasks. In G. Underwood (Ed.), Strategies of information processing (pp. 151–216). New York: Academic Press.

Craik, F.I.M., & Lockhart, R.S. (1972). Levels of processing: a framework for memory research. Journal of Verbal Learning and Verbal Behaviour, 11, 671–684.

Craik, F.I.M., & Tulving, E. (1975). Depth of processing and the retention of words in episodic memory. Journal of Experimental Psychology: General, 104(3), 268–294.

Cutler, A., Mehler, J., Norris, D. & Segui, J. 1986. The syllable's differing role in the segmentation of French and English. Journal of Memory and Language, 25, 385-400.

Danan, M. (1992). Reversed subtitling and dual coding theory: New directions for foreign language instruction. Language Learning, 42(4), 497–527.

Davis, R.S. (1998). Captioned video: Making it work for you. The Internet TESL Journal, 4(3).

Dijkstra, T., Schreuder, R., & Frauenfelder, U.H. (1989). Grapheme context effects on phonemicprocessing. Language and Speech, 32(2), 89–108.

Dodd, B., Oerlemans, M., & Robinson, R. (1988). Cross-modal effects in repetition priming: A comparison of lipread, graphic and heard stimuli. Visible Language, 22(1), 58–77.

Donnenwerth-Nolan, S, Tanenhaus, M.K., & Seidenberg, M.S. (1981). Multiple code activationin word recognition: Evidence from rhyme monitoring. Journal of ExperimentalPsychology: Human Learning and Memory, 7(3), 170–180.

Ellis, N.C. (1994). Vocabulary acquisition: The implicit ins and outs of explicit cognitive mediation. In N.C. Ellis (Ed.), Implicit and explicit learning of languages. London: Academic Press.

Ellis, A.W. (1982). Modality-specific repetition priming of auditory word recognition. Current Psychological Research, 2, 123–128.

Feustel, T.C., Shiffrin, R.N., & Salasoo, A. (1983). Episodic and lexical contributions to the repetition effect in word identification. Journal of Experimental Psychology: General,112, 309–346.

Frost, R. (1998). Toward a strong phonological theory of visual word recognition. Psychological Bulletin, 123(1), 71–99.

Frost, R., Repp, B.H., & Katz, L. (1988). Can speech perception be influenced by simultaneous presentation of print? Journal of Memory and Language, 27, 741–755.

Garza, T.J. (1991). Evaluating the use of captioned video materials in advanced foreign language learning. Foreign Language Annals, 24 (3), 239–250.

Gipson, P. (1984). A study of the long-term priming of auditory word recognition. Unpublished doctoral dissertation, University of Cambridge.

Grainger, G., & Ferrand, L. (1996). Masked orthographic and phonological priming in visual word recognition and naming: cross-task comparisons. Journal of Memory and Language,35, 623–647.

Haist, F., Musen, G., & Squire, L.R. (1991). Intact priming of words and nonwords in amnesia. Psychobiology, 19, 275–285.

Holobow, N.E., Lambert, W.E., & Sayegh, L. (1984). Pairing script and dialogue: combinations that show promise for second or foreign language learning. Language Learning, 34(4), 59–74.

Jackson, A., & Morton, J. (1984). Facilitation of auditory word recognition. Memory and Cognition, 12(6), 568–574.

Jacoby, L.L. (1983). Remembering the data: Analyzing interactive processes in reading. Journal of Verbal Learning and Verbal Behaviour, 22, 485–508.

Jacoby, L.L., & Dallas, M. (1981). On the relationship between autobiographical memory and perceptual learning. Journal of Experimental Psychology: General, 3, 306–340.

Jakimik, J., Cole, R.A. & Rudnicky, A.I. (1985). Sound and spelling in spoken word recognition.Journal of Memory and Language, 24, 165–178.

Jung, U. (1990). The challenge of broadcast videotex to applied linguistics. IRAL, 28(3), 201–219.

Kirsner, K., & Dunn, J.C. (1985). The perceptual record: A common factor in repetition priming and attribute retention. In M.I. Posner & O.S.M. Marin (Eds.), Attention and performance XI (pp. 547–565). Hillsdale, N.J.: Erlbaum.

Kirsner, K., Milech, D., & Standen, P. (1983). Common and modality-specific processes in the mental lexicon. Memory and Cognition, 11(6), 621–630.

Kirsner, K., & Smith, M.C. (1974). Modality effects in word identification. Memory and Cognition, 2(4), 637–640.

Lambert, W.E., Boehler, I., & Sidoti, J. (1981). Choosing the language of subtitles and spoken dialogues for media presentations: Implications for second language education. Applied Psycholinguistics, 2, 133–148.

MacLeod, C.M., & Masson, M.E.J. (2000). Repetition priming in speeded word reading:
Contributions of perceptual and conceptual processing episodes. Journal of Memory and
Language, 42, 208–228.

Marslen-Wilson, W. (1989). Access and integration: Projecting sound onto meaning. In M. Marslen-Wilson (Ed.), Lexical representation and process (pp. 25–74). Cambridge, MA: MIT Press.

Masson, M.E.J. (1995). A distributed memory model of semantic priming. Journal of
Experimental Psychology: Learning, Memory, and Cognition, 21(1), 3–23.

McClelland, J.L., & Rumelhart, D.E. (1981). An interactive activation model of context effects in letter perception: Part 1: an account of the basic findings. Psychological Review, 88, 365–407.

Mendelsohn, D. (1994). Learning to listen. San Diego: Dominie Press.

Monsell, S. (1985). Repetition and the lexicon. in A. Ellis (Ed.), Progress in the psychology of language (pp. 147–195). London: Lawrence Erlbaum.

Morris, C.D., Bransford, J.D., & Franks, J.J. (1977). Level of processing versus transfer appropriate processing. Journal of Verbal Learning and Verbal Behaviour, 16, 519–533.

Morton, J. (1979). Facilitation in word recognition: Experiments causing change in the logogen model. In P.A. Kolers, M.E. Wrolstad, & H. Bouma, Processing models of visible language (pp. 259–268). New York: Plenum.

Perfetti, C.A. (1985). Reading ability. Oxford: Oxford University Press.
Price, K. (1983). Closed caption T.V.: an untapped resource. MATSOL Newsletter, 12, 1–8.

Robinson, P. (1995). Attention, memory, and the “noticing” hypothesis. Language Learning, 45(2), 283–331.

Roediger, H.L., & McDermott, K.B. (1993). Implicit memory in normal human subjects. In H. Spinnler & F. Boller (Eds.), Handbook of neuropsychology vol. 8 (pp. 63–130).
Amsterdam: Elsevier.

Scarborough, D.L., Gerard, L., & Cortese, C. (1979). Accessing lexical memory: The transfer of word repetition effects across task and modality. Memory and Cognition, 7, 3–12.

Schacter, D.L. (1992). Priming and multiple memory systems: Perceptual mechanisms of implicit memory. Journal of Cognitive Neuroscience, 4(3), 244–256.

Schneider, V.I., & Healy, A.F. (1993). Detecting phonemes and letters in text: Interactions between different types and levels of processes. Memory and Cognition, 21, 739-751.

Seidenberg, M.S., & Tanenhaus, M.K. (1979) Orthographic effects on rhyme monitoring. Journal of Experimental Psychology: Human Learning and Memory, 5(6), 546–554.

Van Orden, G.C., Pennington, B.F., & Stone, G.O. (1990). Word identification in reading and the promise of subsymbolic psycholinguistics. Psychological Review, 97, 488–522.

Vanderplank, R. (1988). The value of teletext subtitles in language learning. ELT Journal, 42(4), 272–281.

Vanderplank, R. (1990). Paying attention to the words: practical and theoretical problems in watching television programmes with uni-lingual (CEEFAX) sub-titles. System, 18(2), 221–234.

Vanderplank, R. (1993). A very verbal medium: Language learning through closed captions. TESOL Journal, 3 (1), 10–14.

Ziegler, J.C., Montant, M., & Jacobs, A.M. (1997). The feedback consistency effect in lexical decision and naming. Journal of Memory and Language, 37, 533–554.

The Effect of Bimodal Input on Implicit and Explicit Memory an Investigation Into the Benefits of Within-la...