Dual Coding Theory

Dual Coding Theory is probably one of the most fundamental theories for multimedia learning. The basic idea is that visual and verbal information is processed differently along distinct channels in working memory.

The founder of this theory, Allen Paivio, calls this mental representation of visual information as imagines and mental representation of verbal information as logogens.

Visual imagens consist of objects in a particular environment. Spatial information such as 3D layout is also the part of imagens. Logogens store language information and closely related to understanding and logical thought. Although they use separate channels, Logogens and Imagens can be strongly interlinked just as a word “flower” can bring up the appearance of a flower in mind.

Paivio explains further that it is because visual information is in analogue code while verbal information is in symbolic code. Analogue codes represent the physical stimuli we observe in our environment, such as trees, for instance. On the other hand, symbolic codes are arbitrarily chosen (such as numbers), as opposed to perceptually. Due to the differing nature of the visual and verbal inputs, the human mind creates separate representations for information processed in each channel.

< Information Visualization> by Colin Ware, Figure 9.1

The practical application for this theory is for example, recommending audio narration in informative science animations instead of using subtitles. When viewers need to read subtitles, they tend to miss the rest of the visual input (Imagine you are watching a movie with subtitles,, hard to focus on the actors, isn’t it). This effect is called Split-attention theory. When multiple visual inputs are used for various types of information within the same display, viewers will experience overload in processing the information. Therefore, distributing the amount of information through 2 different channels can help the learning process.

Another good example is a pitch presentation. We find successful presentations (e.g. Sorry, I have to mention Steve Jobs again here) are well synchronized with visual aids like a stage play or TV show. We hear Steve Jobs planned and rehearsed everything including gestures. The key to his successful presentations is, I think, the ability to moderate the amount of information given and to direct viewers’ attention to where he wanted.

Supporting evidence for the Dual Coding Theory comes from a variety of fields including psychology and neuroscience. It has been applied to interface design, the development of educational materials and more. Furthermore, Dual Coding Theory has implications for cognitive sciences.

References

• Information Visualization, Ware 2004
• A Split-Attention Effect in Multimedia Learning: Evidence for Dual processing Systems in Working Memory,  Mayer and Moreno 1998
• Wiki