Knowledge acquisition from verbal and pictorial information

The quality of learning inside and outside schools depends largely on knowledge acquisition from text and pictures. There are two ways to enhance this kind of learning: improving the design of the learning material and improving students’ learning activities. Both need sufficiently deep understanding of what is going on in the mind of the individual when learning from text and pictures. In this chapter, we first describe and discuss theoretical approaches to learning from text and pictures: dual coding theory, conjoint processing theory, cognitive theory of multimedia learning, and the integrated model of text and picture comprehension. We then describe a set of design principles for combining text and pictures: the multimedia principle, the coherence principle, the contiguity principle, the modality principle, the redundancy principle, and the structure mapping principle. Afterward, we point out possibilities of improving students’ learning activities by the decomposition of representations into components, by generating connections between representational components or prior knowledge, by generating referential connections between verbal and pictorial elements, by direct or embedded hints, and by feedback. Finally, we describe methods for research on text-picture integration. Instructional material typically includes combinations of written text and pictures, diagrams, and graphs. Whereas text comprehension has received much attention during the last four decades (cf. Gernsbacher, 1990; Graesser, Millis, & Zwaan, 1997; Kintsch, 1998; van Dijk & Kintsch, 1983; see also contributions to this volume by Kendeou & Trevors [Chapter 12], Kirby, Cain, & White [Chapter 14], and Britt & Rouet [Chapter 13]), research on the comprehension of graphics (pictures, diagrams, and graphs) is still at its beginning (cf. Schnotz & Kulhavy, 1994). Former studies focused primarily on the mnemonic function of pictures illustrating a text (Levie & Lentz, 1982; Levin, Anglin, & Carney, 1987), but further research indicated that combining text with pictures also supports comprehension of complex technical devices, when texts and pictures are explanatory, when verbal and pictorial content are related to each other, when verbal and pictorial information are presented closely together in space or time, and when individuals have low prior knowledge about the subject domain but high spatial cognitive abilities (Mayer, 1997, 2001, 2005).