Boundary extension

• Kristin Michod Gagnier, Department of Psychology, University of Delaware, Newark, DE

• Dr. Helene Intraub, Department of Psychology, University of Delaware, Newark, DE

Figure 1: In the top row are two views of the same scene: a close-up (A) and a wider-angle view (B). In the bottom row are drawing made from memory of the close-up (C) and wide-angle (D) photographs. Note that in both cases, observers remembered having seen beyond all four edges of the photographs. (Based on Intraub & Richardson, 1989, Figure 1).

Boundary extension is an error of commission in which people confidently remember seeing a surrounding region of a scene that was not visible in the studied view (Intraub & Richardson, 1989). For example, in Figure 1, when people drew the picture shown in panel A from memory (panel C), they tended not only to complete the cropped trashcans (i.e., make them whole), but also to include information on all four sides of the picture that had not been visible in the photograph. The most interesting aspect of this false memory is that although it is an error with respect to the stimulus, it is actually a very good prediction of the world that did exist just beyond the edges of the original view.

Boundary extension

Figure 2: The right column shows outline tracings of close-up and wider-angle photographs of a scene in which only the object was traced (B and D); the left column shows tracings of the same views in which the background was included (A and C). (Based on Intraub, Gottesman & Bills, 1998, Figure 1.)

Boundary extension does not occur in response to all types of pictures. Instead, it is evoked by pictures that convey scene structure (Intraub, Gottesman & Bills, 1998; Gottesman & Intraub, 2003). For example, Figure 2 shows two views of a traffic cone on the street.

Figure 3: The edges of the photograph form a view-boundary that surrounds the objects and background. The boundaries of the towel (an object boundary) surround the sandal. Boundary extension was elicited by the view boundary, but not the object boundary. (Based on Gottesman & Intraub, 2003, Figure 1.)

In the right column (B & D), the backgrounds of the pictures are blank, whereas in the left column (A & C), stippling in the background suggests the surface of a street. Boundary extension only occurred when the pictures contained a background surface. This effect was not limited to the content of the visual stimuli, but the interpretation of the picture as depicting a scene. For example, if while viewing the pictures with blank backgrounds (panels B & D) people were instructed to imagine the background in response to a verbal description, boundary extension occurred. An important distinction that has been raised is that boundary extension does not occur for all boundaries within a scene, but appears to be limited to the boundaries of the view (Gottesman & Intraub, 2002, 2003). To illustrate, consider the scene depicted in Figure 3. After viewing that scene, people tended to remember having seen a greater expanse of the grassy field around the objects, but not a greater expanse of the towel around the sandal.

Figure 4: The pail is shown within a rectangular view (panel A), a circular view (panel B), an irregular rectangle view (panel C) and an irregular circular view (panel D).(Based on Daniels & Intraub, 2006, Figure 2).

Boundary extension is a robust phenomenon that has been reported across the lifespan. It has been observed in children as young as 6 years old (Candel, Merckelbach, Houben & Vandyck, 2004; Chapman, Ropar, Mitchell & Ackroyd 2005; Seamon, Schlegel, Heister, Landau, & Blumenthal, 2002) and in adults as old as 84 years of age (Seamon, Schlegel, Heister, Landau, & Blumenthal, 2002). Children with Asperger’s Syndrome showed the same patterns of error as other children who were tested (Chapman, et al., 2005). Patterns of preference in infant looking behavior suggest that infants as young as 3 months old remember seeing beyond the edges of a close-up (see method sections for details: Quinn & Intraub, 2007). In terms of stimulus constraints, boundary extension is not limited to rectangular views (the typical format of photographs). It occurs whether the shape of the view is rectangular, circular or irregular (Daniels & Intraub, 2006; see Figure 4). This adds support to the hypothesis that boundary extension may help viewers to maintain a coherent representation of the world in spite various types of obstructions (e.g., looking through an opening in foliage, or through the mouth of a cave, or through a typical window).

Figure 5: Visual exploration (A) of a scene through a window-like aperture, and haptic exploration (B) of the same scene through a window of the same size.

Indeed, boundary extension is elicited by views of the real 3D world through a window-like aperture (Figure 5, panel A). In contrast to viewing a photograph, under these conditions, viewers have the benefit of parallax [1] and stereopsis [2], as well as seeing normal-sized objects that are within grasping distance. Yet, minutes after studying the scenes (as in panel A), viewers increased the size of the window dramatically, revealing a greater expanse of the background. Across scenes, the mean area increase ranged from 28%-94% (Intraub, 2004). Importantly, boundary extension was not limited to vision. When participants were blindfolded and explored the same 3D scenes with their hands (a form of haptic exploration [3]; see Figure 5, panel B), they remembered having felt beyond the edges of the window. On average, they increased the area by 10-30%. Boundary extension without a visual input does not appear to be mediated by visual imagery, because an individual with Leber’s Syndrome [4], who had been deaf and blind since early life, showed the same error (Intraub, 2004). That is, she too remembered having felt a greater expanse of the scene than she had actually explored within the boundaries of the window.

Methods for Assessing Boundary Extension

It is critical to determine if performance in any memory test reflects an underlying aspect of mental representation or is an artifact of a given method. Therefore, several tests have been used to assess spatial memory for scenes, and these have provided converging evidence for boundary extension. Boundary extension occurs in drawing tasks (free recall), reconstruction tests and in recognition/rating tasks. Other tests include a border-adjustment task using computer graphics (adjusting a “virtual” window to reveal more or less of a scene; Intraub, Hoffman, Wetherhold & Stoehs, 2006), and a loom-zoom technique in which the viewing area is maintained and the picture is expanded or contracted to reveal more or less of the scene (Chapman et al 2005).

Each type of test has advantages and disadvantages. The benefit of drawing is that it allows for free expression, unconstrained by the experimenter. It also provides a rough quantitative assessment of the amount of extended space (e.g., Intraub & Bodamer, 1993). The cons include the length of time it takes to create each sketch (i.e., memory cannot be tested rapidly) and the variability in participants’ artistic ability. Border adjustment tests and loom-zoom tests avoid these problems and allow for a more rapid response than drawings. They also provide a better quantitative assessment the remembered area. However, as the participants make adjustments they are exposed to additional views of the scene and these could affect memory. The recognition/rating task avoids this problem and is best suited for test situations in which rapid, holistic responses are desired. The test, however, is limited to providing a qualitative assessment of the remembered space.

Figure 6: Stimuli for the familiarization/novelty preference test used to test picture memory in infants. The familiarization stimulus (middle), a closer view (top) and a wider view (bottom). (Based on Quinn & Intraub, 2007, Figure 2).

The recognition/rating task, provides viewers with a test picture that is either identical to the studied view, or that differs in that it shows more or less of the scene. The viewers then rate the test pictures on a 5-point scale: “much closer-up (-2)”, “slightly closer-up (-1)”, “the same (0)”, “slightly farther-away (1)” or “much farther-away (2)”. The recognition/ratings task provides two distinctive patterns that are diagnostic of boundary extension:

1. When the stimulus and test pictures are identical close-ups, viewers rate the test picture as “too close-up” compared to the original, thus indicating that the original was remembered with extended boundaries. (As more wide-angle views are presented and tested, the magnitude of the effect decreases, until the wide views reveal little or no directional distortion).

2. When the stimulus and test pictures differ (i.e., the more close-up stimulus followed by a wider-angle test view and vice versa) a rating asymmetry will occur. Although the same pair of pictures is used, when the closer view is presented first (and is remembered with extended boundaries), the wider-angle test view will be rated closer to “same”, than when the wider-angle is the stimulus and close-up is presented at test.

To test boundary extension in infants, a familiarization/novelty preference procedure has been used (Quinn & Intraub, 2007). The test capitalizes on the infant’s preference for novelty and requires three views of the same simple photograph: a “middle” view, a wider-angle and more close-up view (see Figure 6). In the control condition, infants were simultaneously presented with the closest and widest views (Figure 6, top and bottom panels), and exhibited no preference. In the memory condition, infants were familiarized with the middle view (center panel of Figure 6), and then shown the closer and wider views. Unlike the control group, the infants in the memory condition, showed a preference for the close-up. This suggests that the familiarization picture (middle view) was remembered as looking like the wider view (boundary extension), causing the close-up now to appear novel.

A counterintuitive aspect of the boundary extension memory error is that it is greatest under conditions where good memory would be expected. For example, although it occurs in memory for very small stimulus sets (e.g., 3 pictures), it may not be observed if memory is overloaded (e.g., a set larger than approximately 24 pictures). If memory is tested following the presentation of a large set, observers tend to have a poor sense of where the boundaries were and appear to make random errors (inward and outward) in recalling boundary position.

Boundary Extension and Scene-Selective Regions of the Brain

Figure 7: In an fMRI study, when a closer-view was followed later in the sequence by a wider-view of the same scene (left column), attenuation of the neural response occurred in response to the second view in both PPA and RSC (i.e., the red line is lower than the blue line). However, when the order of the stimuli was reversed, in both PPA and RSC, no attenuation occurred (i.e., red line does not differ from the blue line). These responses parallel the behavioral response asymmetry to closer-wider and wider-closer test pairs in the recognition/rating task. (Based on Park, Intraub, Yi, Widders & Chun (2007), Figure 2).

Behavioral research suggested that boundary extension is specific to scene representation. An fMRI study was conducted to determine if boundary extension would cause selective activation of the parahippocampal place area (PPA) and retrosplenial cortex (RSC) which are thought to be scene-selective regions of the cortex (Park, Intraub, Yi, Widders & Chun, 2007). Observers viewed a series of scenes during which each scene would repeat once. In the critical conditions the repetition was not an identical view; a close-up view would later be followed by a wider view of the same scene or vice versa. Observers were simply instructed to remember the scenes. Neural activity can reveal whether two stimuli are treated as the same or different in the brain, in that activity is lower for repeated items as compared to novel items. Patterns of neural attenuation in both the PPA and RSC mirrored the asymmetry obtained in the recognition/rating task (see previous section on methodology) when different views are presented. As shown in Figure 7, when a closer view was followed later by a wider view, the neural response to the wider view decreased (indicating habituation, as in “I’ve seen that before”), however when the wider view was followed by the closer view, the neural response to the second view was just a strong as it was to the first (indicating no habituation, as in “this is a new picture”). No such asymmetry was observed in the lateral occipital cortex (LOC), which is thought to be associated with object recognition; habituation of the neural response occurred irrespective of which view was presented first (as in, “I’ve seen that object before”).

The Possible Role of Boundary Extension during Visual Scanning

The world is continuous but our visual input is not. High acuity is limited to the foveal region and drops off dramatically in the periphery. Because of these constraints, viewers must sample the world with successive movement of their eyes. Each time the eyes move (a saccade), vision is suppressed. Thus, the input to our visual system is a series of discrete snapshots. Yet we have the experience of a coherent and detailed visual world. How the brain creates such a seamless representation is a classic question in perception. It has been suggested that by anticipating space beyond the edges of view, boundary extension may play a role in the integration of successive views. Consistent with this hypothesis, boundary extension occurs when pictures are presented for brief durations that mimic a single fixation (250 ms) or a series of fixation similar to the average fixation frequency of the eye 3 fixations/second (Intraub, Gottesman, Willey & Zuk, 1996). Perhaps more striking, given that errors of commission are thought to occur over relatively long retention intervals in memory, boundary extension occurs when the visual input is disrupted for less than 1/20th of a second -- a duration commensurate with a saccade (Dickinson & Intraub, 2008; Intraub & Dickinson, 2008). This was the case whether the stimulus and test pictures were presented in the same location or appeared on opposite sides of the screen, thus requiring an eye movement. Thus, boundary extension is apparently available in transsaccadic memory, and can survive shifts in attention associated with an eye movement.

Theoretical Explanations of Boundary Extension

There are two different theoretical frameworks that account for boundary extension in two different ways.

Traditional Single-Source Model of Scene Perception. Scene perception is typically approached in terms of a traditional visual information processing model. Representation of the scene is thought of as having a single source (visual input). The visual representation is briefly maintained in a series of very short-term memory buffers (i.e. iconic memory, transsaccadic memory, visual short-term memory, conceptual short-term memory). Memory for the visual input is somewhat impoverished and consequently errors arise. Errors of omission, such as change blindness, (Levin & Simons, 1997; Rensink, O’Regan & Clark, 1997; Simons & Rensink, 2005) occur rapidly because the mental representation in these buffers is not a detailed copy of the original information. Rapid errors of commission such as boundary extension following a 42 ms break in the sensory input (Intraub & Dickinson, 2008), are more difficult to explain. It would require addition of a post-hoc scene-extrapolation process to one or more of these buffers. To address this possibility, a better understanding of the early time-course of boundary extension (i.e., as extrapolation “unfolds”) is needed, and the question of whether or not, at some early point, a veridical (pre-extrapolation) representation exists would need to be answered. A potential pitfall of this modality-specific approach, is that it does not address the possibility of an underlying cause for boundary extension in both the visual and haptic modalities.

A Multi-source Model of Scene Perception. Instead of focusing solely on the visual information, this model takes into account the fact that the world surrounds the observer but can only be explored a part at a time. All scene representation has, at its foundation, a framework of surrounding space. Observers move their eyes, head and body to “fill in” this spatial representation when they explore a scene regardless of modality. According to the multi-source perspective (Intraub & Dickinson, 2008), during scene perception, the sensory input is interpreted within the context of this surrounding spatial framework. Sensory information “fills-in” this representation as one explores a new scene (e.g., through vision or haptics) and also triggers associations and real-world knowledge about the larger context. These and other factors serve to constrain top-down information expected beyond the edges of the view. For example, in vision, amodal continuation [5] of the view beyond the edges of the sensory input contributes highly constrained top-down information to the representation -- supporting the continuation of background surfaces (e.g., Kellman, Yin & Shipley, 1998) and the continuation or completion of any objects that might be cropped by the boundaries (Kanizsa, 1979).

While the stimulus is available, the difference between the currently visible information and the highly expected information beyond the boundaries can be readily discerned by the observer. However, when the sensory input is gone (even in the case of an interruption as brief as 42 ms; Intraub & Dickinson, 2008), what remains is a mental representation that was originally derived from sensory information, constrained top-down continuation, and a semantic context set within a general spatial framework. This representation is “unitary” in the sense that it does not contain “tags” to indicate which portion came from which source. At test, when boundary extension is assessed, viewers must decide which part of this representation was derived from the sensory input alone (e.g., physically seen or touched before). Given this conception of scene perception, a boundary extension task is, in essence, a source monitoring [6] task (Johnson, Hashtroudi, Lindsay, 1993).

The source-monitoring model provides an explanation of why people sometimes misattribute the source of a memory (e.g., mistaking a dream for reality) by suggesting that memories from different sources have qualitatively different characteristics. For example, memory for a perceived event is more likely to have detailed sensory information than memory for an imagined event. Thus, on occasion, a very vivid and detailed event in a dream might be mistakenly attributed to visual perception (i.e., to having actually occurred). According to the multi-source model, boundary extension occurs because observers mistake memory for the highly constrained top down information beyond the edges of the view has having been part of the original sensory information. Thus, instead of the never-seen (or touched) region beyond the edges being rapidly extrapolated post-stimulus, the extrapolated regions, are already part of the representation of the scene prior to the interruption of sensory input. The value of the multi-source model is that it offers a parsimonious explanation for the rapid onset of boundary extension as well as for boundary extension following either visual or haptic perception. It also raises a possible connection between very short-term memory effects and an important model of long-term retention, source monitoring. To test the feasibility of this framework, new spatially based cross-modal studies of boundary extension would need to be undertaken, and it will need to be determined if factors affecting source monitoring, also affect boundary extension.

Implications of Boundary Extension

Scenes cannot be perceived in their entirety all at once. Instead we move our eyes, head and body to accrue specific sensory information. Boundary extension, unlike many other errors of commission (e.g., the misinformation effect; Loftus, Miller, Burns, 1978) appears to be adaptive rather than harmful. This anticipatory error may have the beneficial effect of facilitating the integration of successive views of the world and enhancing our ability to understand scenes as this sampling unfolds. Boundary extension is, in an important sense, an anticipation of surrounding layout that has not yet been explored. By “ignoring” the boundaries of each spurious view and allowing the perceiver to remember having seen beyond the edges of those views, boundary extension may support our understanding of a continuous world, that we can only sample a part at a time.

References

Candel, I., Merckelbach, H., Houben, K. & Vandyck, I. (2004). How children remember neutral and emotional pictures: Boundary extension in children's scene memories. American Journal of Psychology, 117, 249-257.

Chapman, P., Ropar, D., Mitchell, P. & Ackroyd, K. (2005). Understanding boundary extension: Normalization and extension errors in picture memory among adults and boys with and without Asperger's syndrome. Visual Cognition, 12, 1265-1290.

Daniels, K. K. & Intraub, H. (2006). The shape of a view: Are rectilinear views necessary to elicit boundary extension. Visual Cognition, 14, 129-149.

Dickinson, C. A., & Intraub, H. (2008). Transsaccadic representation of layout: What is the time course of boundary extension? Journal of Experimental Psychology: Human Perception and Performance, 34, 543-555.

Intraub, H., and Richardson, M. (1989). Wide-angle memories of close-up scenes. Journal of Experimental Psychology: Learning, Memory and Cognition, 15, 179-187.

Intraub, H., & Hoffman, J.E. (1992). Remembering scenes that were never seen: Reading and visual memory. American Journal of Psychology, 105, 101-114.

Intraub H., & Bodamer, J.L. (1993). Boundary extension: Fundamental aspect of pictorial representation or encoding artifact? Journal of Experimental Psychology: Learning, Memory and Cognition, 19, 1387-1397.

Intraub, H., Gottesman, C.V., Willey, E. V., & Zuk, I.J. (1996). Boundary extension for briefly glimpsed pictures: Do common perceptual processes result in unexpected memory distortions? Journal of Memory and Language, 35, 118-134. (Special Edition, entitled, "Memory Illusions").

Intraub, H., Gottesman, C.V., & Bills, A.J. (1998). Effects of perceiving and imagining scenes on memory for pictures. Journal of Experimental Psychology: Learning, Memory, & Cognition, 24, 186-201.

Intraub, H. (2004). Anticipatory spatial representation in a deaf and blind observer, Cognition, 94, 19-37.

Intraub, H. & Dickinson, C. A. (2008). False memory 1/20th of a second later: What the early onset of boundary extension reveals about perception. Psychological Science, 19, 1007-1014.

Gottesman, C.V. & Intraub, H. (2002). Surface construal and the mental representation of scenes. Journal of Experimental Psychology: Human Perception and Performance, 28, 1-11.

Gottesman, C.V. & Intraub, H. (2003). Constraints on spatial extrapolation in the mental representation of scenes. View-boundaries versus object boundaries. Visual Cognition, 10, 875-893.

Johnson, M.K., Hashtroudi, S., & Lindsay, D.S. (1993). Source monitoring. Psychological Bulletin, 114, 3-28.

Kanizsa, G. (1979). Organization in vision. New York: Praeger.

Kellman, P. J., Yin, C., & Shipley, T. F. (1998). A common mechanism for illusory and occluded object completion. Journal of Experimental Psychology: Human Perception and Performance, 24, 859–869.

Levin, D. T. & Simons, D. J. (1997). Failure to detect changes to attended objects in motion pictures. Psychonomic Bulletin and Review, 4, 501-506.

Loftus, E. F., Miller, D. G., & Burns, H. J. (1978). Semantic integration of verbal information into a visual memory. Journal of Experimental Psychology: Human Learning and Memory, 4, 19-31.

Park, S., Intraub, H., Yi, D-J, Widders, D, & Chun M. M. (2007). Beyond the Edges of a View: Boundary Extension in Human Scene-Selective Visual Cortex, Neuron, 54, 335-342.

Quinn, P.C., & Intraub, H. (2007). Perceiving “outside the box” occurs early in development: Evidence for boundary extension in 3- to 7-month old infants. Child Development, 78, 324-334.

Rensink, R. A,. O'Regan, J. K., & Clark, J. J. (1997). To see or not to see: The need for attention to perceive changes in scenes. Psychological Science, 8, 368-373.

Seamon, J. G., Schlegel, S. E., Hiester, P. M., Landau, S. M. & Blumenthal, B. F. (2002). Misremembering pictured objects: People of all ages demonstrate the boundary extension illusion. American Journal of Psychology, 115, 2, 151-167.

Simons, D. J. & Rensink, R. A. (2005). Change blindness: Past, present, and future. Trends in Cognitive Sciences, 9, 16–20.

Internal references

• Lawrence M. Ward (2008) Attention. Scholarpedia, 3(10):1538.

• Valentino Braitenberg (2007) Brain. Scholarpedia, 2(11):2918.

• Keith Rayner and Monica Castelhano (2007) Eye movements. Scholarpedia, 2(10):3649.

• Seiji Ogawa and Yul-Wan Sung (2007) Functional magnetic resonance imaging. Scholarpedia, 2(10):3105.

• Daniel J. Simons (2007) Inattentional blindness. Scholarpedia, 2(5):3244.

• Howard Eichenbaum (2008) Memory. Scholarpedia, 3(3):1747.

• Rodolfo Llinas (2008) Neuron. Scholarpedia, 3(8):1490.

• John Dowling (2007) Retina. Scholarpedia, 2(12):3487.

See also

False memory, Figure-ground perception, Inattentional blindness, Object recognition

Sponsored by: Eugene M. Izhikevich, Editor-in-Chief of Scholarpedia, the peer-reviewed open-access encyclopedia

Reviewed by: Anonymous

Reviewed by: Daniel J. Simons, Visual Cognition Lab, University of Illionois at Urbana-Champaign, IL

Accepted on: 2009-02-03 18:34:45 GMT