Attention and emotion

• Dr. Luiz Pessoa, Indiana University, Bloomington, IN

• Dr. Mirtes Garcia Pereira, Laboratory of Neurophysiology of Behavior Federal Fluminense University

• Dr. Leticia Oliveira, Universidade Federal Fluminense, Brasil

The processing of emotion-laden visual stimuli is often proposed to take place in an automatic fashion that takes place independently of top-down factors such as attention and awareness. Studies during the 2000s suggest instead that affective processing is, in many circumstances, under the control of attention.

Prioritization

Emotion helps shape information gathering, such that motivationally relevant items receive heightened attention (Lang and Davis, 2006). But how does emotion depend on attention? In this essay, we consider this problem from a cognitive/affective neuroscience perspective and experimental paradigms that have investigated this question by manipulating attention during the processing of emotion-laden visual items. For related reviews, please consult Adolphs (2008) and Vuilleumier (2005); for the role of awareness, please see Pessoa (2005) and Tsuchiya et al. (2009).

The processing of emotion-laden visual stimuli is often proposed to take place in an automatic fashion, namely in manner that is independent of top-down factors such as attention. More generally, although “automaticity” is a concept that is operationalized in quite different ways across studies in cognitive and social psychology, it can be characterized as involving processing occurring independently of the availability of resources, not affected by intentions and strategies, and not necessarily tied to conscious processing (Posner & Snyder, 1975; Jonides 1981). From a more clinical perspective, cognitive models of anxiety also assume the existence of a prioritized and largely automatic threat-processing system and propose that anxiety is characterized by attentional bias that favor the processing of threat cues – and these biases are proposed to be associated with individual differences in anxiety susceptibility (e.g., Williams et al., 1997; Mogg and Bradley, 1998; see also Bishop, 2007). However, studies in the past ten years or so suggest instead that affective processing is, in many circumstances (and possibly always), under the control of attention.

A host of experimental paradigms have documented the many ways in which the processing of emotion-laden stimuli is prioritized. These include detection, search, interference, masking, and the attentional blink, to name just a few. For instance, during the attentional blink paradigm, subjects are asked to report both a first (T1) and a second (T2) visual target within a stream of distractor items. Detecting a second target is believed to be hampered by the initial T1 processing (due to limited processing resources). Interestingly, an emotional T2 item is better detected than a corresponding neutral one, suggesting that the affective dimension of the item counteracts the “blink” (Anderson and Phelps, 2001; Anderson, 2005). Emotional stimuli also elicit “attentional blinks” themselves, suggesting that their processing is prioritized (and leave fewer resources for other stimuli). For instance, negative arousing pictures capture and hold attention, impairing participants’ ability to perform a simple task on a subsequent target stimulus in a rapid stream of visual stimuli (Most el al., 2005).

Emotional state also influences attention. Whereas negative emotions appear to narrow thought and action repertoires, positive emotions broaden them as well as the scope of attention (Fredrickson & Branigan, 2005). Consistent with this notion, a recent neuroimaging study showed that positive and negative states had opposing influences over perceptual encoding in early visual cortices, with positive states broadening and negative states narrowing the field of attention (Schmitz et al., 2009).

The mechanisms underlying affective prioritization continue to be the target of much research, but are generally believed to be related to increased sensory processing to affective stimuli. Indeed, relative to neutral stimuli, emotional stimuli evoke increased fMRI responses across all of ventral occipito-temporal cortex, including “early”, “intermediate”, and “late” visual areas. For instance, Bradley and colleagues (2003) reported more extensive visual cortex activity when participants viewed emotional, compared to neutral, pictures. Later, Padmala and Pessoa (2008) showed a close link between improvements in behavioral performance and trial-by-trial responses in early visual cortex (including primary visual cortex) during the processing of affectively significant visual items. Increased cortical responses in visual cortex to affective stimuli may be due to modulatory signals from the amygdala, consistent with the existence of efferent projections from this structure that reach many levels of the visual cortex (Amaral et al., 1992). Indeed, patients with amygdala lesions failed to exhibit differential responses in visual cortex when viewing emotional faces (Vuilleumier et al., 2004). It is also of relevance that individuals with lesions in the amygdala do not exhibit a reduced attentional blink for T2 emotional words (Anderson and Phelps, 2001).

Attention is not required for emotional perception

Figure 1: Attention is not required for the processing of emotional stimuli. (A) Paradigm employed to manipulate the spatial focus of attention during the presentation of emotional faces. (B) Activation in the left amygdala to fearful vs. neutral faces irrespective of the focus of attention. Adapted from Neuron 30(3), Vuilleumier P, Armony JL, Driver J, Dolan RJ, Effects of attention and emotion on face processing in the human brain: an event-related fMRI study, 829-41, Copyright (2001), with permission from Elsevier.

Affective stimuli effectively divert processing resources and interfere with performance even when they are task irrelevant (Pessoa and Ungerleider, 2004; Vuilleumier, 2005). For instance, reaction times (RTs) when subjects performed auditory tasks (e.g., word discrimination) were slower when they viewed distractor pictures that were unpleasant relative to neutral ones (Bradley et al., 1996; Buodo et al., 2002). Strikingly, such interference has been observed even when the primary task is a very basic one, such as the detection of a simple visual stimulus (Pereira et al., 2006). The interference effects are not only evident in terms of behavioral performance, but are also manifested physiologically. For example, specific event-related potential (ERP) components due to picture viewing were modulated by emotional content even when the main task involved simply detecting a checkerboard stimulus that was interspersed with picture presentation (Schupp et al., 2003). Together, these studies are often interpreted as supporting the notion that the processing of emotional stimuli is obligatory and that increased processing resources are allocated toward their processing although they are irrelevant to the task at hand.

A stronger argument can be advanced that emotional processing is obligatory based on studies in which the spatial focus of attention is explicitly manipulated – note that in the experiments of the previous paragraph, although task irrelevant, emotional stimuli were fully attended. In a well-known study (Vuilleumier et al., 2001), the attentional focus was manipulated by having subjects maintain central fixation while they were asked to compare either two faces or two houses presented eccentrically. On each trial, subjects either compared the faces to each other or the houses to each other (see Fig. 1). Thus, the focus of attention was varied by having subjects attend to the left and the right of fixation (while ignoring top/bottom stimuli) or above and below fixation (while ignoring left/right stimuli). In each case, they indicated whether the attended stimuli were the same or not. When conditions involving fearful faces were contrasted to those involving neutral ones, differential responses in the amygdala – which is often considered as a “signature” of emotional processing – and visual cortex were not modulated by the focus of attention, consistent with the view that the processing of emotional items does not require attention. Similar findings for amygdala automaticity in the processing of fear have been reported by manipulating object-based attention while maintaining the spatial locus of attention constant (Anderson et al., 2003). Responses evoked by fearful faces were equivalent whether or not the faces were attended. Interestingly, however, during unattended conditions, responses evoked by fear and disgust faces were comparable to each other, consistent with the idea that attention is needed for discrimination of emotional content (both expressions evoked greater responses than neutral faces). We now turn to reviewing evidence that suggests that emotional perception requires attention more generally.

Attention is required for emotional perception

Figure 2: Attention is required for the processing of emotional stimuli. (A) Paradigm employed to manipulate the spatial focus of attention. During the attended condition, participants indicated whether faces were male or female. During the unattended condition, subjects indicated whether the peripheral bars were like oriented or not. Stimuli are not drawn to scale. The dashed ellipses indicate the locus of attention, but were not present in the actual experiment. (B) The left map illustrates the responses to the contrast attended vs. unattended when fearful faces were shown, illustrating a robust modulation of amygdala responses by attention (see circles). The panel on the right illustrates evoked responses in the left amygdala as a function of facial expression and attention, showing that differential responses were only observed during the attended condition. ATT: attended; UNATT: unattended. Adapted with permission from the National Academy of Sciences: Pessoa et al. (2002), copyright (2002).

The argument so far can be summarized as follows. Emotional stimuli comprise a privileged stimulus category that is not only prioritized but their processing also takes place in an obligatory fashion that is independent of attention. However, it is also known that, in general, visual processing capacity is limited. Because of this finite capacity, competition among visual items is proposed to “select” the most important information at any given time (Grossberg, 1980; Desimone and Duncan, 1995). When resources are not fully consumed, it has been suggested that spare processing capacity is utilized for the processing of unattended items (Lavie, 1995). This line of reasoning, which has been applied to regular, non-emotional stimuli, suggests that the automaticity of affective processing can be tested by attentional manipulations that more fully consume available processing resources. Thus, in this context, a critical variable in understanding the extent of unattended processing is the attentional load of a task – namely, the extent to which it uses up resources.

Several fMRI studies have attempted to follow the strategy outlined above. For example, the responses evoked by centrally presented emotional faces were evaluated when a very demanding peripheral task was performed. Under these conditions, differential responses to fearful vs. neutral faces were eliminated in both the amygdala and visual cortex (Pessoa et al., 2002). Consistent with the notion that task load was important in determining the extent of processing of the face stimuli, when the difficulty of the peripheral task was parametrically manipulated, a valence effect (i.e,, fearful > neutral) was observed during low task demand conditions, but not during medium or high demand conditions (Pessoa et al, 2005). The dependence of emotional perception on attention was also observed in studies that employed centrally presented, overlapping competing stimuli (i.e., paradigms that manipulate object-based attention), (Mitchell et al., 2007), including emotional stimuli of higher affective significance that were paired with shock (Lim et al., 2008) – or by using highly aversive, mutilation pictures (Erthal et al., 2005). Furthermore, attentional modulation of emotional perception has also been observed for peripherally presented faces (Silvert et al., 2007), a class of stimuli that may be especially effective at engaging the amygdala given that this structure may be sensitive to low spatial frequency information (Vuilleumier et al., 2003).

ERP studies, which unlike fMRI studies offer temporal information on the order of milliseconds, have also investigated how emotional perception depends on attentional factors. In one study (Schupp et al., 2007), the processing of emotional pictures from the International Affective Picture System (IAPS) was strongly attenuated (as measured by an ERP component labeled the “EPN”) when participants performed demanding attention tasks. On the contrary, passively viewing the same emotional images generated enlarged evoked responses relative to those evoked by neutral stimuli. Likewise, differential ERP responses to peripheral, emotional IAPS pictures also relies on the availability of processing resources (De Cesarei et al., 2009). Emotional pictures in the near periphery modulated brain activity only when they were attended (though passively viewed), but not when participants were engaged in a distractor task (determining whether a rectangular outline contained a gap or not).

In closing this section, it is worth pointing out that the prioritization of emotional stimuli can occur for multiple reasons. For example, emotional stimuli appear to draw or capture attention (Fox et al., 2001). These stimuli are also associated with an increased difficulty of disengagement (Fox et al., 2002). At times, the former situations are suggested to indicate a more automatic type of processing, while the latter are in line with the notion that attention is required for the processing of emotional stimuli. Generally, however, this distinction is problematic because both types of processes are resource dependent and need attention to occur. For instance, in a non-emotional context, attentional capture linked to abrupt visual onsets can be suppressed when attention is previously focused on a different spatial location (Yantis and Jonides, 1990).

Attention is not required for emotional perception (again)

The results suggesting that emotional perception is automatic or that it depends on attention can be reconciled by making use of the concept of attentional load (Lavie, 1995). When load is low, “spill over” capacity is available to the processing of task-irrelevant emotional stimuli. As load is increased, however, fewer resources will be available and, in the limit, emotional perception will be eliminated. Whereas this framework can be used to explain a broad set of results, some findings appear to resist this explanation. In one study, subjects performed a difficult target-detection task while task irrelevant, though emotionally arousing, pictures were shown in the background (Muller et al., 2008). Despite the difficulty of the task, emotional pictures interfered behaviorally with the main task. Parallel findings were registered in steady-state visual evoked potentials, which were reduced during the presentation of emotional relative to neutral background images (such reduction was suggested to reflect the withdrawal of processing resources from the main task by the emotional distractors).

Another recent MEG study provided evidence for mandatory processing of fearful faces (Fenker et al., in press). Both low- and high-load conditions were investigated in separate experiments. During the high-load condition, the target was defined by a conjunction of features, such as a red-green (vs. blue-yellow) vertical bar. In the low-load condition, subjects determined the orientation of an oriented bar (vertical vs. horizontal) presented in a given color (indicated at the beginning of the block). During the low-load condition, task-irrelevant fearful faces slowed-down reaction time when they were presented in the same visual field as the bar target (relative to when faces were presented in the opposite visual field). To investigate the neural impact of the task-irrelevant faces, the authors probed the so-called N2pc component, which is believed to reflect attentional focusing in visual search. They observed that lateralized fearful faces elicited an N2pc approximately 240-400 ms in contralateral visual cortex. Importantly, the N2pc was observed during conditions of high load, although no behavioral effects were detected.

Summary

Taken together, results from behavioral and neuroimaging methods suggest that while emotional processing is prioritized, in many contexts it depends on processing resources. These findings come from diverse paradigms, including those employing peripheral emotional stimuli and those in which affective and neutral stimuli are spatially separated. In general, the discrepancy between studies suggesting that emotional perception is automatic and those illustrating the dependence on attention is accounted by the concept of attentional load. Thus, to reveal that emotional perception is not immune to the effects of attention, processing resources need to be largely consumed – otherwise, performance will appear to be relatively automatic. Yet, as outlined in the previous section, this account may not explain all cases, and there may be circumstances in which more true automaticity is observed. At present, the reasons for this discrepancy are unclear, suggesting that it would be profitable for future studies to tackle this issue more directly. One possibility is that individual differences are important predictors of amygdala sensitivity to emotional stimuli and help explain the impact of emotional stimuli. For instance, studies from the literature on anxiety have revealed that anxious participants exhibit greater interference from threat-related stimuli (e.g., MacLeod et al., 1986). More recent studies have investigated the extent to which amygdala responses to threat-related distractors depends upon individual anxiety levels (Bishop et al., 2004). Whereas low- anxious individuals only showed increased amygdala responses to attended fearful faces, high- anxious individuals showed increased amygdala responses to both attended and unattended threat-related stimuli. These findings suggest that the threat value of a stimulus varies as a function of a participant’s anxiety level, although attention is important even for high-anxious individuals (Fox et al., 2005; Bishop et al., 2007).

References

• Adolphs R. Fear, faces, and the human amygdala (2008). Curr Opin Neurobiol 18(2):166-72.

• Amaral DG, Price JL, Pitkanen A, Carmichael ST (1992) Anatomical organization of the primate amygdaloid complex. In: Anggleton JP, editor. The amygdala: neurobiological aspects of emotion, memory and mental dysfunction. New York: Wiley-Liss; p. 1-66.

• Anderson AK (2005) Affective influences on the attentional dynamics supporting awareness. J Exp Psychol Gen 134(2):258-81.

• Anderson AK, Christoff K, Panitz D, De Rosa E, Gabrieli JD (2003) Neural correlates of the automatic processing of threat facial signals. J Neurosci 23(13):5627-33.

• Anderson AK, Phelps EA (2001) Lesions of the human amygdala impair enhanced perception of emotionally salient events. Nature 411(6835):305-9.

• Bishop SJ (2007) Neurocognitive mechanisms of anxiety: an integrative account. Trends Cogn Sci 11:307-316.

• Bishop SJ, Jenkins R, Lawrence AD (2007) Neural processing of fearful faces: effects of anxiety are gated by perceptual capacity limitations. Cereb Cortex 17:1595-1603.

• Bradley MM, Cuthbert BN, Lang PJ (1996) Picture media and emotion: effects of a sustained affective context. Psychophysiology 33(6):662-70.

• Bradley MM, Sabatinelli D, Lang PJ, Fitzsimmons JR, King W, Desai P (2003) Activation of the visual cortex in motivated attention. Behav Neurosci 117(2):369-80.

• Buodo G, Sarlo M, Palomba D (2002) Attentional resources measured by reaction times highlight differences within pleasant and unpleasant, high arousing stimuli. Motivation and Emotion 26: 123–138.

• De Cesarei A, Codispoti M, Schupp HT. (2009) Peripheral vision and preferential emotion processing. Neuroreport 20(16):1439-43.

• Desimone R, Duncan J (1995) Neural mechanisms of selective attention. Annual Review of Neuroscience 18:193-222.

• Erthal FS, de Oliveira L, Mocaiber I, Pereira MG, Machado-Pinheiro W, Volchan E, Pessoa L (2005) Load-dependent modulation of affective picture processing. Cogn Affect Behav Neurosci 5(4):388-95.

• Fenker DB, Heipertz D, Boehler CN, Schoenfeld MA, Noesselt T, Heinze HJ, Duezel E, Hopf JM (in press) Mandatory processing of irrelevant fearful face features in visual search. J Cogn Neurosci.

• Fox E, Russo R, Bowles R, Dutton K. Do threatening stimuli draw or hold attention in subclinical anxiety? (2001) J Exp Psychol Gen 130:681–700.

• Fox E, Russo R, Georgiou GA (2005) Anxiety modulates the degree of attentive resources required to process emotional faces. Cogn Affect Behav Neurosci 5(4):396-404.

• Fox E, Russo R, Georgiou GA (2005) Anxiety modulates the degree of attentive resources required to process emotional faces. Cogn Affect Behav Neurosci 5(4):396-404.

• Fredrickson, B. L., & Branigan, C. (2005). Positive emotions broaden the scope of attention and thought-action repertoires. Cognition and Emotion 19: 313-332.

• Grossberg S (1980) How does a brain build a cognitive code? Psychol Rev 87:1–51.

• Harris, C. R., & Pashler, H. (2004). Attention and the processing of emotional words and names: Not so special after all. Psychological Science 15: 171-178.

• Jonides, J. (1981). Voluntary versus automatic control over the mind's eye's movement. In J. B. Long & A. D. Baddeley (Eds.), Attention and performance IX (pp. 187-203). Hillsdale, NJ: Erlbaum.

• Lang PJ, Davis M. (2006) Emotion, motivation, and the brain: reflex foundations in animal and human research. Prog Brain Res 156:3-29.

• Lavie, N (1995) Perceptual load as a necessary condition for selective attention. J Exp Psychol Human 21: 451–468.

• Lim SL, Pessoa L. (2008) Affective learning increases sensitivity to graded emotional faces. Emotion 8(1):96-103.

• MacLeod C, Mathews A, Tata P. (1986) Attentional bias in emotional disorders. J Abnorm Psychol 95(1):15-20.

• Mitchell DG, Nakic M, Fridberg D, Kamel N, Pine DS, Blair RJ (2007) The impact of processing load on emotion. Neuroimage 34(3):1299-309.

• Mogg, K., Bradley, B.P., 1998. A cognitive-motivational analysis of anxiety. Behaviour Research and Therapy 36: 809–848.

• Most, S. B., Chun, M. M., Widders, D. M., & Zald, D. H. (2005). Attentional rubbernecking: Attentional capture by threatening distractors induces blindness for targets. Psychonomic Bulletin and Review 12: 654–661.

• Müller MM, Andersen SK, Keil A (2008) Time course of competition for visual processing resources between emotional pictures and foreground task. Cereb Cortex 18(8):1892-9.

• Padmala S, Pessoa L (2008) Affective learning enhances visual detection and responses in primary visual cortex. J Neurosci 28(24):6202-10.

• Pereira MG, Volchan E, de Souza GG, Oliveira L, Campagnoli RR, Pinheiro WM, Pessoa L (2006) Sustained and transient modulation of performance induced by emotional picture viewing. Emotion 6(4):622-34.

• Pessoa L, Ungerleider LG (2004) Neuroimaging studies of attention and the processing of emotion-laden stimuli. Prog Brain Res 144:171-182.

• Pessoa L, McKenna M, Gutierrez E, Ungerleider LG (2002) Neural processing of emotional faces requires attention. Proc Natl Acad Sci 99(17):11458-63.

• Pessoa L, Padmala S, Morland T (2005) Fate of unattended fearful faces in the amygdala is determined by both attentional resources and cognitive modulation. Neuroimage 28(1):249-55.

• Posner, M. I. & Snyder, C. R. R. (1975). Attention and cognitive control.(In R. L. Solso (Ed.), Information processing and cognition: The Loyola symposium (pp. 5585). Hillsdale, NJ: Erlbaum.

• Schmitz TW, De Rosa E, Anderson AK.(2009) Opposing influences of affective state valence on visual cortical encoding. J Neurosci 29(22):7199-207.

• Schupp HT, Junghöfer M, Weike AI, Hamm AO (2003) Attention and emotion: an ERP analysis of facilitated emotional stimulus processing. Neuroreport 14(8):1107-10.

• Schupp HT, Stockburger J, Bublatzky F, Junghöfer M, Weike AI, Hamm AO (2007) Explicit attention interferes with selective emotion processing in human extrastriate cortex. BMC Neurosci 8:16.

• Schupp HT, Stockburger J, Bublatzky F, Junghöfer M, Weike AI, Hamm AO (2008) The selective processing of emotional visual stimuli while detecting auditory targets: an ERP analysis. Brain Res 1230:168-76.

• Silvert L, Lepsien J, Fragopanagos N, Goolsby B, Kiss M, Taylor JG, Raymond JE, Shapiro KL, Eimer M, Nobre AC (2007) Influence of attentional demands on the processing of emotional facial expressions in the amygdala. Neuroimage 38(2):357-66.

• Tsuchiya N, Moradi F, Felsen C, Yamazaki M, Adolphs R. (2009) Intact rapid detection of fearful faces in the absence of the amygdala. Nat Neurosci 12(10):1224-5.

• Vuilleumier P, Armony JL, Driver J, Dolan RJ (2001) Effects of attention and emotion on face processing in the human brain: an event-related fMRI study. Neuron 30(3):829-41.

• Vuilleumier P, Armony JL, Driver J, Dolan RJ (2003) Distinct spatial frequency sensitivities for processing faces and emotional expressions. Nat Neurosci 6(6):624-31.

• Vuilleumier P, Richardson MP, Armony JL, Driver J, Dolan RJ (2004) Distant influences of amygdala lesion on visual cortical activation during emotional face processing. Nat Neurosci 7(11):1271-8.

• Vuilleumier P (2005) How brains beware: neural mechanisms of emotional attention. Trends Cogn Sci 9(12):585-94.

• Williams, J.M.G., Watts, F.N., MacLeod, C., Mathews, A., 1997. Cognitive Psychology and Emotional Disorders. Wiley, Chichester.

• Yantis, S. and Jonides, J. (1990). Abrupt visual onsets and selective attention: Voluntary versus automatic allocation. Journal o f Experimental Psychology: Human Perception and Performance 16(1): 121-134.

Internal references

• Joseph E. LeDoux (2008) Amygdala. Scholarpedia, 3(4):2698.

• Lawrence M. Ward (2008) Attention. Scholarpedia, 3(10):1538.

• Kimron L. Shapiro, Jane Raymond, Karen Arnell (2009) Attentional blink. Scholarpedia, 4(6):3320.

• Valentino Braitenberg (2007) Brain. Scholarpedia, 2(11):2918.

• Paul L. Nunez and Ramesh Srinivasan (2007) Electroencephalogram. Scholarpedia, 2(2):1348.

• William D. Penny and Karl J. Friston (2007) Functional imaging. Scholarpedia, 2(5):1478.

• Seiji Ogawa and Yul-Wan Sung (2007) Functional magnetic resonance imaging. Scholarpedia, 2(10):3105.

• Dale Purves (2009) Neuroscience. Scholarpedia, 4(8):7204.

External links

• Luiz Pessoa's website

Sponsored by: Naotsugu Tsuchiya, California Institute of Technology, Pasadena, CA, USA

Reviewed by: Dr. Rebecca M Todd, Department of Psychology, University of Toronto

Reviewed by: Prof. Steve Most, Department of Psychology, University of Delaware

Reviewed by: Anonymous

Accepted on: 2010-02-03 01:58:28 GMT