Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2009 Dec;258(1-2):4-15.
doi: 10.1016/j.heares.2009.03.012. Epub 2009 Apr 2.

The neural basis of multisensory integration in the midbrain: its organization and maturation

Barry E Stein  1 , Terrence R Stanford, Benjamin A Rowland
Affiliations
Review

The neural basis of multisensory integration in the midbrain: its organization and maturation

Barry E Stein et al. Hear Res. 2009 Dec.

Abstract

Multisensory integration describes a process by which information from different sensory systems is combined to influence perception, decisions, and overt behavior. Despite a widespread appreciation of its utility in the adult, its developmental antecedents have received relatively little attention. Here we review what is known about the development of multisensory integration, with a focus on the circuitry and experiential antecedents of its development in the model system of the multisensory (i.e., deep) layers of the superior colliculus. Of particular interest here are two sets of experimental observations: (1) cortical influences appear essential for multisensory integration in the SC, and (2) postnatal experience guides its maturation. The current belief is that the experience normally gained during early life is instantiated in the cortico-SC projection, and that this is the primary route by which ecological pressures adapt SC multisensory integration to the particular environment in which it will be used.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Multisensory enhancement. This multisensory neuron responds to visual and auditory stimuli and exhibits a significant overlap in the visual and auditory receptive fields (shaded regions, top). Responses were recorded to visual, auditory, and spatiotemporally concordant (see maps and stimulus traces on top) visual-auditory stimulus pairs. The effectiveness level of the visual stimulus was altered by adjusting its intensity (effectiveness increases from top to bottom). The impulse rasters, peri-stimulus time histograms, and summary figures showing mean response magnitudes are provided for each stimulus type (visual=white, auditory=gray, multisensory=black). When stimulus effectiveness levels are low, the multisensory responses evidence relatively large enhancements, larger than the sum of the unisensory responses (superadditive). However, as stimulus effectiveness levels are increased, the proportional response enhancement evidenced by the multisensory response decreases, and at the highest levels of effectiveness is not distinguishable from the sum of the unisensory responses. This phenomenon is known as “inverse effectiveness.” From Alvarado et al., 2007b.
Figure 2
Figure 2
The temporal profile of multisensory enhancement. Left: Impulse rasters illustrating the responses of a multisensory SC neuron to visual (V), auditory (A), or combined visual-auditory (VA) stimulation. Right: Two different measures of the response show the same basic principle of initial response enhancement: multisensory responses appear enhanced from their very onset (i.e., as soon as inputs arrive). On top the measure is the mean stimulus-driven cumulative impulse count (qsum), on the bottom it is an instantaneous measure of response efficacy referred to as the event estimate. From Rowland and Stein 2008.
Figure 3
Figure 3
Comparisons between multisensory and unisensory integration in physiology. A: The magnitude of response evoked by a cross-modal stimulus (y-axis) is plotted against the magnitude of the largest response evoked by the component unisensory stimuli when presented alone (x-axis). Most of the observations evidence multisensory enhancement (positive deviation from the solid line of unity). B: The same cannot be said for response magnitudes evoked by two within-modal stimuli. Here, the evoked response is typically not statistically better than the largest evoked by one of the component stimuli individually. This observation is consistent for both multisensory and unisensory neurons (insets on right). From Alvarado et al. 2007b.
Figure 4
Figure 4
Comparisons between multisensory and unisensory integration in behavior. Animals were trained in a spatial localization task and their localization accuracy was tested with brief (50ms) visual and auditory stimuli. There were two potential visual targets (V1 and V2) at each location. They were presented either alone, individually with the auditory stimulus at the same location to assay multisensory integration, or together to assay unisensory integration. A: The mean accuracy of the animals in localizing the auditory (A1) stimulus, the visual stimuli at each location (V1 and V2), the visual stimuli when simultaneously presented (V1V2), and conditions where a visual stimulus was paired with the auditory stimulus (V1A1). While both multisensory and unisensory integration yielded enhancements, multisensory integration was far more efficacious. B: Responses are collapsed across locations and decomposed by type: correct responses (C), no-go responses (NG) where the animal did not move from the starting position, and wrong responses (W) where the animal oriented to the wrong location. While multisensory integration yields robust decreases in both types of errors, unisensory integration decreased only the numbers of NG errors. C: Multisensory and unisensory integration follow parallel but significantly trends of “inverse effectiveness.” Each point on this plot of % enhancement vs. the best unisensory accuracy represents a different test location. From Gingras et al., 2009. In Press.
Figure 5
Figure 5
Multisensory integration in the SC depends on association cortex. SC responses to auditory (A), visual (V), and multisensory (AV) stimuli are recorded before (left) and after (right) the deactivation of association cortex. The visual stimulus is tested at multiple (5) levels of effectiveness. The graphs at the top of the figure provide stimulus traces, impulse rasters, and peri-stimulus time histograms for each response. The bottom graphs provide summaries of the mean response levels (lines) and the percent multisensory enhancement (bars) observed for each of the stimulus pairings. Prior to the deactivation of cortex, the responses evidence the characteristic “inverse effectiveness,” with larger unisensory responses associated with smaller multisensory enhancements. However, after cortical deactivation (shaded region of inset), multisensory enhancements are significantly attenuated and even eliminated at each of the stimulus effectiveness levels tested. From Jiang, et al., 2001.
Figure 6
Figure 6
SC neurons receive converging input from different sensory subregions of association cortex. Neurons in the auditory (FAES) and somatosensory (SIV) subregions of the anterior ectosylvian sulcus were injected with florescent tracers to identify their projections into the deep layers of the SC. It was found that these inputs often converged and crossed one another in the SC, with areas of thickening representing presumptive locations of contact (arrows). From Fuentes et al., 2008.
Figure 7
Figure 7
Dark-rearing prevents the normal development of multisensory integration and the normal contraction of SC RFs. Illustrative receptive field overlap is shown for one neuron (upper) as is its lack of multisensory integration (8% change, NS, t-test). Population analyses showed that although multisensory neurons were common in these animals (pie chart, lower left) evidence of multisensory integration was very rare (4% vs. 81% in normals). The effect of dark rearing on RF size (they contract very little during development) is shown in the lower right where visual, auditory and somatosensory RF sizes are compared to those in normal populations. Adapted from Wallace et al., 2004a.
Figure 8
Figure 8
SC sampling and RF overlap in normal and V-A spatial disparity reared animals. Top: Normal sampling of all SC quadrants yielded visual (red) and auditory (green) receptive field (RF) center distributions. Bottom: % RF overlap in control animals (yellow) was often 91–100%; far exceeding that seen in spatial-disparity reared animals (red, often <10%). The physiology of a characteristic neuron is shown in Figure 9. Adapted from Wallace and Stein 2007.
Figure 9
Figure 9
Rearing with visual-auditory spatial disparity yields anomalous multisensory integration: When the visual (red) and auditory (white) stimuli are spatially coincident in either the visual (blue) or auditory (green) RF of this typical neuron (top and middle), they produce no multisensory integration. But when spatially disparate and in their RFs (bottom) they produce significant enhancement - a striking reversal of the normal condition, but one consistent with the animal’s abnormal multisensory experience. Adapted from Wallace and Stein 2007.
Figure 10
Figure 10
Reversible cortical deactivation using muscimol-impregnated Elvax. AES and rLS were unilaterally deactivated in these animals during a period of early life (postnatal weeks 4–12) during which multisensory integration normally develops in the SC. The cortex reactivated after this time. Animals were then assessed behaviorally in a spatial localization task (top) or physiologically (bottom) at one year of age. Animals show normal multisensory enhancement in their localization of visual (V) or multisensory (VA) stimuli on the side of space ipsilateral to the deactivation (green), but significantly attenuated enhancement on the contralateral side (red). The auditory stimulus was neutral in this task and did not evoke responses on its own. Similarly, a sample of neurons from the ipsilateral SC of these animals shows virtually no multisensory enhancement (bottom left), in contrast to a control sample (bottom right).
See this image and copyright information in PMC

References

    1. Alvarado JC, Stanford TR, Vaughan JW, Stein BE. Cortex mediates multisensory but not unisensory integration in superior colliculus. J Neurosci. 2007a;27:12775–12786. - PMC - PubMed
    1. Alvarado JC, Vaughan JW, Stanford TR, Stein BE. Multisensory versus unisensory integration: contrasting modes in the superior colliculus. J Neurophysiol. 2007b;97:3193–3205. - PubMed
    1. Barth DS, Brett-Green B. Multisensory -evoked potentials in rat cortex. In: Calvert GA, Spence C, Stein BE, editors. The Handbook of Multisensory Processes. Cambridge, MA: MIT Press; 2004. pp. 357–370.
    1. Bernstein LE, Auer ET, Moore JK. Audiovisual speech binding: convergence or association. In: Calvert GA, Spence C, Stein BE, editors. The Handbook of Multisensory Processes. Cambridge, MA: MIT Press; 2004. pp. 203–224.
    1. Bolognini N, Maravita A. Proprioceptive alignment of visual and somatosensory maps in the posterior parietal cortex. Curr Biol. 2007;17:1890–1895. - PubMed

Publication types