Integrating Flexible Normalization into Midlevel Representations of Deep Convolutional Neural Networks

Luis Gonzalo Sánchez Giraldo, Odelia Schwartz

Research output: Contribution to journalArticle

Abstract

Deep convolutional neural networks (CNNs) are becoming increasingly popular models to predict neural responses in visual cortex. However, contextual effects, which are prevalent in neural processing and in perception, are not explicitly handled by current CNNs, including those used for neural prediction. In primary visual cortex, neural responses are modulated by stimuli spatially surrounding the classical receptive field in rich ways. These effects have been modeled with divisive normalization approaches, including flexible models, where spatial normalization is recruited only to the degree that responses from center and surround locations are deemed statistically dependent. We propose a flexible normalization model applied to midlevel representations of deep CNNs as a tractable way to study contextual normalization mechanisms in midlevel cortical areas. This approach captures nontrivial spatial dependencies among midlevel features in CNNs, such as those present in textures and other visual stimuli, that arise from tiling high-order features geometrically. We expect that the proposed approach can make predictions about when spatial normalization might be recruited in midlevel cortical areas. We also expect this approach to be useful as part of the CNN tool kit, therefore going beyond more restrictive fixed forms of normalization.

Original languageEnglish (US)
Pages (from-to)2138-2176
Number of pages39
JournalNeural computation
Volume31
Issue number11
DOIs
StatePublished - Nov 1 2019

Fingerprint

Visual Cortex
Spatial Analysis
Neural Networks
Normalization

ASJC Scopus subject areas

  • Arts and Humanities (miscellaneous)
  • Cognitive Neuroscience

Cite this

Integrating Flexible Normalization into Midlevel Representations of Deep Convolutional Neural Networks. / Giraldo, Luis Gonzalo Sánchez; Schwartz, Odelia.

In: Neural computation, Vol. 31, No. 11, 01.11.2019, p. 2138-2176.

Research output: Contribution to journalArticle

@article{800f05a82e8840f484a2fd57d46ccdb3,
title = "Integrating Flexible Normalization into Midlevel Representations of Deep Convolutional Neural Networks",
abstract = "Deep convolutional neural networks (CNNs) are becoming increasingly popular models to predict neural responses in visual cortex. However, contextual effects, which are prevalent in neural processing and in perception, are not explicitly handled by current CNNs, including those used for neural prediction. In primary visual cortex, neural responses are modulated by stimuli spatially surrounding the classical receptive field in rich ways. These effects have been modeled with divisive normalization approaches, including flexible models, where spatial normalization is recruited only to the degree that responses from center and surround locations are deemed statistically dependent. We propose a flexible normalization model applied to midlevel representations of deep CNNs as a tractable way to study contextual normalization mechanisms in midlevel cortical areas. This approach captures nontrivial spatial dependencies among midlevel features in CNNs, such as those present in textures and other visual stimuli, that arise from tiling high-order features geometrically. We expect that the proposed approach can make predictions about when spatial normalization might be recruited in midlevel cortical areas. We also expect this approach to be useful as part of the CNN tool kit, therefore going beyond more restrictive fixed forms of normalization.",
author = "Giraldo, {Luis Gonzalo S{\'a}nchez} and Odelia Schwartz",
year = "2019",
month = "11",
day = "1",
doi = "10.1162/neco_a_01226",
language = "English (US)",
volume = "31",
pages = "2138--2176",
journal = "Neural Computation",
issn = "0899-7667",
publisher = "MIT Press Journals",
number = "11",

}

TY - JOUR

T1 - Integrating Flexible Normalization into Midlevel Representations of Deep Convolutional Neural Networks

AU - Giraldo, Luis Gonzalo Sánchez

AU - Schwartz, Odelia

PY - 2019/11/1

Y1 - 2019/11/1

N2 - Deep convolutional neural networks (CNNs) are becoming increasingly popular models to predict neural responses in visual cortex. However, contextual effects, which are prevalent in neural processing and in perception, are not explicitly handled by current CNNs, including those used for neural prediction. In primary visual cortex, neural responses are modulated by stimuli spatially surrounding the classical receptive field in rich ways. These effects have been modeled with divisive normalization approaches, including flexible models, where spatial normalization is recruited only to the degree that responses from center and surround locations are deemed statistically dependent. We propose a flexible normalization model applied to midlevel representations of deep CNNs as a tractable way to study contextual normalization mechanisms in midlevel cortical areas. This approach captures nontrivial spatial dependencies among midlevel features in CNNs, such as those present in textures and other visual stimuli, that arise from tiling high-order features geometrically. We expect that the proposed approach can make predictions about when spatial normalization might be recruited in midlevel cortical areas. We also expect this approach to be useful as part of the CNN tool kit, therefore going beyond more restrictive fixed forms of normalization.

AB - Deep convolutional neural networks (CNNs) are becoming increasingly popular models to predict neural responses in visual cortex. However, contextual effects, which are prevalent in neural processing and in perception, are not explicitly handled by current CNNs, including those used for neural prediction. In primary visual cortex, neural responses are modulated by stimuli spatially surrounding the classical receptive field in rich ways. These effects have been modeled with divisive normalization approaches, including flexible models, where spatial normalization is recruited only to the degree that responses from center and surround locations are deemed statistically dependent. We propose a flexible normalization model applied to midlevel representations of deep CNNs as a tractable way to study contextual normalization mechanisms in midlevel cortical areas. This approach captures nontrivial spatial dependencies among midlevel features in CNNs, such as those present in textures and other visual stimuli, that arise from tiling high-order features geometrically. We expect that the proposed approach can make predictions about when spatial normalization might be recruited in midlevel cortical areas. We also expect this approach to be useful as part of the CNN tool kit, therefore going beyond more restrictive fixed forms of normalization.

UR - http://www.scopus.com/inward/record.url?scp=85073576471&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85073576471&partnerID=8YFLogxK

U2 - 10.1162/neco_a_01226

DO - 10.1162/neco_a_01226

M3 - Article

C2 - 31525314

AN - SCOPUS:85073576471

VL - 31

SP - 2138

EP - 2176

JO - Neural Computation

JF - Neural Computation

SN - 0899-7667

IS - 11

ER -