Cortex As A Deep Architecture

Depth of architecture refers to the number of levels of composition of non-linear operations in the function learned. Whereas most current learning algorithms correspond to shallow architectures (1, 2 or 3 levels), the mammal brain is organized in a deep architecture (173) with a given input percept represented at multiple levels of abstraction, each level corresponding to a different area of cortex. Humans often describe such concepts in hierarchical ways, with multiple levels of abstraction. The brain also appears to process information through multiple stages of transformation and representation. This is particularly clear in the primate visual system (173), with its sequence of processing stages: detection of edges, primitive shapes, and moving up to gradually more complex visual shapes.


Based on current knowledge of brain anatomy (173), it appears that the cortex can be seen as a deep architecture, with 5–10 levels just for the visual system.


(173) T. Serre, G. Kreiman, M. Kouh, C. Cadieu, U. Knoblich, and T. Poggio, “A quantitative theory of immediate visual recognition,” Progress in Brain Research, Computational Neuroscience: Theoretical Insights into Brain Function, vol. 165, pp. 33–56, 2007.

-- Yoshua Bengio

from "Learning Deep Architectures for AI"

Quoted on Sat Dec 1st, 2012