The Brain Is Not A Feed Forward Neural Network
Let me give you an analogy to show how far [three-row feed forward artificial] neural networks were from real brains. Imagine that instead of trying to figure out how a brain worked we were trying to figure out how a digital computer worked. After years of study, we discover that everything in the computer is made of transistors. There are hundreds of millions of transistors in a computer and they are connected together in precise and complex ways. But we don't understand how the computer works or why the transistors are connected the way they are. So one day we decide to connect just a few transistors together to see what happens. Lo and behold we find that as few as three transistors, when connected together in certain way, become an amplifier. A small signal put into one end is magnified on the other end. (Amplifiers in radios and televisions are made using transistors in this fashion.) This is an important discovery, and overnight an industry springs up making transistor radios, televisions, and other electronic appliances using transistor amplifiers. This is all well and good, but it doesn't tell us anything about how the computer works. Even though an amplifier and a computer are both made of transistors, they have almost nothing else in common. In the same way, a real brain and a three-row neural network are built with neurons, but have almost nothing else in common.
from "On Intelligence"
Quoted on Fri Dec 23rd, 2011