Artificial Neural Networks: Seeing The Forest For The Trees
The number of layers and number of neurons in the hidden layer [of a feedforward neural network] has been selected experimentally, as there is really no easy way of determining these values. It helps, however, to remember that the ANN [Artificial Neural Network] learns by adjusting the weights, so if an ANN contains more neurons and thereby also more weights it can learn more complicated problems. Having too many weights can also be a problem, since learning can be more difficult and there is also a chance that the ANN will learn specific features of the input variables instead of general patterns which can be extrapolated to other data sets. In order for an ANN to accurately classify data not in the training set, this ability to generalise is crucial – without it, the ANN will be unable to distinguish frequencies that it has not been trained with.
from "Neural Networks Made Simple"
Quoted on Tue Sep 20th, 2011