9/2/2002
Introduction to Neural
Networks
By Lior Elazary
In 1943 McCulloch and Pitts proposed a simple mathematical model of a single neuron, getting the inspiration from a real biological neuron. A biological neuron takes information from other neuron, processes the information and then passes it to other neurons to process. This network of neurons seems to be the fundamental cause of the brain’s ability to perform tasks such as pattern recognition, learning, focusing attention, motion control, etc. It is estimated that the cerebral cortex in a human being contains about 30 billion neurons and 1 million billion connections. If we could count one synapse per second (connection) we would not finish counting for 32 million years.
THE NEURON:
A simple biological neuron consists of a cell body (soma), which performs the calculations, a number of inputs (dendrites), which receives the calculations, and one or more outputs (axon), which passes the information to other neurons. Note that this is a proposed structure of how a simple neuron works, and that there are still many other unanswered questions of how neurons actually work.
Dendrites (input) cell body (soma) Axon
Here is the proposed mathematical structure.
Dendrites (input) cell body (soma) Axon
I1
W1
I2 W2 
Wn
In
The function of the neuron is to take the sum of the inputs multiply by their weights and then pass it through a limiting function. The limiting function could be as simple as a threshold if statement or a sigmoid function. So the output of the neuron can be computed with the following function:
F(x)=1 if
> θ or 0 
Where k is a positive constant that as it approaches ∞ will produce the binary threshold function.
CONNECTION:
The neurons can be connected in many ways to each other. One of the most common ways is a feedforward network. In a feed forward network, input neurons are connected to hidden neurons (other neurons that don’t connect to anything physical in the world), and hidden neurons are connected to output neurons. If the input neurons are directly connected to the output neurons, then the networks will be able to differentiate between linearly separable functions. If hidden layers are added, then the network can differentiate between more complex functions.
INPUT:
HIDDEN:
OUTPUT:
LEARNING:
The network can then learn to recognize simple patterns by modifying the appropriate weights of the network to achieve the desired mapping between the input and the output.
Several algorithms have been developed for this purpose. One of them is backpropagation, which uses the error (the difference between the desired output and the actual output) to propagate back through the network. For a simple two layer network (input, and output) the learning can be computed by the following formula.
USES:
Neural networks have been used successfully in many applications where pattern recognition is requires. These applications include speech recognition, vision processing, stock market prediction, data filtering, robotics applications and many more.
A simple robot based on a neural network.
This robot is based on a robot called “ALDER” by Ulrich Nehmzow. The robot consists of a simple network with two input and two outputs. (Figure 1)
The inputs are the left and right sensors which return a 1 if there is smoothing within a few inches for them or 0 if not. The two output sensors correspond for a left turn and a right turn. The robot main function is to keep the sensors quite. If any of the sensors trigger then the robot passed the values through the network to compute the output. The output with the larger value is tried first and the sensors are once again checked. If the sensors are still triggered then the robot chooses the output neuron with the next largest value and tries that for an extra time. If the sensors are at rest then the network learns the motor output that is required with the sensor input. Thus the robot learns to map its sensors value to corresponding motor outputs.
Wlwlt Wlwrt Wrwlt Wrwrt
