Why would anyone want a `new' sort of computer?
What are (everyday) computer systems good at... .....and not so good at?
Good at Not so good at
Fast arithmetic Interacting with noisy data or data from the environment
Doing precisely what the programmer programs them to do Massive parallelism
Massive parallelism
Fault tolerance
Adapting to circumstances
Where can neural network systems help?
• where we can't formulate an algorithmic solution.
• where we can get lots of examples of the behaviour we require.
• where we need to pick out the structure from existing data.
What is a neural network?
Neural Networks are a different paradigm for computing:
• von Neumann machines are based on the processing/memory abstraction of human information processing.
• neural networks are based on the parallel architecture of animal brains.
Neural networks are a form of multiprocessor computer system, with
• simple processing elements
• a high degree of interconnection
• simple scalar messages
• adaptive interaction between elements
A biological neuron may have as many as 10,000 different inputs, and may send its output (the presence or absence of a short-duration spike) to many other neurons. Neurons are wired up in a 3-dimensional pattern.
Real brains, however, are orders of magnitude more complex than any artificial neural network so far considered.
Example: A simple single unit adaptive network:
The network has 2 inputs, and one output. All are binary. The output is
1 if W0 *I0 + W1 * I1 + Wb > 0
0 if W0 *I0 + W1 * I1 + Wb <= 0
We want it to learn simple OR: output a 1 if either I0 or I1 is 1.
Algorithms and Architectures.
The simple Perceptron:
The network adapts as follows: change the weight by an amount proportional to the difference between the desired output and the actual output.
As an equation:
Δ Wi = η * (D-Y).Ii
where η is the learning rate, D is the desired output, and Y is the actual output.
This is called...