Artificial Neural Network
Artificial Neural Network
Copyright By PowCoder代写 加微信 powcoder
What are artificial neural networks?
Artificial neural networks (ANNs) are simplified models of the human central nervous system.
They are networks of highly interconnected neural computing elements that have the ability to respond to input stimuli and to learn to adapt to the environment.
The building blocks or computing elements are neurons (single nodes in the neural network)
Multilayer perceptron (1)
Multilayer feedforward neural network
with backpropagation learning
Architecture: 1 or 2 hidden layers, no. of nodes per layer
Parameters: learning rate (η) & momentum (α)
Processes at a neuron (1)
Each neuron is a comparator that produces an output when the cumulative effect of the input stimuli exceeds a threshold value
one or more input links, x1, x2, x3.
each input is a positive or negative real value
each input link has a corresponding weight w1, w2, w3, which are positive or negative real values.
The weights, which model the synaptic neural connections in biological nets, act to increase or decrease the input signals to the neuron
Node process 1
combine the inputs usually by summing them: Sum wixi
a threshold value is sometimes applied to the combined input: Sum wixi + threshold. But this can be replaced with an extra input link with an input value of 1 and a weight for the link to provide a bias
Processes at a neuron (2)
Node process 2
apply a function to the combined input to obtain the output.
the neuron uses an activation function to map the input to an output
commonly used functions are the sigmoid (logistic function) and the Tanh (hyperbolic tangent) function
an output which can be a positive or negative real value
Multilayer perceptron (5)
Procedure for backpropagation learning
the weights on the links are initialised to small random values (usually between -1.0 and +1.0, or -0.5 to 0.5)
training cases are presented to the network in some sequence
the input values of a training case are entered in the network, and the input values are propagated forward layer by layer until output values are obtained.
Multilayer perceptron (6)
Procedure for backpropagation learning
the values from the output nodes are compared with the desired or target outputs, and errors are computed:
ei = ti – zi
where zi is the output value of output node i, and ti is the target value
Multilayer perceptron (7)
Procedure for backpropagation learning
Multilayer perceptron (7)
Procedure for backpropagation learning
Multilayer perceptron (6)
Procedure for backpropagation learning
The error is then used in a feedback mechanism to adjust the weights of the links in the network to reduce the error
the adjustments to the weights are propagated backwards from the output to the input layer
the errors computed at the output layer are used to adjust the weights between the output layer and the last hidden layer
likewise an error value computed from the last hidden layer outputs is used to adjust the weights in the next to the last hidden layer, and so on.
in this way, the errors are propagated backwards layer by layer with corrections being make to the corresponding layer weights in an iterative manner.
Potential problems in NN modelling
Overfitting
Use cross validation: Divide the training set into 3 subsets
1 for training,
1 for monitoring the performance of the network and deciding when to stop training, and
the 3rd for final validation
Specifying a multilayer perceptron (1)
no. of input nodes: depends on the number of independent variables
no. of output nodes: depends on the number of dependent variables and categories in the dependent variable
no. of hidden layers: 1 or 2
no. of nodes for each hidden layer: rule of thumb formula: p / (10 (m +n)), where p is the no. of training cases, n & m are the no. of input and output nodes
activation function: usually logistic or tanh
error function: usually mean square error
initial set of weights: random nos in the range [-0.2, 0.2]
Specifying a multilayer perceptron (2)
learning method: Backpropagation, Quickprop or Delta-Bar-Delta
learning rate: start with a larger value (0.9) and gradually reduce it
momentum coefficient: start with a small value (0.1) and increase it
cross-validation: divide the training set into 3 subsets, 1 for training, 1 for monitoring the performance of the network and deciding when to stop training, and the 3rd for final validation
It is desirable to prune the final network by removing noncontributory nodes and links
Data preparation
Scale the input and output values to [-1.0, 1.0] or [0, 1.0]
Convert categorical variables to dummy variables (if the program doesn’t do it automatically)
It is also advisable to introduce some noise into the training samples
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com