Machine Learning I
Machine Learning II
Lecture 3 – NN more in depth
1
1
What is a neural network?
Perceptrons were developed in the 1950s and 1960s by the scientist Frank Rosenblatt, inspired by earlier work by Warren McCulloch and Walter Pitts.
How do perceptrons work?
A perceptron takes several binary inputs, , ,…, and produces a single binary output:
Rosenblatt proposed a simple rule to compute the output. He introduced weights, , ,…, real numbers expressing the importance of the respective inputs to the output.
2
What is a neural network?
The neuron’s output, 0 or 1, is determined by whether the weighted sum is less than or greater than some threshold value.
This can be mathematically shown as:
output=
Example: There is a festival in your city. You want to see if it is a good idea to go to the festival or not? You have some considerations:
Is the weather good?
Does your friend accompany you?
Is the festival near public transit?
3
What is a neural network?
Are all these conditions have equal impact on your decision? Maybe not. How can you show this difference?
Let’s say =6, =2, =2.
Let’s make the threshold 5
output=
By changing the weight and threshold, the decision will be different.
What if the network is more complex?
4
What is a neural network?
Now we can have the same equations on nodes.
We can simplify our notations:
and
output=
5
Example
Sigmoid function
By a small change in bias or weight we may continuously jump between zero and one.
We can overcome this problem by introducing a new type of artificial neuron called a sigmoid neuron. Sigmoid neurons are similar to perceptrons, but modified so that small changes in their weights and bias cause only a small change in their output.
What if I claim, I can estimate this probability by a non linear function as
z large
z small or negative
6
Example: Hand write recognition
How can we train a model that can identify numbers from handwritten document?
How can my brain easily recognize numbers?
What if I split the number to smaller pixels?
7
28
28
Example: Hand write recognition
8
https://gfycat.com
Example: Hand write recognition
9
https://gfycat.com
Activations will let up when they are 1
Example: Hand write recognition
10
https://gfycat.com
Example: Hand write recognition
11
3blue1brown
How do our eyes recognize numbers?
Can we say the second to the last hidden layer, predicts these shapes?
Example: Hand write recognition
12
3blue1brown
By looking at the upper part of input node, the relavant node on hidden layer should be activated
It is common between 8 and 9 so we should find a good combination of two nodes
Example: Hand write recognition
13
3blue1brown
How about recognizing the circle or the straight line?
We can make those through another formula.
= + + +
= + +
Example: Hand write recognition
14
3blue1brown
Do these pictures reflect what is going on first layer?
This breakdown technique is common in many machine learning approaches.
Example: Hand write recognition
15
3blue1brown
What are the parameters that can show the pixels?
Example: Hand write recognition
16
3blue1brown
To make those parameters:
…
Example: Hand write recognition
17
3blue1brown
This sum : () can take any value but remember that we wanted the output to be between zero and one
What is the solution?
We can control some pixels that might be light up by adding a bias parameter
What if we have more nodes?
18
https://Edex.com
Computing the nodes
19
is 5×3
is 3×1
is 5×3
is 3×4
is 4×1
Computing the nodes
20
Computing the nodes
21
Assignment 1
Two sensors are installed in Susquehanna river that can measure the level of the water on two different locations. Their measurements goes through a neural network and the output shows if there is a chance of flood or not (y=1 means there is a chance of flood). Suppose sensor 1 shows 0.4 feet while the second one shows 0.32 feet. Is there any chance of flood?
22
Please use Python to answer to the question.
Please generate Random vectors and matrices (using numpy) for weights and bias. (seed =123)
You must explain your final result.
/docProps/thumbnail.jpeg