Supervised Learning - Classification Week 9 Challenge
1) A neural network is a type of ml model whoch tries to emulate the human brain to solve problems. Basically it is a ml model that can fit lines/curves to complex datasets where linear or logistic models cannot be used.
It does this by passing the input features through node each with each layer containing weight, threshold, activation function. It then combines these layers to get the output.
They are hard to interpret as it contains a lot of functions in them hidden in a "black box".
Types of nn are: Ann, Cnn and Rnn.
2) Deep learning is a branch of ml which is based on artificial neural network.
There we use different interconnected layers called neurons that work together.
In a fully connected Deep neural network, there is an input layer and one or more hidden layers connected one after the other. Each neuron receives input from the previous layer neurons or the input layer. The output of one neuron becomes the input to other neurons in the next layer of the network, and this process continues until the final layer produces the output of the network.
This allows non linear relationships to be captured.
It also gets trained using backpropagation which allows the model to learn from labelled data.
3) An activation function is a mathematical operation applied to the output of each neuron in a neural network. It determines the output of a neuron based on wighted sum. It does the following:
a) Introduces non - linearity to the network so it can learn to represent more complex relationships. Wihtout this it would be a composition of a linear functions which is still a linear function.
b) It facilitates backpropagation used to adjust weights to minimize error by giving gradients.
4) Types of activation functions are:
a) Sigmoid function: Produces values between 0 and 1. It is used for binary classification problems.
f(x)=1/1+e^-x
b) ReLU(rectified linear unit): Gives the input value as output if it is positive else it returns 0. It is used because it is simple and effective.
f(x)=max(0,x)
c) Leaky ReLU: It is similar to relu but this one allows for a small negative value to pass as output preventing dead neurons.
f(x)=max(alpha*x,x)
d) Hyperbolic tan function(tan(h)): Produces values between -1 and 1.
f(x)=(e^x - e^-z)/(e^x + e^-z)
5) Backpropagation is a supervised learning algorithm used to train artificial neural networks. It basically assumes weights for the neurons at first and makes a prediction. Then it uses mean squared error to calculate the error. It then runs this back and makes changes to the weights to each neuron layer by using the chain rule and gradient decent. After updating the weights it then makes a prediction again with the new wights and repeats this process till the error is below a set threshold or the step size is small.