Project 2

OBJECTIVE: Perform the gradient descent for a single layer ANN

SOLUTION PROCEDURE:

• We assume the number of features as 4, number of samples as 4 and number of neurons in the hidden layer as 10
• We create the initial values of the weights for the input to hidden layer & hidden to output layer with the help of the numpy random. rand function.
• First step would be the forward pass which is subdivided into two
  • Hidden layer: we determine the z values by the dot product of the input & input to hidden weights & the activation function value by the sigmoid function.
  • Output layer: we determine the z values by the dot product of the activation function value of hidden layer & hidden to output weights & the activation function value by the sigmoid function.
  • Predicted value is the result of the activation function of the output layer
• Second step would be the backward pass to determine the gradients to update the weights
  • Gradient for hidden to output layer weights: we have three terms such as error, derivative of the activation function by the z & derivative of the z by the hidden to output weights are all multiplied together
  • Gradient for input to hidden layer weights: we have five terms such as error, derivative of the activation function (output layer) by the z, derivative of the z by the activation function (hidden layer), derivative of the activation function (hidden layer) by the z and derivative of the z by the hidden to output weights are all multiplied together.
  • The hidden to output layer weights & input to hidden layer weights are updated by adding the respective gradients for every iteration.

CODE & OUTPUT: Jupyter notebook attached

CONCLUSION:

Objective of the project is to perform the gradient descent for a single layer ANN which has been successfully completed & the final values are closer to the target value.