Supervised Learning - Prediction Week 3 Challenge

import numpy as np
import math


def gradient_descent (x,y):
    
   #slope and intercept
   m_curr = b_curr = 0
   mse_prev = 0
   iterations = 1000
   n=len(x)
   learning_rate = 0.001
   for i in range (iterations):
      # Equ of line
      y_predicted = m_curr*x + b_curr
      # cost function
      mse = (1/n)* sum([val**2 for val in (y-y_predicted)])
      # slope dervidative
      md = -(2/n)*sum(x*(y-y_predicted))
      # intercept derivative
      bd = -(2/n)*sum(y-y_predicted)
      #Update new slope and intercept
      m_curr = m_curr - learning_rate * md
      b_curr = b_curr - learning_rate * bd
      
      if (math.isclose(mse,mse_prev)):
        bestFit += 1
      else:
        bestFit = 0
      
      if (bestFit == 5):
        print ("Best fit m_curr {} , b_curr {}, mse {}".format(m_curr,b_curr,mse))
        break
      mse_prev = mse

#area of house in sq.ft     
x=np.array([1800, 1200,800, 3600, 2400, 5000,2000 ])
#cost
y=np.array([970000, 450000, 203000,1374200,1205000,2120000,115000])
    
gradient_descent (x,y)      

2. L1 and L2 are one of the methods to find the balanced fit models.

L1 and L2 models reduces the variance of dataset by slightly increasing the bias.

L1 and L2 shrink slope and other variables by change the λ.

The main difference is that L1 can shrink slope and other variables to 0, but L2 will shrink close to 0.

L1 excludes useless varables and it is better that L2 by reducing varaince.

L2 tends better when maost varibles are usefull.

3. Different loss functions used to calculate regression are

Mean square Error, Mean Absolute error.

4. Learning rate is like a step function the rate at which the gradient decsent reaches local minima
Learning rate will impact two things
i. How fast the algorith learns or how big the step function is to acheive local minima.
ii. Whether the cost/loss function is reduced or not.

5. Linear regression can be evaluted using cost or loss function

Loss function should be as close to zero, which means residual between actual value and predicted value is optimum and the best line of fit is acheived.

6. Coefficient of determination is a measure of varaiation of output variable with respect to Input variables. In other words coefficient of determination is the measure of closeness of line of fit to actual value and its value ranges from 0 to 1. For Eg if coefficient of determination is 0.9 then predeicted value is 90% accurate to actual value.
If number of Input variables increases even if it doesnt impact output variable then coefficient of determination will increase. But Adjusted R-squared penialize the increase in input variables and gives proper coefficient of determination value.