Supervised Learning - Prediction Week 3 Challenge

1.Perform Gradient Descent in Python with any loss function

def gradient_descent(x,y,iteration=30):

    m_current=b_current=0  #intialize m and b 

    learning_rate =0.0004 # step's 

    n=len(x)

    for i in range(iteration):

        y_predict = (m_current*x)+b_current #y=mx+b

        cost = (1/n) * sum( [val**2  for val in ( y-y_predict )]) # mean_squared_error

        m_next = -(2/n)*sum(x*(y-y_predict)) #(∂/∂m) refer above pic

        b_next = -(2/n)*sum(y-y_predict) #(∂/∂b)

        m_current = m_current - (learning_rate *m_next)

        b_current = b_current - (learning_rate *b_next)

        print("m  {}  ,b  {}  ,cost  {}  ,iteration  {} ".format(m_current,b_current,cost,i)


2. Difference between L1 & L2 Gradient descent method

L1 (Lasso) and L2 (Ridge)
L1 > Help in eliminating Feature that are not important
L2 > When dataset doesn't contain outlier L2 can be used and most case it is prefered if dataset contain outlier then L1 is prefered

3. What are the different loss functions for regression
    1. Mean Squared Error
         It is the mean of square difference between actual(y) and predicted(y̅) value     Σ(y-y̅)²/n
    2. Mean Squared Logarithmic Error Loss
         It will reduce difference between predicted and actual value So it will be easy for to understand
    3. Mean Absolute Error Loss
         In previous method we didn't consider outlier's in this we take all the data and find average of absolute difference between actual and predicted

4. What is the importance of learning rate
    Adoptation to the problem , Larger the learning rate it move rapidly and take large steps while smaller learning rate take step in small manner So it require large epoch to reach the result

5. How to evaluate linear regression
     1. R-square( coefficient of determination)/Adjust R-square
        explain the variance of dependent variable(target {price}) by the independent variable(area,location,rooms) in regression model, where in Adjust R-square we consider number of independent variable also
     2. Mean Square Error(MSE)/Root Mean Square Error(RMSE)
         It is the mean of square difference between actual(y) and predicted(y̅) value Σ(y-y̅)²/n , where RMSE is error rate achieved by square root of MSE
     3.Mean Absolute Error Loss
         In previous method we didn't consider outlier's in this we take all the data and find average of absolute difference between actual and predicted

6 .What is the difference between multiple and adjusted coefficient of determination
   The amount of variation in the response variable(target) that can be explained by the predictor variables. while In adjust r-square number of predictor taken into consideration So if you get large difference in both value then our model is overfitted