Supervised Learning - Prediction Week 3 Challenge
1.Perform Gradient Descent in Python with any loss function
import numpy as np
import pandas as pd
import matplotlib.pyplot as pt
#importing input data
data = pd.read_csv('data.csv')
X = data.iloc[:,0]
Y = data.iloc[:,1]
print(X)
#building a mean square error loss or cost function model
m = 0
c = 0
L = 0.0001 #learing rate
no_of_Iterations = 1000
n = float(len(X))
for i in range(len(X)):
y_predicted = m*X + c #y predicted
cost = (1/n) * sum([val**2 for val in (Y - y_predicted)]) #calculating loss using mean square error loss
derivative_m = (-2/n) * sum(X * (Y - y_predicted))
derivative_c = (-2/n) * sum(Y - y_predicted)
m = m - L*derivative_m
c = c - L*derivative_c
cost = 1/n * sum(pow((Y - y_predicted),2))
print(f'm value = {m}, c value = {c}, loss or cost value = {cost}, iteration value={i}')
print(f'The final m value ={m} and c value ={c}')
2. Difference between L1 & L2 Gradient descent method
L1 regularization technique is called Lasso Regression and model which uses L2 is called Ridge Regression.
L1 regularization:
Derivative of the cost function will estimate around median of the data to evade overfitting.
By including the absoute value of complexity or weights terms in the L1 regularization can add the penalty term in cost function.
Unable to learn complex data patterns.
L2 regularization:
Derivative of the cost function will estimate around mean of the data to evade overfitting.
L2 regularization appends the squared value of weights in the cost function.
Able to learn complex data patterns
3.What are the different loss functions for regression?
a. Root Mean Square Error. RMSE
b. Mean Absolute Error MAE
c. Mean Square Error. MSE
4.What is the importance of learning rate?
-To find the best fitting line. We need to provide a lower learning rate in order to get the lowest loss value or cost that is
near to zero.
-If the learning rate is not provided correctly or is a higher value the cost function will jump as well as the constants in
determining the best fit line. Thus making it impossible to get the best fitting line.
-Learing rate is also essential to train the algorithm to make more accurate predictions.
-It is good for a new model or algorithm to have a low learning rate so that the algorithm can find the low cost value and also
the best fit line for the model. This directly help in model to make accurate predictions.
5.How to evaluate linear regression?
We use loss function or cost fuctions like:
a. Root Mean Square Error. RMSE
b. Mean Absolute Error MAE
c. Mean Square Error. MSE
To determine the best fit line for linear regression and then we compute to check Rsquare or coefficient of determination which
measures the global fit of the model using the best line.We can evaluate linear regression by coefficient of determination R^2
If R squared is close to 1 means the line is best fit line.
R^2 = 1 - (SSregression/ SStotal)
6.What is the difference between multiple and adjusted coefficient of determination?
Coefficient of determination or R squared measures the global fit of the model.
Multiple coefficient of determination
-Multiple coefficient of determination measure of how well a variable can be predicted using linear functions of set of
variables.
-Multiple coefficient of determination takes values between 0 and 1. 1 indicating that the predictions are exactly correct and
a value of 0 indicating that no linear combination of the independent variables is a better predictor than is the fixed.
-Multiple coefficient of determination is known as the square root of the coefficient of determination.
Adjusted coefficient of determination
- Adjusted R2 or R bar squared is is an attempt to account for the phenomenon of the R2 automatically increasing when extra
explanatory variables are added to the model.
-The adjusted R2 can be negative, and its value will always be less than or equal to that of R2.
-Adjusted R2 reaches a maximum, and decreases afterward, would be the regression with the ideal combination of having the
best fit without excess/unnecessary terms.
-if you have a large difference between your multiple and your adjusted Rsquared that indicates you may have overfit your model.