Supervised Learning - Prediction Week 3 Challenge

Q.1:-  Answer in .ipynb file and in screenshots.

Q.2:-

• L1 Regularization, also called a lasso regression, adds the “absolute value of magnitude” of the coefficient as a penalty term to the loss function.

• L2 Regularization, also called a ridge regression, adds the “squared magnitude” of the coefficient as the penalty term to the loss function.

Q.3:-

Loss Functions for Regression:

Regression or Regression Analysis is a supervised learning technique that can be defined as statistical techniques that are used to model the relationship between the dependent real numbered variable “y” and the independent variables “xi”. Based on different scenarios and their importance, we use different types of regression methods. Let’s understand the various possible loss functions that can be used for improving the performance evaluation in regression analysis.

 

1-Mean Squared Error / L2 Loss:

This is the most used loss function as it is very easy to understand and implement. It works well with almost all regression problems. As the name says, “the mean squared error is the mean of the squared errors”. Here error is the difference between the model-predicted value “y_hat” and the actual value “y_real”.

•Squaring is to magnify the large errors and also to observe the variance of our losses.

 

1-Mean Absolute Error / L1 Loss:

Mean absolute error is the mean of the absolute difference between the model-predicted value and the actual value. Mean absolute error is more robust in the case of outliers because it’s not good to square the difference as it will be too huge.

 

3- Root Mean Squared Error:

Root Mean Squared Error is the square root of Mean square error. As we are applying square root to MSE, RMSE comes down to the same unit as the target variable. The loss function of RMSE is exactly the same as MSE, we just brought down the order from 2 to 1 so that correlation can be easily done. The correlation becomes difficult in MSE because it severely penalizes large errors.

 

4- Mean Bias Error:

Mean Bias Error is the exact difference between the predicted value and the actual value without any math function like absolute or square root applied to it. “The major limitation that MBE has is positive and negative errors have a chance of canceling out”. That is why this is the rarely used and less popular loss function.

 

5- Huber Loss:

As we already discussed, there are drawbacks in both MSE and MAE, so here is where Huber's loss comes into the picture. It takes only beneficial things from both MSE and MAE and comes them into a single loss function.

Q.4:- 

This parameter determines how fast or slow we will move towards the optimal weights. If the learning rate is very large we will skip the optimal solution. If it is too small we will need too many iterations to converge to the best values. So using a good learning rate is crucial.

Q.5:-

There are 3 main metrics for model evaluation in regression:

R Square/Adjusted R Square.

Mean Square Error(MSE)/Root Mean Square Error(RMSE)

Mean Absolute Error(MAE)

 Q.6:-

we use the adjusted coefficient of multiple determination, denoted adjustedR2">adjusted R R2">Square , which corrects the overestimation of the coefficient of multiple determination when new independent variables are added to the model. The adjusted coefficient of multiple determination is interpreted in the same way as the coefficient of multiple determination. The adjusted coefficient of multiple determination adjusts the value of R2">R R2">Square to account for the number of independent variables in the model in order to avoid overestimating the impact of adding independent variables to the model.

The adjusted coefficient of multiple determination is calculated from the value of R2">R Square :

adjustedR2=1−((n−1)×(1−R2)n−k−1)">adjusted R2=1−(((n−1)×(1−R2))/n−k−1) 

where n is the number of observations and k">k is the number of independent variables. Although we can find the value of the adjusted coefficient of multiple determination using the above formula, the value of the coefficient of multiple determination is found in the regression summary table.