Supervised Learning - Prediction Week 3 Challenge

.2............Difference between L1 & L2 Gradient descent method

L1 - LASSO :

L2 - RIDGE :

3....................... What are the different loss functions of a regression?

Regression or Regression Analysis is a supervised learning technique that can be defined as statistical techniques that are used to model the relationship between the dependent real numbered variable “y” and the independent variables “xi”. Based on different scenarios and their importance, we use different types of regression methods. Let’s understand the various possible loss functions that can be used for improving the performance evaluation in regression analysis.

1. MSE (L2 Loss function):

This is the most used loss function as it is very easy to understand and implement. It works well with almost all regression problems. As the name says, “the mean squared error is the mean of the squared errors”. Here error is the difference between the model-predicted value “y_hat” and the actual value “y_real”.

2. MAE (Mean Absolute Error):

Mean absolute error is the mean of the absolute difference between the model-predicted value and the actual value. Mean absolute error is more robust in the case of outliers because it’s not good to square the difference as it will be too huge.Irrespective of the direction of the error whether it is positive or negative as there might be chances of them canceling out, in turn, giving no error, we only consider its magnitude in this case.

3. Root Mean Squared Error:

Root Mean Squared Error is the square root of Mean square error. As we are applying square root to MSE, RMSE comes down to the same unit as the target variable. The loss function of RMSE is exactly the same as MSE, we just brought down the order from 2 to 1 so that correlation can be easily done. The correlation becomes difficult in MSE because it severely penalizes large errors.

4. Mean Bias Error:

Mean Bias Error is the exact difference between the predicted value and the actual value without any math function like absolute or square root applied to it. “The major limitation that MBE has is positive and negative errors have a chance of canceling out”. That is why this is the rarely used and less popular loss function.

5. Huber loss:

There are drawbacks in both MSE and MAE, so here is where Huber's loss comes into the picture. It takes only beneficial things from both MSE and MAE and comes them into a single loss function.

4.................... What is the importance of learning rate?

Learning rate in the Gradient descent defines the rate at which the tangent line to be drawn or in simple terms, the step in each tangent line. More the learning rate, the step will be high, which skips the local minima and doesn't give proper solution. If the learning rate is too small, the accuracy will be high but the computational cost will be high. So, optimal learning rate to be chosed before solving a Linear Regression problem.

5.............. How to Evaluate Linear Regression?

To evaluate the Regression problem, we have two techniques,

1. R squared or Co-efficient of Determination:

2. Adjusted R squared

The R squared and the adjsuted R squared should be almost equal to 1 for the good performance of the model.

6........... What is the difference between multiple and adjusted coefficient of determination?

R squared is also known as Co-efficient of determination.

1. R squared or Co-efficient of Determination:

2. Adjusted R squared:

k is the number of predictors or independent variables . So usually R squared value will be high, when Adjusted R squared is used, the value will decrease or increase based on the number of predictors.

When many independant variables are involved it is better to us Adjusted R square than the R square as the number of independant variables can affect the R square value in the Adjusted R square.