Supervised Learning - Prediction Week 3 Challenge

2.Difference between L1 & L2 Gradient descent method

T Here L1 is the mean absolute error

L2 is the mean squared error.

The mean absolute error is an average of the all absolute errors. The mean absolute error is a common measure of estimate error in time series analysis. The mean squared error of an estimator measures the average of the squares of the errors, which means the difference between the estimator and estimated.

3.What are the different loss functions for regression

Mean Squared Error: is the most commonly used regression loss function. MSE is the sum of squared distances between our target variable and predicted values.

Mean Absolute Error:

is another loss function used for regression models. MAE is the sum of absolute differences between our target and predicted variables. So it measures the average magnitude of errors in a set of predictions, without considering their directions.

Huber Loss:

is less sensitive to outliers in data than the squared error loss. It’s also differentiable at 0. It’s basically absolute error, which becomes quadratic when error is small. How small that error has to be to make it quadratic depends on a hyper parameter, 𝛿 (delta), which can be tuned. Huber loss approaches MSE when 𝛿 ~ 0 and MAE when 𝛿 ~ ∞ (large numbers.)

Log Cosh Loss:

Log-cosh is another function used in regression tasks that’s smoother than L2. Log-cosh is the logarithm of the hyperbolic cosine of the prediction error.

Quantile Loss:

In most of the real-world prediction problems, we are often interested to know about the uncertainty in our predictions. Knowing about the range of predictions as opposed to only point estimates can significantly improve decision making processes for many business problems.

Root Mean Squared Error:

It is the sum of square root of squared distances between our target variable and predicted values.

4.What is the importance of learning rate:

we must set the learning rate to an appropriate value. This parameter determines how fast or slow we will move towards the optimal weights. If the learning rate is very large we will skip the optimal solution. If it is too small we will need too many iterations to converge to the best values. So using a good learning rate is crucial.

both low and high learning rates results in wasted time and resources

A lower learning rate means more training time

more time results in increased cloud GPU costs

a higher rate could result in a model that might not be able to predict anything accurately

5. How to evaluate linear regression.

There are 3 main metrics for model evaluation in regression:

1. R Square/Adjusted R Square: R Square measures how much variability in dependent variable can be explained by the model. It is the square of the Correlation Coefficient(R) and that is why it is called R Square.
2. Mean Square Error(MSE)/Root Mean Square Error(RMSE)
3. Mean Absolute Error(MAE)

6.What is the difference between multiple and adjusted coefficient of determination.

The most obvious difference between adjusted R-squared and R-squared is simply that adjusted R-squared considers and tests different independent variables against the stock index and R-squared does not. One of the most essential limits to using this model is that R-squared cannot be used to determine whether or not the coefficient estimates and predictions are biased.