Supervised Learning - Prediction Week 3 Challenge
2.
|
L1 (mean absolute eerror) |
L2 (mean square error) |
| 1. It has a sparse solution | It has no sparse solution |
| 2. It generates simple and interpretable models | It gives more accurate predictions when the output variable is the function of whole input variables |
| 3. Unable to learn complex data patterns | It is able to learn complex data patterns |
| 4. It gives multiple solutions | It has only one solution |
| 5. Panelize the sum of absolute values of weights | Panelize the sum of square weights |
| 6. Computationally inefficient over non-sparse conditions | Computationally efficient because of having analytical solutions |
| 7. Constructed in feature slection | No feature selection |
3. a) Mean Absolute Error (MAE) :
MAE is the simplest error , it just calculates the absolute difference between the actual and predicted values and takes it's mean
Equation :
b) Mean Squared Error (MSE): Here we calculate the square of our error and then take it's mean .
Equation:
c) Root Mean Squared Error (RMSE) :
RMSE is just the square root of MES,which means, it is again , a linear scoring method,but still better than MAE as it gives comparatively more weighted to larger errors.
Equation:
4. Importance of learning rate:
Learning rate is a scalar , a value which tels the machine how fast or slow to arrive at some conclusion . The speed at which a model learns is important and it varies with different applications . A super fast learning algorithm can miss can miss a few data points or correlations which can give better insights on the data. Missing this will eventually lead to wrong classification.
And , to find the next step or the adjacent data point, the gradient descent algorithms mutiply the gradient by learning rate.
For example, if the gradient magnitude is 1.5 and the earning rate is 0.01,,then the gradient descent algorithm will pick the next point 0.015 away from the previous point. If the learning rate is too small,learning will take too long.
And if the learning rate too large , the next point will perpetually bounce haphazardly across the bottom of the valley:
5. There are three main metrics to evaluate the linear regression error:
a. Mean Squared Error: The most common metric for regression tasks is MSE. It has a convex shape. I t is the average of the squared difference between the predicted and actual value. Since it isdifferentiable and has a convex shape,it is easier to optimize.
Formula:
MSE penalizes large errors
b. Mean Absolute Error : It is the average of the absolute difference between the target value and the predicted value by the model.Not preferred in cases where outliers are promient.
Formula:
MAE does not penalizes large errors
c. R-squared or coeeficient of determination:
This metric represents the part of the variance of the dependent variable explained by the independent variable of the model. It measures the strength of the relationship between your model and the dependent variable.
6. Multiple R squared is simply a measure of Rsquared for models that have multiple predictor variables.Therefore it measures the amount of variation in the response variable that can be explained by the predictor variables . The fundamental point is that when you add predictors to your model, the multiple Rsquared will always increase , as a predictor will always explain some portion of the variance.
Adjusted Rsquared controls against this increase,and adds penalities for the number of predictors in the model.Therefore it shows a balance between the most simplest model and the best fitting model .Generally, if you have a large difference between your multiple and your adjusted Rsquared that indicates you may have overfit your model.