Supervised Learning - Prediction Week 3 Challenge
2. Difference between L1 & L2 Gradient descent method
L1 loss (Mean Absolute Error, MAE) and L2 loss (Mean Squared Error, MSE) are two commonly used loss functions in machine learning, particularly in regression tasks. They differ in how they measure the error or the difference between predicted values and actual target values:
L1 Loss (MAE - Mean Absolute Error):
L1 loss, also known as Mean Absolute Error (MAE), measures the average absolute difference between the predicted values and the actual target values.
L2 Loss (MSE - Mean Squared Error):
L2 loss, also known as Mean Squared Error (MSE), measures the average squared difference between the predicted values and the actual target values.
L1 loss (MAE) is less sensitive to outliers due to the absolute value operation, while L2 loss (MSE) gives more weight to larger errors due to squaring.
MAE is easier to interpret since it provides an average of the absolute error in the same units as the target variable, whereas MSE provides an average of the squared error.
3.What are the different loss functions for regression
In regression problems, the goal is to predict a continuous numerical output, and the choice of a loss function is crucial for training a machine learning model. Different loss functions are used based on the specific requirements of the problem and the characteristics of the data. Here are some common loss functions used in regression:
Mean Squared Error (MSE):
Formula:MSE = (1/n) Σ(y_i - ŷ_i)^2
MSE is one of the most widely used loss functions for regression. It calculates the average squared difference between the predicted values (ŷ_i) and the actual target values (y_i). It penalizes large errors heavily and is sensitive to outliers.
Mean Absolute Error (MAE):
Formula: MAE = (1/n) Σ|y_i - ŷ_i|
MAE computes the average absolute difference between predicted values and actual target values. It is less sensitive to outliers compared to MSE.
Huber Loss:
Formula:
If |y_i - ŷ_i| ≤ δ: Huber Loss = (1/2) * (y_i - ŷ_i)^2
If |y_i - ŷ_i| > δ: Huber Loss = δ * |y_i - ŷ_i| - (1/2) * δ^2
Huber loss is a combination of MSE and MAE. It is less sensitive to outliers than MSE and provides a compromise between MSE and MAE by using a hyperparameter δ.
4. What is the importance of learning rate
The learning rate is a hyperparameter that controls the step size during gradient descent. Its importance lies in the following aspects:
If the learning rate is too high, the algorithm may overshoot the minimum, leading to divergence.
If it's too low, the algorithm may converge very slowly or get stuck in local minima.
Proper tuning of the learning rate is crucial for efficient and effective gradient descent.
5. How to evaluate linear regression
Evaluating a linear regression model is essential to assess its performance and determine how well it fits the data. Several metrics and techniques can be used to evaluate a linear regression model:
1.Coefficient of Determination (R-squared, R⊃2;):
R-squared measures the proportion of the variance in the dependent variable (target) that is explained by the independent variables (features) in the model.
R-squared values range from 0 to 1, with higher values indicating a better fit.
However, R-squared alone doesn't provide insights into whether the model's predictions are biased or have high errors.
2.Mean Squared Error (MSE):
MSE calculates the average squared difference between the predicted values and the actual target values.
Lower MSE values indicate better model performance.
It's important to consider the scale of the MSE, as it depends on the scale of the target variable.
3.Mean Absolute Error (MAE):
MAE computes the average absolute difference between the predicted values and the actual target values.
Like MSE, lower MAE values are preferable.
MAE is more interpretable than MSE since it's in the same units as the target variable.
4.Root Mean Squared Error (RMSE):
RMSE is the square root of the MSE, which provides a measure of error in the same units as the target variable.
Like MSE, lower RMSE values indicate better model performance.
5.Adjusted R-squared:
Adjusted R-squared takes into account the number of independent variables in the model.
It penalizes the addition of unnecessary variables, helping to prevent overfitting.
A higher adjusted R-squared suggests a better trade-off between model complexity and goodness of fit.
6. What is the difference between multiple and adjusted coefficient of determination
The multiple coefficient of determination (R-squared) and the adjusted coefficient of determination (adjusted R-squared) are both metrics used to evaluate the goodness of fit of a regression model, particularly multiple linear regression models with multiple predictor variables. However, they serve slightly different purposes and have some key differences:
while both R-squared and adjusted R-squared measure the proportion of variance explained by the predictors in a multiple linear regression model, adjusted R-squared provides a more conservative estimate by penalizing the inclusion of unnecessary predictors. It helps strike a balance between model complexity and goodness of fit, making it a useful metric when comparing models with different numbers of predictors. Adjusted R-squared is often preferred when selecting the best model from a set of candidate models.