Supervised Learning - Prediction Week 3 Challenge
2. Difference between L1 & L2 Gradient descent method
L1 regularization penalizes the sum of absolute values of the weights, whereas L2 regularization
penalizes the sum of squares of the weights. The L1 regularization solution is sparse. The L2 regularization solution is non-sparse
3. What are the different loss functions for regression
- Mean Squared Error/L2 Loss: This is the most used loss function as it is very easy to understand and implement.
- Mean Absolute Error / L1 Loss
- Root Mean Squared Error
- Mean Bias Error
- Huber Loss
4. What is the importance of learning rate
The learning rate controls how quickly the model is adapted to the problem.
Smaller learning rates require more training epochs given the smaller changes made to the weights each update,
whereas larger learning rates result in rapid changes and require fewer training epochs.
6. What is the difference between multiple and adjusted coefficient of determination
Multiple R squared: is simply a measure of Rsquared for models that have multiple predictor variables.
Therefore it measures the amount of variation in the response variable that can be explained by the predictor variables.
The fundamental point is that when you add predictors to your model, the multiple Rsquared will always increase,
as a predictor will always explain some portion of the variance.
Adjusted Rsquared: controls against this increase, and adds penalties for the number of predictors in the model.
Therefore it shows a balance between the most parsimonious model, and the best fitting model. Generally,
if you have a large difference between your multiple and your adjusted Rsquared that indicates you may have overfit your model.