Supervised Learning - Prediction Week 3 Challenge
Q1: Perform Gradient Descent in Python with any loss function.
A1:
Q2: Difference between L1 & L2 Gradient descent method.
A2:
| L1 Gradient Descent (Lasso) | L2 Gradient Descent (Ridge) |
| 1. Penalizes the sum of absolute values of weights. | 1. Penalizes the sum of squares of weights. |
| 2. Solution is sparse. | 2. Solution is non-sparse. |
| 3. Has built-in feature selection since weights are reduced to zero. | 3. Doesn't perform feature selection since weights are reduced to values near zero. |
| 4. It is robust to outliers. | 4. It is not robust to outliers. |
| 5. Computatinally more expensive. | 5. Computatinally less expensive. |
Q3: What are the different loss functions for regression ?
A3:
- Mean Absolute Error (MAE)
- Mean Squared Error (MSE)
- Root Mean Squared Error (RMSE)
- Huber Loss
- Log-Cosh Loss
Q4: What is the importance of learning rate ?
A4: The learning rate is a tuning parameter in an optimization algorithm that determines the step size at each iteration while moving toward a minimum of a loss function. Since it influences to what extent newly acquired information overrides old information, it metaphorically represents the speed at which a machine learning model "learns". Learning rate is a configurable hyperparameter used in the training of neural networks that has a small positive value, often in the range between 0.0 and 1.0.
The learning rate controls how quickly the model is adapted to the problem. Smaller learning rates require more training epochs given the smaller changes made to the weights each update, whereas larger learning rates result in rapid changes and require fewer training epochs.
Q5: How to evaluate linear regression ?
A5: A linear regression model can be evaluated using any one of the metrics mentioned below:
- R Square/ Adjusted R Square
- Mean Squared Error/ Root Mean Squared Error
- Mean Absolute Error
Q6: What is the difference between multiple and adjusted coefficient of determination ?
A6: R-square represents the proportion of the variance in the response variable of a regression model that can be explained by the predictor variables. This value ranges from 0 to 1. It tells us how useful the predictor variables are at predicting the value of the response variable.
However, R-square has a drawback, each time we add a new predictor variable to the model the R-squared is guaranteed to increase even if the predictor variable isn't correlated to the response variable.
To overcome this drawback a modified version of R-square called Adjusted R-square is used, that adjusts for the number of predictors in a regression model. Adjusted R-square imposes a penalty for adding variables which do not improve the goodness of the existing model.
When building multiple linear regression models, it is best to use Adjusted R-square to check the goodness of the model. In case the model has only one input variable, then the R-square & Adjusted R-square would be exactly the same.