Supervised Learning - Prediction Week 3 Challenge
1.
def gradient_descent(X, y, theta, alpha, num_iters, loss_func, gradient_func):
m = len(y)
for _ in range(num_iters):
gradient = gradient_func(X, y, theta)
theta = theta - alpha * gradient
cost = loss_func(X, y, theta)
print(f'Cost: {cost}')
return theta
2.
The main difference lies in the regularization term added to the cost function:
- L1 regularization (Lasso) adds the sum of the absolute values of the coefficients: \(\lambda \sum_{j=1}^m |w_j|\).
- L2 regularization (Ridge) adds the sum of the squared values of the coefficients: \(\lambda \sum_{j=1}^m w_j^2\).
3.
Common loss functions for regression include:
- Mean Squared Error (MSE): \(\frac{1}{n} \sum_{i=1}^n (y_i - \hat{y}_i)^2\)
- Mean Absolute Error (MAE): \(\frac{1}{n} \sum_{i=1}^n |y_i - \hat{y}_i|\)
- Huber Loss: A combination of MSE and MAE, which is less sensitive to outliers.
4.
The learning rate determines the size of the steps taken during optimization. A too small learning rate leads to slow convergence, while a too large one can cause divergence or oscillation. Finding the right learning rate is crucial for the algorithm to converge efficiently.
5.
Common metrics to evaluate linear regression include:
- Mean Squared Error (MSE): Average of the squares of the errors.
- R-squared (Coefficient of Determination): Proportion of the variance in the dependent variable that is predictable from the independent variables.
6.
- Multiple Coefficient of Determination (R-squared): Measures the proportion of the variance in the dependent variable that is predictable from the independent variables.
- Adjusted Coefficient of Determination: Penalizes excessive use of independent variables in the regression model. Adjusted R-squared increases only if the addition of the independent variable improves the model more than would be expected by chance.