Supervised Learning - Prediction Week 3 Challenge

1. Perform Gradient Descent in Python (Example Code)

# Example: Minimizing Mean Squared Error (MSE) Loss

import numpy as np

X = np.array([1, 2, 3])

y = np.array([2, 4, 6])

theta = 0

learning_rate = 0.01

iterations = 100

for _ in range(iterations):

gradient = -(2 / len(y)) * np.sum((y - (theta * X)) * X)

theta -= learning_rate * gradient

print(theta)

2. Difference Between L1 and L2 Gradient Descent Methods

• L1 (Lasso): Adds absolute value of weights to cost; leads to sparse models (some weights = 0).
• L2 (Ridge): Adds squared value of weights to cost; shrinks weights, helping avoid overfitting.

3. Loss Functions for Regression

• MSE (Mean Squared Error)
• MAE (Mean Absolute Error)
• Huber Loss (less sensitive to outliers)

4. Importance of Learning Rate

Controls step size in gradient descent:

• Too high: Diverges.
• Too low: Converges too slowly.

5. How to Evaluate Linear Regression

• R-squared: How well the model explains the variance.
• RMSE/MSE: Average error size.
• Residual Plots: Check for randomness.

6. Multiple vs. Adjusted Coefficient of Determination ($R^2$R2)

• Multiple $R^2$R2: How much variance is explained.
• Adjusted $R^2$R2: Adjusts for the number of predictors to avoid overfitting.