Challenge

Supervised Learning - Prediction Week 3 Challenge

1. Gradient Descent with any Loss Function:

Gradient Descent is an iterative optimization algorithm used to minimize a loss function.

import numpy as np

def gradient_descent(X, y, learning_rate, epochs):

m, b = 0, 0 # Initial guess

n = len(X)

for epoch in range(epochs):

y_pred = m * X + b

loss = np.mean((y_pred - y)**2)

gradient_m = 2/n * np.sum(X * (y_pred - y))

gradient_b = 2/n * np.sum(y_pred - y)

m -= learning_rate * gradient_m

b -= learning_rate * gradient_b

print(f'Epoch {epoch+1}/{epochs}, Loss: {loss:.4f}')

return m, b

2. Difference between L1 & L2 Gradient Descent:

L1 and L2 refer to regularization terms added to the loss function to prevent overfitting.
L1 regularization (Lasso): Adds the absolute values of the coefficients. Encourages sparsity.
L2 regularization (Ridge): Adds the squared values of the coefficients. Tends to shrink coefficients.

3. Loss Functions for Regression:

4. Importance of Learning Rate:

The learning rate controls the size of the steps taken during optimization.
Too high a learning rate may cause overshooting, and too low a learning rate may result in slow convergence or getting stuck in local minima.
It's crucial to choose an appropriate learning rate for efficient and stable convergence.

5. Evaluation of Linear Regression:

6. Difference between Multiple and Adjusted Coefficient of Determination:

Multiple Coefficient of Determination (R-squared) measures the proportion of the variance in the dependent variable that is predictable from the independent variables.
Adjusted R-squared adjusts R-squared for the number of predictors in the model, penalizing for adding irrelevant predictors. It provides a more accurate measure when comparing models with different numbers of predictors.

Ship work like this
in your own course.