Supervised Learning - Prediction Week 3 Challenge
-
import pandas as pd import numpy as np import matplotlib.pyplot as plt from sklearn.metrics import mean_squared_error df_train = pd.read_csv("data.csv") # getting the values for X X = df_train.iloc[:,0] # getting the values for y y = df_train.iloc[:,1] print(X, y) def gradient_descent(X, y, lr=0.001, epoch=1000): ''' X is the features y is output lr is the learning rate epoch is the number of times our parameters get updated ''' m, b = 0, 0 # parameters mse = [] # lists to store learning process N = len(X) # number of samples for _ in range(epoch): f = y - (m*X + b) # Updating m and b m -= lr * (-2 * X.dot(f).sum() / N) b -= lr * (-2 * f.sum() / N) # calculating and storing the value of mse mse.append(mean_squared_error(y, (m*X + b))) return m, b, mse m,b,mse = gradient_descent(X,y) print(mse) - The difference between the L1 and L2 regularization is that L1 regularization tries to estimate the median of the data while the L2 regularization tries to estimate the mean of the data to avoid overfitting.
- The different loss functions are: Mean square error, mean absolute error, mean bias error, cross-entropy loss, Hinge loss.
- The learning rate controls how quickly the model is adapted to the problem. Smaller learning rates require more training epochs given the smaller changes made to the weights each update, whereas larger learning rates result in rapid changes and require fewer training epochs.
- Ways to evaluate linear regression are: R square or adjusted R square, Mean square error, Mean absolute error.
- When we add predictors to your model, the multiple R-squared will always increase, as a predictor will always explain some portion of the variance. Adjusted R-squared controls against this increase, and adds penalties for the number of predictors in the model.