-
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from sklearn.metrics import mean_squared_error
df_train = pd.read_csv("data.csv")
# getting the values for X
X = df_train.iloc[:,0]
# getting the values for y
y = df_train.iloc[:,1]
print(X, y)
def gradient_descent(X, y, lr=0.001, epoch=1000):
'''
X is the features
y is output
lr is the learning rate
epoch is the number of times our parameters get updated
'''
m, b = 0, 0 # parameters
mse = [] # lists to store learning process
N = len(X) # number of samples
for _ in range(epoch):
f = y - (m*X + b)
# Updating m and b
m -= lr * (-2 * X.dot(f).sum() / N)
b -= lr * (-2 * f.sum() / N)
# calculating and storing the value of mse
mse.append(mean_squared_error(y, (m*X + b)))
return m, b, mse
m,b,mse = gradient_descent(X,y)
print(mse)
- The difference between the L1 and L2 regularization is that L1 regularization tries to estimate the median of the data while the L2 regularization tries to estimate the mean of the data to avoid overfitting.
- The different loss functions are: Mean square error, mean absolute error, mean bias error, cross-entropy loss, Hinge loss.
- The learning rate controls how quickly the model is adapted to the problem. Smaller learning rates require more training epochs given the smaller changes made to the weights each update, whereas larger learning rates result in rapid changes and require fewer training epochs.
- Ways to evaluate linear regression are: R square or adjusted R square, Mean square error, Mean absolute error.
- When we add predictors to your model, the multiple R-squared will always increase, as a predictor will always explain some portion of the variance. Adjusted R-squared controls against this increase, and adds penalties for the number of predictors in the model.