Supervised Learning - Prediction Week 3 Challenge

1.Perform Gradient Descent in Python with any loss function?

import time
start_time=time.time()
import pandas as pd
import numpy as np

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import mean_squared_error

import seaborn as sns

data=pd.read_csv('pmsm_temperature_data.csv')

data2=data.copy()

print(data.info())
print(data.describe())

data['ambient'].value_counts()

data.isnull().sum()
print('Data columns with null values:\n',data.isnull().sum())

# finding cross corelation
data_stat=data.corr()
print(data_stat)

print(round(data_stat,2))

df_select=data.select_dtypes(include=['float64','int64'])
for i in range(0,len(df_select.columns),3):
sns.pairplot(data=df_select,x_vars=df_select.columns[i:i+3],y_vars=['motor_speed'])

#Regression

features=list(set(data.columns)-set(['motor_speed']))
target=list(['motor_speed'])
print(features)
print(target)

x=data.loc[:,features].astype(float)
y=data.loc[:,target].astype(float)

train_x,test_x,train_y,test_y=train_test_split(x,y,test_size=0.3,random_state=1)

base_pred=np.mean(test_y)
print(base_pred)

base_pred=np.repeat(base_pred,len(test_y))

base_root_mean_square_error=mean_squared_error(test_y,base_pred)**0.5
print(base_root_mean_square_error)

scaler_x=StandardScaler()
scaler_y=StandardScaler()

scaler_x.fit(train_x)
scaler_y.fit(train_y)

train_x=scaler_x.transform(train_x)
test_x=scaler_x.transform(test_x)
train_y=scaler_y.transform(train_y)

#linear regression

lgr=LinearRegression(fit_intercept=False)
x=lgr.fit(train_x,train_y)
x.coef_

predicted_y_lgr=lgr.predict(test_x)
y_predicted_in_original_scale=scaler_y.inverse_transform(predicted_y_lgr)
lr_rmse=mean_squared_error(test_y, y_predicted_in_original_scale)**0.5

print(lr_rmse)

from matplotlib import pyplot
pyplot.plot(test_y,'b*')
pyplot.plot(y_predicted_in_original_scale,'r*')

def r_square(test_y,predicted_y):
RSS=np.sum((predicted_y-test_y)**2)
TSS=np.sum((test_y-np.mean(test_y))**2)
r_2=1-(RSS/TSS)
return r_2

r2_test_data=r_square(test_y,y_predicted_in_original_scale)
print(r2_test_data)

#regression using stats model

import statsmodels.api as sm
model=sm.OLS(train_y,train_x).fit()
#predictions=model.predict(test_x)
#y_predicted_in_original_scale2=scaler_y.inverse_transform(predictions)
model.summary()

#lr_rmse2=mean_squared_error(test_y,y_predicted_in_original_scale2)**0.5

#print(lr_rmse2)

print("--%s seconds--"%(time.time()-start_time))

2.Difference between L1 & L2 Gradient descent method?

L1:Mean Absolute Error (MAE): MAE calculates the average absolute difference between the predicted and actual values. MAE is less sensitive to outliers compared to MSE and provides a measure of average prediction error.

L2:Root Mean Squared Error (RMSE): RMSE is the square root of MSE and provides the evaluation metric in the same units as the dependent variable.

3.What are the different loss functions for regression?

There are several commonly used loss functions for regression.

• Mean Squared Error (MSE): MSE is one of the most widely used loss functions for regression. It computes the average squared difference between the predicted and actual values.
• Root Mean Squared Error (RMSE): RMSE is the square root of MSE and provides the evaluation metric in the same units as the dependent variable.
• Mean Absolute Error (MAE): MAE calculates the average absolute difference between the predicted and actual values. MAE is less sensitive to outliers compared to MSE and provides a measure of average prediction error.

4.What is the importance of learning rate

The learning rate is a parameter for optimization in gradient descent.

Convergence: The learning rate affects how quickly or slowly the optimization algorithm converges to the optimal solution.

Speed of Learning: The learning rate impacts the speed at which the model learns from the data.

Robustness to Noisy Data: The learning rate can also influence the model's robustness to noisy or outlier data points.

5.How to evaluate linear regression?

Linear regression can be evaluated using various metrics to assess the model's performance

1. Mean Squared Error (MSE): MSE measures the average squared difference between the predicted and actual values.

2. Root Mean Squared Error (RMSE): RMSE is the square root of MSE and provides the evaluation metric in the same units as the dependent variable.

3. Mean Absolute Error (MAE): MAE calculates the average absolute difference between the predicted and actual values. It provides a measure of average prediction error.

4. R-squared (R^2): R-squared represents the proportion of the variance in the dependent variable explained by the independent variables. It ranges from 0 to 1, with higher values indicating a better fit.

6.What is the difference between multiple and adjusted coefficient of determination?

The coefficient of determination (R-squared) is a statistical measure that represents the proportion of the variance in the dependent variable that can be explained by the independent variables in a regression model. It is a common metric used to evaluate the goodness of fit of a regression model.

Multiple R-squared,(R^2), measures the proportion of the variance in the dependent variable that is explained by all the independent variables in a multiple regression model. It ranges from 0 to 1, where 0 indicates that none of the variance is explained by the independent variables, and 1 indicates that all the variance is explained.

multiple R-squared indicates the overall goodness of fit of a multiple regression model, while adjusted R-squared adjusts for the number of predictors and sample size, providing a more reliable measure of model fit when comparing models with different numbers of predictors.

Adjusted R-squared, (R^2_adj), is an adjusted version of the multiple R-squared that takes into account the number of predictors (independent variables) and the sample size.