Supervised Learning - Prediction Week 3 Challenge
1.Perform Gradient Descent in Python with any loss function
Kindly find attached programs with and without using sklearn.
2.Difference between L1 & L2 Gradient descent method
When determining the best fit line in linear regression, there may be a chance that line can be underfitting or overfitting. In this place regularization prevents it.
L1->Lasso Regression
L2->Ridge Regression
Lasso: 1/nΣ(yi-yp)+λ*|slope| (Magnitude of Slope)
It prevents the overfitting of line and select featuring. Through this regression slope can moves towards 0 or 0.
Also it excludes the slope of less value which results in selecting featuring.
Ridge: 1/nΣ(yi-yp)+λ* (Slope)^2
It adds the lambda*square of slope to a cost function which would help us to find the another best fit line with less variance.It reduces the weight towards 0 but not actually to 0. It penalizes the steep slope basically.
3.What are the different loss functions for regression?
Kindly find attached Question-3 file.
4.What is the importance of learning rate
Learning Rate (LR) plays a important role in predictions accurately. Its a value which tells the program/machine how fast or slow to get a conclusion. Speed at which the model learns is also important.
If lesser the Learning Rate, higher would be the learning time. If higher the learning rate, model may miss data and its correlation.
5.How to evaluate linear regression
Have provide answer below and attached Question-5_1 and Question-5_2 image as well.
Linear regression can be evaluated using R-Square,Mean Square error/Root mean square error,Mean actual error.
R-square:it shows the variability/ difference in a dependent value. Its a square of correlation coefficient.
Rsquare=1-(Σ(yi-yp))^2/(Σ(yi-ymean)^2)
MSE/RMSE:Both are same but dividing the square value by square root of it shows the actual variance value.
MSE/RMSE=(1/nΣ(yi-yp))^2)/(1/n`sqrt Σ(yi-yp))^2`)
MAE: This shows the absolute value of error
MAE=1/nΣ|ya-yp|
6.What is the difference between multiple and adjusted coefficient of determination.
Multiple R^2 shows the variance of dependent variable from actual to predicted. It have values from 0-1.
But if predicted values are increased, then R^2 values will also be increased. So modified version of R^2 was developedto provide the metrics to say how the predictions of model is useful.
R^2=1-(Σ(yi-yp))^2/(Σ(yi-ymean)^2)
R^2=1-[(1-R^2)*(n-1)/(n-k-1)]
n->number of observations, k->number of predicted values.