Supervised Learning - Prediction Week 3 Challenge

Q1)Perform Gradient Descent in Python with any loss function

ANS:Explanation about the code first we need to take all the requiered libraries and import it ,after that Initial data and parameters are initialized, next run a main body ,after that ,If the condition J>tol is true, then the while loop will be executed next Inside the while loop, there is a for loop which fetches each value present in x and y and calculates the error and change in the loss function with respect to the parameters m and b

After exiting the for loop, the sum of the terms are calculated and this process keeps repeating till the error is lesser than the given tolerance value

For a learning rate of 0.0008, it takes 7352 iterations for the error to be lesser than the tolerance and the m and b values are found to be 1.006 and 0.982

Q2)Difference between L1 & L2 Gradient descent method

Mean Absolute Error:

ANS: L2 Gradient descent method is nothing but the mean Absolute error .The mean absolute error (MAE) is a quantity used to measure how close predictions are to the outcomes.the mean absolute error is an average of the all absolute errors. The mean absolute error is a common measure of estimate error in time series analysis

Mean Squared Error:

L1 Gradient descent method is nothing but the Mean square Error .The mean squared error of an estimator measures the average of the squares of the errors, which means the difference between the estimator and estimated.

MSE is a function, equivalent to the expected value of the squared error loss or quadratic loss. The difference occurs because of the randomness. The MSE is a measure of the quality of an estimator, it is always positive, and values which are closer to zero are better. The MSE is the second moment of the error, and includes both the variance of the estimator and its bias. For an unbiased estimator, the MSE is the variance of the estimator.

Q3)What are the different loss functions for regression

ANS:

The different loss functions for regression are listed given below

• Root mean square error
• Mean square error
• Mean absolute error

Q4)What is the importance of learning rate

ANS:

Giving a high learning rate will cause the function to overshoot beyond its lowest value and will never reach its lowest value or will oscillate before arriving at one.

Giving a low learning rate will cause the function to approach the lowest value very slowly leading to a high number of iterations.

In machine learning, we deal with two types of parameters;

1) machine learnable parameters and

2) hyper-parameters.

The Machine learnable parameters are the one which the algorithms learn/estimate on their own during the training for a given dataset. In equation-3, β₀, β₁ and β₂ are the machine learnable parameters. The Hyper-parameters are the one which the machine learning engineers or data scientists will assign specific values to, to control the way the algorithms learn and also to tune the performance of the model. Learning rate, a hyper-parameter used to control the rate at which an algorithm updates the parameter estimates or learns the values of the parameters.

Q5)How to evaluate linear regression

ANS:

Linear regression can be evaluated by determining how good the fit is, i.e. what is the error between the predicted value and the data set. The most common metric for evaluating linear regression model performance is called root mean squared error, or RMSE. The basic idea is to measure how bad/erroneous the model’s predictions are when compared to actual observed values. So a high RMSE is “bad” and a low RMSE is “good”.'

Q6)What is the difference between multiple and adjusted coefficient of determination

ANS:

When the dependent variable depends on only one independent variable then it is known as the linear coefficient of determination, but when it depends on multiple Independent variables then it is multiple coefficient of determination.

The Adjusted coefficient of determination is the method in which the parameters which do not contribute towards the predicted value are ignored. These parameters are called insignificant independent variables.

Adding new variables to the system will affect the R-squared value and the adjusted R-squared disregards the infinitesimally small increments and only considers the parameters which contribute to the output in a significant way

R Squared is a demographical measuring use to represents the contradictions between dependent and independent variables. The variances which are proportional are the dependent variable had described by the independent variable.

R^2 = 1-(RSS/TSS)

R^2 = coefficient determination

RSS = Sum of Squares of Residuals

TSS = Total Sum of Squares