Supervised Learning - Prediction Week 3 Challenge

1.Perform Gradient Descent in Python with any loss function?

2.Difference between L1 & L2 Gradient descent method?

sol:

• In supervised machine learning, the ML models get trained training data and there are the possibilities that the model performs accurately on training data but fails to perform well on test data and also produces high error due to several factors such as collinearity, bias-variance impact and over modeling on train data. 
• For example, when the model learns signals as well as noises in the training data but couldn’t perform appropriately on new data upon which the model wasn’t trained, the condition/problem of overfitting takes place.   
• Overfitting simply states that there is low error with respect to training dataset, and high error with respect to test datasets.  
• Various methods can be adopted, for avoiding overfitting of models on training data, such as cross-validation sampling, reducing number of features, pruning, regularization and many more.
• In regression analysis, the features are estimated using coefficients while modelling. Also, if the estimates can be restricted, or shrinked or regularized towards zero, then the impact of insignificant features might be reduced and would prevent models from high variance with a stable fit. 
• In simple words, it avoids overfitting by panelizing the regression coefficients of high value. More specifically, It decreases the parameters and shrinks (simplifies) the model
• Both L1 and L2 can add a penalty to the cost depending upon the model complexity, so at the place of computing the cost by using a loss function, there will be an auxiliary component, known as regularization terms, added in order to panelizing complex models. 

• What is L1 Regularization?

   

   L1 regularization is the preferred choice when having a high number of features as it provides sparse solutions. Even, we obtain the computational advantage because features with zero coefficients can be avoided.The regression model that uses L1 regularization technique is called Lasso Regression.

• What is L2 regularization?

   L2 regularization can deal with the multicollinearity (independent variables are highly correlated) problems through constricting the coefficient and by keeping all the variables. 

   L2 regression can be used to estimate the significance of predictors and based on that it can penalize the insignificant predictors.

3.What are the different loss functions for regression?

• Mean Absolute Error (MAE).
• Mean Squared Error (MSE).
• Root Mean Squared Error (RMSE).

• Mean Absolute Error (MAE): sol:It can be called as arithmetic average of absolute errors, i.e. absolute difference between actual and predicted paired data points. 

• Mean Squared Error (MSE):sol:MSE measures the average of the sum of squares of the errors. It averages squared difference between the estimated values and the actual value. It is a kind of risk function where it calculates the deviation from the actual value with the predicted value which is squared and averaged with the number of instances a model has.The MSE values closer to zero are better as this refers that model has less error.

• Root Mean Squared Error (RMSE):sol:Root mean squared error (RMSE) is the square root of the mean of the square of all of the error. 

4.What is the importance of learning rate?

sol:Learning rate gives the rate of speed where the gradient moves during gradient descent. Setting it too high would make your path instable, too low would make convergence slow. Put it to zero means your model isn't learning anything from the gradients.

learning rate should be 0.001

5.How to evaluate linear regression?

sol: The linear regression model can be evaluated by using R squared value it should between 0 to 1 ,if the value is close to one our model performing good .if we have multiple features we have to use adjusted R sqaured value instead of R square value.

6.What is the difference between multiple and adjusted coefficient of determination?

sol:Multiple R squared is simply a measure of Rsquared for models that have multiple predictor variables. Therefore it measures the amount of variation in the response variable that can be explained by the predictor variables. The fundamental point is that when you add predictors to your model, the multiple Rsquared will always increase, as a predictor will always explain some portion of the variance.

Adjusted Rsquared controls against this increase, and adds penalties for the number of predictors in the model. Therefore it shows a balance between the most parsimonious model, and the best fitting model. Generally, if you have a large difference between your multiple and your adjusted Rsquared that indicates you may have overfit your model.