1. From a practical standpoint, L1 tends to shrink coefficients to zero whereas L2 tends to shrink coefficients evenly.
L1 is therefore useful for feature selection, as we can drop any variables associated with coefficients that go to zero.
L2, on the other hand, is useful when you have collinear/codependent features.
2.1. BINARY CROSS-ENTROPY LOSS / LOG LOSS:
This is the most common loss function used in classification problems. The cross-entropy loss decreases as the predicted probability converges to the actual label.
It measures the performance of a classification model whose predicted output is a probability value between 0 and 1.
When the number of classes is 2, it’s binary classification.
common-loss-functions
When the number of classes is more than 2, it’s multi-class classification.
2. HINGE LOSS
The second most common loss function used for classification problems and an alternative to the cross-entropy loss function is hinge loss,
primarily developed for support vector machine (SVM) model evaluation.
Hinge loss penalizes the wrong predictions and the right predictions that are not confident.
It’s primarily used with SVM classifiers with class labels as -1 and 1. Make sure you change your malignant class labels from 0 to -1.
3.MEAN SQUARE ERROR / QUADRATIC LOSS / L2 LOSS
We define MSE loss function as the average of squared differences between the actual and the predicted value. It’s the most commonly used regression loss function.
4.MEAN ABSOLUTE ERROR / L1 LOSS
We define MAE loss function as the average of absolute differences between the actual and the predicted value. It’s the second most commonly used regression loss function. It measures the average magnitude of errors in a set of predictions, without considering their directions.
common-loss-functions
1. Mean Square Error / Quadratic Loss / L2 Loss We define MSE loss function as the average of squared differences between the actual and the predicted value. It’s the most commonly used regression loss function. ...
2. Mean Absolute Error / L1 Loss ...
3. Huber Loss / Smooth Mean Absolute Error ...
4. Log-Cosh Loss ...
5. Quantile Loss ...
3.The learning rate is an important parameter in training a neural network, and it can have a significant impact on the performance of the network.
If the learning rate is too low, the training process will be slow and the network may not converge to a solution
4.Mean Absolute Error(MAE)
Root Mean Square Error(RMSE)
Coefficient of determination or R2
Adjusted R2
5.Adjusted Coefficient of Determination:
Adjusted coefficient of determination is the adjusted value of the coefficient of determination in which the number of variables of the data set is taken into consideration.
It determines the fitting of the multiple regression equations for the sample data. It is also known as adjusted or
. It determines the proportion of variation which is determined by the estimated line of regression.
The adjusted coefficient of determination takes the values between 0 and 1.
It explains the percentage of variation of the independent variables that affect the dependent variables.
If the adjusted coefficient of determination is closer to 1, it indicates that the estimated equation of regression fits the data.
The value of the adjusted coefficient of determination increases only when new independent variables are added which in turn increases the power of the regression equation.
Hence, adjusted coefficient of determination is widely used in multiple regression analysis. In this way, it is very useful measure than the coefficient of determination.