Supervised Learning - Prediction Week 3 Challenge
1.
f(w)f(w),w=[w1,w2,…,wn]Tw=[w1,w2,…,wn]T
2.
overfitting is a phenomenon that occurs when a machine learning or statistics model is tailored to a particular dataset and is unable to generalise to other datasets. This usually happens in complex models, like deep neural networks.
Regularisation is a process of introducing additional information in order to prevent overfitting. The focus for this article is L1 and L2 regularisation.
There are many explanations out there but honestly, they are a little too abstract, and I’d probably forget them and end up visiting these pages, only to forget again. In this article, I will be sharing with you some intuitions why L1 and L2 work by explaining using gradient descent. Gradient descent is simply a method to find the ‘right’ coefficients through iterative updates using the value of the gradient. (This artical shows how gradient descent can be used in a simple linear regression.)
L1 and L2 regularisation owes its name to L1 and L2 norm of a vector w respectively. Here’s a primer on norms
3.
- Regression Loss Functions
- Mean Squared Error Loss
- Mean Squared Logarithmic Error Loss
- Mean Absolute Error Loss
- Binary Classification Loss Functions
- Binary Cross-Entropy
- Hinge Loss
- Squared Hinge Loss
- Multi-Class Classification Loss Functions
- Multi-Class Cross-Entropy Loss
- Sparse Multiclass Cross-Entropy Loss
- Kullback Leibler Divergence Loss
4.
Learning rate is a hyper-parameter that controls the weights of our neural network with respect to the loss gradient. It defines how quickly the neural network updates the concepts it has learned.
A desirable learning rate is low enough that the network converges to something useful, but high enough that it can be trained in a reasonable amount of time.
5.
Evaluation metrics are a measure of how good a model performs and how well it approximates the relationship. Let us look at MSE, MAE, R-squared, Adjusted R-squared, and RMSE.