Supervised Learning - Classification Week 8 Challenge

2.What are the pros and cons of knn

Advantages of KNN:
*No Training Period : KNN is called Lazy Learner (Instance based learning). It does not learn anything in the training
period. It does not derive any discriminative function from the training data. In other words, there is no training period
for it. It stores the training dataset and learns from it only at the time of making real time predictions. This makes
the KNN algorithm much faster than other algorithms that require training e.g. SVM, Linear Regression etc.
* Since the KNN algorithm requires no training before making predictions, new data can be added seamlessly which will not
impact the accuracy of the algorithm.
*Very easy to implement for multi-class problem: Most of the classifier algorithms are easy to implement for binary
problems and needs effort to implement for multi class whereas K-NN adjust to multi class without any extra efforts.
*Can be used both for Classification and Regression: One of the biggest advantages of K-NN is that K-NN can be used both
for classification and regression problems.
*Variety of distance criteria to be choose from: K-NN algorithm gives user the flexibility to choose distance while
building K-NN model.
1.Euclidean Distance
2.Hamming Distance
3.Manhattan Distance
4.Minkowski Distance
*KNN is very easy to implement. There are only two parameters required to implement KNN i.e. the value of K and the
distance function (e.g. Euclidean or Manhattan etc.)

Disadvantages of KNN:
*Does not work well with large dataset : In large datasets, the cost of calculating the distance between the new point and
each existing points is huge which degrades the performance of the algorithm.
*Does not work well with high dimensions : The KNN algorithm doesn't work well with high dimensional data because with large
number of dimensions, it becomes difficult for the algorithm to calculate the distance in each dimension.
*Need feature scaling: We need to do feature scaling (standardization and normalization) before applying KNN algorithm to
any dataset. If we don't do so, KNN may generate wrong predictions.
*Sensitive to noisy data, missing values and outliers: KNN is sensitive to noise in the dataset. We need to manually
impute missing values and remove outliers.
*Imbalanced data causes problems : k-NN doesn’t perform well on imbalanced data. If we consider two classes, A and B, and
the majority of the training data is labeled as A, then the model will ultimately give a lot of preference to A. This
might result in getting the less common class B wrongly classified.