Supervised Learning - Classification Week 8 Challenge

1. Refer attcahement.

2.

Pros of K-nearest neighbors (KNN):

1. Simplicity: KNN is a straightforward algorithm and easy to understand. It does not require complex mathematical calculations or assumptions about the underlying data distribution.

2. No training phase: KNN is an instance-based algorithm, meaning it does not explicitly learn a model or make assumptions about the data. This makes the training phase very fast since there is none.

3. Versatility: KNN can be used for both classification and regression problems. It is capable of handling non-linear decision boundaries and can adapt to complex patterns in the data.

4. Interpretability: KNN provides interpretability in the sense that it assigns a class label or predicts a value based on the actual data points closest to the query point. This can be helpful in explaining the reasoning behind the predictions.

5. Robust to outliers: KNN is generally robust to outliers since it considers the proximity of data points rather than relying on their absolute values. Outliers have less influence on the final prediction.

Cons of K-nearest neighbors (KNN):

1. Computationally expensive: During the prediction phase, KNN requires calculating the distances between the query point and all the training data points. As the dataset grows larger, the computational cost increases significantly.

2. Memory-intensive: KNN needs to store the entire training dataset in memory for making predictions. This can be a challenge when dealing with large datasets, especially if the dataset has high dimensionality.

3. Sensitivity to feature scaling: KNN calculates distances between data points, and if the features have different scales, those with larger scales may dominate the distance calculation. It is important to preprocess and normalize the features to ensure fair comparisons.

4. Curse of dimensionality: KNN performance can degrade when the number of features (dimensions) in the dataset is high. In high-dimensional spaces, data points become more sparse, and the notion of distance becomes less meaningful, leading to reduced accuracy.

5. Lack of automatic feature selection: KNN considers all the features equally when making predictions. It does not perform automatic feature selection or feature weighting, which means irrelevant or noisy features can negatively affect the algorithm's performance.