Supervised Learning - Classification Week 8 Challenge

Answer no. 1 subitted in .ipynb form.

Answer no.2

The k-nearest neighbors (KNN) algorithm is a non-parametric classification and regression method that operates on the principle of finding the k nearest data points in the training set to a given test point and using the majority vote (for classification) or averaging (for regression) to make predictions. Here are some pros and cons of the KNN algorithm:

Pros of KNN:

1. Simplicity: KNN is relatively simple to understand and implement. It does not make any assumptions about the underlying data distribution and does not require extensive model training.

2. No training phase: KNN is an instance-based learning algorithm, meaning it does not explicitly train a model. The training phase involves only storing the training data, making the algorithm efficient in terms of training time.

3. Adaptability to new data: KNN can easily adapt to new data points without retraining the entire model. This makes it useful in scenarios where the data distribution is non-stationary or when new data is continuously added.

4. Non-linearity: KNN can capture complex nonlinear relationships in the data without explicitly assuming any functional form or using complex mathematical equations.

Cons of KNN:

1. Computational complexity: As the size of the training set grows, the computation time for KNN increases significantly. Since KNN requires calculating distances between the test point and all training points, the algorithm can become computationally expensive for large datasets.

2. Memory requirements: KNN requires storing the entire training dataset in memory, which can be a challenge when dealing with datasets that have millions or billions of records. This can limit the scalability of the algorithm.

3. Sensitivity to feature scaling: KNN relies on the concept of distance, and the choice of distance metric can be affected by the scale of the features. If the features have different scales, it is important to normalize or standardize them to ensure fair comparisons.

4. Determining the optimal value of k: The performance of KNN heavily depends on the choice of the parameter k, which represents the number of nearest neighbors to consider. Selecting an appropriate value for k requires careful consideration and often involves experimentation.

5. Imbalanced data: KNN can be biased towards the majority class when dealing with imbalanced datasets. This is because the majority class tends to dominate the nearest neighbors, leading to biased predictions.