Supervised Learning - Classification Week 8 Challenge

1. Apply KNN to the “Surface defects in stainless steel plates” and identify the differences

Refer to python code. KNN outperforms RandomForest and GridSearchCV. Data is imbalance hence stratified sampling was done and output had many variables so it was first put into numpy array and mode of the data was used. Graphs have been plotted appropriately.

2. What are the pros and cons of knn

Pros and Cons of the K-Nearest Neighbors (KNN) Algorithm

---

✅ Pros of KNN

1. Simple and Easy to Implement:

   • KNN is intuitive and straightforward, making it ideal for beginners.
   • No assumptions about data distribution.

2. No Training Phase:

   • KNN is a lazy learner, meaning it doesn't require a training phase.
   • All computations happen during prediction, which simplifies the model.

3. Flexible with Distance Metrics:

   • Works well with various distance metrics (Euclidean, Manhattan, Minkowski), allowing adaptability based on data types.

4. Effective for Multi-Class Problems:

   • Handles multi-class classification well without needing significant modifications.

5. Naturally Adapts to New Data:

   • New data can be easily added to the model without retraining.

6. Good for Small Datasets:

   • Performs well when the dataset is small and well-structured.

---

❌ Cons of KNN

1. Computationally Intensive:

   • Since it calculates distances for every prediction, it's slow for large datasets.
   • The prediction phase can be expensive in terms of time and memory.

2. Sensitive to Irrelevant Features:

   • KNN doesn't perform feature selection, so irrelevant or redundant features can reduce accuracy.

3. Curse of Dimensionality:

   • As the number of features increases, the distance between data points becomes less meaningful, reducing accuracy.

4. Sensitive to Imbalanced Data:

   • If some classes are underrepresented, KNN may favor the dominant classes.

5. Feature Scaling Required:

   • Since KNN relies on distance calculations, features need to be normalized to prevent bias from large-scale features.

6. Storage and Memory Concerns:

   • KNN stores the entire training dataset, which can lead to high memory usage.

7. Choosing the Right 'K':

   • Selecting the optimal number of neighbors (K) is critical but often requires extensive experimentation.

---

When to Use KNN:

• Small to medium-sized datasets.
• When interpretability and simplicity are important.
• When the data is well-structured and low-dimensional.