Supervised Learning - Classification Week 8 Challenge

Q1.

Performing KNN Classifier on the "Surface defects in Steel plates" dataset:

In this process we first need to import the data and then start to apply the algorithm.

The classification process will be performed in the following manner:

• Data Preparation and Cleaning.
• Data Visualization.
• Applying K-NN algorithm to the dataset.

Data preparation and cleaning:

In the first process we first need to import all the necessary libraries and the dataset on which we are going to perform the algorithm

Now dividing the data set into 'Faults' and 'Attributes' sets.

Apply KNN Algorithm

• In this process we first import the necessary sklearn libraries for the K-NN algorithm.
• Then we will scaled the data as data is too large it is difficult for the processor to run fast.
• After that we will split the data into test and training subset.
• Then after we will apply the K-NN algorithm to the dataset.
• We will check the efficiency of the model.
• We will try to predict the test model.
• Lastly we will create a heatmap of the confusion matrix

model accuracy when n_neighbors=40

Generating a heatmap of the confusion matrix for this model.

code in ipynb

Q2.

Pros of KNN

• Simplicity and Ease of Implementation: KNN is simple to understand and implement. It does not require any complex assumptions or parameters, making it an ideal choice for beginners in machine learning.

• No Training Phase (Lazy Learning): KNN is a "lazy learner" because it does not build an explicit model during training. Instead, it stores all the training data and makes predictions based on the most similar instances. This makes it very fast during training since no actual computation is needed.

• Adaptability to Different Types of Data: KNN can handle both classification and regression problems. It is versatile and can be used for multi-class problems as well. It also works well with a wide variety of distance or similarity metrics (e.g., Euclidean, Manhattan, Minkowski).

• Intuitive Interpretation: The predictions made by KNN can be intuitively understood since they are based on the similarity of test samples to known training samples. This makes it easier to explain the model's predictions to stakeholders or non-technical audiences.

• No Need for a Parametric Model: KNN does not assume any underlying distribution of data. Unlike some parametric models (e.g., linear regression), it does not need any prior assumption about the distribution of data, making it flexible to use in various domains.

• Good Performance with Sufficient Data:

  KNN can provide high accuracy and good performance with enough representative data, especially when the decision boundary is non-linear and complex.
  

Cons of KNN

• Computationally Expensive at Prediction Time: Since KNN stores all training data and performs computations during the prediction phase, it can be very slow when dealing with large datasets. The time complexity is O(n) for each prediction, where n is the number of training samples.

• Memory Intensive:

  KNN requires storing all training data, which can be memory-intensive, especially for large datasets. This can be a significant drawback when dealing with high-dimensional data or large-scale applications.

• Sensitive to Irrelevant or Redundant Features:

  KNN is sensitive to the presence of irrelevant or redundant features since all features contribute equally to the distance calculation. This can degrade performance if there are noisy or non-informative features. Feature scaling (e.g., normalization or standardization) and feature selection can mitigate this problem.

• Poor Performance with Imbalanced Data:

  KNN can perform poorly when dealing with imbalanced datasets where some classes are significantly more prevalent than others. In such cases, the algorithm may be biased toward the majority class. It requires additional techniques like adjusting class weights or using distance-weighted voting.

• Sensitive to the Choice of K:

  The performance of KNN heavily depends on the choice of the hyperparameter K (the number of nearest neighbors). Choosing a value that is too small can make the model noisy and overfit the training data, while a value that is too large can over-smooth the decision boundary and underfit the data.

• Curse of Dimensionality:

  KNN is sensitive to the "curse of dimensionality." As the number of features (dimensions) increases, the distance between data points becomes less meaningful, and the algorithm’s performance may degrade. This is because, in high-dimensional spaces, data points tend to be equidistant from each other, making it harder to find true nearest neighbors.

• Does Not Learn an Explicit Model:

  KNN does not provide a learned, explicit model that can be analyzed or inspected. It lacks the interpretability of models like decision trees or linear regression, where you can see how input features influence the output prediction.