Supervised Learning - Classification Week 8 Challenge
1) Applying KNN to the “Surface defects in stainless steel plates”
Steps:
#Importing libraries
#Importing dataframe
#Performing Exploratory data analysis
#checking missing values
The dataset consists of 1941 instances and 34 columns. It does not have any null values. The results are in one hot-encoded format.
* The last 7 columns were target classification. The column names were assigned to label_columns and all the values were indexed using df.iloc and assigned to variable targets.
#printing targets
'
#the last 7 columns were dropped and created a new column which consists of all targets
Printing target counts:
Visualizing the data with pie chart and histogram plot
Visualizing the correlation with heatmap
**With the help of a heatmap, we can conclude that there is a high correlation between X minimum and X maximum, Y minimum and Y maximum, and type of steel A300 and A400.
Based on our conclusions from the heatmap, we drop type of steel A400, X minimum and Y minimum.
The feature values are assigned to variable x and target values to variable y.
The x and y values are then split into training and testing sets. The test size is 30% of the size of the complete dataset.
Applying KNN to data and predicting the accuracy
**The accuracy of model is 50.42% with test size of 30%
Conclusion: KNN model is able to predict bumps, K scratch and Z scratch more accurately but fails to predict 'other faults' correctly.
Comparing to SVM model KNN accuracy is lesser.
2) Pros & Cons of KNN:
| Pros | Cons |
|
K-NN is pretty intuitive and simple: K-NN algorithm is very simple to understand and equally easy to implement. To classify the new data point K-NN algorithm reads through whole dataset to find out K nearest neighbors. |
K-NN slow algorithm: K-NN might be very easy to implement but as dataset grows efficiency or speed of algorithm declines very fast. |
|
K-NN has no assumptions: K-NN is a non-parametric algorithm which means there are assumptions to be met to implement K-NN. Parametric models like linear regression has lots of assumptions to be met by data before it can be implemented which is not the case with K-NN. |
Curse of Dimensionality: KNN works well with small number of input variables but as the numbers of variables grow K-NN algorithm struggles to predict the output of new data point. |
|
No Training Step: K-NN does not explicitly build any model, it simply tags the new data entry based learning from historical data. New data entry would be tagged with majority class in the nearest neighbor. |
K-NN needs homogeneous features: If you decide to build k-NN using a common distance, like Euclidean or Manhattan distances, it is completely necessary that features have the same scale, since absolute differences in features weight the same, i.e., a given distance in feature 1 must means the same for feature 2. |
|
It constantly evolves: Given it’s an instance-based learning; k-NN is a memory-based approach. The classifier immediately adapts as we collect new training data. It allows the algorithm to respond quickly to changes in the input during real-time use. |
Optimal number of neighbors: One of the biggest issues with K-NN is to choose the optimal number of neighbors to be consider while classifying the new data entry. |
|
Very easy to implement for multi-class problem: Most of the classifier algorithms are easy to implement for binary problems and needs effort to implement for multi class whereas K-NN adjust to multi class without any extra efforts. |
Imbalanced data causes problems: k-NN doesn’t perform well on imbalanced data. If we consider two classes, A and B, and the majority of the training data is labeled as A, then the model will ultimately give a lot of preference to A. This might result in getting the less common class B wrongly classified. |
|
Can be used both for Classification and Regression: One of the biggest advantages of K-NN is that K-NN can be used both for classification and regression problems. |
Outlier sensitivity: K-NN algorithm is very sensitive to outliers as it simply chose the neighbors based on distance criteria. |
|
One Hyper Parameter: K-NN might take some time while selecting the first hyper parameter but after that rest of the parameters are aligned to it. |
Missing Value treatment: K-NN inherently has no capability of dealing with missing value problem. |
|
Variety of distance criteria to be choose from: K-NN algorithm gives user the flexibility to choose distance while building K-NN model.
|