Classification of Surface defects in stainless steel plates dataset with different algorithms

1) Apply knn to the “Surface defects in stainless steel plates” and identify the differences

Aim:

To apply the knn algorithm to the "faults" dataset and to compare the results obtained from other algorithms

Method :

Predictive analysis :

Code :

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
#Functions to calculate training and test accuracy
def model_score_tr(model,x_train,x_test,y_train,y_test):
    model.fit(x_train,y_train)
    return model.score(x_train,y_train)
def model_score_ts(model,x_train,x_test,y_train,y_test):
    model.fit(x_train,y_train)
    return model.score(x_test,y_test)

df=pd.read_csv("faults.csv")
print(df)
print(df.columns)
print(df.info())
print(df.describe(include="all"))
print(df.isnull().values.any())
#plt.figure(figsize=(15,10))
#sns.heatmap(df.corr(),cmap="coolwarm")
df=df.drop(['X_Maximum','Y_Maximum','X_Perimeter','Log_X_Index','Sum_of_Luminosity','Log_Y_Index','Outside_Global_Index'],axis=1)
print(df.shape)
#df.hist(figsize=(15,15))
#df.plot(kind='box',layout=(6,5),subplots=True,sharex=False,sharey=False,figsize=(15,15))
columns_label=df.columns.values[-7:]
print(columns_label)
targets=(df.iloc[:,-7:]==1).idxmax(1)
print(targets)
#sns.countplot(targets)
x=df.iloc[:,0:21]
print(x)
from sklearn.preprocessing import StandardScaler
sc=StandardScaler()
X=sc.fit_transform(x)
from sklearn.model_selection import train_test_split
x_train,x_test,y_train,y_test=train_test_split(X,targets,test_size=0.2,random_state=9)
#Training score
trs=[]
#Test score
ts=[]
#Different types of algorithms
from sklearn import linear_model
trs=model_score_tr(linear_model.LogisticRegression(),x_train,x_test,y_train,y_test)
ts=model_score_ts(linear_model.LogisticRegression(),x_train,x_test,y_train,y_test)
from sklearn import tree
trs=np.append(trs,model_score_tr(tree.DecisionTreeClassifier(),x_train,x_test,y_train,y_test))
ts=np.append(ts,model_score_ts(tree.DecisionTreeClassifier(),x_train,x_test,y_train,y_test))
from sklearn import ensemble
trs=np.append(trs,model_score_tr(ensemble.RandomForestClassifier(n_estimators=40),x_train,x_test,y_train,y_test))
ts=np.append(ts,model_score_ts(ensemble.RandomForestClassifier(n_estimators=40),x_train,x_test,y_train,y_test))
from sklearn import svm
trs=np.append(trs,model_score_tr(svm.SVC(),x_train,x_test,y_train,y_test))
ts=np.append(ts,model_score_ts(svm.SVC(),x_train,x_test,y_train,y_test))
from sklearn.neighbors import KNeighborsClassifier
trs=np.append(trs,model_score_tr(KNeighborsClassifier(n_neighbors=5),x_train,x_test,y_train,y_test))
ts=np.append(ts,model_score_ts(KNeighborsClassifier(n_neighbors=5),x_train,x_test,y_train,y_test))
print(trs)
print(ts)

The accuracy for different algorithms were as follows

Algorithm	Training accuracy	Testing accuracy
Logistic regression	0.79	0.8
Decision tree	1	0.8
Random forest	1	0.83
Support vector machine	0.84	0.83
K nearest neighbours	0.85	0.83

Prescriptive analysis :

The accuracy can was highest when the k value is 5, There is a small underfitting issue in the knn algorithm which can resolved by taking more data samples. The accuracy can be also improved by performing a k-fold cross validation which gives us the average accuracy for all different train and test split.

Other algorithms like Decision tree, Random forest were faced with overfitting issues. SVM and Knn produce the best results for this dataset.

2) What are the pros and cons of knn

Pros :

• No training period required hence also called as lazy learning algorithm
• Easy to implement
• Many new data can be added without any changes to the model

Cons :

• Does not work well with high dimensional data
• Feature scaling is required
• Sensitive to noisy data and outliers