Project 6 Mini Project

Implement an image classification model using MobileNet architecture

MobileNet is a lightweight Convolutional neural network to train models and run inference. It uses depthwise convolution for each colour channel R, G and B. MobileNet Architecture follows the pattern given below:

1. A Pointwise convolutional layer with stride 2
2. A convolutional depthwise layer with stride 1
3. A pointwise convolutional layer with a stride of 1
4. Another depthwise layer with stride 2
5. Pointwise layer with stride 1

This pattern is repeated multiple times to generate a convolutional neural network. A standard convolution layer/pointwise convolution layer consists of the following components:

1. A 3 x 3 Convolution filter
2. A Batch Normaliser
3. A ReLu activation function

The Depthwise convolution layer consists of

1. A 3 x 3 Depthwise Convolution filter
2. A Batch Normaliser
3. A ReLu activation function
4. A 1 x 1 Convolution filter
5. A Batch normaliser
6. A ReLu activation function

The Input Image dimensions of a MobileNet architecture are 224 × 224 × 3. The last layer of the MobileNet architecture is a SoftMax Classifier with a stride of 1.

Tensorflow and Keras were used to train the MobileNet Architecture based on the provided custom dataset. The following versions of Tensorflow and Keras were used:

tensorflow 2.6.5,
tensorflow-datasets 4.8.2,
tensorflow-estimator 2.6.0,
tensorflow-gcs-config 2.9.1,
tensorflow-hub 0.12.0,
tensorflow-io-gcs-filesystem 0.30.0,
tensorflow-metadata 1.12.0 ,
tensorflow-probability 0.17.0,
keras 2.6.0,
keras-vis 0.4.1

The Image preparation to run inference is given below:

def prepare_image(file_path: str):
    # using pre_processing module of keras library image is loaded and expanded and pre- 
    # processed
    img = image.load_img(file_path, target_size=(224, 224))
    img_array = image.img_to_array(img)
    img_array_expanded_dims = np.expand_dims(img_array, axis=0)
    return keras.applications.mobilenet.preprocess_input(img_array_expanded_dims)

Dataset Preparation

The Given custom dataset consists of two classes of images.

• The "cats" in the training_set folder had a total of 964 images
• The "dogs" in the training_set folder had a total of 4006 images

A small part of the training set was moved to the test_set. Literature suggests that the ratio of the test set and training set should be in the ratio 9:1. The dataset summary after preparation is given below:

Class Name	Number of images in test_set	Number of images in training_set
cats	100	864
dogs	400	3606

A total of 4469 images belonging to 2 classes were used to train the MobileNet architecture.

The ImageDataGenerator was used to generate the dataset for training. The following code was used to implement the same:

train_datagen=ImageDataGenerator(preprocessing_function=preprocess_input)
training_paths = '/content/gdrive/My Drive/Skill Lync/Week_6_Mini_project/images/training_set/training_set/'
train_generator=train_datagen.flow_from_directory(training_paths,
                                                 target_size=(224,224),
                                                 color_mode='rgb',
                                                 batch_size=32,
                                                 class_mode='categorical',
                                                 shuffle=True)

The training was done using the following block of code:

# A base model of the MobileNet architecture is instantiated
base_model=MobileNet(weights='imagenet',include_top=False) 

# The next last few layers are re-implemented based on MobileNet architecture
x=base_model.output
print(x)
x=GlobalAveragePooling2D()(x)
x=Dense(1024,activation='relu')(x) 
x=Dense(1024,activation='relu')(x) 
x=Dense(512,activation='relu')(x) 
preds=Dense(2,activation='softmax')(x) 

# The New model is set up
model=Model(inputs=base_model.input,outputs=preds)

# set the last 20 layers of the model as trainable
for layer in model.layers[:20]:
    layer.trainable=False
for layer in model.layers[20:]:
    layer.trainable=True

# Compile the model
model.compile(optimizer='Adam',loss='categorical_crossentropy',metrics=['accuracy'])

# step size is calculated from the previous step
step_size_train=train_generator.n//train_generator.batch_size

# Training started
model.fit_generator(generator=train_generator,
                   steps_per_epoch=step_size_train,
                   epochs=10)

Running Inference

Once the model has finished training, the inference is run on the model to generate by passing an image to the model. This is done using the following block of code:

img_path = '/content/gdrive/My Drive/Skill Lync/Week_6_Mini_project/dog_1.jpg'
new_image = load_image(img_path, show=True)
pred = model.predict(new_image)
print(pred)

The images are loaded using the following helper function:

def load_image(img_path: str, show=False):
    # using image module in keras.preprocessing load the image and expand the image 
    # so that inference can be run on the image.
    img = image.load_img(img_path, target_size=(150, 150))
    img_tensor = image.img_to_array(img)                   
    img_tensor = np.expand_dims(img_tensor, axis=0)         
    img_tensor /= 255.                                      

    if show:
        plt.imshow(img_tensor[0])                           
        plt.axis('off')
        plt.show()

    return img_tensor

The predictions are done using the following helper function:

def out_predictions(path: str, show=False):
  # image path is passed to the function load_image() 
  # model.predict() method is used to get the predictions
  new_image = load_image(path, show=show)
  pred = model.predict(new_image)
  return pred

Evaluation / Classification Metrics

The classification score is generated using sklearn.metrics module.

The y_true and y_pred are computed using the following code:

# A mapping array is identified
mapping_array = {0: 'cat', 1: 'dog'}
test_master_path = '/content/gdrive/My Drive/Skill Lync/Week_6_Mini_project/images/test_set/test_set'
y_true, y_pred, y_label = [], [], []

# for all images in cat folder in the test_set
cat_paths = os.path.join(test_master_path, f'cats')
cat_pathlist = Path(cat_paths).rglob('*.jpg')

for path in cat_pathlist:
  path_in_str = str(path)
  y_true.append('cat')

  # run predictions find the index of the largest prediction and 
  # find the index and append to y_label
  pred = out_predictions(path_in_str)
  highest_pred = pred[0].max()
  index = np.where(np.isclose(pred[0], highest_pred))[0][0]
  label = mapping_array.get(index)
  y_label.append(label)

# for all images in dog folder in the test_set
dog_paths = os.path.join(test_master_path, f'dogs')
dog_pathlist = Path(dog_paths).rglob('*.jpg')

for dog_path in dog_pathlist:
  path_in_str = str(dog_path)
  y_true.append('dog')

  # run predictions find the index of the largest prediction and 
  # find the index and append to y_label
  pred = out_predictions(path_in_str)
  highest_pred = pred[0].max()
  index = np.where(np.isclose(pred[0], highest_pred))[0][0]
  label = mapping_array.get(index)
  y_label.append(label)

The outputs of the various metrics are summarised below:

• The Accuracy score of the classification of the model is 0.952
• The Summary of the other metrics such as F1 Score, Precision Score, and Recall are:

Class name	Precision Score	F1 Score	Recall Score
dog	0.9454976303317536	0.9708029197080293	0.9975
cat	0.9871794871794872	0.8651685393258428	0.77

The links for the drive and the Colab are attached below:

Drive link - https://drive.google.com/drive/folders/11JxMfMRoPZEOjsoWY1K7rKR8bCj4L0wd?usp=share_link

Colab link - https://colab.research.google.com/drive/1UIvWpGWVoy8ciIEPwH1KUFfwzQbgH88h?usp=share_link