Project 9

AIM: To track objects in a video using Deep Sort and draw bounding boxes with object id.

Procedure:

Deep SORT: Deep Simple OnlineRealTime Tracking is a highly efficient algorithm for tracking detected objects with bounding boxes and assigning an id to every detected object in a sequence.

1. Detection
2. Estimation
3. Association, &
4. Track Identity creation and destruction.

Code:

1. For object detection, the TensorFlow centernet has been used with classes of 80.

2. KalmanFilter, Iou and cost matrix coding were borrowed from the GitHub

(!git clone https://github.com/niconielsen32/ObjectTracking.git )

3. Main Code to track objects in 'traffic.mp4'

1. Importing necessary modules

2. import cv2
   from google.colab import files
   import centernet
   import os
   import sys
   sys.path.append('/content/ObjectTracking')
   
   import numpy as np
   import PIL.Image
   import matplotlib.pyplot as plt
   import shutil
   from tqdm.auto import tqdm

3. import SORT function from sort.py to use kalmanFilter and import create_video and show_video and VisTrack from lib.py to create video and draw boxes and ids on frames of videos

4. from sort import Sort, KalmanBoxTracker
   from lib import VisTrack, show_video, create_video
   
   vt = VisTrack()

5. Detection of classes using centernet

6. #numofclasses = 80 by default
   obj = centernet.ObjectDetection(num_classes = 80)
   #preloaded coconames will be laoded so weights_path is none
   obj.load_weights(weights_path = None)

7. reading video and creating folder for output

8. cap = cv2.VideoCapture('traffic.mp4')
   
   folder_out = 'Track'
   if os.path.exists(folder_out):
     shutil.rmtree(folder_out)
   os.mkdir(folder_out)

9. Calling SORT to apply KalmanFIlter with Intersection over Union to predict object flow and assign the same id in subsequent frames for the same object

10. sort = Sort(max_age=1, min_hits=3, iou_threshold=0.4)

11. Reading frame by frame and predict the object using centernet and update using SORT and drawing bounding boxes with object ids

    #process bar
    pbar = tqdm(total=length)
    i=0
    while True:
      ret, frame = cap.read()
      if frame is None:
        print('process done')
        break
      
      boxes, classes, scores = obj.predict(frame)
      detections_in_frame = len(boxes)
    
      if detections_in_frame:
        idxs = np.where(classes)[0]
        boxes=boxes[idxs]
        scores = scores[idxs]
        classes=classes[idxs]
      else:
        boxes = np.empty((0,5))
      # cv2.rectangle(frame, (int(boxes[0]), int(boxes[1])), (int(boxes[2]), int(boxes[3])), (0,0,255),2)
    
      # cv2.imshow('frame', frame)
      # key =cv2.waitKey(1)
      # if key ==27:
      #   break
      #print(scores[:,np.newaxis])
      det = np.hstack((boxes, scores[:, np.newaxis]))
      #print(det)
    
      res = sort.update(det)
      
      boxes_track = res[:,:-1]
      #print(boxes_track)
      boxes_ids = res[:,-1].astype(int)
      #print(boxes_ids)
      p_frame = PIL.Image.fromarray(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))
    
      if detections_in_frame:
        p_frame = vt.draw_bounding_boxes(p_frame, boxes_track, boxes_ids, scores)
      p_frame.save(os.path.join(folder_out, f"{i:03d}.png"))
      
      i+=1
      pbar.update(1)​

12. creating video from saved images using create_video and show video function in lib.py

13. tracking_video = 'tracking.mp4'
    create_video(frames_patten='Track/%03d.png', video_file = 'tracking.mp4', framerate=25)
    show_video('tracking.mp4')

Results:

The above coding was performed in google colabs

and the output is as shown in the video link