Easy video object detection Process using Tensorflow

Here I explain complete end to end tenorflow object detection Deployment set up.

Below are the steps we are gonna follow:

1. Setting up the Tensorflow object detection api
2. Building a basic video object detection model using pretrained models
3. Building a basic video number plate recognition model using pretrained weights
4. Set up the Tensorboard for visualization of graph
5. Set up the Tensorflow serving for deployment
6. Object detection using Tensorflow serving
7. Reportbee Docker Image for Machine Learning and Data Science.

1. Setting up the Tensorflow object detection api:

Step1: Clone the Tensorflow model:

git clone https://github.com/tensorflow/models

Step2: Install the slim

cd models/research/slim/
sudo pip install -e .

Step 3: Set the system path

# From tensorflow/models/research/export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim

Step 4: protobuf-compiler installation

# From tensorflow/models/research/
protoc object_detection/protos/*.proto --python_out=.

Note:In case if above gives error use below, it is because of old version , download latest version as below.

Manual protobuf-compiler installation and usage

If you are on linux:

Download and install the 3.0 release of protoc, then unzip the file.

# From tensorflow/models/research/
wget -O protobuf.zip https://github.com/google/protobuf/releases/download/v3.0.0/protoc-3.0.0-linux-x86_64.zip
unzip protobuf.zip

Run the compilation process again, but use the downloaded version of protoc

# From tensorflow/models/research/
./bin/protoc object_detection/protos/*.proto — python_out=.

Step 5:Check the completion of the installation from research folder

python object_detection/builders/model_builder_test.py

Reference:

https://github.com/tensorflow/models/blob/master/research/delf/INSTALL_INSTRUCTIONS.md

https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/installation.md

2. Building a basic video object detection model using pre-trained models

Step1: Select the pre-trained model from model zoo https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md

Step2: Follow below link for image object recognition

https://github.com/tensorflow/models/blob/master/research/object_detection/object_detection_tutorial.ipynb

Step3: Apply on video

To apply above code on video you can use below implementation:

from moviepy.editor import VideoFileClip
write_output = 'output_video.mp4'
clip1 = VideoFileClip("input_video.mp4")
white_clip = clip1.fl_image(function_name_to_apply_on_image)
%time white_clip.write_videofile(write_output, audio=False)

Note:If in case it gives error ,Try adding libraries to python path.
Run the following command from tensorflow/models directory.

# From tensorflow/models/
export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim

Now the code will run successfully.This command needs to run from every new terminal you start. If you wish to avoid running this manually, you can add it as a new line to the end of your ~/.bashrc file.

You can find your object detection video output in ‘output_video.mp4'.

Find my complete implementation in the below link:

https://github.com/kalpa-subbaiah/Vehicle-Detection/blob/master/Vehicle_detection_Tollgate_video.ipynb

Output Video:

3. Building a basic video number plate recognition model using pre-trained weights

Step1:

Follow matthewearl’s blog http://matthewearl.github.io/2016/05/06/cnn-anpr/ to understand the number plate recognition approach using Tensorflow .

Step2:

1. Download pre-trained model weights.npz from here given by AnjieZheng.

https://github.com/AnjieZheng/Tensorflow-Number-Plate-Recognition.

2. Place weights.npz in the same directory of source.py.

3. Run the below command:

./detect.py in.jpg weights.npz out.jpg

4. To convert this code for video use VideoFileClip approach as mentioned above.

You can find your number plate recognition video output in ‘numplate_output_video.mp4’.

Find my complete implementation in the below link:

https://github.com/kalpa-subbaiah/Number-plate-recognition--Tensorflow/blob/master/Number_Plate_Recognition.ipynb

Output Video:

4. Set up the Tensorboard for visualization of graph

Step 1:Add the below commands in sess.run to get the summary in logs

writer = tf.summary.FileWriter(‘logs’)
writer.add_graph(sess.graph)

Step 2: To run TensorBoard, use the following command

tensorboard --logdir=’logs’

Step 3: Verify the files generated in the logs folder

Check whether the event file ‘events.out.tfevents.gpu’ is stored in logs.

Step 4:Launch Tensorboard using below url:

http://172.16.0.205:6006/

Note:In case if it says ‘Tried to connect to port 6006, but address is in use’

Get the Tensorboard pid using tensorboard –logdir pid and kill the earlier process id.

Step 7: To view the pre-trained model graph in Tensorboard use below code:

python import_pb_to_tensorboard.py — model_dir=’faster_rcnn_nas_lowproposals_coco_2017_11_08/frozen_inference_graph.pb’ --log_dir=’fastercnnlogs’
tensorboard --logdir=fastercnnlogs

Refer https://neptune.ai/blog/tensorboard-tutorial

5. Set up the Tensorflow serving for deployment

Step 1:Follow below link to Install docker https://docs.docker.com/glossary/?term=installation

Step 2:Download the TensorFlow Serving Docker image and repo

docker pull tensorflow/serving
git clone https://github.com/tensorflow/serving

Step 3:Location of demo models

TESTDATA="$(pwd)/serving/tensorflow_serving/servables/tensorflow/testdata"

Step 4:Start TensorFlow Serving container and open the REST API port

Test the Tensorflow serving using below model and test data

docker run -t --rm -p 8501:8501 \
-v "$TESTDATA/saved_model_half_plus_two_cpu:/models/half_plus_two" \
-e MODEL_NAME=half_plus_two \
tensorflow/serving &

Step 5:Query the model using the predict API

curl -d '{"instances": [1.0, 2.0, 5.0]}' \
   -X POST http://localhost:8501/v1/models/half_plus_two:predict

Step 6:Returns the predictions

{ "predictions": [2.5, 3.0, 4.5] }

6. Set up the Tensorflow serving for object detection

Step 1: Add below script and get a container wrapping our model.

https://gist.github.com/adabb12c69cced99d3864d601d033001.git

Step2:Choose a model to deploy in the model zoo.

For our example, we will use “ssd_resnet_50_fpn_coco”

Step3:Note the url of the model in zoo (you don’t have to download this file): MODEL_URL=http://download.tensorflow.org/models/object_detection/ssd_resnet50_v1_fpn_shared_box_predictor_640x640_coco14_sync_2018_07_03.tar.gz

Step4: Build a custom image

git clone https://gist.github.com/adabb12c69cced99d3864d601d033001.git object-detect
cd object-detect
cp Tensorflow-Object-Serving.Dockerfile DockerFile
docker build -t object-detect — build-arg model_url=$MODEL_URL .
docker run -p 8080:8080 -p 8081:8081 object-detect

You now have a REST API on localhost:8080 serving you model.

Step5: Predict the image by executing below python file

import PIL.Image
import numpy
import requests
from pprint import pprint

image = PIL.Image.open("dog.jpg")  
image_np = numpy.array(image)  
payload = {"instances": [image_np.tolist()]}

res=requests.post("http://localhost:8080/v1/models/default:predict", json=payload)
pprint(res.json())

It gives the detection boxes , detection_scores,detection_classes and num_detection as output

7. Reportbee Docker Image for Machine Learning and Data Science:

Easy object detection using Reportbee Docker image

1. Pull image — docker pull reportbee/datascience
2. Run image — docker run --rm -d -p 6006:6006 -p 8888:8888 reportbee/datascience:latest
3. Open your browser and goto localhost:8888 and use the password reportbee to login to the jupyter notebook

4. The jupyter notebook has the models folder cloned from repo (https://github.com/tensorflow/models/tree/master/research/object_detection). You can navigate to the object detection folder (models/research/object_detection) and directly execute the object_detection_tutorial.ipynb notebook.