Image Classification using Torchvision Pre-trained Models in a Single YonoArc Block

Recently, I decided to learn how to implement machine learning (ML) algorithms, especially in the computer vision field, using the well-known package Pytorch. One of the useful resources, to break the ice with such a powerful ML package, is the PyTorch for Beginners series of tutorials offered by Learn OpenCV. The series contains some core topics that should be understood by the computer vision developers who are using Pytorch.

I follow the second tutorial with the title: PyTorch for Beginners: Image Classification using Pre-trained models. I am surprised by the amount of the architectures supported by the Torchvision module for image classification.

Torchvision package consists of popular datasets, model architectures, and common image transformations for computer vision. Basically, if you are into Computer Vision and using PyTorch, Torchvision will be of great help!

I thought it will be a beneficial idea to use YonoArc App in the YonoHub platform to encapsulate all the supported image classification architectures, by torchvision package, in a single block. The developers will have the facility of using only one YonoArc block to evaluate and benchmark the different performances of the well-known classifiers with its pre-trained weights.

YonoHub is the first cloud-based system for designing, sharing, and evaluating complex systems, such as Autonomous Vehicles, ADAS, and Robotics. Yonohub features a drag-and-drop tool to build complex systems consisting of many blocks, a marketplace to share and monetize blocks, a builder for custom environments, and much more.

How does the block work?

To have a good hand-on example, I use in the following demo the nuScenes dataset which can be streamed in the YonoArc pipeline using its supported player block. For more information about nuScenes-YonoArc package, you can read my previous article here.

Image Classification Demo

Torchvision Image Classifier YonoArc block has one input which intuitively of type Image. The source of the input images is the nuScenes Dataset Player as shown above. On the other hand, the block has two different outputs. The first output port is an image after writing the classes that the model is certain about. The other port is the classes associated with its confidence that the model predicted. The type of this output port is a custom ROS message which is included in the perception_msgs package I implemented for interacting with the YonoArc blocks in the field of computer vision.

The image classifier block has a property of choosing the model which you want to evaluate. As shown below, you have a drop list, under the title Model **Options, **with 31 different image classification models. You can reach this drop list by clicking on the setting icon of the block.

Image Classification Model Options

One great thing about YonoArc is that each block can use different resource models in the same pipeline. For example, the image classifier block uses GPU as a resource model. However, an eight-core CPU is the resource model of both the nuScenes player and the video viewer blocks.

Source Code

During the implementation of the presented block, I make use of the LearnOpenCV tutorial code. I manipulate it to be general and suitable for the YonoArc blocks code structure. YonoHub Docs is very useful to understand how to make a python 3 API YonoArc block. You can read this part of the docs to understand the changes happen to the tutorial code. You can check the source code below,

Results

After running the above pipeline, the below results have been produced. AlexNet model is chosen to predict the shown estimated classes.

Image Classification Results

Finally, I tried to demonstrate how to use YonoArc App in the YonoHub platform to implement the image classification tutorial by Learn OpenCV. Merging the benefits of both Pytorch, especially Torchvision, and YonoArc App to have an encapsulated image classification library of models in the shape of a block. You can purchase the Torchvision Image Classifier block for free, through YonoStore, and try the different models available without dealing with the code details. In addition, you can clone the above source code from here to reuse it for such blocks. Follow the steps in the referred tutorial to learn how to implement a YonoArc block.

Next, I will work on the semantic segmentation tutorial to have a similar reusable YonoArc block. It’s easy to try out Yonohub. New users receive $25 free credits. Sign up on Yonohub!