training, prediction, and evaluation of machine visions models in validation mode

a technology-agnostic abstraction for machine vision models.

“I trained a new model X with data from Y” you have heard this sentence. Someone said it during the daily scrum. There is a “new” model, you are told. The python code that trains the model is updated, and someone should validate it and push it to staging and then to production — but how? more importantly, what is the minimum feature set that turns a piece of code into a “machine vision model”?

what is the minimum feature set that turns a piece of code into a “machine vision model”?

A machine vision model should support three key functionalities: training, prediction, and evaluation. The prediction function is deployed when the model is served. The training function produces what is to be served, and the evaluation function is run at a frequency to track the model's performance and invoke additional training if required.

Every training, prediction, or evaluation starts with an ingest. Thus every ingest has a purpose: training, prediction, or evaluation.

You ingest data with a purpose: to train a model, run some prediction on a model, or evaluate a model. Ingest connects to the backend databases and runs a set of queries on the metadata to find the relevant images, videos, annotations, and datasets on the cloud.

blue_driver is a 3-class TensorFlow image classifier that takes in 128x128 colour images and produces turn_left/turn_right/no_action decisions — more.

Code needs to be validated constantly, and models are not exempt. You need to train, run prediction on, and evaluate models when there is a change in either the model code or the code that consumes the model. This is where validation comes in. There is a need for training, prediction, and evaluation versions that run quickly to validate the latest code/model changes.

Validation training starts with ingesting the smallest amount of data that allows training and runs the train function to produce a validation model. A validation model is code-complete and enables the validation of the pipeline pieces further down into the model serve. Validation training also validates the training code.

A validation prediction ingests the smallest amount of data that allows a prediction and then runs the prediction function on it.

Similarly, a validation evaluation ingests the smallest amount of data that allows an evaluation and validates the evaluation code on it.

Here is a summary:

nurah runs game of life simulations on frames it receives through sqs from a Flir Lepton thermal image sensor 80x60 — simulation code courtesy of golpy.

This abstraction of machine vision models is technology-agnostic. I have applied it to semantic segmentation in TensorFlow, object detection in PyTorch, and image classification in TensorFlow for vehicle guidance.

see this repo for implementation of this abstraction for Kaggle vegetable classification and recognition using Transfer Learning on EfficientNet.