Dec 19, 2021
machine vision: a personal story
if you’ve known me for the past 3–4 years you know that I’m working on blue — blue is a cloud-connected rover that applies a tensorflow semantic segmentation deep learning convolutional neural network on images that it receives from its camera to maneuver safely in its environment.
fast forward to the end of 2021 blue uses S3 for transferring assets, such as images and models, and SQS and EC2 for communication and model train/eval/predicts.
This is a story of technology — and a personal one.
This story begins on November 8, 2018. At the time I worked for a medical startup that had an Android-based RDT reader with machine vision capabilities in the market. To keep the product running we needed to train and release a set of models per RDT type: for detection and analysis and so on. Multiply that with hundreds and thousands of RDTs that are in the market and that are introduced over time and it was clear that we needed a backend: a cloud backend.
This is a story of finding the tools necessary for building a cloud backend: my preferred software stack is python on top of bash. This is also a story of finding the necessary cloud services to carry data and metadata, execute code on data and metadata, and communicate between the machines that are running the code on the cloud and at the edge, i.e. a rover in the field. This is a story of finding the hardware components and learning how to 3d print using FDM and SLA.
A cloud backend is a nice thing to have and you can do cool things with it. Here are two examples, one pure software and one involving hardware.
This is an ongoing story. Next time I will write about validating that using this code I can control a Sphero RVR using the keyboard. Then I will refactor the code to add the cloud pieces and so on.
