Efficient/Edge AI

Summary

Designing the whole pipeline – edge based neural network architecture, pruning and quantization techniques and automatic embedded code generation.

The story

The work in this project was done mainly under a technology demonstrator (or proof of concept) for CeADAR - Ireland’s Center for Applied AI, which can be checked here. The whole pipeline starting from creating a deep neural network with low memory and computational footprint to pruning and quantizing that network for deployment and finally automatically generating embedded code to deploy the network on an embedded device running ARM processor was created. The diagram below shows this whole pipeline.

Fig. 1: Overview of complete Edge AI pipeline

Collaborating with Dr. Deepu John, the problem we tackled was anomaly detection in heartbeats and proposed a novel architecture inspired from auto-encoders. Adding additional neurons for feature augmentation to auto-encoders and chopping off the decoded layer we got a classifier with very low memory and computational footprint, making it easier to deploy on edge. The diagram below shows the architecture.

Fig. 2: Architecture of deep neural network deployed at the edge

We also created a novel pruning technique based on the ability of neurons to separate classes. This technique was patented and published and the diagram below shows the metric we proposed class-separation score (css). For more details check the paper here.

Fig. 3: Class-separation-score, a metric we designed to prune neural networks

Also, a collaboration with UCD Laboratory for Advanced Manufacturing Simulation and Robotics led to the paper, “Review and application of Edge AI solutions for mobile collaborative robotic platforms”, in Procedia CIRP, which can be found here.

This is still an exciting field with lots of new inventions coming frequently, like more specialized hardware and Neural Architecture Search (NAS) for the edge.