ARTIFICIAL INTELLIGENCE
Improving operational efficiency
with AI and Deep learning capabitlities

Network Training

The training phase of the neural network determines the weight coefficients of each neuron of the network by performing the necessary inference calculations. Only the inference deployment phase requires optimization on the actual embedded platforms being deployed onboard. The training phase of the neural network can be achieved offline on servers in the cloud using popular frameworks such as TensorFlow, Caffe, Pytorch.

Network Accelerators

Determining the type of neural network accelerator to be utilized in the embedded system is necessary as relying only on the CPU might saturate its capability and increase the power dissipation. This is demonstrated when using the ResNet-50 standard benchmark for comparing inference execution times and associated CPU workload.

Accelerators options

• Use a vector engine inside the CPU
• GPU, either external or discrete
• VPU (Vision Processor Unit)
• Use a FPGA

Inference Framework

This phase can be particularly complex. However, all Intel devices capable of calculating inferences are supported over a single inference framework toolkit named OpenVINO™ (Intel® Open Visual Inference and Neural network Optimization). With this inference framework, trained neural networks are converted and optimized depending on the target hardware, whether it is a CPU, a CPU with vector acceleration, a CPU with an integrated VPU or GPU, a CPU hosting a dedicated external accelerator, or a FPGA.

With a powerful and dedicated API supported by OpenVINO™, only a few lines of code of Python or C++ are necessary to acquire successive images from a video, scale them appropriately, do some early classic image processing (light, contrast, resizing…), run through the network, and finally rebuild a video stream annotated with detections performed by the neural network. One of the major benefits of OpenVINO™ is the execution of inferences on any Intel® platforms with minimum changes necessary to the application, with or without taking advantage of the different acceleration available on the hardware. Later, during the application design, designers can decide whether, or not, to include external hardware depending on the actual total workload and overall power dissipation.

Movidius/Intel VPU accelerating deep learning inferences

• 1 Tops of Deep Neural Network (DNN) operations, only 2 Watts
• Intel OpenVINO toolkit uses models from Tensor Flow, CAFFE, …
• OpenVINO human pose demo running on TRACe platforms
• 7 FPS , 2W accelerator
• Multiple accelerators x2 or x4 possible
• Reduce power consumption/workload of main Intel CPU/GPU

White-paper coming soon ! Accelerating Neural Networks for Public Transportation