Part 3 :
Design

Understand the operation & potential of optronic systems

Computing power & interoperability

Now that the main concepts have been mastered, we know that to implement a powerful vision system, especially if it’s to be embedded, it is necessary to use the latest technologies in order to benefit from adaptive computing powers and evolutive systems.

For example, if the image sensor is a 2M pixel sensor, running at 25 frames / second, it will have to process 2 million pixels x 25, or 50 million pixels / second (this is approximate and is just to give an idea of the computing capacity required).

The goal is to get the system working in real-time and interacting with its environment.
At Nexvision, we don’t hesitate to implement the latest technologies straight out of the lab, be it in terms of image sensors, graphic processors or any other type of component.

As a matter of fact, we are one of the few companies in Europe who were the early adopters of Nvidia GPUs, and have been working very closely with Photonis to develop their EBCMOS sensors for night vision.

We not only develop complete systems, beginning with image sensors, but also include the development of an entire tailor-made system that goes with it; this being, the hardware as well as software, including the processing and / or image analysis algorithms.

In most cases, we create embedded electronic designs where CPU, GPU, NPU and FPGA constitute the heart of the system and the computing power.

Note:

Depending on the design, there may be:

  • Multiple parallel image sensors (with the same frequency band or different bands to capture different spectra)
  • Remote heads with embedded image pre-processing systems
  • Other types of sensors (e.g. radar, lidar) that complement the system and its interface to carry out the fusion of sensors, making the overall system even more precise.

ex : 1 camera design

ex : n camera design

CPU - Central Processing Unit

The Conductor !

Runs the operating system (OS) and performs algorithmic processing at a high level. Makes decisions, manages the human-machine interface, and the application in relation to a given profession.

GPU - Graphical Processing Unit

The analyst !

Does some image processing but mostly image analysis e.g.: can deal with point clouds => the GPU takes these characteristic points and does an image analysis to extract relevant information.
On the technical side, it does floating calculations.

FPGA - Field-Programmable Gate Array

FPGA is row computing power !

It takes raw images and performs pre-processing by improving and extracting their characteristic points. It is a true performer that does «simple» calculations on a large-scale.
On the technical side, it does full calculations.

HardWare: sequential logic in HW: strong parallelized data processor with access to defined memory (non-random) => implementation of algorithms in the form of a sequencer HW: not coming from the software.

Things to know

Coding an algorithm on FPGA takes on average 20 times longer than for a CPU / GPU (that is, 20 times more difficult = more expensive).
The first design selection starts with a CPU / GPU and if there’s not enough computing power, one or more FPGAs are added.

FPGA / ASIC / GPU / CPU / MANY CORES / NPU comparison