PIXTERA

PIXTERA
Image Processing Library & algorithms

---

Overview
Features
IP samples
Applications
Markets
Image & Video processing power
HDL coding development
Memory
Video Input / Output
Control & Data I/O
Power supply
Typical processing algorithms
Image enhancement
Image analysis
Projects

OVERVIEW

---

• Hardware video processing, up to a tera operations per second (1012), FPGA/HDL based
• User accessible image processing framework environment to speed-up algorithm portage
• Nexvision’s image pre-processing advanced algorithm library
• Nexvision’s technical support for MATLAB or C to HDL coding optimization

FEATURES

---

• High performance FPGA (Field Programmable Gate Array), based on latest 28nm process, low power
• Ultrafast & High resolution frame grabbing and processing – 12Mpixel@180fps
• Low Size, Weight and Power (SWaP)
• Support up to 4 independent video sensor boards

IP SAMPLES

• Deconvolution
• Fusion of multiple shoot
• Optical enhancement and corrections
• Vibration stabilisation
• Compression/decompression
• Human body detection
• Shape, character recognition
• Traffic accident detection, behaviour & flow control
• Depth map
• 3D scene reconstruction

APPLICATIONS

• Opto-electronics devices
• Multimedia processing
• Gyrostabilised video (broadcast, defense) on UAV
• Vision based vehicle guidance system or driver augmented reality
• Multispectral video boarder surveillance
• Multitarget video tracking
• Night vision, 3D range gating active imaging
• Blind Deconvolution
• Crypto accelerator
• Printed board, circuit board and LCD inspection
• Cinema video camera with real time effects
• Video post-production processing
• Automated license plate recognition (ANPR)
• High end digital signage or real time interactive showcase
• Compact low weight, low power consuming vision solutions
• Video camera with CPU intensive image processing (hand held smartcam)
• Multispectral / Immersive / Omnidirectional 360° camera (virtual speedome)
• Embedded video enhancement and image analysis, indexing before compression & transmission

MARKETS

• Vision
• Defense
• Medical image processing
• Transport
• High end video surveillance (boarders, urban, CCTV)
• Industrial process supervision and visual inspection
• Law enforcement & forensic
• High end multimedia
• High-Performance Computing (HPC)

IMAGE & VIDEO PROCESSING POWER

• Unprecedented video pixel-crunching abilities
• Highly parallelized/vectorial pixel processing
• Multicore Single-Instruction-Multiple-Data (SIMD) processing
• High bandwidth pixel-video random access streams on local FPGA SRAM & external QDR-SRAM
• High bandwidth pixel-video linear access streams on external DDR3-SDRAM
• Very low latency

HDL CODING DEVELOPMENT ENVIRONMENT

• Developer is free to focus on algorithm development and optimization (C/MATLAB/software/HDL coding only), as the hardware video architecture is already defined.
• Reduced integration time
• Upgrade path for new features and performance enhancements

MEMORY

DDR- SDRAM (high capacity, high speed)

• 6 x banks DDR3 1066MHz
• 2Gbits per bank, 16bits bus

SRAM (high speed, low random read access latency)

• 2 x QDR III @ 600MHz
• 144Mb per bank, 18bits bus

VIDEO INPUT/OUTPUT 30 pin FPC connectors

VIDEO INPUT

• 4 independant video sensor board , each up to 200Mpixel/s, 16 bits
• Free format : RAW bayer 10/12 bits, RGB422, ITU-BT1120, …

VIDEO OUTPUT

• 2 x 200Mhz 16 bits

CONTROL & DATA INPUT/OUTPUT

• PCI Express x4 Interface

POWER SUPPLY & PHYSICAL DIMENSIONS

• Input : 5VDC, 20W max
• Processing Board : 93mm (L) x 60mm (l) x 8mm (h)
• Temperature : 0°C to 50°C (-40°C to +85°C optional)
• Humidity : 10 – 90% non condensing

TYPICAL PROCESSING ALGORITHMS

IMAGE ENHANCEMENT

Image sensor processing

• CFA Bayer pattern to RGB (demosaïcing), state of art non linear algorithms for very high quality color interpolation (*)
• Auto exposure : fast adaptive for highly changing scene illumination conditions (*)
• Multi-exposure or multiresolution, on a frame by frame basis (*)
• Color matrix correction : dynamic, scene and illumination measurement based (*)
• Automatic white balance
• Application specific (*)
• Gamma conversion & YUV/HSI color conversion, histogram, logic, LUT mapping, segmentation, and thresholding (*)
• Dead pixel correction
• IR sensors non uniformity correction (*)
• Anti flickering (*)

Image shoot enhancement

• Dynamic local tone mapping (Shadows and highlights)
• High dynamic range (16bits resolution based)
• 3D noise Filter (spatial-temporal). (*)
• Contrasts and edges enhancement (algorithm type : USM – Unsharp mask) (*)
• Spatial filters

Deconvolution (*)

• Image reconstitution model which integrate bayer pattern and color aberration (*)
• Parallel algorithm on fpga which deconvolves the image using recursive algorithm which converge in few iterations. (new patent pending approach) (*)
• Process of Deconvolution which integrate the sensor’s noise and which improve the denoising or the deblurring in function of user’s goal (*)
• Myopic deconvolution to estimate the psf of optical and atmospherical aberration (based on fractal found in natural image) (*)
• Lens defocused mechanically to estimate the psf (*)
• Motion of camera could be integrated in deconvolution process (*)
• Multi-channels deconvolution (*)

Frequency domain transformation

Fusion

• Multiple shoot and/or multi angle of view in multiple spectral band : visible, Shortwave InfraRed and Thermal InfraRed (MWIR/LWIR), Terahertz, etc… )
• Fusion with non-linear co-registration warping algorithm that corrects for visible+VNIR+SWIR versus thermal IR parallax and optical distortions

Autofocus : real time focus tracking with lens control loop (*)

Optical enhancement and corrections

• Optic’s aberrations corrections and super-resolution (*)
• Atmospherical aberration correction (*)
• Lens distorsion correction
• Lens distorsion lateral and longitudinal chromatic aberrations, vignettage (relative illumination) correction
• Lens barrel distorsion distorsion correction
• Ultra wide angle lens projection correction (360° « FishEye » circle image real time dewarping) (*)
• Co-optronics designed optics : extended depth of view/digital autofocus – wave front coding
• Anisotropic 2D image scaling

Vibration Correction – Video stabilisation

• Close control loop multilevel stabilisation depending on amplitude-period (mechanical, optical, electronic and software)
• Lens/mirror piezoactuator stabilisation control for angular movement correction (yaw and pitch)
• Piezo-microactuator image sensor micro-scanning stabilisation control (close loop with 6 axis gyro-accelero sensors Nexvision’s “MULTILINK” board) (*)
• Motion compensation : virtual windows counter motion centring using feature points video tracking
• Viewer pointed, automatic target tracking (gyrostabilized pan-tilt-zoom)

Virtual pan-tilt-zoom

• Ultra high resolution video (X-HDTM) (*) with Nexvision IMA12MC
• Multiple video sensors stitching to create panoramic images (*) with IMAPANO
• 360° high-resolution, real-time dewarp video (*)

2D/3D calibration

Compression/decompression

Studio visual effects

Back ground discrimination, correlation, gradient operation, Hough transformation, morphology, projection, edge thinning, line verification, rule based post processing, convolution, motion adaptive deinterlace, image restoration, etc…

User define algorithms

IMAGE ANALYSIS

• Feature points extraction and analysis
• Fire detection
• Pattern matching
• Texture recognition
• Suspicious stationary object detection
• Motion detection (*)
• Human body detection
• Shape, character recognition
• Visitors/pedestrian counting
• Multi-modal gesture recognition
• Content based retrieval & Query-by-content
• Sensors fusion (GIS positioning, accelero-gyro, ultrasound, radar, ..)
• Multispectral band facial biometric recognition (Visible/SWIR/LWIR)
• Depth map
• 3D scene reconstruction
• Registration – unified scene alignment – translation & scaled perspective
• Traffic accident detection, behaviour & flow control
• Color analysis
• 1D and 2D measurements
• 1D and 2D code reading and verification