Skip to content Skip to footer

Science & Technology

Close
Science & technology

Re-inventing Spectral Imaging with the new, 4K, tunable filter and snapshot technologies

Hyperspectral Imaging (HSI)

  • Generates a 3D data structure that includes spectral and spatial info, known as Spectral Cube (SC)
  • SC images unveil invisible structures, while SC spectra inform about the identity and the composition of natural or manmade materials
  • HSI trained with application-specific Machine Learning models, enables the fully automated and instant material/object/condition identification and sorting

It is the technology of choice for examining spatially heterogeneous objects, remotely and nondestructively

  • HSI is like having super-powered vision. Normal cameras see the world in just red, green, and blue.
  • HSI go way beyond, seeing hundreds of colors, even the ones our eyes can’t detect
  • HSI maps the composition and characteristics of the imaged area, pushing the boundaries of what we can observe and understand about our world

  • Multispectral cameras can be customized with lesser, dedicated spectral bands to detect the presence and the quantity of a known target (e.g., chlorophyl content).

  • Hyperspectral cameras are the first choice for chemical imaging and for detecting multiple structures in complex environments

  • Evolution of spectral imaging is bio-inspired. For example, mantis shrimp can see 12 colors, extended to ultraviolet spectrum
  • HSI has been evolved through two paths by:
    1. Adding the spatial dimension to conventional spectrometers, through spatial scanning
    2. Adding the spectral dimension to cameras through full frame optical filtering  
  1.  

HSI is a booming business,  rapidly evolving along the second path, towards achieving spectral (hyper-) vision in e.g.,  robots, car sensors, cell phones, medical endoscopes   

Science & technology

Conventional HSI Technologies

A. Spatial Scanning Spectrometers : HSI evolved from spectroscopy

  • Known as Push-Broom systems, acquire spectra in a line-by-line fashion scanning the entire Field-of-View (FOV).
  • The spectral Images are subsequently constructed from the raw spectra
  • A two-dimensional sensor is utilized to record the spectra of one pixel line at a time, sharing the same optics with conventional spectrometers
  • Relative motion between the camera and the object is an absolute requirement to acquire a full SC. Thus , in static scene applications a mechanical translator is required to move the camera in relation to the object

Push Broom cameras: from hyperspectral to multispectral!

  • A hyperspectral camera detects hundreds of continuous wavelength bands using prism or grating-based spectrometer technology.
  • Light passes through a slit that spatially limits its incoming light and is then dispersed by a grating. The dispersed light is projected onto the sensor. The spectral Images are subsequently constructed from the raw spectra
  • The Grading-based push broom systems acquire hundreds of wavelengths, with no option to select the most informative subset. A significant amount of the acquired data is redundant, which delay and complicate analysis
  • Due to the line scan output their use is limited to applications in which either the target or the camera is in motion at a constant speed. In these applications however, the informative spectral bands are fewer and known. Due to this fact, the industry tends to adopt multispectral push broom technologies with a series of discreate filters attached on a 2D sensor

B. Spectral Scanning Imagers: HSI evolved from monochrome imaging

  • Known as Staring systems, they utilize a tunable band-pass filter to acquire full-image frame images at different wavelengths in a time sequential manner
  • The spectra are then calculated from the co-registered pixel values of the captured spectral image pack
  • The core technology of Staring systems is an electrooptic tunable optical filter, coupled with an imaging sensor. Narrow band images are captured in a plurality of filter’s tuning steps across the spectrum. Typical filter type technologies are: Fabry Perot, Liquid Crystall, Acousto-optic filters
  • The Staring HSI cameras need no mechanical scanners to acquire a SC. However, filter tuning for spectral scanning happens within a sub minute time frame, during which the imaging geometry shall remain stationary

Tunable filter example: Fabry-Perot (F-P) interferometry HSI Cameras

  • The imaging white light rays are trapped in a cavity formed by two mirrors, forming a resonant optical cavity.
  • Piezoelectric elements vary the distance between the mirrors, selecting the transited wavelengths as the ones experiencing constructive interference
  • F_P cameras are characterized by compactness and high tuning speed
  • F-P HSI suffer from a series of inherent limitations, such as remarkable cross-talking between the bands, resulting to inaccurate spectroscopy and poor spectral contrast. Moreover, it has limited tuning range (~300nm), significant loss due to multiple reflection, angle of incidence dependence requiring fixed lens

C. Non scanning/instant Imagers : HSI evolved from color imaging

  • Known as Snapshot systems, they instantly capture a set of full-frame images at different wavelengths. They share the same principle of operation with color cameras
  • Unlike conventional color or monochrome imagers acquiring single images, Snapshot cameras acquire packs of spectral images (Spectral Cubes) at video rate
  • The core technology of Snapshot systems is based on either a repeating filtering pattern of single pixels, using a single objective, or a tile filtering of pixel blocks, using lens array. They are suitable for analyzing dynamic phenomena or moving targets
  • One major drawback of this class of HSI is that in both cases the increase of the spectral information compromise the spatial information (resolution)

Snapshot cameras: new solutions under investigation

  • New snapshot camera solutions are currently under investigation. Typical examples are the coded aperture concept (up ) and various types of computational spectral imaging, such as metasurface filters or deep learning algorithms (bottom) for demultiplexing spatial from spectral data, for reconstructing the SC.
  • However, their current level of development display unsatisfactory performance. The complex task of SC reconstruction set impractical computational power requirements, with the consequence of prohibiting real-time operation and high-resolution imaging.

Trivial, suboptimal multispectral solutions

a. filter wheel revolving Infront of camera sensor
b. miniature lenses focusing on multiple sensors
c. miniature lenses focusing on a single, large area image array
d. active systems comprising a standard camera and a LED array
    

The deficiencies of the conventional HSI technologies …

Push broom Standard tunable filters Snapshot
Due to line scanning, spatial motion is an operational requirement Limited spectral scanning range (~300nm), requiring multiple units to scan an adequate spectral range Spectral resolution increases at the cost of spatial resolution (trade-off)
No control over acquisition data volume, redundant data Fixed, unchangeable objective lens, limits applications The wavelength of the acquired spectral bands is not customizable
Moderate spatial resolution Moderate spatial resolution Low spatial resolution: typical fraction of megapixel
Do not provide real-time spectral image acquisition for live inspection Broad and overlapping spectral bands, low and variable light throughput Delayed (off-line) data display due to post capturing processing or low-resolution live display due to limited bandwidth and spatial resolution
Low light throughput due to slit/dispersion element, absorption filters Delicate, error-prone design Low light throughput due to light rejecting band pass filters

...are the Spectricon’s innovation drivers

Science & technology

MUSES9 cameras: HSI re-invented

A. The MUSES9 HS product family: the tunable filter superiority

  • MUSES9 HS is based on a novel electro-optic tunable filter (EOTF) design, which offers the world’s: a) broader tunability range (370nm-1700nm); b) maximum light throughput (94%) and c) minimum, practically zero inter-band cross talking (5 OD blocking efficiency) 
  • The EOF encapsulates a specially constructed continuously variable bandpass filter, which is used as a mask in specially designed Fourier optics. The image collected by the objective lens passes through the EOTF, and when it is tuned, the different locations of the CVF transmit different wavelengths and block all the other wavelengths. The transmitted wavelengths are linearly varying with the location of the CVF in relation to a pinhole mask. Typical dispersion value is 2nm/mm
  • It acquires up to 315 spectral bands at 6.4 megapixel/per band, while offering real time inspection at any desired wavelength.

B. The MUSES9 SnS product family: the world’s first 4K snapshot HSI

  • The MUSES9-SnS innovative technology combines the real-time acquisition of a sufficient set of 4K level spectral images, with an AI-based spectral estimation technique to compose a full hypercube cube, without compromising spectral or spatial resolution.
  • It is bases on polychroic mirrors and optics to project imaging information to a set of imaging sensors. It produces up to 4 spectral images per imaging sensor. There is no light and spatial resolution loses
  • From the captured set, a new set of spectral images can be calculated with advanced spectral estimation models

MUSES9 Product Family

MUSES9 Camera Accessories

Deficiencies of competing scanning technologies Spectricon’s MUSES9 HS product family
Due to line scanning, spatial motion is an operational requirement Full frame spectral scanning requires no mechanical translators, making it field deployable
No control over acquisition data volume, redundant data It can acquire any set of spectral images selected with the SW
Moderate spatial resolution Industry’s best: 6.4 megapixels
Do not provide real-time spectral image acquisition for live inspection it acquires and real-time displays any selected spectral image
Low light throughput 94% transmittance
Limited spectral scanning range (~300nm), requiring multiple units to scan an adequate spectral range  with a single camera, up to 1300nm tuning range is achieved
Fixed, unchangeable objective lens, limits applications  Supports C-mount or F-mount universal lens giving freedom to select FOV distance etc
Broad and overlapping spectral bands, low and variable light throughput  The out-of-band light is efficiently blocked (5 OD blocking efficiency)
Delicate, error-prone design  Robust design offering high reproducibility
Deficiencies of competing snapshot technologies Spectricon’s MUSES9 SNS product family
Spectral resolution increases at the cost of spatial resolution There is no trade of between spatial and spectral resolution
The wavelength of the acquired spectral bands is not customizable It is customizable to a certain extent
Low spatial resolution: typical fraction of megapixel 6.4 megapixels
Delayed (off-line) data display due to post capturing processing or low bandwidth  Display with up to 45 frames/s up to 22 bands at 6.4-megapixel resolution each
Low light throughput due to light rejecting band pass filters High light throughput due to mirror-based filtering