Xperi develops world-first In-cabin monitoring technologies on Neuromorphic vision systems

New sensing technologies complement standard vision system capabilities, enhancing occupant safety, enabling increased personalization, advancing in-cabin experience to tomorrow’s third space experience

DTS®, a global leader in next-generation audio, imaging and sensing technology, and a wholly owned subsidiary of Xperi Holding, announced a world-first neuromorphic driver monitoring solution (DMS), powered by Prophesee Metavision® – Event-Based Vision sensor.

Using the raw feed from the Metavision® – Event-Based Vision sensor, the DTS AutoSense* team developed driver monitoring centric features such as gaze tracking, head pose, identification, and eyelid opening.

With more than 20 years of world experience in neural solutions and computer vision, and with billions of products powered by their solutions, the Xperi team maps and adapts in-cabin monitoring (ICM) technologies to existing and future sensor centric trends.

“Intelligent sensing is required by the next-generation in-cabin experience. The critical path to this is a sensor fusion-focused advanced research strategy,” said Petronel Bigioi, chief technology officer, Xperi. “DTS AutoSense’s advanced research team is centered on state-of-the-art sensing solutions and neuromorphic sensing is one of the technologies that can shape the future of the industry.”

The neuromorphic sensors capture information at an equivalent frame rate of 10,000 frames per second without requiring active illumination. This enables better low light performance for driver monitoring features as well as never seen before capabilities such as saccadic eye movement or micro-expressions, crucial next steps in the next-generation in-cabin experiences.

Xperi’s implementation has 12 patents pending and relies on an end-to-end data generation and training system built to address the particularities of the sensor. The training data set was generated based on Xperi’s extensive computer vision infrastructure, reusing ground-truth from the visible and near infrared spectrums, synthesizing a completely novel approach in neuromorphic based sensing.   

“Being able to instantaneously detect the subtlest, almost imperceptible, face and eye motions can be lifesaving,” said Petronel Bigioi. “The low light powerful vision processing capabilities, fast response times and low power operation that this pioneering neuromorphic-based technology application enables makes it ideal for the safety-critical applications of driver monitoring systems, while also broadening the ability to further enhance and personalize the vehicle cabin.”