Qualcomm’s AR/VR efforts are accelerating. Following the launch of stand-alone Vive with Snapdragon VR 835 platform, the company launched Spectra ISP camera module for smartphone and headset manufacturers. Goal is to enable several new features which may be the key to simplification of how users around the world use their smartphones and headsets.
While we mostly know Qualcomm as the manufacturer of Snapdragon processors, the company is also leading manufacturer of image processing systems, i.e. cameras. Second generation Spectra ISP proves that leadership with innovations such as active or passive depth sensing and biometric recognition (iris scan, facial features) – enabling future smart devices to unlock themselves automatically. Perhaps the most interesting feature is the depth sensing, available for both mid-range and high-end smartphones. Perhaps Spectra is best explained through the video below:
As you can see above, depth sensing can now identify individual fingers and could be one of main drivers for the upcoming headsets such as Microsoft Hololens 3, which switched from Intel processors to custom silicon. Thanks to a number of companies which now offer dual lens cameras, Qualcomm had an easier job to make active depth sensing work, and perhaps this is going to be the key feature for our navigation in virtual environment.
The way how passive depth sensing works is also ingenious – Spectra ISP records the video through both cameras, adding the bokeh effect (background blurring). Once focus is on moving objects such as hands, the camera combines two images – one being black & white one to process the elements of our extremities and the way how they interact with perspective-correct digital assets. Active depth sensing is whole another ball game. Using three camera modules, Spectra ISP uses the third camera for IR (infra-red) mode. The LED-based IR illuminator system sends IR rays which are detected and re-created digitally inside the system. By using depth maps, developers can precisely position our extremities in virtual or augmented environment, leaving the “uncanny valley” behind, enabling full immersion.
Moving away from AR/VR into the field of security, Spectra now supports facial expressions, and eye scans for automatic unlock of the phone. Colored contact lens users are safe, as the camera adopts to iris pattern rather than the eye color. The company uses AI, i.e. machine learning (ML) algorithms to “make sure you are you”.
Devices using the Snapdragon VR platform with Spectra ISP are expected to debut on 2018 edition of Mobile World Congress, to be held at the tail end of February in Barcelona, Spain. If you’re intrigued by this announcement, don’t forget that the company is hosting a motion tracking webinar on Wednesday, August 16th at 9AM Pacific.
