A pair of UCLA bioengineers and a former postdoctoral scientist have developed a new class of bionic 3D camera systems that can mimic the multiview vision of flies and the natural sonar detection of bats, resulting in multidimensional imaging with an extraordinary depth range that also can scan through blind spots.
Powered by computational image processing, the camera can decipher the size and shape of objects hidden around corners or behind other items. The technology could be built into autonomous vehicles or medical imaging tools with detection capabilities well beyond what is considered state-of-the-art today. This research was published in nature communication.
In the dark, bats can visualize a vivid image of their environment by using some form of echolocation or sonar. Their high-frequency beeps bounce off their surroundings and are picked up again by their ears. The tiny differences in how long it takes for the echo to reach the nocturnal animals and the intensity of the sound tell them in real time where things are, what’s in the way and the proximity of potential prey.
Many insects have geometrically shaped compound eyes, where each “eye” is made up of hundreds to tens of thousands of individual vision units, making it possible to see the same thing from multiple lines of sight. For example, the spherical compound eyes of flies give them nearly 360-degree vision, although their eyes have a fixed focal length, making it difficult for them to see anything far away, such as a fly swatter held aloft.
Inspired by these two natural phenomena in flies and bats, the UCLA-led team set to work designing a powerful 3D camera system with advanced capabilities that would leverage these benefits, but also address nature’s shortcomings.
“While the idea itself has been tried out, seeing over a range of distances and around occlusions has been a major hurdle,” said study leader Liang Gao, an associate professor of bioengineering at UCLA Samueli School of Engineering. “To address that, we have developed a new computational imaging framework, which for the first time enables wide and deep panoramic view with simple optics and a small number of sensors.”
Called “Compact Light-field Photography” or CLIP, the frame allows the camera system to “see” at a greater depth range and around objects. In experiments, the researchers showed that their system can “see” hidden objects that are undetected by conventional 3D cameras.
The researchers also use a type of LiDAR, or “Light Detection And Ranging,” in which a laser scans the environment to create a 3D map of the area.
Conventional LiDAR, without CLIP, would take a high-resolution snapshot of the scene, but miss hidden objects, just like our human eyes would.
Using seven LiDAR cameras with CLIP, the array creates a lower-resolution image of the scene, processes what individual cameras see, and then reconstructs the combined scene in high-resolution 3D imaging. The researchers showed that the camera system could display a complex 3D scene with different objects, all at different distances.
“If you cover one eye and look at your laptop, and there’s a coffee mug just behind it, you might not see it because the laptop is blocking the view,” explains Gao, who is also a member of the California NanoSystems Institute. “But if you use both eyes, you’ll find that you can see the object better. That’s kind of what’s happening here, but now imagine seeing the mug with the compound eye of an insect. Now multiple views are possible .”
According to Gao, CLIP helps the camera array understand what’s hidden in a similar way. In combination with LiDAR, the system is able to achieve the bat echolocation effect, allowing one to detect a hidden object by how long it takes for the light to bounce back to the camera.
The co-lead authors of the published research are UCLA bioengineer student Yayao Ma, who is a member of Gao’s Intelligent Optics Laboratory, and Xiaohua Feng – a former UCLA Samueli postdoc who works in Gao’s lab and now a research scientist at the Research Center for Humanoid Sensing at Zhejiang Laboratory in Hangzhou, China.
New Research ‘Reveals’ High-Resolution Hidden Objects
Xiaohua Feng et al, Compact light field photography towards versatile three-dimensional vision, nature communication (2022). DOI: 10.1038/s41467-022-31087-9
Provided by the University of California, Los Angeles
Quote: Insect eyes and bat sonar: Bio-engineers turn to the animal kingdom to create bionic super 3D cameras (2022, August 12) retrieved August 13, 2022 from https://phys.org/news/2022-08- bug-eyes-sonar-bioengineers-animal.html
This document is copyrighted. Other than fair dealing for personal study or research, nothing may be reproduced without written permission. The content is provided for informational purposes only.