Weijian Yang and Feng Tian developed a camera that uses a thin microlens array and new image-processing algorithms to capture 3D information about multiple objects in single exposure. The raw sub-images from the microarray are displayed on the monitor. [Image: S. Luy, University of California, Davis]
Two researchers at the University of California, Davis, USA, have designed a lensless camera that captures 3D information about macroscopic objects in a single exposure (Opt. Express, doi: 10.1364/OE.465933). The new camera is the first of its kind to image objects behind opaque obstacles, according to the pair. It employs a thin, flexible microlens array and advanced image-processing algorithms, and it could find many applications from robotics to entertainment.
Computational imaging
From the start of the instrument’s design process, electrical engineering professor Weijian Yang and his doctoral student Feng Tian considered the camera’s hardware together with its deep-learning image-reconstruction algorithm. According to Yang, he and Tian wanted to overcome two big challenges that have dogged previous lensless cameras: slow image-reconstruction speed and the necessity for extensive calibration of the device’s point-spread function.
Many existing neural networks are “completely black boxes,” Yang says. “They can perform the designated tasks, but the underlying mechanism is difficult to explain.” Instead, the pair’s deep-learning software uses an embedded physical model to teach itself how to map information that it collects from the image back to the objects in a scene. After being “taught,” the camera can produce high-quality image reconstructions at a very high speed without the need for further calibration.
Flexible lens
Because the researchers’ camera doesn’t use the bulk lenses that conventional cameras employ, they consider their version lensless, Yang says. Although some previous lensless cameras have used multi-aperture arrays or diffusers as their optical phase masks, Yang and Tian designed and incorporated an array of 37 3-mm-diameter microlenses made out of flexible polymer. Taken as a unit, the microlens array had a focal length of 15 mm and an overall radius of curvature of 6.5 mm. “Because each microlens can observe objects from different viewing angles,” Yang says, “it can accomplish complex imaging tasks such as acquiring 3D information from objects partially obscured by objects closer to the camera.”
The single-shot image acquired by the new camera can be refocused and reconstructed to reveal object scenes at different distances. [Image: F. Tian, University of California, Davis]
Yang notes that most lensless cameras, including his own laboratory’s past efforts, have demonstrated 3D imaging of microscopic objects, such as biological samples. In the current study, Yang and Tian had to take into account the differences in properties of biological materials and those of hard macroscopic objects, such as the plastic toys they imaged.
According to Yang, while the lensless camera can reconstruct the scenes with 3D objects, the images still contain artifacts that need to be further suppressed. “This could be potentially realized by developing an optimization platform to simultaneously learn the best optics design and the best reconstruction algorithm,” he says, adding that the current optics design is still empirical. Also, since this proof-of-concept experiment used a consumer-grade camera body and sensor chip without the commercial lenses, Yang and Tian say they would like to reduce the size of the sensor package so it could eventually fit into a mobile phone or similar handheld device.