However, even high-resolution fluorescence microscopes have a hard limit to the amount of detail they can show. Within the past few decades, methods that yield “super-resolution” images have broken that barrier, revealing details at the sub-cellular level, even smaller than one ten-millionth of a meter — an advance that won the 2014 Nobel Prize in Chemistry.
But those strategies come with their own drawbacks: They can be expensive and complex, and they sometimes involve high-intensity light that is toxic to the cells being studied.
Now, UCLA researchers have created a new technique that uses deep learning to transform lower-resolution fluorescence microscopy images into super resolution. The framework takes images from a simple, inexpensive microscope and produces images that mimic those from more advanced and expensive ones. “We need better microscopes to enable discovery at the micro- and nanoscale and allow us to make observations that are otherwise impossible,” UCLA Chancellor’s Professor of Electrical and Computer Engineering said, adding that the technology could be an inexpensive and easy-to-use solution for scientists who are researching the molecular workings of cells and other microscopic systems, but who lack the resources to purchase or use more sophisticated equipment.
The scientists’ work could make advanced microscopy more readily accessible to researchers and open paths of discovery throughout science and engineering.
During the experiments, the researchers fed a computer thousands of images of cells and other microscopic structures, taken by five types of fluorescence microscopes. The images were presented in matched pairs, with the object shown in lower resolution and super resolution.
To learn from those images, the system uses a “generative adversarial network,” a model for artificial intelligence in which two algorithms compete. One algorithm tries to create computer-generated super-resolution images from a low-resolution input image, while the second algorithm tries to differentiate between those computer-generated images and existing super-resolution images that are obtained from advanced microscopes.
That “training” needs to be done only once for each type of subject the system needs to learn. After that, the network can improve a low-resolution image it has never “seen” before to match the image resolution from a super-resolution microscope, which eliminates the need for an expensive, high-resolution microscope.
In the study, the UCLA-developed system successfully enhanced the resolution, contrast and depth of field of original images, which were of cell and tissue samples. “Using a super-resolution microscope requires precise technical skills and expertise,” said Laurent Bentolila, a co-author of the study and the director of the Advanced Light Microscopy/Spectroscopy Laboratory at CNSI. “Seeing that you can now get the same results using deep learning without an advanced and delicate instrument is truly amazing.”
The new approach avoids some of the disadvantages of other super-resolution techniques. For instance, scientists do not need to illuminate the sample with intense light, which can alter cells’ behavior, or even damage or kill them. In addition, it improves resolution based only upon image data. In the study, this method outperformed other resolution enhancement algorithms that depend on assumptions that can prove flawed. “Our system learns various types of transformations that you cannot model because they are random in some sense, or very difficult to measure, enabling us to enhance microscopy images at a scale that is unprecedented,” Hongda Wang, a UCLA graduate student, said.