Deep learning takes fluorescence microscopy into super resolution

Scientists studying the mysteries of life sometimes rely upon fluorescence microscopy to get a close look at living cells. The technique involves dyeing parts of cells so that they glow under special lighting, revealing cellular structures that measure smaller than one-millionth of a meter.

Photo
The technique transforms low-resolution images from a fluorescence microscope (a) into super-resolution images (b) that compare favorably with those from high-resolution equipment (c). Images on the bottom row are closeups of those on the top row.
Source: UCLA

However, even high-resolution fluorescence microscopes have a hard limit to the amount of detail they can show. Within the past few decades, methods that yield “super-resolution” images have broken that barrier, revealing details at the sub-cellular level, even smaller than one ten-millionth of a meter — an advance that won the 2014 Nobel Prize in Chemistry.

But those strategies come with their own drawbacks: They can be expensive and complex, and they sometimes involve high-intensity light that is toxic to the cells being studied.

Now, UCLA researchers have created a new technique that uses deep learning to transform lower-resolution fluorescence microscopy images into super resolution. The framework takes images from a simple, inexpensive microscope and produces images that mimic those from more advanced and expensive ones. “We need better microscopes to enable discovery at the micro- and nanoscale and allow us to make observations that are otherwise impossible,” UCLA Chancellor’s Professor of Electrical and Computer Engineering said, adding that the technology could be an inexpensive and easy-to-use solution for scientists who are researching the molecular workings of cells and other microscopic systems, but who lack the resources to purchase or use more sophisticated equipment.

The scientists’ work could make advanced microscopy more readily accessible to researchers and open paths of discovery throughout science and engineering.

During the experiments, the researchers fed a computer thousands of images of cells and other microscopic structures, taken by five types of fluorescence microscopes. The images were presented in matched pairs, with the object shown in lower resolution and super resolution.

To learn from those images, the system uses a “generative adversarial network,” a model for artificial intelligence in which two algorithms compete. One algorithm tries to create computer-generated super-resolution images from a low-resolution input image, while the second algorithm tries to differentiate between those computer-generated images and existing super-resolution images that are obtained from advanced microscopes.

That “training” needs to be done only once for each type of subject the system needs to learn. After that, the network can improve a low-resolution image it has never “seen” before to match the image resolution from a super-resolution microscope, which eliminates the need for an expensive, high-resolution microscope.

In the study, the UCLA-developed system successfully enhanced the resolution, contrast and depth of field of original images, which were of cell and tissue samples. “Using a super-resolution microscope requires precise technical skills and expertise,” said Laurent Bentolila, a co-author of the study and the director of the Advanced Light Microscopy/Spectroscopy Laboratory at CNSI. “Seeing that you can now get the same results using deep learning without an advanced and delicate instrument is truly amazing.”

The new approach avoids some of the disadvantages of other super-resolution techniques. For instance, scientists do not need to illuminate the sample with intense light, which can alter cells’ behavior, or even damage or kill them. In addition, it improves resolution based only upon image data. In the study, this method outperformed other resolution enhancement algorithms that depend on assumptions that can prove flawed. “Our system learns various types of transformations that you cannot model because they are random in some sense, or very difficult to measure, enabling us to enhance microscopy images at a scale that is unprecedented,” Hongda Wang, a UCLA graduate student, said.

Subscribe to our newsletter

Related articles

Microscopy: AI converts 2D images into 3D

Microscopy: AI converts 2D images into 3D

Researchers have devised a technique that extends the capabilities of fluorescence microscopy, which allows scientists to precisely label parts of living cells and tissue with dyes that glow under special lighting.

AI finds patterns of mutations in tumour images

AI finds patterns of mutations in tumour images

Researchers have developed an AI algorithm that uses computer vision to analyze tissue samples from cancer patients.

Dentistry: AI helps localize the mandibular canals

Dentistry: AI helps localize the mandibular canals

Researchers have developed a new model that accurately and automatically shows the exact location of mandibular canals.

AI outperform doctors: Experts express concerns

AI outperform doctors: Experts express concerns

Many studies claiming that AI is as good as (or better than) human experts at interpreting medical images are of poor quality and are arguably exaggerated, warn researchers in The BMJ.

Smart insoles unlock the secrets of your sole

Smart insoles unlock the secrets of your sole

Researchers at Stevens Institute of Technology have developed an AI-powered, smart insole that instantly turns any shoe into a portable gait-analysis laboratory.

Smart algorithm analyses whole brain vasculature

Smart algorithm analyses whole brain vasculature

Scientists have developed a technique for visualising the structures of all the brain's blood vessels including any pathological changes.

AI may spot newborns at risk of blinding disease

AI may spot newborns at risk of blinding disease

An AI device may help identify newborns at risk for aggressive posterior retinopathy of prematurity (AP-ROP).

Deep learning model accurately identifies sleep stages

Deep learning model accurately identifies sleep stages

A deep learning model can identify sleep stages as accurately as an experienced physician.

Deep learning, 3D tech create better drugs

Deep learning, 3D tech create better drugs

Researchers have designed a novel approach to use deep learning to better understand how proteins interact in the body.

Popular articles