A representation of a neural network provides a backdrop to a fish larvas...
A representation of a neural network provides a backdrop to a fish larva's beating heart.
Source: A representation of a neural network provides a backdrop to a fish larva's beating heart.

AI makes great microscopes better than ever

Machine learning helps some of the best microscopes to see better, work faster, and process more data.

To observe the swift neuronal signals in a fish brain, scientists have started to use a technique called light-field microscopy, which makes it possible to image such fast biological processes in 3D. But the images are often lacking in quality, and it takes hours or days for massive amounts of data to be converted into 3D volumes and movies.

Now, EMBL scientists have combined artificial intelligence (AI) algorithms with two cutting-edge microscopy techniques - an advance that shortens the time for image processing from days to mere seconds, while ensuring that the resulting images are crisp and accurate.

"Ultimately, we were able to take 'the best of both worlds' in this approach," says Nils Wagner, one of the paper's two lead authors and now a PhD student at the Technical University of Munich. "AI enabled us to combine different microscopy techniques, so that we could image as fast as light-field microscopy allows and get close to the image resolution of light-sheet microscopy."

Although light-sheet microscopy and light-field microscopy sound similar, these techniques have different advantages and challenges. Light-field microscopy captures large 3D images that allow researchers to track and measure remarkably fine movements, such as a fish larva's beating heart, at very high speeds. But this technique produces massive amounts of data, which can take days to process, and the final images usually lack resolution.

Light-sheet microscopy homes in on a single 2D plane of a given sample at one time, so researchers can image samples at higher resolution. Compared with light-field microscopy, light-sheet microscopy produces images that are quicker to process, but the data are not as comprehensive, since they only capture information from a single 2D plane at a time.

To take advantage of the benefits of each technique, EMBL researchers developed an approach that uses light-field microscopy to image large 3D samples and light-sheet microscopy to train the AI algorithms, which then create an accurate 3D picture of the sample.

"If you build algorithms that produce an image, you need to check that these algorithms are constructing the right image," explains Anna Kreshuk, the EMBL group leader whose team brought machine learning expertise to the project. In the new study, the researchers used light-sheet microscopy to make sure the AI algorithms were working, Anna says. "This makes our research stand out from what has been done in the past."

Robert Prevedel, the EMBL group leader whose group contributed the novel hybrid microscopy platform, notes that the real bottleneck in building better microscopes often isn't optics technology, but computation. That's why, back in 2018, he and Anna decided to join forces. "Our method will be really key for people who want to study how brains compute. Our method can image an entire brain of a fish larva, in real time," Robert says.

He and Anna say this approach could potentially be modified to work with different types of microscopes too, eventually allowing biologists to look at dozens of different specimens and see much more, much faster. For example, it could help to find genes that are involved in heart development, or could measure the activity of thousands of neurons at the same time.

Next, the researchers plan to explore whether the method can be applied to larger species, including mammals.

The findings are published in Nature Methods.

Subscribe to our newsletter

Related articles

AI finds patterns of mutations in tumour images

AI finds patterns of mutations in tumour images

Researchers have developed an AI algorithm that uses computer vision to analyze tissue samples from cancer patients.

AI challenge aims to improve mammography accuracy

AI challenge aims to improve mammography accuracy

AI techniques, used in combination with the evaluation of expert radiologists, improve the accuracy in detecting cancer using mammograms.

AI-driven platform “HRMAn” never sleeps

AI-driven platform “HRMAn” never sleeps

Researchers have developed a new AI-driven platform that can analyse how pathogens infect our cells with the precision of a trained biologist.

AI Eve augments genetic tests

AI Eve augments genetic tests

AI model called EVE shows remarkable capacity to interpret the meaning of gene variants in humans as benign or disease-causing.

AI picks up mutations in colorectal cancers

AI picks up mutations in colorectal cancers

A deep learning algorithm picks up molecular pathways and the development of key mutations more accurately than existing methods.

An AI that ‘thinks’ like humans

An AI that ‘thinks’ like humans

Creating human-like AI is about more than mimicking human behaviour – technology must also be able to process information, or ‘think’, if it is to be fully relied upon.

Enabling AI-driven advances without sacrificing privacy

Enabling AI-driven advances without sacrificing privacy

Secure AI Labs is expanding access to encrypted health care data to advance AI-driven innovation in the field.

Researchers psychoanalyse artificial intelligence

Researchers psychoanalyse artificial intelligence

We can run tests and experiments, but we cannot always predict and understand why AI does what it does.

The progress and risks of artificial intelligence

The progress and risks of artificial intelligence

Artificial intelligence has reached a critical turning point in its evolution, according to an international panel of experts.

Popular articles

Subscribe to Newsletter