Neural networks: artificial brains need sleep too

No one can say whether androids will dream of electric sheep, but they will almost certainly need periods of rest that offer benefits similar to those that sleep provides to living brains, according to new research from Los Alamos National Laboratory.

Photo
Intelligent machines of the future may need to sleep as much as we do.
Source: Los Alamos National Laboratory

“We study spiking neural networks, which are systems that learn much as living brains do,” said Los Alamos National Laboratory computer scientist Yijing Watkins. “We were fascinated by the prospect of training a neuromorphic processor in a manner analogous to how humans and other biological systems learn from their environment during childhood development.”

Watkins and her research team found that the network simulations became unstable after continuous periods of unsupervised learning. When they exposed the networks to states that are analogous to the waves that living brains experience during sleep, stability was restored. “It was as though we were giving the neural networks the equivalent of a good night’s rest,” said Watkins.

The discovery came about as the research team worked to develop neural networks that closely approximate how humans and other biological systems learn to see. The group initially struggled with stabilizing simulated neural networks undergoing unsupervised dictionary training, which involves classifying objects without having prior examples to compare them to.

“The issue of how to keep learning systems from becoming unstable really only arises when attempting to utilize biologically realistic, spiking neuromorphic processors or when trying to understand biology itself,” said Los Alamos computer scientist and study coauthor Garrett Kenyon. “The vast majority of machine learning, deep learning, and AI researchers never encounter this issue because in the very artificial systems they study they have the luxury of performing global mathematical operations that have the effect of regulating the overall dynamical gain of the system.”

The researchers characterize the decision to expose the networks to an artificial analog of sleep as nearly a last ditch effort to stabilize them. They experimented with various types of noise, roughly comparable to the static you might encounter between stations while tuning a radio. The best results came when they used waves of so-called Gaussian noise, which includes a wide range of frequencies and amplitudes. They hypothesize that the noise mimics the input received by biological neurons during slow-wave sleep. The results suggest that slow-wave sleep may act, in part, to ensure that cortical neurons maintain their stability and do not hallucinate.

The groups’ next goal is to implement their algorithm on Intel’s Loihi neuromorphic chip. They hope allowing Loihi to sleep from time to time will enable it to stably process information from a silicon retina camera in real time. If the findings confirm the need for sleep in artificial brains, we can probably expect the same to be true of androids and other intelligent machines that may come about in the future.

Subscribe to our newsletter

Related articles

Optimizing neural networks on a brain-inspired computer

Optimizing neural networks on a brain-inspired computer

Research shows how so-called “critical states” can be used to optimize artificial neural networks running on brain-inspired neuromorphic hardware.

AI challenge aims to improve mammography accuracy

AI challenge aims to improve mammography accuracy

AI techniques, used in combination with the evaluation of expert radiologists, improve the accuracy in detecting cancer using mammograms.

AR allows researchers to see patients’ real-time pain

AR allows researchers to see patients’ real-time pain

Researchers have developed a technology to help clinicians "see" and map patient pain in real-time, through special augmented reality glasses.

Machine learning could enhance scientific peer review

Machine learning could enhance scientific peer review

Researchers make the case that Artificial Intelligence tools have the potential to help researchers separate the wheat from the chaff.

The embedded ethics approach in AI development

The embedded ethics approach in AI development

Alena Buyx, Professor of Ethics in Medicine and Health Technologies at TUM, explains the embedded ethics approach.

AI offers clues to a 500-year old mystery

AI offers clues to a 500-year old mystery

Researchers used AI and genetic analyses to examine the structure of the inner surface of the heart using 25 000 MRI scans.

Neural network helps doctors explain relapses of heart failure

Neural network helps doctors explain relapses of heart failure

Researchers have developed an algorithm that not only predicts hospital readmissions of heart failure patients, but also tells you why these occur.

Network medicine makes drug repurposing effective

Network medicine makes drug repurposing effective

Artificial intelligence can increase the effectiveness of drug repositioning or repurposing research.

BCI training reduces phantom limb pain

BCI training reduces phantom limb pain

Scientists used brain-computer-interface to train the brains of patients to reduce phantom-hand pain.

Popular articles