Neural networks: artificial brains need sleep too

No one can say whether androids will dream of electric sheep, but they will almost certainly need periods of rest that offer benefits similar to those that sleep provides to living brains, according to new research from Los Alamos National Laboratory.

Photo
Intelligent machines of the future may need to sleep as much as we do.
Source: Los Alamos National Laboratory

“We study spiking neural networks, which are systems that learn much as living brains do,” said Los Alamos National Laboratory computer scientist Yijing Watkins. “We were fascinated by the prospect of training a neuromorphic processor in a manner analogous to how humans and other biological systems learn from their environment during childhood development.”

Watkins and her research team found that the network simulations became unstable after continuous periods of unsupervised learning. When they exposed the networks to states that are analogous to the waves that living brains experience during sleep, stability was restored. “It was as though we were giving the neural networks the equivalent of a good night’s rest,” said Watkins.

The discovery came about as the research team worked to develop neural networks that closely approximate how humans and other biological systems learn to see. The group initially struggled with stabilizing simulated neural networks undergoing unsupervised dictionary training, which involves classifying objects without having prior examples to compare them to.

“The issue of how to keep learning systems from becoming unstable really only arises when attempting to utilize biologically realistic, spiking neuromorphic processors or when trying to understand biology itself,” said Los Alamos computer scientist and study coauthor Garrett Kenyon. “The vast majority of machine learning, deep learning, and AI researchers never encounter this issue because in the very artificial systems they study they have the luxury of performing global mathematical operations that have the effect of regulating the overall dynamical gain of the system.”

The researchers characterize the decision to expose the networks to an artificial analog of sleep as nearly a last ditch effort to stabilize them. They experimented with various types of noise, roughly comparable to the static you might encounter between stations while tuning a radio. The best results came when they used waves of so-called Gaussian noise, which includes a wide range of frequencies and amplitudes. They hypothesize that the noise mimics the input received by biological neurons during slow-wave sleep. The results suggest that slow-wave sleep may act, in part, to ensure that cortical neurons maintain their stability and do not hallucinate.

The groups’ next goal is to implement their algorithm on Intel’s Loihi neuromorphic chip. They hope allowing Loihi to sleep from time to time will enable it to stably process information from a silicon retina camera in real time. If the findings confirm the need for sleep in artificial brains, we can probably expect the same to be true of androids and other intelligent machines that may come about in the future.

Subscribe to our newsletter

Related articles

Brain-on-a-chip would need little training

Brain-on-a-chip would need little training

A neural network that mimics the biology of the brain can be loaded onto a microchip for faster and more efficient artificial intelligence.

Brain-computer interface generates personally attractive images

Brain-computer interface generates personally attractive images

Researchers have succeeded in making an AI understand our subjective notions of what makes faces attractive.

Designing medical deep learning systems

Designing medical deep learning systems

Researchers have analysed whether better design of deep learning studies can lead to the faster transformation of medical practices.

How to train a robot - using AI and supercomputers

How to train a robot - using AI and supercomputers

Computer scientists use TACC systems to generate synthetic objects for robot training.

Biomedical research: deep learning outperforms machine learning

Biomedical research: deep learning outperforms machine learning

Deep-learning methods have the potential to offer substantially better results, generating superior representations for characterizing the human brain.

Teaching artificial intelligence to adapt

Teaching artificial intelligence to adapt

Researchers have used a computational model of brain activity to simulate this process more accurately than ever before.

Blind spots at the intersection of AI and neuroscience

Blind spots at the intersection of AI and neuroscience

Is it possible to read a person's mind by analyzing the electric signals from the brain? The answer may be much more complex than most people think.

How humans use objects to solve problems

How humans use objects to solve problems

What's SSUP? The Sample, Simulate, Update cognitive model developed by MIT researchers learns to use tools like humans do.

A computer reads and predicts thoughts

A computer reads and predicts thoughts

Researchers at the University of Helsinki have developed a technique in which a computer models visual perception by monitoring human brain signals.

Popular articles

Subscribe to Newsletter