The experiment was performed on a prototype of the BrainScales-2 chip,...
The experiment was performed on a prototype of the BrainScales-2 chip, Schematic representation of a neural network and Results for simple and complex tasks (from left to right)
Source: Heidelberg University and MPID

Optimizing neural networks on a brain-inspired computer

Neural networks in both biological settings and artificial intelligence distribute computation across their neurons to solve complex tasks. New research now shows how so-called “critical states” can be used to optimize artificial neural networks running on brain-inspired neuromorphic hardware. The study was carried out by scientists from Heidelberg University working within the Human Brain Project, and the Max-Planck-Institute for Dynamics and Self-Organization (MPIDS).

Many computational properties are maximized when the dynamics of a network are at a “critical point”, a state where systems can quickly change their overall characteristics in fundamental ways, transitioning e.g. between order and chaos or stability and instability. Therefore, the critical state is widely assumed to be optimal for any computation in recurrent neural networks, which are used in many AI applications.

Researchers from the HBP partner Heidelberg University and the Max-Planck-Institute for Dynamics and Self-Organization challenged this assumption by testing the performance of a spiking recurrent neural network on a set of tasks with varying complexity at – and away from critical dynamics. They instantiated the network on a prototype of the analog neuromorphic BrainScaleS-2 system. BrainScaleS is a state-of-the-art brain-inspired computing system with synaptic plasticity implemented directly on the chip. It is one of two neuromorphic systems currently under development within the European Human Brain Project.

First, the researchers showed that the distance to criticality can be easily adjusted in the chip by changing the input strength, and then demonstrated a clear relation between criticality and task-performance. The assumption that criticality is beneficial for every task was not confirmed: whereas the information-theoretic measures all showed that network capacity was maximal at criticality, only the complex, memory intensive tasks profited from it, while simple tasks actually suffered. The study thus provides a more precise understanding of how the collective network state should be tuned to different task requirements for optimal performance.

Mechanistically, the optimal working point for each task can be set very easily under homeostatic plasticity by adapting the mean input strength. The theory behind this mechanism was developed very recently at the Max Planck Institute. “Putting it to work on neuromorphic hardware shows that these plasticity rules are very capable in tuning network dynamics to varying distances from criticality”, says senior author Viola Priesemann, group leader at MPIDS. Thereby tasks of varying complexity can be solved optimally within that space.

The finding may also explain why biological neural networks operate not necessarily at criticality, but in the dynamically rich vicinity of a critical point, where they can tune their computation properties to task requirements. Furthermore, it establishes neuromorphic hardware as a fast and scalable avenue to explore the impact of biological plasticity rules on neural computation and network dynamics. “As a next step, we now study and characterize the impact of the spiking network’s working point on classifying artificial and real-world spoken words”, says first author Benjamin Cramer of Heidelberg University.

The results have been published in Nature Communications.

Subscribe to our newsletter

Related articles

Neural networks: artificial brains need sleep too

Neural networks: artificial brains need sleep too

States that resemble sleep-like cycles in simulated neural networks quell the instability that comes with uninterrupted self-learning in artificial analogs of brains.

AI challenge aims to improve mammography accuracy

AI challenge aims to improve mammography accuracy

AI techniques, used in combination with the evaluation of expert radiologists, improve the accuracy in detecting cancer using mammograms.

Towards an AI diagnosis like the doctor's

Towards an AI diagnosis like the doctor's

Researchers show how they can make an AI show how it's working, as well as let it diagnose more like a doctor, thus making AI-systems more relevant to clinical practice.

AI tool speeds up COVID-19 research

AI tool speeds up COVID-19 research

Computer scientists are aiming to speed up treatments and vaccines for COVID-19 — by making researchers’ jobs easier.

AI and probabilistic modelling applied to oncology

AI and probabilistic modelling applied to oncology

Improving the prediction of survival indicators in patients with breast cancer using tools from artificial intelligence and probabilistic modelling is the aim of ModGraProDep.

Neural networks help cardiac diagnostics

Neural networks help cardiac diagnostics

Artificial intelligence may be an aid to interpreting ECG results, helping healthcare staff to diagnose diseases that affect the heart.

AI model accurately classifies colorectal polyps

AI model accurately classifies colorectal polyps

An AI model for automated classification of colorectal polyps could benefit cancer screening programs by improving efficiency, reproducibility, and accuracy.

AI enables fast-track review of COVID-19 research proposals

AI enables fast-track review of COVID-19 research proposals

An AI tool to help funders identify specialists to peer-review proposals for emergency COVID-19 research has been developed.

AI outperform doctors: Experts express concerns

AI outperform doctors: Experts express concerns

Many studies claiming that AI is as good as (or better than) human experts at interpreting medical images are of poor quality and are arguably exaggerated, warn researchers in The BMJ.

Popular articles