Gradient-free optimization of artificial and biological networks using learning to learn

  • Gradient-freies Optimieren von künstlichen und biologischen Netzwerken mittels des learning to learn Verfahrens

Yegenoglu, Alper; Morrison, Abigail Joanna Rhodes (Thesis advisor); Herty, Michael (Thesis advisor)

Jülich : Forschungszentrum Jülich GmbH, Zentralbibliothek, Verlag (2023)
Book, Dissertation / PhD Thesis

In: Schriften des Forschungszentrums Jülich. IAS series 55
Page(s)/Article-Nr.: 1 Online-Ressource (136 Seiten) : Illustrationen, Diagramme

Dissertation, RWTH Aachen University, 2023


Understanding intelligence and how it allows humans to learn, to make decision and form memories, is a long-lasting quest in neuroscience. Our brain is formed by networks of neurons and other cells, however, it is not clear how those networks are trained to learn to solve specific tasks. In machine learning and artificial intelligence it is common to train and optimize neural networks with gradient descent and backpropagation. How to transfer this optimization strategy to biological, spiking networks (SNNs) is still a matter of research. Due to the binary communication scheme between neurons of an SNN via spikes, a direct application of gradient descent and backpropagation is not possible without further approximations. In my work, I present gradient-free optimization techniques that are directly applicable to artificial and biological neural networks. I utilize metaheuristics, such as genetic algorithms and the ensemble Kalman Filter, to optimize network parameters and train networks to learn to solve specific tasks. The optimization is embedded into the concept of meta-learning and learning to learn respectively. The learning to learn concept consists of a two loop optimization procedure. In the first, inner loop the algorithm or network is trained on a family of tasks, and in the second, outer loop the hyper-parameters and parameters of the network are optimized. First, I apply the EnKF on a convolution neural network, resulting in high accuracy when classifying digits. Then, I employ the same optimization procedure on a spiking reservoir network within the L2L framework. The L2L framework, an implementation of the learning to learn concept, allows me to easily deploy and execute multiple instances of the network in parallel on high performance computing systems. In order to understand how the network learning evolves, I analyze the connection weights over multiple generations and investigate a covariance matrix of the EnKF in the principle component space. The analysis not only shows the convergence behaviour of the optimization process, but also how sampling techniques influence the optimization procedure. Next, I embed the EnKF into the L2L inner loop and adapt the hyper-parameters of the optimizer using a genetic algorithm (GA). In contrast to the manual parameter setting, the GA suggests an alternative configuration. Finally, I present an ant colony simulation foraging for food while being steered by SNNs. While training the network, self-coordination and self-organization in the colony emerges. I employ various analysis methods to better understand the ants’ behaviour. With my work I leverage optimization for different scientific domains utilizing meta-learning and illustrate how gradient-free optimization can be applied on biological and artificial networks.


  • JARA-CSD (Center for Simulation and Data Science) [080031]
  • Department of Computer Science [120000]
  • Neural Computation Teaching and Research Area [124920]