Learning and decision making in closed loop simulations of plastic spiking neural networks

Weidel, Philipp; Morrison, Abigail (Thesis advisor); Lakemeyer, Gerhard (Thesis advisor)

Aachen : RWTH Aachen University (2020, 2021)
Dissertation / PhD Thesis

Dissertation, RWTH Aachen University, 2020

Abstract

To understand how animals and humans learn, form memories and make decisions is along-lasting goal in many fields of science. Many models of learning and decision making were developed in the fields of machine learning, artificial intelligence, and computational neuroscience. Although these models aim to describe similar mechanisms, they do not all pursue the same goal. These models can be differentiated between models aiming to reach optimal performance on a specific task (or set of tasks) and models trying to explain how animals and humans learn. Some models of the first-class use biologically inspired methods (such as deep learning) but are usually biologically not realistic and are therefore not well suited to explain the function of the brain. Models in the second class focus on being biologically plausible to explain how the brain works, but often demonstrate their capability on too simplistic tasks and yield low performance on well-known tasks from machine learning. This thesis aims to close the gap between these two types of models. In the first part of this work, tools are developed that allow the combination of biologically plausible neural network models together with powerful toolkits known from machine learning and robotics. To this end, MUSIC, the middleware for spiking neural network simulators such as NEST and NEURON is interfaced with ROS, a middleware for robotic hardware and simulators such as Gazebo. This toolchain is extended with interfaces to reinforcement learning toolkits such as the OpenAI Gym. In the second part, a spiking neural network model for action selection in the basalganglia is developed and interfaced to a simulated robot being able to reproduce basic behavioral experiments. In this work, the dynamics and roles of different neuron types in parts of the basal ganglia are investigated which sheds light on the mechanism how action selection is implemented in the brain. The last part of this thesis addresses the question of how the brain can represent its environment in the neural substrate of the cortex and how a realistic model of reinforcement learning can make use of these representations. To this end, a spiking neural network model of unsupervised learning is presented which is able to learn its input projections such that it can detect and represent repeating patterns. By using an actor-critic reinforcement learning architecture driven by a realistic dopamine modulated plasticity rule the model can make use of the representations and learn a range of well known and difficult tasks.

Institutions

• Department of Computer Science [120000]
• Chair of Computer Science 3 (Software Engineering) [121510]
• Neural Computation Teaching and Research Area [124920]