Research team
Neuromorphic multi-drone perception.
Abstract
The trend towards autonomous drones is currently driving the integration of an increasing number of sensors for safe navigation under all circumstances, forcing algorithms and hardware to be energy efficient and fast. When drone technology continues to mature, deploying swarms of them will enable even more advanced use cases, for example in precision agriculture. Swarms also offer the possibility of sharing both sensory and compute resources, making the swarm act and respond as a single collaborative entity with overall better performance. In this PhD project, we work with real-world multi-sensory data collected by multiple drones and develop a spike-based neuromorphic fusion solution running on custom imec hardware. More specifically, we will focus on the following research questions: - Can we build a low power sensor fusion solution based on spiking neural networks for autonomous drone navigation and obstacle avoidance, running on imec hardware. We will investigate different solutions to perform spike encodings and carry out the learning. A trade off will be made vs power consumption and hardware. - How can collaborative drones, each with their own spike-based neuromorphic fusion solution, communicate with each other in a timely and resource efficient way? Which sensor fusion tasks need to be performed by which nodes in a collaborative setting? - Can we develop efficient techniques for distributed training across multiple spike- based drones to reduce each drone's individual memory and power requirement and, at the same time, lower the convergence time of the swarm?Researcher(s)
- Promoter: Latré Steven
- Fellow: Van Damme Laurens
Research team(s)
Project type(s)
- Research Project
Training Spiking Neural Networks using Temporal Logic.
Abstract
In the emerging field of low power AI for deployment at edge devices, Spiking Neural Networks (SNN) are gaining traction as prime candidate technology due to initial results of spiking neuromorphic systems saving up to one or two order of magnitude in energy for inference tasks. While today's SNNs are typically trained in the cloud using variants of the traditional backpropagation method, future applications will benefit from on-device adaptation and learning capabilities. Spike-Timing-Dependent Plasticity (STDP), an interesting brain-inspired local learning alternative that uses the temporal factor of spike events for learning, has shown promising results for unsupervised feature learning, and can be deployed for on-device learning. However, for training on specific tasks, STDP needs to be extended with a third factor in the form of a success signal to steer the learning process. The existing three-factor learning rules can be characterized by having different and somewhat ad-hoc definitions for the third factor which may or may not work well in particular applications. This proposal will investigate new SNN training methods that combine STDP learning with formal methods from Temporal Logic to define structured reward signals that are applicable to a wide range of supervised, self-supervised and reinforcement learning applications, and allow for distributed deployment. Enhanced SNNs will open up a wealth of opportunities for smart industries, health, environment etc.Researcher(s)
- Promoter: Latré Steven
- Fellow: Van Damme Laurens
Research team(s)
Project type(s)
- Research Project