Applying Multi-Agent RL to SLAM with Graph Pose for Sampled-Data MPC and CPN of Autonomous Drone Swarms

2020 NYU CUSP Capstone Project

An example of 3D simulation. (The code for the 3D simulation is taken and modified from

Developing methods that allow drones to autonomously navigate in different environments has been a topic of extensive research in recent years. One research topic of interest in autonomous drone navigation is to explore the maneuverability and capability of drones to navigate inaccessible environments and situations that might be too risky for human access. Using a swarm of drones that autonomously navigate a post-catastrophe scenario in order to optimally map the disaster zone, i.e independently and efficiently identify and map the structural damage across a geographic site, has been a problem less explored. Detection and mapping changes across a post-catastrophe site enables a more robust estimation of structural damage.

This project attempted to explore and simulate a reinforcement learning approach to enable drones to perform task assignment and scheduling in order to efficiently maximize coverage for identifying and mapping structural changes within the post-catastrophe environment. The primary objective of the simulation was to focus on the exploration of ad-hoc decentralized task assignment and scheduling by one or more drone(s) at the edge with minimal connectivity aside from local communication between nearest neighbors. Other workstreams in the project explored satellite and aerial imagery, seismic structural damage equation models, and generative adversarial networks (GANs) related to the Port-au-Prince 2010 Haiti earthquake site as a use case and attempt to explore methods that might be utilized to identify structural changes from satellite images, using generative synthetic data and estimated fragility equations in order to address uncertainty and ambiguity in the detection of discrepancies in edges related to damage.

Satellite/Aerial Images

Satellite and aerial images are taken before and after the Haiti earthquake (2010/01/13~2010/01/21) by Google maps and GeoEye and a sample is shown on the left. Click Learn More (redirects to an official webpage hosted by Google) to know more about the dataset.

What we did?

There are multiple facets to this project which are elucidated in detail below.

Change Detection

The change detection algorithm is used to quantify how much the pre-catastrophe environment has changed due to the disaster from images.

Learn more

Generative Adversarial Networks

GAN is used to produce synthetic data which can increase the amount of paired pre and post-catastrophe images to train the agents (i.e drones).

Learn more

Enhancing pre-disaster images

The upsampling methods are attempted to explore emulating the post-disaster image resolution, although fully acknowledging the joint hypothesis problem as a limitation to this approach.

Learn more

Simulations & Reinforcement Learning

Custom simulation is built to replicate our tasks, i.e. optimally map the entire environment, in order to train and evaluate the learning algorithm.

Learn more

Damage Equations

To evaluate the seismic effects on structures, damage equations are used to make a likelihood estimation of damage in a cell/raster.

Learn more


Rohith Gandhi Ganesan Yue Jin Kunru Lu Elmon Toraman


Dr. David K A Mordecai, Professor
Dr. Giuseppe Loianno, Professor

Support: RiskEcon® Lab for Decision Metrics @ Courant NYU
Agile Robotics and Perception Lab (ARPL) @ Tandon NYU

We would like to sincerely thank Joseph Bullock (AI Researcher at UNGP and PhD Researcher at Durham University), Dr. Debra Laefer (New York University) and Dr. Ufuk Hancilar (Bogazici University) for their helpful feedback and suggestions to this project, and thank our project advisors for their guidance throughout the project. Any remaining errors are our own.

For more information:

  • Address

    370 Jay Street, 13th Floor
    Brooklyn, NY 11201
  • Email
  • Phone