Generalization through Simulation: Integrating Simulated and Real Data into Deep Reinforcement Learning for Vision-Based Autonomous Flight

arXiv paper link

Click below to view the video

Crazyflie Setup

Our quadrotor setup consists of a Crazyflie modified with an onboard camera. See here for the list of parts and here for for instructions on building the Crazyflie and running the software. The ROS code is in the ros directory included in this repository, and is a standalone package.

Software Setup

We run our code using docker.

Build and start the docker image:

$ cd docker
$ ./gcg-docker.sh build Dockerfile-gibson
$ ./gcg-docker.sh start

Your main interface to the docker will be through ssh:

$ ./gcg-docker.sh ssh

If you wish to change the docker image, you must run stop, build and start again:

$ ./gcg-docker.sh stop
$ ./gcg-docker.sh build Dockerfile-gibson
$ ./gcg-docker.sh start

Running our experiments

Download our data and models and place to <path to GtS>/data

The relevant experiment files are in the <path to GtS>/configs.

To evaluate our method in simulation:

$ cd scripts
$ python run_gcg_eval.py eval_in_sim -itr 800

To train from scratch in simulation:

$ python run_gcg.py train_in_sim

To train on the simulation data we gathered, enter '<path to GtS>/data/tfrecords' to the 'offpolicy' parameter in configs/train_tf_records.py and then run:

$ python run_gcg_train.py train_tf_records

To evaluate our pre-trained GtS model in the real world:

$ python run_gcg_eval.py eval_in_rw -itr 6

Note: you may need to run this on your local machine (not docker) because of ROS. See the Dockerfile-gibson for relevant system and python dependencies.

References

Katie Kang*, Suneel Belkhale*, Gregory Kahn*, Pieter Abbeel, Sergey Levine. "Generalization through Simulation: Integrating Simulated and Real Data into Deep Reinforcement Learning for Vision-Based Autonomous Flight." ICRA 2019