The simulator used is AirSim from Microsoft, window identification was implmented on the Unreal version of AirSim and precision landing wa implemented in the Unity version.
This is achieved by a proportional visual controller that identifies the center of the marker and moves the drone in a parallel plane to the marker until it is centered enough, then it descends and this process is repeated until the dron is at a safa landing distance.
landig.mp4
The idea was to implment a neural network able to identify windows and then fly trough them, however due to time constraints only the identification of windows was implemented.
Data was collected using AirSim ́s API, as shown bellow. The data and images provided by the simulaton were written to a directory in pascal_voc format.
window.Detect.speed.up.mp4
Thanks to the simulation, a lot of information was gathered with different climatic and illumination conditions. Using Google Colab, a version of EfficientDet was trained.
Simulated environment: image
Data collected: image
Network test with dust storm: image
Network test in simulation: image
