Building façade style classification from UAV imagery using a Pareto-optimized deep learning network

Abstract

The article focuses on utilizing unmanned aerial vehicles (UAV) to capture and classify building façades of various forms of cultural sites and structures. We propose a Pareto-optimized deep learning algorithm for building detection and classification in a congested urban environment. Outdoor image processing becomes difficult in typical European metropolitan situations due to dynamically changing weather conditions as well as various objects obscuring perspectives (wires, overhangs, posts, other building parts, etc.), therefore, we also investigated the influence of such ambient “noise”. The approach was tested on 8768 UAV photographs shot at different angles and aimed at very different 611 buildings in the city of Vilnius (Wilno). The total accuracy was 98.41% in clear view settings, 88.11% in rain, and 82.95% when the picture was partially blocked by other objects and in the shadows. The algorithm’s robustness was also tested on the Harward UAV dataset containing images of buildings taken from above (roofs) while our approach was trained using images taken at an angle (façade still visible). Our approach was still able to achieve acceptable 88.6% accuracy in building detection, yet the network showed lower accuracy when assigning the correct façade class as images lacked necessary façade information.