A Three-Layered Approach to Facade Parsing

Anelo Martinovi¹, Markus Mathias¹, Julien Weissenberg², and Luc Van Gool^{1, 2}

¹ESAT-PSI/VISICS, KU Leuven, Belgium

²Computer Vision Laboratory, ETH Zurich, Switzerland

Abstract. We propose a novel three-layered approach for semantic segmentation of building facades. In the first layer, starting from an oversegmentation of a facade, we employ the recently introduced machine learning technique Recursive Neural Networks (RNN) to obtain a probabilistic interpretation of each segment. In the second layer, initial labeling is augmented with the information coming from specialized facade component detectors. The information is merged using a Markov Random Field. In the third layer, we introduce weak architectural knowledge, which enforces the final reconstruction to be architecturally plausible and consistent. Rigorous tests performed on two existing datasets of building facades demonstrate that we significantly outperform the current-state of the art, even when using outputs from earlier layers of the pipeline. Also, we show how the final output of the third layer can be used to create a procedural reconstruction.

LNCS 7578, p. 416 ff.

Full article in PDF | BibTeX

lncs@springer.com
© Springer-Verlag Berlin Heidelberg 2012