In this paper, a multi-image region growing algorithm to obtain planar 3-D surfaces in the object space from multiple dense
disparity maps, is presented. A surface patch is represented by a plane equation and a set of pixels in multiple images. The
union of back projections of all pixels in the set onto the infinite plane, forms the surface patch. Thanks to that hybrid
representation of planar surfaces, region growing (both region aggregation and region merging) is performed on all images
simultaneously. Planar approximation is done in object space by linear least square estimation using all data points of the
region under question in all images. Linear edge segments detected on colour images are used for constraining the region growing
during the region aggregation phase as well as for detection of borders of surface patches. Experimental results on disparity
maps obtained from high-resolution aerial images of urban areas demonstrate the performance of the algorithm.