varcity.eu ICPR 2016, Cancún, Mexico Efficient Volumetric Fusion of Airborne and Street-Side Data for Urban Reconstruction András Bódis-Szomorú, Hayko Riemenschneider, Luc Van Gool (Extended slides) Street-side mobile mapping DENSE 3D POINT CLOUD FROM IMAGERY  no upper floors / roofs  uncovered pedestrian-only areas  undriven districts  no courtyards  occlusions (trees, parking cars, fences)  often no ground INCOMPLETE MAP COVERAGE ICPR 2016 Andras Bodis-Szomoru 2 Airborne 3D acquisition MULTI-VIEW STEREO FROM 15-CENTIMETER NADIR IMAGERY CAPTURED AT 3 KM TYPICAL DEFECTS DUE TO MISSING DATA  no detail on street-side  walls not visible  shadows  smoothed (blurry) walls  defects (holes) ICPR 2016 Andras Bodis-Szomoru 3 Idea of airborne/street-side fusion COMPLETENESS ROOFS HIGH FLOORS COURTYARDS MISSING STREETS STREET-SIDE 3D (A MOBILE MAPPING EXAMPLE) AIRBORNE NADIR 3D DETAIL Requirements: efficient + watertight + large-scale ICPR 2016 Andras Bodis-Szomoru 4 This paper: efficient volumetric airborne/streetside fusion INPUT AIRBORNE POINT CLOUD INPUT STREET-SIDE POINT CLOUD LINES OF SIGHT (SPARSE SFM OR DENSE MVS) LINES OF SIGHT Assumptions:  visibility data (lines of sight) given  point clouds are accurately geo-registered (via precise GPS/IMU or GCP’s) ICPR 2016 Andras Bodis-Szomoru 5 This paper: efficient volumetric airborne/streetside fusion AIRBORNE ONLY (FOR COMPARISON) OUR FUSION RESULT ICPR 2016 Andras Bodis-Szomoru 6 This paper: efficient volumetric airborne/streetside fusion OUR FUSION RESULT AIRBORNE ONLY (FOR COMPARISON) ICPR 2016 Andras Bodis-Szomoru 7 Related work General surface reconstruction  explicit methods, e.g. zippering  depth map integration via TSDF, e.g. VRIP, KinectFusion (Curless & Levoy SG‘96, Izadi et al. ’06)  s/t cut over voxel grid (unsigned, UDF) (Hornung& Kobbelt EG’06, Lempitsky& Boykov CVPR’07)  Poisson Surface Reconstruction (octree) (Kazhdan EG’06)  convex variational (voxels or height map), e.g. TV-L1, TGV-fusion  cell complex (Chauve CVPR’10)  3DT approaches (Labatut et al. ICCV’07, Lafarge & Alliez EG’13) Street-side & airborne data  superpixel meshes (Bodis et al. CVIU’16)  most address geo-localization / registration  DSM + street-side LiDAR (Fruh & Zakhor CGA’03)  Octree & Dual Contouring (Fiocco et al. 3DIM’05)  Poisson Surface Reconstruction (Shan et al. 3DPVT’13) Superpixel meshes (Bodis et al. CVIU’16) ICPR 2016 Andras Bodis-Szomoru 8 Contributions  1st to propose 3DT-fusion for airborne/street-side  Point cloud blending against gross ray conflicts  Techniques to reduce workload (large urban scenes)  Many experiments on detail vs. workload (sparse & dense input) ICPR 2016 Andras Bodis-Szomoru 9 Volumetric surface reconstruction via 3DT UNKNOWN SURFACE TO RECONSTRUCT ICPR 2016 Andras Bodis-Szomoru 10