Aerial 6DoF localization typically relies on precise GNSS signals or radiometrically rich 3D reconstructions, limiting scalability and on-board deployment. We propose SemCityLoc, a semantic–geometric alignment system that reframes aerial pose estimation as structured surface registration between foundation-model-derived visual priors and standardized LoD-compliant 3D city models.
Instead of matching sparse contours or dense texture, our method aligns semantic surfaces and monocular depth with lightweight semantic 3D building models, increasing pose discriminability in repetitive and occluded urban environments. To enable accurate evaluation, we introduce SemCityLockeD, the first real-world benchmark combining centimeter-accurate UAV poses with standardized LoD1–LoD3 semantic city models and challenging low-altitude imagery.
Experiments demonstrate substantial improvements over existing map-based approaches, improving recall by up to 36% and reducing mean positional error from 9.89 m to 2.62 m in challenging urban canyons. Our results indicate that semantically structured geometry provides sufficient and scalable constraints for high-precision aerial localization without radiometric scene reconstructions.
SemCityLoc follows a coarse-to-fine semantic–geometric alignment strategy. Given a query image and an initial pose prior, we first perform a 4D semantic cost-volume search to obtain a coarse pose estimate, then refine it via joint semantic and depth alignment using a particle-filter optimization scheme to recover the final 6DoF camera pose.
Two-stage pose matching. In the coarse stage, a DINOv3-based semantic segmentation model and batch rendering construct a 4D cost volume for coarse pose estimation. In the fine stage, a particle filter fuses semantic and monocular depth (MoGe-2) cues for refined pose estimation.
A frozen DINOv3 ViT backbone with a lightweight DPT decoder predicts semantic masks for the query image. Candidate poses are sampled around the prior over (x, y, z, ψ) — roll and pitch are fixed from gravity. For each pose we render the LoD semantic mask and score it against the query mask with a per-class IoU cost, building a 4D cost volume whose maximum yields the coarse pose.
We add monocular depth from MoGe-2, aligning estimated and rendered depth via a robust scale–shift term. The final cost combines semantic and depth terms. A particle filter iteratively perturbs the coarse hypothesis, evaluates the cost, and updates the pose in a weighted manner — enabling stable, efficient convergence without exhaustive high-dimensional grid search.
Unlike contour-based alignment over sparse edges, semantic surface partitions add region-level structural constraints that increase pose observability — transforming localization from edge matching into structured surface registration.
SemCityLockeD is the first real-world benchmark pairing centimeter-accurate UAV poses with standardized LoD semantic 3D city models and challenging low-altitude imagery. It captures a densely built urban environment with closely spaced buildings, narrow urban canyons, and diverse architecture (19th–21st century).
(a) Swiss-EPFL (>75 m, sub-urban)
(b) UAVD4L (high altitude)
(c) SemCityLockeD (nadir)
(d) SemCityLockeD (oblique)
Georeferenced keypoints projected with ground-truth poses align closely with image structures in (c) and (d), indicating high pose consistency under substantially more challenging close-range urban-canyon geometry than the high-altitude (a) and (b).
Projecting LoD geometry with the estimated poses reveals noticeable misalignment for LoD-Loc, especially in oblique views and facade-occluded urban canyons. SemCityLoc produces geometrically consistent overlays and accurate depth alignment (especially apparent in view 3).
| Method | 2m–2° | 3m–3° | 5m–5° | Yaw (°) | XYZ (m) |
|---|---|---|---|---|---|
| MC-Loc (DINOv2) | 5.32 | 8.51 | 18.09 | 5.90 | 10.71 |
| MC-Loc (RoMa) | 0 | 1.06 | 3.19 | 6.67 | 18.61 |
| LoD-Loc | 35.11 | 47.87 | 53.19 | 1.78 | 9.89 |
| SemCityLoc (no coarse sel.) | 41.49 | 56.38 | 72.34 | 1.11 | 5.94 |
| SemCityLoc (no refine) | 56.38 | 70.21 | 81.91 | 1.04 | 3.20 |
| SemCityLoc (full) | 69.15 | 84.04 | 89.36 | 0.42 | 2.62 |
Recall in %, yaw in degrees, position in meters. CAD-Loc (e-LoFTR / RoMa) score 0 across thresholds and are omitted. Bold = best, underline = second best. Both coarse selection and semantic–depth refinement are complementary and jointly critical.
| Method | 2m–2° | 5m–5° | XYZ (m) |
|---|---|---|---|
| LoD-Loc | 88.18 | 99.54 | 1.38 |
| SemCityLoc | 89.64 | 99.95 | 1.26 |
| Method | 2m–2° | 5m–5° | XYZ (m) |
|---|---|---|---|
| LoD-Loc | 17.41 | 48.55 | 12.54 |
| SemCityLoc | 35.36 | 89.18 | 3.09 |
The semantic–geometric alignment generalizes across heterogeneous environments — in the challenging out-of-Place setting it nearly doubles recall and cuts mean positional error from 12.54 m to 3.09 m.
SemCityLoc converges within a few epochs and stabilizes after ~15 epochs. Fine-tuning a lightweight head on pretrained features with an IoU-based objective yields smoother gradients than reprojection-based losses trained from scratch.
Performance improves from LoD1 to LoD3. Notably, semantic LoD2 matches or surpasses LoD3 despite lower geometric resolution — semantically partitioned surfaces enhance pose observability and partially compensate for missing fine-grained geometry.
@inproceedings{semcityloc2026,
author = {Mao, Jingfeng and Chen, Xuyang and Zhang, Qilin and Dhaouadi, Oussema
and Wang, Guangming and Sheil, Brian and Cremers, Daniel
and Xia, Yan and Wysocki, Olaf},
title = {SemCityLoc: Aerial 6DoF Localization Using Semantic 3D City Models},
booktitle = {European Conference on Computer Vision (ECCV)},
year = {2026}
}