HoliCity: A City-Scale Data Platform for Learning Holistic 3D Structures

We present HoliCity, a city-scale 3D dataset with rich structural information. Currently, this dataset has 6,300 real-world panoramas of resolution 13312 × 6656 that are accurately aligned with the CAD model of downtown London with an area of more than 20 km², in which the median reprojection error of the alignment of an average image is less than half a degree. This dataset aims to be an all-in-one data platform for research of learning abstracted high-level holistic 3D structures that can be derived from city CAD models, e.g., corners, lines, wireframes, planes, and cuboids, with the ultimate goal of supporting real-world applications including city-scale reconstruction, localization, mapping, and augmented reality.

Sampled Viewpoints

Advantage of HoliCity

Coverage Map (Click to Enlarge)

HoliCity covers an area of more than 20km² of the London city from 6,300 viewpoints, which dwarfs previous datasets based on LIDAR technology. Here, we show the coverage map that is aligned with Google Maps and compares it against the Oxford RobotCar dataset under the same scale.

Accuracy

We carefully align the panoramas with the 3D models. We show the reprojection error of annotated points between the images and the CAD model. We find that for an average image, the median reprojection error is less than half a degree and the 95th percentile does not exceed 1.2 degrees.

Features and Comparison

Applications

HoliCity provides labels to support learning-based methods for outdoor environments. We demonstrate the results on tasks such as holistic surface segmentation and normal estimation. Please refer to our paper for more details.

Downloads

This dataset is distributed by Professor Yi Ma's group at the EECS Department of UC Berkeley. It is made available for non-commercial purposes only, such as academic research, teaching, scientific publications, and personal experimentation. You must agree to the HoliCity Terms of Use before downloading.

Instructions:	[github] [arxiv]
Downloads (scene):	[cad] [panorama] [split]
Downloads (perspective):	[image] [camera] [depth] [normal] [plane] [semantic] [vanishing points]

Disclaimer: The street-view panorama imagery is owned and copyrighted by Google Inc. The CAD models used to make the dataset are owned by AccuCities Inc. The refinement of images' geographic information and rendering of CAD models are done at UC Berkeley. For commercial usage of the dataset, one must seek explicit permissions from each of these rightful owners.

Authors

Yichao Zhou
UC Berkeley

Jingwei Huang
Stanford

Xili Dai
UC Berkeley

Linjie Luo
Bytedance

Zhili Chen
Bytedance

Yi Ma
UC Berkeley

Contacts: Open an issue on Github or mail to Yichao Zhou ([email protected]).

Acknowledgment

This work is sponsored by a generous grant from Sony Research US. We'd also like to thank Sandor Petroczi and Michal Konicek from AccuCities for the help of their London CAD models.

Reference

If you find HoliCity useful in your research, please consider citing:


@article{zhou2020holicity,
    author={Zhou, Yichao and Huang, Jingwei and Dai, Xili and Luo, Linjie and Chen, Zhili and Ma, Yi},
    title={{HoliCity}: A City-Scale Data Platform for Learning Holistic {3D} Structures},
    year = {2020},
    archivePrefix = "arXiv", 
    note = {arXiv:2008.03286 [cs.CV]},
}