Scalable RGBD scanning for 3D environments with articulated objects

Yongsen Mao  •   Yiming Zhang  •   Hanxiao Jiang  •   Angel X. Chang  •   Manolis Savva

We introduce MultiScan, a scalable RGBD dataset construction pipeline leveraging commodity mobile devices to scan indoor scenes with articulated objects and web-based semantic annotation interfaces to efficiently annotate object and part semantics and part mobility parameters. We use this pipeline to collect 273 scans of 117 indoor scenes containing 10957 objects and 5129 parts. The resulting MultiScan dataset provides RGBD streams with per-frame camera poses, textured 3D surface meshes, richly annotated part-level and object-level semantic labels, and part mobility parameters. We validate our dataset on instance segmentation and part mobility estimation tasks and benchmark methods for these tasks from prior work. Our experiments show that part segmentation and mobility estimation in real 3D scenes remain challenging despite recent progress in 3D object segmentation.






Stars: 44

View Code


Version: 1.0


MultiScan Dataset

Size: 364 GB

View Instruction

Benchmark Dataset

Size: 4.1 GB

View Instruction
  • 2022-11-30: MultiScan initial release.
Project Video
Paper and Bibtex


author = {Mao, Yongsen and Zhang, Yiming and Jiang, Hanxiao and Chang, Angel X and Savva, Manolis},

title = {MultiScan: Scalable RGBD scanning for 3D environments with articulated objects},

booktitle = {Advances in Neural Information Processing Systems},

year = {2022}



Yongsen Mao, Yiming Zhang, Hanxiao Jiang, Angel X. Chang, Manolis Savva. MultiScan: Scalable RGBD scanning for 3D environments with articulated objects In NeurIPS 2022.


This work was funded in part by a Canada CIFAR AI Chair, a Canada Research Chair, NSERC Discovery Grant, a research grant by Facebook AI Research, and enabled by support from WestGrid and Compute Canada. The iOS and Android scanning apps were developed by Zheren Xiao and Henry Fang. We thank Qirui Wu for his help in re-implementing PointGroup with the Minkowski engine and contributions to the minsu3D code repository. We also thank Zhen (Colin) Li for collecting additional scans, and Henry Fang, Armin Kavian, Han-Hung Lee, Zhen (Colin) Li, Weijie (Lewis) Lin, Sonia Raychaudhuri, and Akshit Sharma for helping to annotate data.

Last updated: 2022-10-23