CMU, Meta Researchers Develop Universal System for 3D Reconstruction
The Breakdown:
- Researchers developed a model that converts data into precise 3D maps.
- Trained on real-world scenes, the model captures both small details and large spaces with high precision.
- The research moves robotics closer to human-like spatial reasoning.
***
Researchers from the Carnegie Mellon University Robotics Institute (RI) and Meta Reality Labs have built a powerful model that reconstructs images, camera data or depth scans into accurate 3D maps within a unified system.
The model, called MapAnything, has a wide range of flexibility, making it useful for everything from augmented reality to environmental mapping by providing a fast, simple way to turn visual data into accurate 3D representations. This technology means robots can better navigate cluttered environments and researchers can quickly generate maps of complex real-world scenes.
“When humans interact with the world, we constantly estimate distances –– how far away something is, whether we will run into it or how far we need to reach out to grasp it,” said Nikhil Keetha, a Ph.D. student in the RI and co-lead researcher on MapAnything. “But for robots, that kind of spatial reasoning is difficult to do from 2D images alone. MapAnything moves into 3D, a foundational step toward giving machines a more humanlike understanding of their environments.”
The research team assembled a large-scale dataset with more than 200,000 real-world scenes to train the MapAnything model, including indoor spaces like classrooms and offices and outdoor paths, forests and dynamic nature scenes. In tests, the model proved to be both flexible and high-performing.
“ We wanted to enable 3D metric scene geometry in a very flexible way,” Keetha said. “ Current systems are either slow or only support a particular task. With MapAnything, we can handle any number of images or data types and build accurate 3D maps in one step. That means it could just as easily reconstruct a small water bottle on a desk or map an entire office.”
For more information on MapAnything, visit the project website.
For More Information: Aaron Aupperlee | 412-268-9068 | aaupperlee@cmu.edu