Carnegie Mellon Develops SPOT System for Human-Like Robot Planning

Researchers at Carnegie Mellon University have introduced a groundbreaking system that enhances robotic capabilities for navigating complex environments. Named Search over Point cloud Object Transformations (SPOT), this innovative approach allows robots to interpret their surroundings and execute tasks, such as organizing objects, with a level of intuition previously reserved for humans.

SPOT equips robots with the ability to analyze 3D camera data, enabling them to understand spatial relationships and object shapes in real time. This advancement addresses a significant challenge in robotics: operating effectively in cluttered settings like kitchens, classrooms, and offices. By facilitating goal-driven planning, SPOT represents a crucial step toward integrating robots into everyday life.

David Held, Associate Professor at the Robotics Institute (RI), along with Professor Maxim Likhachev, guided a team of students to develop this system. Their objective was to improve how robots coordinate movements involving multiple objects—essential for tasks like putting away dishes or organizing shelves. Effective planning is critical; for instance, a robot must remove items in the correct order to avoid collisions and ensure safe handling.

Rather than relying on symbolic descriptions that require exhaustive rules, SPOT operates directly from raw sensory data. According to Amber Li, a Ph.D. student and co-lead researcher, “SPOT operates directly in the point cloud space with raw sensory input from one camera and needs no additional information about the scene or the objects.” This capability allows the robot to perceive its environment in three dimensions, significantly enhancing its ability to plan movements even in cluttered or partially obstructed spaces.

In practical applications, the research team tested SPOT using a Franka robotic arm and plastic dishes. The results demonstrated that the system could effectively rearrange the dishes into various configurations. SPOT outperformed traditional planning methods by intuitively determining which objects to move first to reach the desired outcome. Kallol Saha, a master’s student and co-lead researcher, remarked, “When humans organize our homes, we don’t have a set of rules in our minds that we follow before rearranging objects. We just look, plan, then act. SPOT brings that kind of intuitive decision-making to robots.”

The significance of SPOT extends beyond academia. The system was accepted for presentation at the 2025 Conference on Robotic Learning in Seoul, South Korea, highlighting its potential impact on the field. Funding for this research was provided by the Toyota Research Institute and the Office of Naval Research.

For further insights into SPOT and its applications, interested parties can visit the project website. The advancements made through this research could pave the way for more versatile robotic assistants, enhancing their ability to engage with and support humans in everyday environments.