A scalable multi-agent gantry system (Figure 1) will be used to demonstrate the performance of multi-agent planners (Figure 2) in the real environment, allowing for a sim2real approach. This hardware-in-the-loop system has the ability to interact with mobile robots within the workspace, allowing for agents to execute learned policies to achieve a common goal. For instance, constructing a pyramid of blocks with two, four degree-of-freedom manipulators in a shared workspace (Figure 3). This scalable setup can be adapted to scenarios for construction, manufacturing and warehousing solutions.