Tangible Swarm: See The Unseen

Tangible Swarm builds upon the idea of creating a closed-loop control between a camera that acts as the sensing part (e.g., performing tag tracking, pattern recognition, etc.),  an agent-based simulator as the computational element, and a projector that fulfills the actuation component.  The proposed tool can be extended to a wide range of use cases ranging from novel robotic art installations to changing urban scenarios.  

Fig. 1  shows a bird's eye view of the arena built to conduct the experiments to demonstrate the proposed tool. The arena is a rectangular area of 2.5 x 2.5 m2. This experimental area is divided into a 5 x 5 grid where each cell corresponds to a 0.5 x 0.5 m2 area.

Fig. 1 - Bird's eye view of the arena built for the experiments. The arena is a rectangular area of 2.5 x 2.5 m2 delimited by 4 wooden planks attached to each other using hot melt glue.

Fig. 2  shows a diagram of the arrangement of the sensing module; a camera performing the tracking of the robots and of the actuation component; a projector displaying information on the arena.

Fig. 2 - Diagram with the physical disposition of the sensing (camera) and actuation (projector) elements

Fig. 3 shows the information workflow that takes place during these experiments. First, each robot (e-puck) carries a tag; this tag encodes a specific robot ID used in the system. Second, tags are recognized by a vision-based tracking system using a camera right on top of the arena. Third, (x,y,z) coordinates for each tag are calculated with respect to the bottom-left corner of the arena and sent to a ROS multi-master network where the robots are connected in a decentralized manner.  Robots publish information such as their battery level, neighbor IDs, sensor inputs, etc. into this network. Then, this information is extracted from the network and sent to Gama, an agent-based simulation software. With this information, Gama is able to create a virtual representation of the current state of the arena which is projected on the arena itself by using a short throw projector. 

Fig. 3 - Information workflow for the Tangible Swarm tool.

Finally, Fig. 4 shows a snapshot of an experiment where a rich virtual representation of the arena (e.g., colored cells, communication patterns between the robots, robot's internal variables, etc.) is projected on the arena and robots in real-time. 

Detailed information about the system setup as well as configuration parameters can be found in the supplementary material of the paper titled "Secure and secret cooperation in robotic swarms".