The iCub is continuously updated to integrate ED technology. A SoC/FPGA-based modular infrastructure, AER serialisation , and YARP middleware, support the integration of visual, tactile and auditory ED sensors, neuromorphic computational platforms (SpiNNaker and DYNAP) and software modules for seamless integration on the robot.
Neuromorphic iCub Infrastructure
Amongst the latest developments, we implemented a new vision system integrating upgraded ED and frame-based sensors. The low spatial resolution, large field of view and motion sensitive ED sensors coupled with low temporal but high spatial resolution and small field of view frame-based sensors parallels the organisation of the primate's foveated vision. Coarse large field of view periphery can be used to detect salient regions in the scene, that guide sequential saccades that put the region of interest in the high acuity fovea for detailed stimulus processing. To explore ED tactile sensing, we are working on the emulation of ED encoding using the current capacitive sensors integrated on the iCub. Besides the improvement in communication bandwidth thanks to the sensor compression and use of the serial AER protocol, the final goal of this activity is to acquire asynchronous data from different types of sensors (vision and skin at first) and study the use of temporal correlations for multi-sensory integration.
The integration of event-driven systems into the iCub also includes the development of the full software pipeline to intelligently control the robot based on the event-driven perception. The robot infrastructure requires such a framework to be distributed across multiple computers, that don’t necessarily have to be mounted within the robot. The Event Driven Perception for Robotics group develops and maintains the code for embedded systems and operating system drivers, as well as the open-source libraries for event decoding, communication, multiple processing algorithms, visualization and robot control. The scientific research performed by the EDPR group is realised as a documented software packages as well as applications to demonstrate the results, live on the robot. These packages are made open-source to promote the replication of results for our research collaborators. The core libraries are developed in such a way as to promote fast communication between modules and ensure real-time operation for on-line and safe robotics applications. The libraries are fully compatible with the yet-another-robot-platform (YARP) that provides the software framework for all iCub robots.
Publications
Glover, A., Vasco, V., Iacono, M., & Bartolozzi, C. The event-driven software library for YARP — with algorithms and iCub applications. Frontiers in Robotics and AI
Bartolozzi, C., Ros, P. M., Diotalevi, F., Jamali, N., Natale, L., Crepaldi, M., & Demarchi, D. Event-driven encoding of off-the-shelf tactile sensors for compression and latency optimisation for robotic skin. IEEE/RSJ International Conference on Intelligent Robots and Systems
Ros, P. M., Crepaldi, M., Bartolozzi, C., & Demarchi, D. Asynchronous DC-free serial protocol for event-based AER systems. IEEE International Conference on Electronics, Circuits, and Systems
Ros, P. M., Laterza, M., Demarchi, D., Martina, M., & Bartolozzi, C. Event-driven encoding algorithms for synchronous front-end sensors in robotic platforms. IEEE Sensors Journal
De Marcellis, A., Stanchieri, G. D. P., Faccio, M., Palange, E., Ros, P. M., Martina, M., ... & Bartolozzi, C. Live Demonstration: Event-Driven Serial Communication on Optical Fiber. IEEE International Symposium on Circuits and Systems
Rovere, G., Bartolozzi, C., Imam, N., & Manohar, R. Design of a QDI asynchronous AER serializer/deserializer link in 180nm for event-based sensors for robotic applications. IEEE International Symposium on Circuits and Systems
Publications
Glover, A., Vasco, V., Iacono, M., & Bartolozzi, C. The event-driven software library for YARP — with algorithms and iCub applications. Frontiers in Robotics and AI
DOI
Bartolozzi, C., Ros, P. M., Diotalevi, F., Jamali, N., Natale, L., Crepaldi, M., & Demarchi, D. Event-driven encoding of off-the-shelf tactile sensors for compression and latency optimisation for robotic skin. IEEE/RSJ International Conference on Intelligent Robots and Systems
DOI
Ros, P. M., Crepaldi, M., Bartolozzi, C., & Demarchi, D. Asynchronous DC-free serial protocol for event-based AER systems. IEEE International Conference on Electronics, Circuits, and Systems
DOI
Ros, P. M., Laterza, M., Demarchi, D., Martina, M., & Bartolozzi, C. Event-driven encoding algorithms for synchronous front-end sensors in robotic platforms. IEEE Sensors Journal
DOI
De Marcellis, A., Stanchieri, G. D. P., Faccio, M., Palange, E., Ros, P. M., Martina, M., ... & Bartolozzi, C. Live Demonstration: Event-Driven Serial Communication on Optical Fiber. IEEE International Symposium on Circuits and Systems
DOI
Rovere, G., Bartolozzi, C., Imam, N., & Manohar, R. Design of a QDI asynchronous AER serializer/deserializer link in 180nm for event-based sensors for robotic applications. IEEE International Symposium on Circuits and Systems
DOI