Prediction allows to compensate for delays in the perception pipeline – data acquisition, transmission, pre-processing and tracking – and for an earlier and better planning of the action.
Inspired by the “Perception-Action” paradigm, we aim at applying the same principles to robot learning and control. Using pre-existing information, we want to build a model of the task at hand and, combining the latter with what is perceived through the event-cameras, determine how the robot should act based on a predicted outcome, and then using the observed outcome to update the model. We targeted human handing actions and bouncing ball.
We asynchronously feed standard deep learning architectures (LSTMs) with the updated spatial and temporal position of the target every time it moves of a fixed number of pixels, to infer both spatial and temporal coordinates of future points.
The asynchronous, motion-driven sampling strategy provides a lower error with respect to a synchronous time-driven one, thanks to the more regular and informative samples related to the actual dynamics being addressed. The computational load varies also with the latter, allowing to save computation when this is not needed.
Event-driven tracking, Deep Learning, LSTM Networks, TensorFlow 2.0.
Monforte M., Arriandiaga A., Glover A., Bartolozzi C. Where and When: Event-Based Spatiotemporal Trajectory Prediction from the iCub's Point-Of-View, Proceedings - IEEE International Conference on Robotics and Automation
DOI
Monforte M., Arriandiaga A., Glover A., Bartolozzi C. Exploiting Event Cameras for Spatio-Temporal Prediction of Fast-Changing Trajectories, Proceedings - 2020 IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS)
DOI
