Google researchers are hoping to further the state of deep learning to offer smoother, more human-like motions to robots, giving them the ability to seamlessly plan for next actions while executing the current one.
Groove like humans
In a paper titled “Thinking while moving: Deep reinforcement learning with concurrent control”, the researchers successfully developed an algorithmic framework to help a robot more closely mimic how an actual person or an animal move.
While the authors acknowledged past successes in utilizing deep reinforcement learning for in-hand manipulation tasks, they note that these rely on a “blocking” observe-think-act paradigm.
Such an approach will not work well in real-world environments, though, due to the highly dynamic nature of our world. After observing the environment and computing an action, for instance, the robot might find that the current environment has evolved from what was initially observed. Imagine a humanoid robot trying to walk to the other side of the room, and having a kid dashing across its path mid-stride.
“[Previous work] use a blocking observe-think-act paradigm: the agent assumes that the environment will remain static while it thinks, so that its actions will be executed on the same states from which they were computed. This assumption breaks in the concurrent real world, where the environment state evolves substantially as the agent processes observations and plans its next actions,” wrote the authors in their paper.
The researchers successfully tackled the challenge by extending existing value-based deep reinforcement learning algorithms, using a continuous-time formulation of a dynamic programming equation (Bellman Equations) that is optimized to incorporate an awareness of inherent system delays.
The result? Pick-up arms that move faster and smoother than using traditional methods. Additional information on their work – including irresistible videos of robot arms picking up items, can be found here.
Photo credit: iStockphoto/StockRocket