Pushy Robotics Learns about Object Manipulation

The idea of directing a sensor to explore a space and its contents is useful.   Now an element of how objects in the space react.

Pushy robots learn the fundamentals of object manipulation
MIT researchers have compiled a dataset that captures the detailed behavior of a robotic system physically pushing hundreds of different objects. Using the dataset — the largest and most diverse of its kind — researchers can train robots to “learn” pushing dynamics that are fundamental to many complex object-manipulation tasks, including reorienting and inspecting objects, and uncluttering scenes.

To capture the data, the researchers designed an automated system consisting of an industrial robotic arm with precise control, a 3D motion-tracking system, depth and traditional cameras, and software that stitches everything together. The arm pushes around modular objects that can be adjusted for weight, shape, and mass distribution. For each push, the system captures how those characteristics affect the robot’s push.

The dataset, called “Omnipush,” contains 250 different pushes of 250 objects, totaling roughly 62,500 unique pushes. It’s already being used by researchers to, for instance, build models that help robots predict where objects will land when they’re pushed.

“We need a lot of rich data to make sure our robots can learn,” says Maria Bauza, a graduate student in the Department of Mechanical Engineering (MechE) and first author of a paper describing Omnipush that’s being presented at the upcoming International Conference on Intelligent Robots and Systems. “Here, we’re collecting data from a real robotic system, [and] the objects are varied enough to capture the richness of the pushing phenomena. This is important to help robots understand how pushing works, and to translate that information to other similar objects in the real world.” ... '