Just as Siri and other natural language processing software learns from its human overseers -- the more you speak to it, the better it will become -- the Washington team wanted to see how learning from multiple human sources could speed up a robot's ability to construct Lego models. Of course.
Maya Cakmak, UW assistant professor of computer science and engineering, explained: "Because our robots use machine-learning techniques, they require a lot of data to build accurate models of the task. The more data they have, the better model they can build. Our solution is to get that data from crowdsourcing."
To test the theory, the UW team first asked 14 participants to build Lego models of simple things -- trees, snakes and turtles -- in front of the robot. The robot used was a Gambit arm-and-gripper model, which is precise enough to pick up and place chess pieces. It was used in conjunction with a Microsoft Kinect, to measure the depth of the building blocks and designs. When asked to replicate the models built by the participants in the room, the robot failed to do so effectively.
Turning to the crowd, via Amazon Mechanical Turk, the robot posted a job asking "How would you make a shape (turtle, person, car, etc.) using these coloured blocks?" and received more than 100 answers and images back. Using pre-programmed algorithms, the robot could then hunt for the model that was most similar to one of the original designs -- the car, tree, turtle, snake and a few more -- and separate out the most difficult or simple features of the building process. Participants were also asked to rate the designs, and this fed into the robots decision-making. It then proceeded to build the model that most likely ticked all the boxes -- closest to the original, highly rated and simple to make -- with its robotic pincers.
The Washington team had programmed the robot to compare spatial features -- but it was in using the more than 100 model images/demos/explanations uploaded by those on Amazon Mechanical Turk that the Gambit was able to start making the same conclusions a human would, and put that into practice. Being able to compare a perfect model against a handful of models presented to it, does not give the robot enough to go on. On top of this, by having a vaster number of models to refer to, the robot could decide which it was most capable of building. If only presented with one option, of a difficult build, it would be less likely to build a perfect replica. Giving the robot more than 100 options allows it to select the model that is not only close to the original, but also within its power to create.
That is why the finished models were simpler versions of the original. "The end result is still a turtle, but it's something that is manageable for the robot and similar enough to the original model, so it achieves the same goal," said Cakmak.
Although the robot could not learn sufficiently from watching the 14 participants, it's clear this first step is needed. Combining that experience with the opinions of 100 others not in the room, was what gave the results.
"Future work will focus on exploring the application of the approach to more challenging tasks such as 3D object assembly and tool use, self-learning of heuristic measures of task-difﬁculty, and endowing the robot with the ability to use decision theoretic methods to decide when to crowdsource and when to ask the physical user for more examples," write the authors in a paper presented at the 2014 IEEE International Conference on Robotics and Automation.