Simulations ran on specialized AI chips from Nvidia instead of the general-purpose chips used in computers and servers. As a result, the researchers say they were able to train the robot in less than one-hundredth of what would normally be required.
Using specialized chips also presents challenges. Nvidia’s chips excel at the computations that are important for rendering graphics and running neural networks, but they are not well suited for simulating physics properties such as climbing and sliding. So the researchers had to come up with some clever software workarounds, says Rev Leberedian, Nvidia’s vice president of simulation technology. “It’s taken us a long time to fix this,” he says.
Simulation, AI and specialized chips have the potential to advance robotic intelligence. Nvidia has evolved software tools Which makes it easy to simulate and control industrial robots using its chips. The company has also established a Robotics Research Lab in Seattle. and sells chips and software For use in self-propelled vehicles.
Unity Technologies, which makes software for creating 3D video games, has also branched out into creating software suitable for roboticists to use. Danny Lang, the company’s senior vice president for artificial intelligence, says Unity saw how many researchers were using the company’s software to run simulations, so they made it more realistic and compatible with other robotics software. Unity is now working with Swedish company Algoryx, which is testing whether reinforcement learning and simulation can Train forestry robots to collect logs.
reinforcement learning is done over the decades But recently some have made remarkable AI milestones, others thanks to advances in technology. In 2015, reinforcement learning was used to train a computer to play Go, a subtle and intuitive board game with superhuman skills. It has recently been put into practical use, covering the automated aspects of chip design that require experience and judgment. The trouble is that learning this way requires a lot of time and data.
For example, it took the company Open AI more than 14 days to train a robotic arm to manipulate the Rubik’s Cube in a crude way with reinforcement learning, using multiple CPUs running simultaneously. Waiting two weeks each time to retrain robots can discourage companies from using robots.
Early attempts to train robots with reinforcement learning split the process into several parts real world robots. Improvements in physical simulation have made it possible to accelerate learning in virtual environments.
The new work is “extremely exciting for end users”, says Andrew Spielberg, an MIT student who has used similar simulation methods to create new physical designs for robots. He noted that a research group at Google has done related work, Speeding up robot learning by splitting it In one of the company’s custom tensor processing unit chips.
Tully Foote, which manages the widely used open source robot operating system Open Robotics Foundation, says that simulation is increasingly important for commercial users. “Validating software in realistic scenarios before deploying hardware saves a lot of time and money,” he says. “It can run faster than real time, never breaks the robot, and can be reset automatically and immediately when an error occurs.”
But Tully says transferring robotic learning to the real world is much more challenging. “There’s a lot of uncertainty in the real world,” he says. “Dirt, lighting, weather, hardware non-uniformity, wear and tear, all need to be tracked.”
Labradian at Nvidia says the kind of simulation that is used to train walking robots could ultimately also influence the design of the algorithms involved. “The virtual world is valuable for just about everything,” he says. “But certainly one of the most important ones is to create the playground or training ground for the AI that we want to build.”
- The latest on tech, science and more: Receive our newsletter!
- Is Becky Chambers Science Fiction’s Last Hope?
- a fraction from Each, Dave Eggers’ new novel
- Why James Bond doesn’t use the iPhone
- Now’s the Time to Buy Your Holiday Gifts
- Religious exemption should not exist for vaccine mandate
- ️ Explore AI like never before with our new database
- Wired Games: Get the Latest Tips, Reviews, and More
- From robotic vacuums to affordable mattresses to smart speakers, customize your home life with the best picks from our gear team