Robotics Training Gets a Reality Check From Simulation, Code, Vision, and Touch

Robotics is attacking the training problem from several directions. Simulation assets, coding agents, hand-capture data, and tactile models are converging around a practical goal: making robots perform useful work without requiring endless human demonstrations.

One example starts with a can, a robot arm, and Isaac Sim. SimReady means an OpenUSD asset has the properties required for a defined simulation use case, including collisions, mass, joints, and joint drives. That distinction matters because a robot can look correct while stationary yet move or interact with objects incorrectly once the simulation starts.

SimReady Central, an open asset library on Hugging Face, provides robot and object assets with authored simulation properties and profile-validation evidence. SimReady Foundation connects three levels—Profile, Features, and Requirements—and validation checks the asset settings against the profile’s requirements. In other words, the asset is not merely a handsome digital prop.

Building a Pick-and-Place Task

The example combines the UR17G arm, Robotiq 2F-140 gripper, and beverage can in Isaac Sim. The task is simple enough to explain and unforgiving enough to expose bad setup: grasp the can, lift it, move it to a target, and release it.

The three packages include the models and simulation settings. Developers assemble the scene, define the arm–gripper assembly, add a work surface, set starting positions, and write controls for moving the arm and opening or closing the gripper. The task code can be created and run with GPT-6 Sol in an open Isaac Sim scene, although another agent and model combination may work if it can load the required skills and execute Python against Isaac Sim.

The required software includes Isaac Sim 6.2, the version used for the example, along with its system requirements. Agent Skills documentation makes the bundled skills available to the agent, while the Hugging Face CLI is needed only for the command-line asset download option. Developers can use the USD Search endpoint, find the three assets in SimReady Central, and select Download before saving them to a folder.

Those folders need to stay intact so Isaac Sim can find their supporting files. The asset versions linked in the example should be used, with local USD file paths supplied for the arm, gripper, and beverage can; SimReady Central Space search can also locate OEM curated content.

Before running pick-and-place, test the joints, movement, and contact behavior. Review the arm and gripper’s joint limits and drive settings, try a small arm movement, open and close the gripper, and confirm that the can rests on the work surface instead of falling through it. An agent can perform these checks, or developers can do them manually in Isaac Sim.

During the task, watch whether the beverage can stays in the gripper as the arm moves. Asset structures vary, so the settings in the loaded scene should be the starting point—not assumptions imported from another package.

From Agentic Code to Robotic Touch

At the 2026 Conference on Robot Learning, UC Berkeley and NVIDIA are announcing agentic robotics, an approach that uses AI coding tools to program robots for practical industrial tasks. The method returns to a basic idea: understand how the robot works, then write human-readable code that makes it useful, with no data and no humans required.

For decades, model-based methods grounded robot behavior in detailed physics and mathematics. Imitation learning has reached a plateau, according to Ken Goldberg, a UC Berkeley professor of robotics. In UC Berkeley’s testing, agentic robotics approaches reached success rates of up to 99 percent, compared with only 20 percent for baseline imitation learning techniques.

The near-term target is industrial manufacturing and manipulation, where precision matters more than humanoid theatrics. Semiconductor partners need to place components within 0.3 mm, and Spencer Huang, NVIDIA’s Director of Product for Robotics, says agentic robotics can reduce robot retasking from a minimum of 8 hours at Wistron in Taiwan to a minute or two.

That approach still meets the oldest problem in robotics: simulation and physical contact are hard. Agents can generate flawed code, so NVIDIA uses adversarial setups in which one agent designs a system and another tries to find its weaknesses. The robot may be programmable, but the real world remains a hostile test suite.

Vision and touch are advancing alongside code. Noitom Robotics released HiPHI-Hand on Hugging Face with 11 hours of synchronized dual-fisheye egocentric video, camera extrinsics referenced to the headset IMU, and 3D hand joint positions from optical motion capture. Captured with micro-markers on the fingers and the SOL ego-vision headset, the data provides sub-millimeter 3D hand ground truth even when hands are occluded and was used to train NVIDIA EgoHand, a monocular 3D hand reconstruction model released under the OpenMDW license.

Robbyant, an embodied AI company within Ant Group, released LingBot-Touch as an open-source tactile foundation model. It turns magnetic tactile, visual tactile, and piezoresistive signals into unified tactile tokens, using nearly 5,000 hours of real-world tactile data across three sensing modalities.

Across 26 real-world tests using the Sharpa Wave and XHand dexterous hands, LingBot-Touch improved four action models—LingBot-VLA, π0.5, Diffusion Policy, and T-Rex—by approximately 14 to 26 percentage points on average. In one bottle-cap test, T-Rex rose from 56.25 percent without tactile input and 62.5 percent with raw sensor signals to 90 percent with LingBot-Touch’s representation.

All 26 tests used LingBot-Touch-Tiny, which contains approximately 5.3 million parameters and adds only about 0.13 percent to 0.16 percent to total model size when integrated with LingBot-VLA, π0.5, and T-Rex. The technical report and code are open source, with the model available through Hugging Face and GitHub.

Stay connected via Google News

Follow us for the latest AI News