OpenAI models train a virtual Microduck robot to find a cat in hours

OpenAI models GPT-6 Sol and Astra were used to train a virtual Microduck robot from Pollen Robotics and Hugging Face to walk around a room and identify a bengal cat, work completed in a few hours that previously might have taken weeks. The test used Nvidia Isaac Sim, a Nous Research agent harness called Hermes and a Nvidia DGX Spark computer. The result points to faster robot training for manufacturing and e-commerce, though reliable real-world deployment remains difficult.

How the virtual duck learned to walk and watch

The author trained the simulated Microduck to move without toppling over, using GPT-6 Sol to feed instructions to a control model, watch videos of the virtual robot and suggest changes to its training. The plan is to have a physical Microduck dispense a treat for a pet bengal named Leona, possibly by bowing its head to release one from a 3D-printed head-mounted chamber. An attempt to teach the robot to moonwalk from a GIF of Michael Jackson produced only a jaunty backward jog. The vision system for identifying cats is described as not good enough for a real product.

University of California, Berkeley professor Ken Goldberg and his students tested Astra after its release this past September for building detailed simulations needed for robot training. One lab member gave Astra access to Nvidia tool Newton, showed it a video clip of a real robot, and asked for a simulation of the scene. Within an hour it produced a full 3D simulation with camera position, lighting, background and materials set, Goldberg said. Astra also automated much of the routine work of simulation training by running experiments and examining results.

What faster robot training means for work

Nvidia researcher Jim Fan, who has collaborated with Goldberg, said AI models can now control robots in a way that mirrors software agents, using visual language action models for physical actions as language models use software tools. This week Fan, Goldberg and others open sourced an agentic framework called Aspire that lets a robot discover and use new skills. At the Conference on Robot Learning in Austin, Texas, Fan showed Astra controlling a robot arm to paint the Golden Gate Bridge with nearby paint and brushes, and cited an example where Astra learned to use robot arms to solve a Rubik’s cube. Goldberg said industrial robotics and automation firms have shown interest in the approach.

The account is a single journalist test and lab demonstrations, not proof that simulated behavior will transfer faithfully to physical robots. Sophisticated tasks such as dancing humanoids or grasping robot hands still require extremely detailed simulations and significant engineering expertise. The findings apply to simulation and prototyping workflows, not to consumers or workplaces without access to the same models, Nvidia software and compute.