A source-aware directory of embodied AI agents for robotic manipulation. It tracks GPT-6 Astra and other LLM/VLM systems across robot arms, grasping, simulation, and real-world demonstrations, with links to their decisions, execution interfaces, available code, and supporting evidence. As GPT-6 Astra brings renewed attention to this direction, the collection follows how the field develops across models and arms.
An embodied-AI agent adds a reasoning and tool-use layer to the robot-control stack. An LLM/VLM may interpret a task, choose robot skills, and delegate precise motion to a low-level controller or policy. This differs from an end-to-end policy or VLA that maps observations directly to actions.
Entries identify the robot or simulator, execution interface, available code or demonstrations, and important evidence limits. Results remain author-reported unless a documented maintainer rerun is available.
Robot Manipulation with GPT-6-Astra: Body Knowledge, Experience Reuse, Emergent Skills, and Sim2Real Transfer
GPT-6 Astra observes camera images and writes programs against an XLeRobot low-level control API, using robot geometry, synchronized action/state demonstrations, and reusable visual-feedback skills with fixed model weights. In an elevator-button task, the authors report 30 fixed-start simulation trials, 18 trials comparing new starting positions, 27 local-skill simulation trials, and 12 real-robot trials. Simulation assets and experience reduce same-start real-robot mean completion time by 53.0% and 49.9%, respectively. The repository includes prompts, per-trial data, simulation and hardware-control sources, calibration documentation, and offline analysis scripts. This is an arXiv preprint with author-reported results from a narrow task: physical success means operator-confirmed gripper-tip contact, while simulation requires button activation; the reusable skills were refactored by researchers from an agent-generated routine.
Source: arXiv preprint · Code, trial data, and demonstrations
ARSTAG uses a hierarchy of language agents to turn a single RGB image and a natural-language instruction into task-scoped simulation scenes, robot-feasible demonstrations, and task-consistent data expansion. A coordinator agent manages feedback and recovery across the pipeline. The paper reports sim-to-real transfer across seven grasping, placement, and stacking tasks on a dual-arm robot, with π₀.₅ reaching a reported 74.6% average real-world success rate. This is author-reported research evidence; the result is for generated-data-assisted visuomotor policies rather than direct LLM control of the robot, and the public release should be checked for exact hardware, data-generation settings, and evaluation artifacts.
Source: Paper · Project page
Figure: ARSTAG supplementary-video overview comparing simulation demonstrations with real-robot execution across task-specific manipulation tasks. User-provided image based on the project presentation.
SafeHarness adds two safety mechanisms to a coding-agent manipulation harness: it plans and geometrically verifies waypoint routes with replanning, and selects obstacle-aware contact poses for grasping and placement. On the SafeLIBERO simulation benchmark, the authors report 71.9% task success and 87.5% collision avoidance over 32 tasks with 10 seeds each, using GPT-6 with a frozen π₀.₅ policy and Harness VLA skills. In the same-agent ablation, adding SafeHarness raises task success from 31.0% to 71.9% and collision avoidance from 59.0% to 87.5%. This is simulation evidence, not a real-robot result; 12.5% of episodes still displaced the obstacle. The paper does not currently link a public implementation.
Figure: SafeHarness route-planning and contact-execution examples and reported SafeLIBERO results. Image provided by the contributor; performance figures are author-reported simulation results.
Teach-and-Grow Learning (TGL) turns a few successful demonstrations into reusable closed-loop Skill Blocks. A multimodal agent retrieves and composes these blocks, grounds them in the current scene, selects robot-native tools or policies, checks physical outcomes, and revises the remaining route. A persistent Skill Library and Experience Memory let the system add local capabilities without task-specific policy retraining. The paper reports LIBERO results and controlled studies of skill induction, save-and-reload, feedback-driven replanning, and local capability growth. The current evidence is primarily simulation-based; no public code or real-robot validation was confirmed in the paper record, and its proposed scaling law remains a hypothesis rather than a longitudinal result.
Figure: Teach and Grow overview showing reusable skills, experience memory, scene changes, and outcome checking. User-provided image based on the project presentation.
RoboDawn gives a frozen VLM a compact interface of discrete translation, rotation, and gripper commands, then uses visual observations and execution feedback for closed-loop manipulation. The authors report 73.6% success for GPT-6 Astra with one in-context demonstration versus 53.2% zero-shot on RoboTwin 2.0 C2R, and 47.17% versus 35.67% on RoboDojo. The technical report also describes zero-shot physical trials on Franka (9/10 block-in-basket, 5/10 block stacking) and dual Piper cloth folding (0/10). These are author-reported results; code is marked as forthcoming, and the real-robot trial counts are small.
Source: Project and technical report · Recorded episodes and results · Report PDF
Figure: RoboDawn simulation recording showing the task scene, agent trace, tool call, and observation views. Screenshot provided by the contributor.
AGP places task planning and execution under a general-purpose coding agent: it reads robot-camera observations and state, writes and runs programs, issues Cartesian or joint-space commands through a documented interface, and revises actions from physical feedback. On an I2RT YAM setup, the authors report 57 successful trials out of 62 across eight main-result configurations; a separate released dataset contains 162 real-robot trials and should not be conflated with that headline evaluation. The results are author-reported and each configuration has limited trials; the project publishes per-trial material for inspection.
Source: Project page and results · Paper · Code · Trial dataset
Figure: AGP demonstration interface showing sequential and simultaneous task variations, human demonstration input, target images, and robot-camera observations during cloth folding. User-provided image based on the project presentation.
A comparative study of GPT-6 Astra as a direct robot policy and as a high-level reviewer or correction layer for the learned π₀.₅ policy. The public evaluation covers RoboDojo tasks and compares direct end-effector control with hybrid System-2 reasoning plus System-1 sensorimotor skills. The repository reports Astra Direct at 26% success with a mean score of 37.81, and the hybrid policy at 48% success with a mean score of 62.60; GPT corrections were applied to 14.4% of executed control steps. These are author-reported evaluation results across selected cases, not evidence of general robot competence, and the public release does not represent every deployment or evaluation artifact.
Source: Repository and code · Project page · Astra embodied-AI index
Figure: GPT-6 Astra correction in the hybrid-control loop, with task state, decision rationale, command details, and recorded end-effector control. User-provided image based on the project presentation.
An open-source closed-loop framework for in-context robot learning with VLM agents. The agent can use demonstrations, target images, interaction history, and execution feedback without gradient updates or task-specific parameter changes. The public repository provides adapters for ARX X5 and I2RT/YAM robot arms, structured robot-tool actions, IK checks, gripper and waypoint control, feedback-driven replanning, and append-only run recording. The paper reports author-run real-robot results across six tasks; the public tree does not include all deployment hosts, private prompts, calibration, run recordings, or the complete evaluation environment.
Source: Repository and code · Paper · Project page
Figure: GPT-Policy combines task information with human video, robot video and action, goal images, interaction feedback, and self-history to guide robot actions. User-provided image based on the project presentation.
Cortex / InternVLA-M1.5 — A Bidirectionally Aligned Embodied Agent Framework for Long-Horizon Manipulation
An embodied-agent framework for long-horizon manipulation that aligns a high-level System-2 planning agent with low-level execution through a shared subtask interface. The project describes 32 canonical manipulation skill primitives and releases long-horizon subtask annotations across robot datasets and benchmarks. Dataset coverage, model results, and real-robot claims should be checked against the project paper and released evaluation setup.
Source: Repository and code
Figure: Cortex demonstrations and system overview for long-horizon robot manipulation. User-provided image based on the project presentation.
A released, model-flexible VLM harness for robot-arm manipulation. The VLM selects bounded action units; robot-specific interpreters turn them into motion for Franka or Piper arms, or for supported simulators. The recent public release adds the GUMI data collectors, plugins, training pipeline, LoRA adapters, and demonstration data. The authors provide code, setup guides, and model/data links. A public run guide and reported zero-shot or fine-tuned results are starting points, not maintainer-verified hardware evidence.
Source: Project and code
A closed-loop VLM orchestrator that routes high-level reasoning through frozen robot skills. The project reports LIBERO-PRO simulation results and Franka FR3 real-robot tasks, with code and a paper linked from the project page.
Source: Project and demos
Project-page screenshot supplied by the contributor. Author-reported demonstration: remember five dolls’ positions, look away while they are shuffled, then restore the scene using the stored positions.
An open agentic framework for coding agents to improve real-world robot policies through repeated reset, execution, verification, recording, and refinement. It provides hardware-free examples plus workflows for calibrated physical stations; real-robot use requires the project's own safety and authorization preflight.
Source: Repository
An author-reported real-robot preview of GPT-6 Astra using a demonstration video and live visual feedback for plug insertion. The project and videos are public; code and systematic evaluation are still planned. The checked README does not specify the robot model or exact action interface.
GPT-6 Astra plans and calls motion skills through Inspect Robots on real YAM arms. The report and trial records describe a two-task comparison with Claude models. The authors report 19/20 bowl-task completions and 2/20 puzzle-task completions for Astra; these tasks do not establish general robot competence.
Source: Report and trial records · Video
Tools for connecting or evaluating robot agents.
- RoboDojo — an evaluation-only sim-and-real manipulation benchmark with 42 simulation tasks and 18 physical-robot tasks across three embodiments. The September 16–17, 2026 maintenance update corrected an observation-frame discrepancy and RGB byte-channel ordering; the maintainers report reruns left results essentially unchanged. Use XPolicyLab commit
bb9a0b5or later for the aligned RGB ordering. - FailBench — benchmark for VLM-based robot failure detection; tests whether VLMs can judge manipulation success across diverse real and simulated sources. The best reported mean balanced accuracy is 0.77, with performance near chance on contact-intensive tasks.
- ENACT — benchmark and dataset for evaluating VLM embodied cognition through forward and inverse world modeling of egocentric interaction. It includes code, data, a viewer, and a leaderboard; its scope is broader mobile manipulation and world modeling rather than a robot-arm execution interface.
- Inspect Robots — connects an agent or policy to a robot, task, and run logs. Version 0.58.0 adds on-demand camera mode for agents, exposes affirmative operator verdicts as a public scorer API, and improves validation and retention of rollout/evaluation logs. Its mock quick start checks the pipeline, not physical manipulation; see the LLM-agent guide and v0.58.0 release notes.
- ROS MCP Server — exposes ROS topics, services, and actions to an MCP client. It provides an interface, not the robot controller itself.
- StationeryBench — five bimanual desk-manipulation tasks (uncap a marker, retrieve an eraser, extract a sticky pad, pour paper clips, and hand over a ruler), with setup references, human demos, checklists, and 20-episode real-YAM run instructions. Built on Inspect Robots for both VLA and LLM-agent policies. The included mock has no physics, and its scripted oracle uses privileged state; neither is evidence of a model's physical manipulation ability. The package's binary operator score differs from the staged video grading in the associated report.
- FluxVLA — a VLA training, evaluation, and deployment platform used as a low-level policy/backend in some agent workflows, not an LLM agent itself. September 2026 updates add GPT-6 inference and checkpoint-free LIBERO evaluation, native GR00T N1.7 training/evaluation, and horizon-aligned RoboCasa action denormalization. These are platform capabilities; they do not establish LLM-agent performance or new physical-robot results.
- OpenAI: GPT-6 Astra — the original announcement and system overview.
- Robot-Use Agents — a perspective on using language models as tools for robots, including the distinction between high-level reasoning and low-level control.
- Awesome Robot Use Agent — a broader catalogue of robot-use-agent papers, projects, benchmarks, and community demonstrations; useful as a discovery index alongside this robot-arm-focused collection.
Author-created demonstrations; code and technical details are linked when available.
| Media | Source | Reported task |
|---|---|---|
 |
Jiafei Duan — Original post MolmoAct2 sim_eval |
Tabletop grasp and place in simulation |
 |
RotekSong — Original post | G1 cola-bottle pick-up in Isaac Sim |
 |
Kaifeng Zhang — Original post | Real-robot keyboard interaction with visual feedback; Astra learns to type after repeated attempts |
 |
star大小变 — Original post | Low-level robot-arm marker grasp |
 |
Axel — Original post Code PR |
Video-based in-context learning for mobile manipulation across environments, camera views, and layouts; Astra chooses end-effector or joint-space control without a text prompt |
 |
Lucas Cassiano — Original post | Rapid adaptation to an unseen robot embodiment |
 |
Tonghe Zhang — Original post ENPIRE |
Robot in-context learning through the ENPIRE harness |
 |
thijs — Original post | Painting task from a semantic prompt |
 |
Jikun — Original post FluxVLA |
Astra planning with a pretrained embodied policy |
 |
盒子桥 — Original post LoopMaster |
Real-arm cucumber slicing through Loop-ROS |
 |
应物而无累 — Original post | Video-to-simulation asset reconstruction followed by robot-arm control |
 |
Loule — Original post | Real-arm bread pick and place using end-effector poses |
 |
虽然不但是 — Original post | Piper pick and place with GPT-6 and RealSense |
 |
Qineng Wang — Original post Methods and evidence Interactive demo |
GPT-6 Astra spatial-constraint demonstrations: unlocking interlocked parts and threading a rope through three rings in Dual ALOHA kinematic replay; the page describes pregrasped motion with ideal grasps, so this is not physical-hardware validation |
Demonstrations can be expanded into full entries when supporting code or technical details become available.
Suggest a project related to LLM/VLM agent participation in robot-arm manipulation through an issue or the contribution instructions. Include the model's decision, execution interface, robot or simulator, available code or demos, and key evidence limits.
The MIT license covers this collection's original text and templates only. Linked code, data, media, and trademarks remain subject to their original terms. Entries are source-based and author-reported, not endorsements or maintainer-verified hardware results. This is an independent collection, unaffiliated with listed projects or providers.