Diversity-driven agentic framework integrated with self-reflection and memory

RoboGene: Diversity-Driven Agentic Task Generation Framework for VLA Pre-training

Accepted by CoRL 2026

Yixue Zhang^1,2,*, Kun Wu^1,*, Zhi Gao³, Zhen Zhao¹, Pei Ren¹, Zhiyuan Xu¹, Fei Liao¹, Xinhua Wang¹, Shichao Fan¹, Di Wu^1,4, Qiuxuan Feng^1,4, Meng Li¹, Zhengping Che^1,†,‡, Chang Liu^2,‡, Jian Tang^1,‡

¹Beijing Innovation Center of Humanoid Robotics, Beijing, China

²School of Advanced Manufacturing and Robotics, Peking University, Beijing, China

³Beijing Institute of Technology, Beijing, China

⁴State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University, Beijing, China

^*Equal contribution. ^†Project lead. ^‡Corresponding authors.

Paper Hugging Face ModelScope

An automated framework for generating diverse, physically plausible manipulation tasks to enhance Vision-Language-Action model pre-training

Abstract

The pursuit of general-purpose robotic manipulation is hindered by the scarcity of diverse, real-world interaction data. Unlike data collection from the web in vision or language, robotic data collection is an active process incurring prohibitive physical costs.

Existing manual methods are unscalable and biased toward common tasks. To address this, we introduce RoboGene, an agentic framework designed to automate the generation of diverse, physically plausible manipulation tasks across single-arm, dual-arm, and mobile robots.

RoboGene integrates three core components: diversity-driven sampling, self-reflection mechanisms, and human-in-the-loop refinement. We conduct datasets of 18k trajectories and introduce novel metrics to assess task quality and diversity.

Methodology

STAGE 01

Diversity-Driven Sampling

Implements LFU strategy to explore task space comprehensively, ensuring coverage of edge cases often overlooked by manual curation.

STAGE 02

Self-Reflection Validation

Enforces physical constraints through simulation, filtering out infeasible trajectories and ensuring tasks are executable.

STAGE 03

Human-in-the-Loop

Continuous refinement through human feedback, enabling the system to adapt to specific robot embodiments.

Experimental Setup

Evaluated across multiple robot platforms including single-arm, dual-arm manipulators, and mobile manipulators.

18k+Trajectories Collected

3Robot Platforms

300+Manipulation Tasks

2xBetter than Baselines

Diverse Real-World Tasks

Automated task generation across multiple embodiments including mobile manipulators, single-arm, and dual-arm systems.

Water Plant

Organize Desk

Assembling Crucible

Replenish Shelves

Sort Buttons

Pour Water

Handover Tape

Open Medical Box

Rinse Lettuce

Downstream Fine-tuning Performance

Quantitative assessment of pre-training efficacy and real-world execution demonstrations.

Pre-training Efficacy

Grill Skewers

Weigh beakers

Build Blocks

Lubricate Gears

Zero-Shot Generalization & Robustness Analysis

Figure: Generalization Evaluation. Compare RoboGene (Red) against baselines across six categories.

Novel Objects

Illumination Changes

Static Distractors

Background Variations

Instruction Changes

In-Distribution

Citation

If you find this project useful in your research, please consider citing:

@inproceedings{zhang2026robogene,
  title={RoboGene: Diversity-Driven Agentic Task Generation Framework for {VLA} Pre-training},
  author={Zhang, Yixue and Wu, Kun and Gao, Zhi and Zhao, Zhen and Ren, Pei and Xu, Zhiyuan and Liao, Fei and Wang, Xinhua and Fan, Shichao and Wu, Di and Feng, Qiuxuan and Li, Meng and Che, Zhengping and Liu, Chang and Tang, Jian},
  booktitle={Proceedings of the 10th Conference on Robot Learning (CoRL)},
  year={2026}
}