Stage execution
30 stages × 50 episodes in the training scene. Each stage starts from its own valid reset distribution.
Leaderboard
Score protocol
Three evaluation tracks, explicit aggregation and a reproducible evidence trail.
Manuscript baselines, not a verified community leaderboard
The current scores are the six paper baselines. Public submission intake, remote execution and independent verification are not yet available.
Evaluation units
Cross-scene
30 stages × 3 native scenes × 50 episodes. Freeze the policy; retain the background manifest for every run.
Long-horizon
10 sequences × 50 episodes. Reset once per episode, never between stages; measure final sequence completion.
How scores are aggregated
Stage and long-horizon averages weight tasks equally. For cross-scene results, average stage success within each scene first, then report the mean and standard deviation across the three scene averages. The ± value is scene variation, not a training-seed confidence interval.
The original manuscript records ACT/DP stage seeds 50000–50049 and GR00T seeds 10001–10050. New comparison sweeps should use matched episode seeds across methods and record the manifest; do not imply the paper used identical seed ranges for every method.
Keep task horizons and physical success predicates fixed. Persistent hold counters or latches are part of the task definition. π0.5 switches stage models for composition; GR00T uses a pooled task model with stage instructions.
Reproduction package
- IdentityEnvironment/task revision, robot UID, controller, horizon and seeds.
- PolicyCheckpoint hash, adapter revision, training-data scope and observation preprocessing.
- ScenesBackground IDs, camera profile and source/geometry manifests.
- OutcomesPer-episode success, steps, stage trace, logs, videos and aggregation script.