// PROJECTS · 29 WEEKS

Ten projects from a home lab.

Start on the laptop, then move to the arm on the desk, then to a rover in the garage. Projects 1–7 are the spine. Projects 8–10 are optional depth if the core is solid. Each one ends with a folder you can open, a short clip, and a row in the shared eval log.

PLANNEDP10 · Advanced · advanced

Whole-body / VLA

Fine-tune a VLA on the arm you own, compare it to P4 ACT, and do not fake a humanoid.

Weeks 29+ · SO-ARM101 real · humanoid stays in sim · $0 · 10_wbc_vla/

MuJoCo (arm / neck) + Isaac Lab (humanoid locomotion baselines)Train · RTX 5090 (SmolVLA fits; GR00T is tighter)Run · SO-ARM101 real + simBuy · $0. No humanoid. No VR headset.ROS · NoGazebo · No
Whole-body / VLA
PLANNED
DEMO 10 · PLANNED
DEMO 10 · PLANNEDVideo lands here when this milestone ships.
VLA on the real SO-ARM101 next to the P4 ACT baseline table.
G1 in sim (stays in sim)
G1 in sim (stays in sim)
Tabletop object handling
Tabletop object handling
Teleop / VLA path
Teleop / VLA path
SmolVLAGR00TACT baselineNo fake humanoid
01

Goal

Recommended route: SmolVLA (~450M, shortest public SO-101 path) or GR00T N1.7 on the LeRobot format. Compare against the P4 ACT baseline on the same held-out cases. Plot a data-scaling curve at 10 / 25 / 50 demos. Alternative A (TWIST2, honestly scoped): reproduce in sim (they publish MuJoCo models, including a $250 2-DoF neck) and deploy the manipulation half on the real SO-ARM101. Alternative B (cross-embodiment): one architecture and one dataset format across arm, M3 Pro, rover, quadruped, documenting where reuse works. Do not fake the humanoid. Restraint is the point.

02

Why it matters

Best skill-to-cost ratio in the plan. You get onto the current frontier using hardware you already have. By arrival you will have built the halves separately: tracker in P9, diffusion in P4, runtime in P5. What you will not have is a Unitree G1.

03

The work

  1. 01

    Fine-tune SmolVLA or GR00T N1.7 on the P4 (and later) LeRobot datasets.

  2. 02

    Compare vs P4 ACT on identical held-out cases: success, generalization, latency.

  3. 03

    Data-scaling curve: 10 / 25 / 50 demonstrations.

  4. 04

    Write what the VLA gave you that ACT did not, and what it cost.

  5. 05

    Optional: TWIST2 in sim + real-arm manipulation half, or the cross-embodiment study.

04

Done looks like

  • A fine-tuned VLA running on the real SO-ARM101.
  • Comparison vs P4 ACT on identical held-out cases: success, unseen objects/positions, inference latency.
  • Data-scaling curve at 10 / 25 / 50 demonstrations.
  • Written account of what the VLA bought, and what it cost in compute and latency.
05

Stack card

SimulatorMuJoCo (arm / neck) + Isaac Lab (humanoid locomotion baselines)
Trains onRTX 5090 (SmolVLA fits; GR00T is tighter)
Runs onSO-ARM101 real + sim
Buy$0. No humanoid. No VR headset.
ROS 2No
GazeboNo
Compute / targetSO-ARM101, the real arm. Everything humanoid stays in simulation.
Installlerobot + SmolVLA, or NVIDIA/Isaac-GR00T; optionally GMR for retargeting study
06

Repos to study

07

Watch out

  • Don't fake a humanoid hardware demo.
  • No ROS 2. LeRobot + MuJoCo + Isaac Lab.
08

Workstreams

BUILD STREAM

Fine-tune, real-arm deploy, optional TWIST2-in-sim.

EVAL STREAM

ACT comparison, scaling curve, writeup.

Cross-check. Same held-out cases as P4. No new evaluation set that flatters the VLA.

09

What this proves

Frontier skill on hardware you own, without overclaiming a humanoid.

The credible line is a cross-embodiment pipeline on hardware you own, not “we shipped a humanoid.”

10

Study videos

11

Gear / BOM

Unitree G1 (not buying)
Unitree G1 (not buying)Humanoid stays in sim. Do not fake a hardware demo.
not buyingOPTIONAL
Gears: hardware + stack →