
Hardware we own. Software we run.
The plan needs one real robot (the SO-ARM101 we already have) and a rented 4090. Everything else is sim. The stack lives here so we install the same things.
- Local: MacBook M4 Max. MuJoCo, LeRobot recording, control station. Not the Isaac engine.
- Train box: RTX 5090 workstation (Blackwell sm_120). Isaac Lab 4096-env jobs, ACT / Diffusion / VLA fine-tunes.
- Owned robot: SO-ARM101 leader/follower (Feetech STS3215). Used in P3, P4, P8, P10.
- Edge + rover: Yahboom ROSMASTER M3 Pro with Jetson Orin NX Super 16GB. Shared by P5 and P6; P7 rides the same base.
- Print: Bambu H2C for the collector intake. The rover itself is an integrated ROS 2 robot, not a printed chassis.
| Tool | What it is | Used for |
|---|---|---|
| MuJoCo | Contact physics on the Mac (Apple Silicon) | Foundations, sysid, arm dry-runs (P1, P3, P4) |
| Isaac Lab | GPU-parallel simulator + RL (thousands of envs) | Factory lift, legged RL (P2, P9) |
| ROS 2 + Gazebo | Runtime + sim for robots that move through space | Rover and ball collector only (P6, P7) |
| LeRobot | SO-101 teleop, ACT, Diffusion, SmolVLA | Imitation and VLA on owned hardware (P4, P8, P10) |
MuJoCo foundations
Sim only · MacBook · $0

MacBook M4 Max
IN PROGRESSMuJoCo, LeRobot recording, control station. Not the Isaac engine
Isaac Lab factory
RTX 5090 workstation · $0 extra

RTX 5090 workstation
PLANNEDIsaac Lab 4096-env jobs, ACT / Diffusion / VLA fine-tunes. Blackwell sm_120
System ID on the arm
SO-ARM101 you own · $0

SO-ARM101 (leader + follower)
IN PROGRESSOwned arm: sysid, imitation, desk payload, VLA. Not the rover arm.
Where to buy ↗Imitation learning on the arm (tethered)
SO-ARM101 · optional second webcam ~$40

2× different USB cameras
PLANNEDGripper-cam + scene-cam. Two different models so the OS does not swap paths
Where to buy ↗Autonomous rover + cloud skill loop
Yahboom M3 Pro (Orin NX 16GB) · ~$2.3k

Yahboom M3 Pro (Orin NX 16GB)
PLANNEDIntegrated mobile manipulator for ROS 2 navigation, cloud planning, memory and the correction-skill loop. P5 and P6 run in parallel on this robot.
Where to buy ↗External SSD for full-disk clone
PLANNEDClone the M3 Pro image before changing anything
Spare arm servos
PLANNEDM3 Pro 6-DOF arm replacements
DIY alternative (not used)
Replaced by an integrated ROS 2 robot so time goes into autonomy rather than base assembly.
Tennis-ball collector
P6 M3 Pro + intake · ~$90

Collector intake kit
PLANNEDRoller, hopper, beam-break, tennis balls on the P6 M3 Pro
Robot menagerie
M3 Pro from P6 + Bittle X · ~$300

Petoi Bittle X
PLANNEDCheapest honest quadruped; reuse the P6 M3 Pro for mobile manipulation
Legged sim-to-real
$0 on Bittle · or +$1,600 Go2

Unitree Go2 EDU (optional)
OPTIONALBuy only if P9 becomes the specialization. Default is Bittle.
Whole-body / VLA
SO-ARM101 real · humanoid stays in sim · $0

Unitree G1 (not buying)
OPTIONALHumanoid stays in sim. Do not fake a hardware demo.
Where each piece runs
Mac is the control station. Heavy sim is remote. Filter by project or machine.
Environment
Simulators & Physics
RL & Training
Imitation Learning & Manipulation
iDP3 (Improved 3D Diffusion Policy)
TWIST2 high-level policy lineage. Fold in if we reuse that path
Docs ↗Teleop, Retargeting & VLA (P10)
Datasets & Model Hubs
Compute & Remote Access
Dev Tools
Hardware (Product Pages)
P01 · MuJoCo foundations: Laptop only. $0. mjpython / make view on the M4 Max. Cartpole is not the project. The fish + harness v0 is.
P02 · Isaac Lab factory: RTX 5090. No ROS. Blackwell sm_120 gate first. Lift-Cube at 4096 envs. Train headless; replay headful.
P03 · System ID: Owned SO-ARM101. Direct serial. Middleware corrupts the measurement. common/sysid/ is written once here.
P04 · Real-arm IL: Highest-employability project. LeRobot on the Mac for record; train on the 5090. Two different webcams; watch the feed, not the follower.
P05 · Edge: M3 Pro Orin NX 16GB, shared with P06. P5 and P6 run in parallel on the same robot. Measure p99, not average FPS.
P06 · Rover: Yahboom M3 Pro ~$2.2k (verify). ROS 2 Humble + Gazebo Fortress. The project those tools exist for. Own the TF tree before you tune Nav2.
P07 · Ball collector: +$90 on the P6 rover. Vision is the gift. The intake jam rate and the battery are the project.
P08 · Menagerie: M3 Pro from P6 + Bittle ~$300 extra. Skip TurtleBot 4. Point common/ at robots you did not design. A Go2 decision waits until week 20.
P09 · Legged RL: $0 on Bittle, or +$1,600 Go2. A large attributed gap is a result. An unmeasured clean clip is a claim.
P10 · VLA / WBC: Real arm. Humanoid stays in sim. Compare against P4 ACT on the same cases. Do not fake metal.