// GLOSSARY

Words you will meet in this lab

Short definitions for the jargon on the project pages. Skim by project, or jump A–Z. Written so you can follow along without a separate textbook open.

// LAB-WIDE

Lab-wide

Words you will see on almost every project page.

Action

What the policy outputs each step: usually joint targets, torques, or gripper commands.

Used in P01, P04

Domain randomization

domain rand

Randomize sim parameters (mass, friction, delays) during training so the policy survives messier reality. Measure whether it actually helps.

Used in P09

Embodiment

The physical body the policy controls: arm, rover, quadruped, and so on. Code that assumes one body often breaks on another.

Used in P08, P10

Episode

One complete attempt at a task, from start until success, failure, or timeout.

Used in P04, P07

Eval harness

common/eval_harness/

Our shared logger across projects. Every run gets a row: task, metrics, how it failed, and whether it was sim, real, or paired.

Used in P01, P03, P04, P08

Gap report

GR01–GR05

A short writeup of where sim and reality disagree, with numbers. The signature artifact of this lab.

Used in P03, P04, P06, P07, P09

Held-out set

Test cases you freeze before training. You do not tune on them. That is how you know the result is real.

Used in P01, P02, P04, P09

Imitation learning

IL / learning from demos

The robot learns a task by watching demonstrations, not by exploring with a hand-written reward.

Used in P04, P08, P10

Observation

What the policy sees each step: joint angles, images, velocities, and so on. Write this down early. Wrong units here break everything later.

Used in P01, P04

Policy

The neural network (or controller) that maps what the robot senses into the next action.

Used in P01, P02, P04, P05, P09, P10

Reinforcement learning

RL

The agent learns by trial and error in an environment, getting rewarded for getting closer to a goal.

Used in P01, P02, P09

Retargeting

Map a human motion onto a robot whose body proportions differ. Optional study path in P10.

Used in P10

Sim-to-real

sim2real

Train in simulation, then run the same skill on a physical robot. The gap between those two is what we measure.

Used in P03, P04, P06, P09

Teleoperation

teleop (not teleportation)

You move the robot in real time, like puppeteering. In LeRobot that usually means a leader arm driving a follower arm.

Used in P04, P10

Whole-body control

WBC

Controlling arms, torso, and legs together as one coordinated system, not each half on its own.

Used in P10

// MODELS & INTERPRETABILITY

Models & interpretability

How people talk about looking inside neural nets. Useful background for VLAs and foundation models (P10), and for reading modern AI papers.

Activation

hidden state / feature

The numbers flowing through a layer while the model runs. Interpretability work often starts by asking what a particular activation pattern means.

Used in P10

Alignment

AI safety (narrow sense)

Making sure the system’s behavior stays aimed at what we actually want, especially as models get more capable. Auditing tools matter because full decoding may never happen.

Used in P10

Auditing

model audit

Systematic checks on a trained system: probes, scratchpad review, held-out tasks. The practical goal when you cannot fully decode every weight.

Used in P10

Black box

A model you can query but cannot easily explain. Modern nets are grown from data, so the insides are huge arrays of numbers, not hand-designed rules.

Used in P10

Chain of thought

CoT / scratchpad

When a model writes intermediate reasoning before the final answer. Safety researchers watch it for cheating, test-awareness, or misaligned plans. Mostly an LLM idea; VLAs may expose less of this.

Used in P10

Circuit

In mechanistic interpretability: a small subgraph of neurons and weights that implements one recognizable computation inside the larger net.

Used in P10

Direction (in activation space)

A vector inside the model’s hidden state that tends to line up with one concept. Steering or reading that direction is a common interpretability move.

Used in P10

Fine-tune

Take a pretrained model and train it a bit more on your data so it adapts to your robot and task.

Used in P04, P10

Interpretability

Research that tries to understand what is going on inside a trained model, instead of treating it only as input → output.

Used in P10

Linear probing

Train a simple linear classifier on frozen internal activations to test whether a concept (for example “gripper open”) is already encoded there.

Used in P10

Mechanistic interpretability

A stricter flavor of interpretability: reverse-engineer specific circuits or computations inside the network, not just correlate inputs with outputs.

Used in P10

Sparse autoencoder

SAE

A tool used in interpretability work. It expands a model’s internal activations into many mostly-off features so individual concepts are easier to spot.

Used in P10

Test-awareness

eval awareness

When a model seems to notice it is being evaluated and changes behavior (for example plays nice under tests). Chain-of-thought audits try to catch this.

Used in P10

// P01

P01 · MuJoCo foundations

Physics, control, and a first learned policy on the laptop.

Open project →

Action

What the policy outputs each step: usually joint targets, torques, or gripper commands.

Used in P01, P04

CPG

Central Pattern Generator

A rhythmic pattern generator. For the fish: sinusoidal joint commands with a phase offset down the body, tracked by PD.

Used in P01

CPG gait

The swimming (or walking) motion produced by a CPG. In P1 it is the scripted baseline the PPO policy has to beat.

Used in P01

Eval harness

common/eval_harness/

Our shared logger across projects. Every run gets a row: task, metrics, how it failed, and whether it was sim, real, or paired.

Used in P01, P03, P04, P08

Gymnasium

The standard Python API for RL environments (reset, step, observation, reward). Stable-Baselines3 talks to it.

Used in P01

Held-out set

Test cases you freeze before training. You do not tune on them. That is how you know the result is real.

Used in P01, P02, P04, P09

MJCF

MuJoCo XML / model file

The XML format that describes bodies, joints, actuators, geoms, and contacts for MuJoCo.

Used in P01, P03

MuJoCo

Multi-Joint dynamics with Contact

A fast physics engine for robots and contacts. Runs well on a Mac. Used for foundations, sysid, and arm dry-runs.

Used in P01, P03, P04

Observation

What the policy sees each step: joint angles, images, velocities, and so on. Write this down early. Wrong units here break everything later.

Used in P01, P04

PD

Proportional–Derivative control

A simple controller: push harder when error is large (P), and damp the motion using velocity (D). We write this by hand in P1.

Used in P01, P03

Policy

The neural network (or controller) that maps what the robot senses into the next action.

Used in P01, P02, P04, P05, P09, P10

PPO

Proximal Policy Optimization

A popular reinforcement-learning algorithm. Stable, widely used, and what we run for the small fish policy and later locomotion.

Used in P01, P02, P09

PPO policy

The neural network trained with PPO. Same idea as “policy,” just naming the training method.

Used in P01, P09

Reinforcement learning

RL

The agent learns by trial and error in an environment, getting rewarded for getting closer to a goal.

Used in P01, P02, P09

Reward term

One piece of the reward function (for example lift height or action penalty). Change one at a time and keep before/after numbers.

Used in P01, P02, P09

Stable-Baselines3

SB3

A Python library of RL algorithms. Handy for laptop-scale PPO on the fish.

Used in P01

// P02

P02 · Isaac Lab factory

GPU-parallel RL and the Franka lift task.

Open project →

Blackwell / sm_120

The RTX 5090 GPU architecture. get_device_capability() should print (12, 0). Wrong PyTorch builds waste the card.

Used in P02

Headful

Run with the viewer on. Slower. Use it to record clips and check that the policy looks right.

Used in P02

Headless

Run the sim without opening a visual window. Faster for overnight training.

Used in P02

Held-out set

Test cases you freeze before training. You do not tune on them. That is how you know the result is real.

Used in P01, P02, P04, P09

Isaac Lab

RL and robotics framework on top of Isaac Sim. Built for thousands of parallel environments.

Used in P02, P09

Isaac Sim

NVIDIA’s GPU robotics simulator. Heavy, needs a serious GPU. The world Isaac Lab trains inside.

Used in P02, P09

Parallel environments

vectorized envs

Thousands of copies of the same task running at once on the GPU. That is how Isaac Lab gets high sample throughput.

Used in P02, P09

Policy

The neural network (or controller) that maps what the robot senses into the next action.

Used in P01, P02, P04, P05, P09, P10

PPO

Proximal Policy Optimization

A popular reinforcement-learning algorithm. Stable, widely used, and what we run for the small fish policy and later locomotion.

Used in P01, P02, P09

Reinforcement learning

RL

The agent learns by trial and error in an environment, getting rewarded for getting closer to a goal.

Used in P01, P02, P09

Reward term

One piece of the reward function (for example lift height or action penalty). Change one at a time and keep before/after numbers.

Used in P01, P02, P09

RSL-RL

The RL library Isaac Lab often uses under the hood (PPO and friends), originally from Robotic Systems Lab.

Used in P02, P09

Seed

The random-number starting point for a training run. Multiple seeds show whether a result is stable or lucky.

Used in P09, P02

TFLOPS

Trillions of floating-point operations per second. A rough measure of GPU compute. We log a GEMM number as a capability check.

Used in P02

VRAM

GPU memory

How much memory the graphics card has. Limits batch size and number of parallel envs.

Used in P02, P05

// P03

P03 · System ID on the arm

Making the sim match the real SO-ARM101.

Open project →

Armature / rotor inertia

Extra inertia from motors and gears that the sim must include or the arm will look too snappy.

Used in P03

Eval harness

common/eval_harness/

Our shared logger across projects. Every run gets a row: task, metrics, how it failed, and whether it was sim, real, or paired.

Used in P01, P03, P04, P08

Gap metric

The quantitative discrepancy between matching sim and real runs: position error, timing, overshoot, or another frozen measure. A correction survives only if this improves on held-out runs.

Used in P06, P03

Gap report

GR01–GR05

A short writeup of where sim and reality disagree, with numbers. The signature artifact of this lab.

Used in P03, P04, P06, P07, P09

MJCF

MuJoCo XML / model file

The XML format that describes bodies, joints, actuators, geoms, and contacts for MuJoCo.

Used in P01, P03

MuJoCo

Multi-Joint dynamics with Contact

A fast physics engine for robots and contacts. Runs well on a Mac. Used for foundations, sysid, and arm dry-runs.

Used in P01, P03, P04

Numerical fitting

parameter optimization

Use an optimizer to estimate precise values such as friction, damping, latency, or wheel radius. The model chooses what to fit; math finds the number.

Used in P06, P03

PD

Proportional–Derivative control

A simple controller: push harder when error is large (P), and damp the motion using velocity (D). We write this by hand in P1.

Used in P01, P03

Phase lag

How much the real joint lags behind the commanded or simulated motion, usually in degrees or milliseconds.

Used in P03

RMS error

Root Mean Square

A single number for how far sim and real disagree on a joint trajectory. Lower after a good fit.

Used in P03

Sim-to-real

sim2real

Train in simulation, then run the same skill on a physical robot. The gap between those two is what we measure.

Used in P03, P04, P06, P09

SO-ARM101

SO-101

The open leader/follower arm we own. Used for sysid, imitation, desk-side edge work, and VLA. Not the rover arm.

Used in P03, P04, P05, P08, P10

STS3215

The Feetech smart servos inside the SO-ARM101. They run an internal controller you cannot see. Do not model them as ideal torque sources.

Used in P03, P04

System identification

sysid

Measure the real robot, then tune the simulator (friction, damping, and so on) until held-out trajectories match.

Used in P03, P08

// P04

P04 · Imitation on the real arm

Demos, ACT, Diffusion, and the 2×2 experiment.

Open project →

2×2 experiment

In P4: ACT vs Diffusion, each trained on real-data vs sim-data, all scored on the same 20 real-arm cases.

Used in P04

Ablation

Turn one ingredient off (for example domain rand) and compare. That is how you know what mattered.

Used in P09, P04

ACT

Action Chunking Transformer

An imitation policy that predicts a short sequence of actions at once. LeRobot’s usual first policy for the SO-101.

Used in P04, P10

Action

What the policy outputs each step: usually joint targets, torques, or gripper commands.

Used in P01, P04

Data-scaling curve

Plot performance at 10 / 25 / 50 demos (or similar). Shows whether more data still helps.

Used in P10, P04

Demonstration

demo

One recorded human (or teleop) example of the task. P4 aims for about 50 real pick-and-place demos.

Used in P04, P10

Diffusion Policy

An imitation method that generates actions by denoising, similar in spirit to image diffusion models. Our P4 comparison policy next to ACT.

Used in P04

Episode

One complete attempt at a task, from start until success, failure, or timeout.

Used in P04, P07

Eval harness

common/eval_harness/

Our shared logger across projects. Every run gets a row: task, metrics, how it failed, and whether it was sim, real, or paired.

Used in P01, P03, P04, P08

Failure taxonomy

A fixed list of failure types (grasp slip, collision, timeout, …). You count them instead of only saying “it failed.”

Used in P04

Fine-tune

Take a pretrained model and train it a bit more on your data so it adapts to your robot and task.

Used in P04, P10

Gap report

GR01–GR05

A short writeup of where sim and reality disagree, with numbers. The signature artifact of this lab.

Used in P03, P04, P06, P07, P09

Held-out set

Test cases you freeze before training. You do not tune on them. That is how you know the result is real.

Used in P01, P02, P04, P09

Imitation learning

IL / learning from demos

The robot learns a task by watching demonstrations, not by exploring with a hand-written reward.

Used in P04, P08, P10

LeRobot

Hugging Face’s open stack for teleop, datasets, ACT, Diffusion, and SmolVLA on hardware like the SO-101.

Used in P04, P08, P10

MuJoCo

Multi-Joint dynamics with Contact

A fast physics engine for robots and contacts. Runs well on a Mac. Used for foundations, sysid, and arm dry-runs.

Used in P01, P03, P04

Observation

What the policy sees each step: joint angles, images, velocities, and so on. Write this down early. Wrong units here break everything later.

Used in P01, P04

Policy

The neural network (or controller) that maps what the robot senses into the next action.

Used in P01, P02, P04, P05, P09, P10

Schema (dataset)

The agreed fields and types for every recorded episode (cameras, timestamps, joint names). Validate it before collecting 50 demos.

Used in P04

Sim-to-real

sim2real

Train in simulation, then run the same skill on a physical robot. The gap between those two is what we measure.

Used in P03, P04, P06, P09

SO-ARM101

SO-101

The open leader/follower arm we own. Used for sysid, imitation, desk-side edge work, and VLA. Not the rover arm.

Used in P03, P04, P05, P08, P10

STS3215

The Feetech smart servos inside the SO-ARM101. They run an internal controller you cannot see. Do not model them as ideal torque sources.

Used in P03, P04

Teleoperation

teleop (not teleportation)

You move the robot in real time, like puppeteering. In LeRobot that usually means a leader arm driving a follower arm.

Used in P04, P10

// P05

P05 · Edge deployment

Moving the policy from the 5090 onto a Jetson.

Open project →

FP32 / FP16 / INT8

Number formats: 32-bit float (full), 16-bit float (half), 8-bit integer. Lower precision is faster and riskier.

Used in P05

Heartbeat

A repeated signal that says the planner or controller is still alive. If it goes stale, the edge watchdog stops or enters a defined safe state.

Used in P06, P05

JetPack

NVIDIA’s OS + library bundle for Jetson (CUDA, TensorRT, and friends).

Used in P05

Jetson Orin NX Super 16GB

Orin NX

The NVIDIA module on the Yahboom M3 Pro carrier. Shared by P5 and P6: it runs the exported policy, plus safety, perception, SLAM, and Nav2 offline. 16GB is shared — keep the LLM off-board.

Used in P05, P06, P07

Jitter

Variation in loop timing from step to step. Bad jitter feels like a shaky or lunging robot.

Used in P05

ONNX

Open format for exporting a trained model out of PyTorch so other runtimes can load it.

Used in P05

p50 / p95 / p99

Latency percentiles. p99 is the slow tail. A policy that averages 20 Hz but hits 4 Hz at p99 will make the robot lurch.

Used in P05

Policy

The neural network (or controller) that maps what the robot senses into the next action.

Used in P01, P02, P04, P05, P09, P10

Quantization

FP16 / INT8

Store weights with fewer bits so inference is faster and smaller. Measure accuracy drift against FP32 on identical inputs.

Used in P05

SO-ARM101

SO-101

The open leader/follower arm we own. Used for sysid, imitation, desk-side edge work, and VLA. Not the rover arm.

Used in P03, P04, P05, P08, P10

TensorRT

NVIDIA’s inference engine. Takes an ONNX model and optimizes it for Jetson or discrete GPUs.

Used in P05

VRAM

GPU memory

How much memory the graphics card has. Limits batch size and number of parallel envs.

Used in P02, P05

Watchdog

Safety logic that watches for stale cameras, commands, heartbeats, NaNs, and missed deadlines, then runs a defined response such as a safe stop.

Used in P05, P06

Yahboom ROSMASTER M3 Pro

M3 Pro

Integrated ROS 2 mobile manipulator: Orin NX Super 16GB, dual T-mini Plus ToF LiDAR, binocular depth camera, 6-DOF arm (~30 cm, 410 g), mecanum base. Sold as the no-display Orin NX 16GB kit. Replaced the DIY LeKiwi plan.

Used in P05, P06, P07

// P06

P06 · Autonomous rover + cloud skill loop

ROS 2 navigation on the edge, grounded cloud plans, persistent memory, and measured correction skills.

Open project →

Action vocabulary

shared goal schema

The small allowed set of rover actions, such as navigate, observe, manipulate, report, and stop. Cloud output must match this schema before the edge accepts it.

Used in P06

Cloud orchestrator

The FastAPI service around the Claude API. It turns a command plus relevant memory into validated goals and writes correction skills. It never drives motors.

Used in P06

Correction skill

A readable, versioned file describing the observed sim-to-real gap, a hypothesis, simulator edits, the fitting routine, and measured before/after metrics.

Used in P06

Costmap

A grid of “how bad is it to drive here.” Obstacles inflate into lethal or expensive cells.

Used in P06

Edge-cloud boundary

The safety line in P6: the M3 Pro’s Orin NX owns fast physical actions and safe stopping; the cloud may reason slowly, return structured artifacts, or disappear without making the rover unsafe.

Used in P06

Gap metric

The quantitative discrepancy between matching sim and real runs: position error, timing, overshoot, or another frozen measure. A correction survives only if this improves on held-out runs.

Used in P06, P03

Gap report

GR01–GR05

A short writeup of where sim and reality disagree, with numbers. The signature artifact of this lab.

Used in P03, P04, P06, P07, P09

Gazebo

Gazebo Fortress

Robot simulator that plays nicely with ROS 2 Humble. P6 uses Fortress with a calibrated M3 Pro model. Jazzy + Harmonic is a later migration.

Used in P06, P07

Grounding

SayCan-style grounding

Restrict a language model’s plan to actions the robot can actually execute and places that really exist on its map.

Used in P06

Heartbeat

A repeated signal that says the planner or controller is still alive. If it goes stale, the edge watchdog stops or enters a defined safe state.

Used in P06, P05

IMU

Inertial Measurement Unit

Sensor for acceleration and rotation. Helps odometry and balance estimates.

Used in P06

Jetson Orin NX Super 16GB

Orin NX

The NVIDIA module on the Yahboom M3 Pro carrier. Shared by P5 and P6: it runs the exported policy, plus safety, perception, SLAM, and Nav2 offline. 16GB is shared — keep the LLM off-board.

Used in P05, P06, P07

LeKiwi

DIY mobile base that mounts an SO-101. Considered, then replaced by the M3 Pro so time goes into autonomy rather than base assembly. Not used.

Used in P06

LiDAR

Laser scanner that returns distances around the robot. Core sensor for SLAM on the rover.

Used in P06

Nav2

Navigation2

ROS 2 navigation stack: go from pose A to pose B with costmaps and recovery behaviors.

Used in P06, P07

Numerical fitting

parameter optimization

Use an optimizer to estimate precise values such as friction, damping, latency, or wheel radius. The model chooses what to fit; math finds the number.

Used in P06, P03

Odometry

/odom

The robot’s estimate of how it has moved, usually from wheel encoders (and maybe IMU). Drift is normal.

Used in P06

Persistent memory

Information saved outside the current task so the rover can use it tomorrow. P6 keeps exact facts in a normal database and fuzzy recall in a vector database.

Used in P06

ROS 2

Robot Operating System 2

Middleware for robots with many sensors and processes. Used here for the rover and collector, not for the arm stack.

Used in P06, P07

ros2_control

ROS 2 framework for talking to motors and hardware interfaces in a standard way.

Used in P06

Sim-to-real

sim2real

Train in simulation, then run the same skill on a physical robot. The gap between those two is what we measure.

Used in P03, P04, P06, P09

SLAM

Simultaneous Localization and Mapping

Build a map while figuring out where you are inside it. We use SLAM Toolbox for live maps.

Used in P06

Structured goals

An ordered, schema-validated JSON list sent from cloud planning to the ROS 2 orchestrator. Goals name high-level actions and parameters, never wheel speeds.

Used in P06

TF / TF tree

transforms

The tree of coordinate frames (map → odom → base_link → sensors). Own this before you tune Nav2.

Used in P06

URDF

Unified Robot Description Format

XML description of the robot’s links, joints, and sensor frames. Wrong frames break everything downstream.

Used in P06

Vector database

semantic memory

A store searched by meaning rather than exact words. It retrieves relevant past observations before the cloud planner builds a new plan.

Used in P06

Watchdog

Safety logic that watches for stale cameras, commands, heartbeats, NaNs, and missed deadlines, then runs a defined response such as a safe stop.

Used in P05, P06

Yahboom ROSMASTER M3 Pro

M3 Pro

Integrated ROS 2 mobile manipulator: Orin NX Super 16GB, dual T-mini Plus ToF LiDAR, binocular depth camera, 6-DOF arm (~30 cm, 410 g), mecanum base. Sold as the no-display Orin NX 16GB kit. Replaced the DIY LeKiwi plan.

Used in P05, P06, P07

// P07

P07 · Tennis-ball collector

Vision + intake on top of the rover.

Open project →

Beam-break

A cheap sensor that fires when something crosses a light beam. We use it to confirm a ball entered the intake.

Used in P07

Blob tracking

Find a connected patch of pixels (the ball) and track its centroid. Simple, brittle in hard lighting.

Used in P07

Episode

One complete attempt at a task, from start until success, failure, or timeout.

Used in P04, P07

Gap report

GR01–GR05

A short writeup of where sim and reality disagree, with numbers. The signature artifact of this lab.

Used in P03, P04, P06, P07, P09

Gazebo

Gazebo Fortress

Robot simulator that plays nicely with ROS 2 Humble. P6 uses Fortress with a calibrated M3 Pro model. Jazzy + Harmonic is a later migration.

Used in P06, P07

HSV

Hue Saturation Value

A color space that often makes colored objects (tennis balls) easier to threshold than raw RGB.

Used in P07

Jam rate

How often the intake clogs per N balls. Often the real performance number, not the detector score.

Used in P07

Jetson Orin NX Super 16GB

Orin NX

The NVIDIA module on the Yahboom M3 Pro carrier. Shared by P5 and P6: it runs the exported policy, plus safety, perception, SLAM, and Nav2 offline. 16GB is shared — keep the LLM off-board.

Used in P05, P06, P07

Nav2

Navigation2

ROS 2 navigation stack: go from pose A to pose B with costmaps and recovery behaviors.

Used in P06, P07

Precision / recall

Precision: of detections, how many were real balls. Recall: of real balls, how many we caught. Trade them off on purpose.

Used in P07

ROS 2

Robot Operating System 2

Middleware for robots with many sensors and processes. Used here for the rover and collector, not for the arm stack.

Used in P06, P07

Yahboom ROSMASTER M3 Pro

M3 Pro

Integrated ROS 2 mobile manipulator: Orin NX Super 16GB, dual T-mini Plus ToF LiDAR, binocular depth camera, 6-DOF arm (~30 cm, 410 g), mecanum base. Sold as the no-display Orin NX 16GB kit. Replaced the DIY LeKiwi plan.

Used in P05, P06, P07

YOLO

You Only Look Once

A fast object detector. We train a small one on our own court footage and compare it to HSV.

Used in P07

// P08

P08 · Robot menagerie

New robots, shared code, interoperability.

Open project →

Bittle X

Petoi’s small quadruped. Cheap honest legs for characterization and later sim-to-real (P9).

Used in P08, P09

Cross-embodiment

One architecture or dataset format reused across arm, mobile base, rover, quadruped. Document where reuse actually works.

Used in P10, P08

Embodiment

The physical body the policy controls: arm, rover, quadruped, and so on. Code that assumes one body often breaks on another.

Used in P08, P10

Eval harness

common/eval_harness/

Our shared logger across projects. Every run gets a row: task, metrics, how it failed, and whether it was sim, real, or paired.

Used in P01, P03, P04, P08

Imitation learning

IL / learning from demos

The robot learns a task by watching demonstrations, not by exploring with a hand-written reward.

Used in P04, P08, P10

Interoperability

interop

Whether the same tools (harness, deploy, sysid) work across robots you did not design. If not, write down why.

Used in P08

LeRobot

Hugging Face’s open stack for teleop, datasets, ACT, Diffusion, and SmolVLA on hardware like the SO-101.

Used in P04, P08, P10

SDK

Software Development Kit

The vendor’s libraries and docs for talking to a specific robot. Unfamiliar SDKs are the point of P8.

Used in P08

SO-ARM101

SO-101

The open leader/follower arm we own. Used for sysid, imitation, desk-side edge work, and VLA. Not the rover arm.

Used in P03, P04, P05, P08, P10

System identification

sysid

Measure the real robot, then tune the simulator (friction, damping, and so on) until held-out trajectories match.

Used in P03, P08

// P09

P09 · Legged sim-to-real

Locomotion RL and domain randomization.

Open project →

Ablation

Turn one ingredient off (for example domain rand) and compare. That is how you know what mattered.

Used in P09, P04

Bittle X

Petoi’s small quadruped. Cheap honest legs for characterization and later sim-to-real (P9).

Used in P08, P09

Domain randomization

domain rand

Randomize sim parameters (mass, friction, delays) during training so the policy survives messier reality. Measure whether it actually helps.

Used in P09

Gap report

GR01–GR05

A short writeup of where sim and reality disagree, with numbers. The signature artifact of this lab.

Used in P03, P04, P06, P07, P09

Held-out set

Test cases you freeze before training. You do not tune on them. That is how you know the result is real.

Used in P01, P02, P04, P09

Isaac Lab

RL and robotics framework on top of Isaac Sim. Built for thousands of parallel environments.

Used in P02, P09

Isaac Sim

NVIDIA’s GPU robotics simulator. Heavy, needs a serious GPU. The world Isaac Lab trains inside.

Used in P02, P09

Parallel environments

vectorized envs

Thousands of copies of the same task running at once on the GPU. That is how Isaac Lab gets high sample throughput.

Used in P02, P09

Policy

The neural network (or controller) that maps what the robot senses into the next action.

Used in P01, P02, P04, P05, P09, P10

PPO

Proximal Policy Optimization

A popular reinforcement-learning algorithm. Stable, widely used, and what we run for the small fish policy and later locomotion.

Used in P01, P02, P09

PPO policy

The neural network trained with PPO. Same idea as “policy,” just naming the training method.

Used in P01, P09

Reinforcement learning

RL

The agent learns by trial and error in an environment, getting rewarded for getting closer to a goal.

Used in P01, P02, P09

Reward term

One piece of the reward function (for example lift height or action penalty). Change one at a time and keep before/after numbers.

Used in P01, P02, P09

RSL-RL

The RL library Isaac Lab often uses under the hood (PPO and friends), originally from Robotic Systems Lab.

Used in P02, P09

Seed

The random-number starting point for a training run. Multiple seeds show whether a result is stable or lucky.

Used in P09, P02

Sim-to-real

sim2real

Train in simulation, then run the same skill on a physical robot. The gap between those two is what we measure.

Used in P03, P04, P06, P09

Velocity tracking

Locomotion objective: make the robot follow commanded forward/turning speeds without falling.

Used in P09

// P10

P10 · Whole-body / VLA

Frontier policies on hardware you own.

Open project →

ACT

Action Chunking Transformer

An imitation policy that predicts a short sequence of actions at once. LeRobot’s usual first policy for the SO-101.

Used in P04, P10

Activation

hidden state / feature

The numbers flowing through a layer while the model runs. Interpretability work often starts by asking what a particular activation pattern means.

Used in P10

Alignment

AI safety (narrow sense)

Making sure the system’s behavior stays aimed at what we actually want, especially as models get more capable. Auditing tools matter because full decoding may never happen.

Used in P10

Auditing

model audit

Systematic checks on a trained system: probes, scratchpad review, held-out tasks. The practical goal when you cannot fully decode every weight.

Used in P10

Black box

A model you can query but cannot easily explain. Modern nets are grown from data, so the insides are huge arrays of numbers, not hand-designed rules.

Used in P10

Chain of thought

CoT / scratchpad

When a model writes intermediate reasoning before the final answer. Safety researchers watch it for cheating, test-awareness, or misaligned plans. Mostly an LLM idea; VLAs may expose less of this.

Used in P10

Circuit

In mechanistic interpretability: a small subgraph of neurons and weights that implements one recognizable computation inside the larger net.

Used in P10

Cross-embodiment

One architecture or dataset format reused across arm, mobile base, rover, quadruped. Document where reuse actually works.

Used in P10, P08

Data-scaling curve

Plot performance at 10 / 25 / 50 demos (or similar). Shows whether more data still helps.

Used in P10, P04

Demonstration

demo

One recorded human (or teleop) example of the task. P4 aims for about 50 real pick-and-place demos.

Used in P04, P10

Direction (in activation space)

A vector inside the model’s hidden state that tends to line up with one concept. Steering or reading that direction is a common interpretability move.

Used in P10

Embodiment

The physical body the policy controls: arm, rover, quadruped, and so on. Code that assumes one body often breaks on another.

Used in P08, P10

Fine-tune

Take a pretrained model and train it a bit more on your data so it adapts to your robot and task.

Used in P04, P10

GR00T

Isaac GR00T / N1.7

NVIDIA’s humanoid / foundation-model stack. We may fine-tune on LeRobot-format data; humanoid hardware stays in sim.

Used in P10

Imitation learning

IL / learning from demos

The robot learns a task by watching demonstrations, not by exploring with a hand-written reward.

Used in P04, P08, P10

Interpretability

Research that tries to understand what is going on inside a trained model, instead of treating it only as input → output.

Used in P10

LeRobot

Hugging Face’s open stack for teleop, datasets, ACT, Diffusion, and SmolVLA on hardware like the SO-101.

Used in P04, P08, P10

Linear probing

Train a simple linear classifier on frozen internal activations to test whether a concept (for example “gripper open”) is already encoded there.

Used in P10

Mechanistic interpretability

A stricter flavor of interpretability: reverse-engineer specific circuits or computations inside the network, not just correlate inputs with outputs.

Used in P10

Policy

The neural network (or controller) that maps what the robot senses into the next action.

Used in P01, P02, P04, P05, P09, P10

Retargeting

Map a human motion onto a robot whose body proportions differ. Optional study path in P10.

Used in P10

SmolVLA

A relatively small (~450M) VLA with a short public path onto the SO-101 via LeRobot.

Used in P10

SO-ARM101

SO-101

The open leader/follower arm we own. Used for sysid, imitation, desk-side edge work, and VLA. Not the rover arm.

Used in P03, P04, P05, P08, P10

Sparse autoencoder

SAE

A tool used in interpretability work. It expands a model’s internal activations into many mostly-off features so individual concepts are easier to spot.

Used in P10

Teleoperation

teleop (not teleportation)

You move the robot in real time, like puppeteering. In LeRobot that usually means a leader arm driving a follower arm.

Used in P04, P10

Test-awareness

eval awareness

When a model seems to notice it is being evaluated and changes behavior (for example plays nice under tests). Chain-of-thought audits try to catch this.

Used in P10

VLA

Vision-Language-Action model

A policy that takes images (and often language) and outputs robot actions. SmolVLA and GR00T are the routes we consider.

Used in P10

Whole-body control

WBC

Controlling arms, torso, and legs together as one coordinated system, not each half on its own.

Used in P10

// A–Z

Full index

Every term once, alphabetically.