Edge deployment
Unplug the 5090 (unplug the workstation). The arm keeps doing the task. The exact P4 policy now runs untethered on a $249 Jetson at 15 W.
Weeks 8–10 · Yahboom M3 Pro Orin NX 16GB (shared with P06) · 05_edge_runtime/



Goal
Training on a 5090 and inferring on a 5090 is only half the job. Export PyTorch → ONNX → TensorRT. Measure accuracy drift at FP16 and INT8 against FP32 on identical inputs. Measure the loop properly: inference latency p50/p95/p99, camera-to-command latency, jitter. Not average FPS. Document the rate hierarchy: policy inference ≠ control loop ≠ servo update. Write the watchdog for stale frames, stale commands, NaN / out-of-range, and missed deadlines, with a defined response for each. Trigger every branch on purpose. Why not Jetson Thor ($3,499)? Being able to answer that in one sentence is judgment.
Why it matters
P5 inherits the exact policy from P4. Its whole job is to make that policy survive untethered on the Jetson. P5 and P6 run in parallel on the same M3 Pro. In the remote-5090 setup, this is where even the local Mac leaves the loop. The policy runs on the board bolted to the robot. A robot in the field has no 5090. It has a small ARM board, a hard deadline, and no one watching. Three problems only show up here: the policy shrinks under quantization, the loop misses its deadline and the robot lurches, and when a sensor drops out something has to decide the next move. common/deploy/ is written once and becomes the brain of P6–P9. ROS 2 here would add latency to the thing you are measuring.
The work
- 01
PyTorch → ONNX → TensorRT. Measure FP16 and INT8 drift against FP32 on identical inputs.
- 02
Latency: inference p50/p95/p99, end-to-end camera-to-command, jitter. Not average FPS.
- 03
Write the rate hierarchy: policy rate ≠ control rate ≠ servo rate.
- 04
Watchdog state machine. Trigger stale camera, stale command, and injected NaN. Three clips, three log rows.
- 05
Same 20 evaluation cases: workstation vs Jetson. Attribute any drop to quantization or latency with evidence.
Done looks like
- Demo: P4 pick-and-place on the real arm, laptop and workstation physically disconnected.
- Quantization table: FP32 / FP16 / INT8 × output error × task success over 20 trials each.
- Latency histogram with p50/p95/p99 marked, and the control rate you can actually guarantee.
- Safety demo: unplug the camera mid-task and get a defined stop. Repeat for stale commands and injected NaN.
- Same 20 cases, workstation vs Jetson, with an evidence-backed explanation of any drop.
Stack card
| Simulator | None |
|---|---|
| Trains on | RTX 5090 to export |
| Runs on | Yahboom M3 Pro Orin NX Super 16GB |
| Buy | No extra board. P5 and P6 share the M3 Pro. |
| ROS 2 | No |
| Gazebo | No |
| Compute / target | the M3 Pro's Orin NX Super 16GB (shared with P06) |
| Install | JetPack 6.x, TensorRT, onnx, onnxruntime, pycuda; torch.onnx export on the workstation |
Repos to study
Watch out
- Cutting torque is not universally safe: the arm drops the payload; a balancer falls over.
- Do not buy Jetson Thor for this. The $249 Super is the point.
Workstreams
Export path, JetPack bring-up, watchdog, disconnected demo.
Quantization table, latency histogram, workstation-vs-Jetson on 20 cases.
Cross-check. Eval stream can reproduce the p99 number from committed logs.
What this proves
Model export, ARM/embedded Linux, real-time scheduling, and failure-mode engineering. Measured, not claimed.
A policy that averages 20 Hz and drops to 4 Hz at p99 is a robot that lurches.