MSN-009 · 2026 · edge-ai, robotics, lora, jetson
First known deployment of π₀.₅ on a 6-DOF Piper arm running entirely on a Jetson AGX Orin. 29.5x latency reduction and 65% task success rate from 50 kinesthetic demonstrations, using VLASH async inference with no datacenter GPU at runtime.
π₀.₅ is a flow-matching VLA model from Physical Intelligence, built on a PaliGemma backbone. Getting it running on a real arm at usable speed, on embedded hardware, with no datacenter GPU at inference time, required two things: porting VLASH (MIT HAN Lab’s asynchronous inference framework) to a JetPack 6 aarch64 stack, and building an open-source training pipeline from scratch so the process could be reproduced. That second artifact became VLASH Forge.
VLA models are slow. π₀.₅ takes around 5,444 ms per inference chunk on a Jetson AGX Orin. A 6-DOF arm running at that cadence effectively stops between every prediction — the synchronous baseline achieved 5% task success on a pick-and-place task, mostly because the arm was standing still while the model thought.
VLASH’s Temporal Delay Augmentation fixes this by training the model on stale observations. During training, observations are randomly delayed by d ~ U[0, 8] steps, so the model learns to predict actions given an observation that will be slightly out of date by the time the action executes. At inference time, the arm keeps moving through the previous chunk while the model is computing the next one. Chunk boundaries disappear from the motion trajectory.
Async (left) vs synchronous (right) inference at 00:02 elapsed — the async arm has already made contact while the synchronous arm is still computing its first prediction
Async inference: 184.5 ms mean latency, 65% task success, 52 s mean task time. Synchronous baseline: 5,444 ms latency, 5% success, 153 s. 29.5x latency reduction, 2.9x faster task completion.
100 pick-and-place episodes collected via kinesthetic demonstration — physically guiding the Piper arm through each attempt by hand, letting the arm record its own joint states and camera observations at 30 Hz throughout. Each episode ran about 30 seconds. After collection the dataset was pushed to HuggingFace Hub.
Fine-tuning used LoRA rank-16 on the full π₀.₅ model, with `max_delay_steps=8` and `shared_observation=true` to match the async inference training distribution. Training ran on NSCC ASPIRE 2A across 64 A100s using DeepSpeed ZeRO-2. The dataset, checkpoint, and training config are published on HuggingFace.
Overhead camera view during a demonstration episode — arm approaching the pick target on the grey pad
Getting VLASH (MIT HAN Lab) and lerobot_piper (WeGo Robotics) to coexist on a JetPack 6 aarch64 system was the bulk of the engineering work. Five distinct compatibility failures surfaced:
- Dependency version conflicts between the two packages - Module layout divergence causing import failures - rerun API renames between the versions each package expected - draccus config registry misses on cold start - A JetPack-specific GLIBCXX runtime conflict unique to aarch64 that didn’t surface on x86
None required modifying either upstream package’s core logic. The fixes went into shims and the environment definition. The full stack is reproducible in one command via a Pixi lockfile that pins JetPack-specific PyTorch wheels from `pypi.jetson-ai-lab.io/jp6/cu126` alongside both packages as editable installs.
The fine-tuning infrastructure for this deployment didn’t exist publicly. The original VLASH paper released the idea; the container, distributed training config, and HPC scripts were not. VLASH Forge is that missing piece, released as an open-source project.
It packages the full training stack into a single Docker and Singularity image: VLASH and LeRobot as editable installs, DeepSpeed ZeRO-2 for LoRA training (fits π₀.₅ in 12 GB VRAM), and PyTorch FSDP for full fine-tuning across multiple GPUs. The same image runs on a cloud VM, a SLURM or PBS HPC cluster, or a local workstation with one command:
./scripts/train.sh examples/train/pi05/cloud.yaml
The launcher detects Docker vs Singularity automatically and binds persistent scratch storage so weights and checkpoints survive container restarts. A Kubernetes Job manifest is included for cloud-native deployments.
Docs at frieddeli.github.io/VLASH-Forge. Container image at `frieddeli/vlash-forge` on Docker Hub.
Dual Intel RealSense D435i cameras — one wrist-mounted on the Piper end-effector, one overhead on a mini tripod positioned above the workspace. Both streams feed into the π₀.₅ observation encoder simultaneously. Inference runs in bfloat16 on the Jetson AGX Orin (64 GB LPDDR5, CUDA 12.6, JetPack 6.2.2). No network calls at inference time; the model weights live entirely on-device.
Top-down overview of the full rig — overhead camera on tripod, pick source (grey pad, centre) and target (black pad, right), Jetson module at left
Side view — Piper arm with wrist-mounted RealSense, overhead camera on tripod, pick targets laid out
Full test rig from bench level — overhead RealSense on tripod, wrist-mounted camera on the Piper arm, pick targets on the bench
At inference time each camera frame is encoded, a chunk of 50 action steps is sampled via flow-matching, and the arm begins executing immediately. The next chunk starts computing on the Jetson GPU while the current chunk plays out. With `max_delay_steps=8` during training, the policy was trained to expect observation lag — so the stale observation at each chunk boundary is the training distribution, not a distribution shift.
Wrist camera mid-pick — gripper descending toward the ball on the source pad
Wrist camera — ball placed on the black target pad, gripper disengaging
Wrist camera confirmation frame — ball seated on black pad, task complete
- π₀.₅ (PaliGemma backbone, flow-matching action head) - LoRA rank-16 via HuggingFace PEFT - VLASH Temporal Delay Augmentation (`max_delay_steps=8`) - DeepSpeed ZeRO-2 (LoRA training) / PyTorch FSDP (full fine-tuning) - Agile X Robotics Piper 6-DOF arm - Dual Intel RealSense D435i - NVIDIA Jetson AGX Orin 64 GB (CUDA 12.6, JetPack 6.2.2) - NSCC ASPIRE 2A (64× A100) for training - Pixi for reproducible aarch64 environment management - Docker + Singularity for portable training containers