VLASH-Piper: VLA Edge Deployment on Jetson AGX Orin

π₀.₅ is a flow-matching VLA model from Physical Intelligence, built on a PaliGemma backbone. Getting it running on a real arm at usable speed, on embedded hardware, with no datacenter GPU at inference time, required two things: porting VLASH (MIT HAN Lab’s asynchronous inference framework) to a JetPack 6 aarch64 stack, and building an open-source training pipeline from scratch so the process could be reproduced. That second artifact became VLASH Forge.

The latency problem

VLA models are slow. π₀.₅ takes around 5,444 ms per inference chunk on a Jetson AGX Orin. A 6-DOF arm running at that cadence effectively stops between every prediction; the synchronous baseline achieved 5% task success on a pick-and-place task, mostly because the arm was standing still while the model thought.

VLASH’s Temporal Delay Augmentation fixes this by training the model on stale observations. During training, observations are randomly delayed by d ~ U[0, 8] steps, so the model learns to predict actions given an observation that will be slightly out of date by the time the action executes. At inference time, the arm keeps moving through the previous chunk while the model is computing the next one. Chunk boundaries disappear from the motion trajectory.

Results

Async inference: 184.5 ms mean latency, 65% task success, 52 s mean task time. Synchronous baseline: 5,444 ms latency, 5% success, 153 s. 29.5x latency reduction, 2.9x faster task completion.

Data collection and fine-tuning

100 pick-and-place episodes collected via kinesthetic demonstration, physically guiding the Piper arm through each attempt by hand, letting the arm record its own joint states and camera observations at 30 Hz throughout. Each episode ran about 30 seconds. After collection the dataset was pushed to HuggingFace Hub.

Fine-tuning used LoRA rank-16 on the full π₀.₅ model, with max_delay_steps=8 and shared_observation=true to match the async inference training distribution. Training ran on NSCC ASPIRE 2A across 64 A100s using DeepSpeed ZeRO-2. The dataset, checkpoint, and training config are published on HuggingFace.


The integration work

Getting VLASH (MIT HAN Lab) and lerobot_piper (WeGo Robotics) to coexist on a JetPack 6 aarch64 system was the bulk of the engineering work. Five distinct compatibility failures surfaced:

• Dependency version conflicts between the two packages
• Module layout divergence causing import failures
• rerun API renames between the versions each package expected
• draccus config registry misses on cold start
• A JetPack-specific GLIBCXX runtime conflict unique to aarch64 that didn’t surface on x86

None required modifying either upstream package’s core logic. The fixes went into shims and the environment definition. The full stack is reproducible in one command via a Pixi lockfile that pins JetPack-specific PyTorch wheels from pypi.jetson-ai-lab.io/jp6/cu126 alongside both packages as editable installs.

VLASH Forge

The fine-tuning infrastructure for this deployment didn’t exist publicly. The original VLASH paper released the idea; the container, distributed training config, and HPC scripts were not. VLASH Forge is that missing piece, released as an open-source project.

It packages the full training stack into a single Docker and Singularity image: VLASH and LeRobot as editable installs, DeepSpeed ZeRO-2 for LoRA training (fits π₀.₅ in 12 GB VRAM), and PyTorch FSDP for full fine-tuning across multiple GPUs. The same image runs on a cloud VM, a SLURM or PBS HPC cluster, or a local workstation with one command:

./scripts/train.sh examples/train/pi05/cloud.yaml

The launcher detects Docker vs Singularity automatically and binds persistent scratch storage so weights and checkpoints survive container restarts. A Kubernetes Job manifest is included for cloud-native deployments.

Docs at frieddeli.github.io/VLASH-Forge. Container image at frieddeli/vlash-forge on Docker Hub.

Hardware setup

Dual Intel RealSense D435i cameras, one wrist-mounted on the Piper end-effector, one overhead on a mini tripod positioned above the workspace. Both streams feed into the π₀.₅ observation encoder simultaneously. Inference runs in bfloat16 on the Jetson AGX Orin (64 GB LPDDR5, CUDA 12.6, JetPack 6.2.2). No network calls at inference time; the model weights live entirely on-device.

Inference

At inference time each camera frame is encoded, a chunk of 50 action steps is sampled via flow-matching, and the arm begins executing immediately. The next chunk starts computing on the Jetson GPU while the current chunk plays out. With max_delay_steps=8 during training, the policy was trained to expect observation lag, so the stale observation at each chunk boundary is the training distribution, not a distribution shift.


Results



Stack

• π₀.₅ (PaliGemma backbone, flow-matching action head)
• LoRA rank-16 via HuggingFace PEFT
• VLASH Temporal Delay Augmentation (max_delay_steps=8)
• DeepSpeed ZeRO-2 (LoRA training) / PyTorch FSDP (full fine-tuning)
• Agile X Robotics Piper 6-DOF arm
• Dual Intel RealSense D435i
• NVIDIA Jetson AGX Orin 64 GB (CUDA 12.6, JetPack 6.2.2)
• NSCC ASPIRE 2A (64× A100) for training
• Pixi for reproducible aarch64 environment management
• Docker + Singularity for portable training containers