2026.04.12 · 9 min · llm, rocm, gpu, homelab
Qwen3.5-35B-A3B crashed on every GPU load. Qwen-VL-30B-A3B ran fine at 75 t/s on the same card. The difference was a buggy pre-compiled kernel in rocBLAS: building it from scratch fixed it.
I had been running Qwen-VL-30B-A3B on the MI50 for several weeks without incident, pulling clean inference runs at 75 tokens per second. But when Qwen 3.5 dropped, I immediately pulled `Qwen3.5-35B-A3B` to benchmark the new hybrid architecture. The moment I attempted to offload layers to the GPU, `llama-server` crashed with an instant segmentation fault. Dropping `-ngl 0` to force CPU-only execution ran cleanly, confirming the GGUF quantization weights were intact and the fault lived strictly within the GPU offload path.
AMD Instinct MI50: 32GB HBM2, gfx906
The MI50 is an enterprise workhorse: 32GB of ultra-wide HBM2 across a 4,096-bit bus, GCN 5.1 (Vega 20 microarchitecture), and target ISA `gfx906`. AMD officially dropped ROCm support for Vega 20 after ROCm 5.7, so running modern inference stacks on it is always an exercise in systems archaeology. But 32GB of raw VRAM with 1,024 GB/s theoretical memory bandwidth in a homelab server is simply too good to leave on the table.
The crash was deterministic: an immediate segmentation fault (`SIGSEGV`, exit code 139) deep inside `libamdhip64.so` during device slot initialization, before the engine ever processed a single prompt token.
The GDB stack trace pointed into the HIP runtime dispatch rather than high-level llama.cpp parsing logic. Setting `-ngl > 0` triggered it every time; `-ngl 0` ran clean. The issue was triggered exclusively during GPU-accelerated tensor dispatch for this specific model family.
$ ./build/bin/llama-server -m qwen3.5-35b-a3b.gguf -ngl 99
...
ggml_hip_init: found 1 ROCm devices:
Device 0: AMD Instinct MI50, compute capability 9.0, VMM: no
...
Segmentation fault (core dumped)
Qwen3.5-35B-A3B uses a Hybrid MoE-SSM (State Space Model) architecture with selective state scan operations, contrasting sharply with the standard dense attention and vision projection pipelines of the VL model. Tracing execution through GDB and `rocprof` isolated the crash to a specialized rocBLAS triangular matrix solver and GEMM dispatch kernel (`solve_tri` / Tensile GEMM routines) heavily exercised by the SSM recurrent projection passes.
The VL model never hit that specific code path in the same configuration, which explained why one ran at 75 tok/s while the other died at initialization.
The root cause was inside the pre-compiled Debian/Ubuntu `rocBLAS` 7.x distribution package. AMD's build pipeline compiles rocBLAS GEMM routines via Tensile, an automated code-generator that emits thousands of unrolled assembly kernels across matrix shapes and target ISAs. In the generic vendor binary package, the pre-compiled Tensile kernels for `gfx906` contained a fatal code-generation bug: illegal vector register (VGPR) addressing and register file allocation boundaries on Vega 20's GCN 5.1 microarchitecture. When `llama.cpp` dispatched the SSM GEMM kernel, the execution wavefront attempted an illegal memory/register access outside its physical allocation, the GPU MMU tripped a hardware page fault, and `libamdhip64.so` aborted the process.
The upstream apt distribution of rocBLAS 7.x contains a code-generation bug in pre-compiled Tensile GEMM and `solve_tri` kernels targeting `gfx906`. The compiled assembly emits illegal VGPR addressing on Vega 20 silicon, triggering an immediate hardware page fault and segfault inside `libamdhip64.so`. CPU inference is unaffected because it bypasses the HIP/rocBLAS runtime entirely.
Because the pre-packaged binary shipped with broken Tensile assembly, the only clean solution was compiling `rocBLAS` directly from source with `-DAMDGPU_TARGETS=gfx906`.
By invoking `./install.sh -id -a gfx906`, I forced Tensile to run locally against the exact target architecture. Instead of bundling generic or miscompiled assembly objects, Tensile synthesizes tens of thousands of unrolled kernels strictly aligned to Vega 20 hardware parameters: 64-wide wavefronts (`wave64`), 256 vector general-purpose registers (VGPRs) per SIMD unit, and exact L1/L2 cache line granularities. Restricting compilation to `gfx906` only takes about 5 to 10 minutes on a multi-threaded host.
Grab the source matching your installed ROCm release:
# Example for ROCm 7.2.0
wget https://github.com/ROCm/rocm-libraries/releases/download/rocm-7.2.0/rocblas.tar.gz
tar -xzf rocblas.tar.gz
cd rocblas
Build it targeting `gfx906` explicitly:
# -d: installs build dependencies
-i: installs the finished library to /opt/rocm/
-a: forces Tensile to generate kernels specifically for gfx906
./install.sh -id -a gfx906
The `-a gfx906` flag is the linchpin. Generating the assembly locally against Vega 20's register file constraints sidesteps the generic packaging bug entirely.
Next, wipe the previous `llama.cpp` build cache so the binary dynamically links against the freshly installed rocBLAS library:
cd llama.cpp
rm -rf build
cmake -B build -DGGML_HIP=ON -DAMDGPU_TARGETS='gfx906'
cmake --build build --config Release -j $(nproc)
Confirm full GPU offload with `llama-bench`:
./build/bin/llama-bench -m qwen3.5-35b-a3b.gguf -ngl 99
The model loaded without hesitation, allocated all KV layers into HBM2, and ran completely fault-free. If you ever hit `Segmentation fault (core dumped)` in `libamdhip64.so` or `librocblas.so` while offloading novel architectures on retired Instinct or legacy Radeon silicon, don't blame the model weights: inspect the pre-compiled Tensile assembly and build rocBLAS against your raw silicon.