AMD’s Ryzen AI MAX+ 395 (Strix Halo) with the integrated Radeon 8060S (gfx1150 / RDNA 3.5) and 128 GB of unified LPDDR5x memory is a sweet spot for local LLM inference. Because the CPU and GPU share the same physical memory pool, you can expose 100 GiB directly to the GPU via the Graphics Translation Table (GTT)—enough to run gpt-oss-120b (MXFP4) or Qwen3.6-35B-Q8 with a 262k context window completely offloaded to the GPU.

Getting RDNA 3.5 (gfx1150), ROCm 7.1, llama.cpp, and multi-model routing working reliably on Ubuntu involves a few sharp edges. To make the machine (max) 100% reproducible, the entire setup lives in a 4-stage Ansible playbook suite (workerNode/site.yaml).

workerNode/
├── site.yaml                 # Master playbook (runs all 4 stages in order)
├── llama_ROCm.yaml           # Stage 1: Kernel GTT/TTM, ROCm 7.1, llama.cpp build & systemd router
├── llama_router.yaml         # Stage 2: Downloads the 4 active GGUF models using hf CLI
├── llama_https.yaml          # Stage 3: Generates SAN TLS certs & configures HTTPS (:8443)
└── opencode_bootstrap.yaml   # Stage 4: Installs OpenCode CLI, opencode.json, agents & 30 skills

Stage 1: Unlocking a 100 GiB GPU GTT Aperture

Out of the box, the Linux amdgpu driver only exposes a portion of system RAM to the GPU. If you try to load a 65+ GB model with --n-gpu-layers 999, allocation fails or falls back to CPU RAM.

To allocate a 100 GiB GPU aperture, llama_ROCm.yaml configures both amdgpu.gttsize (in MiB) and the Linux kernel Translation Table Maps (ttm) page limits (in 4 KiB pages):

# Unified Memory Architecture (UMA) & TTM Tuning (128GB LPDDR5x on Ryzen AI MAX+ 395)
amdgpu_gtt_size_mb: 102400      # 100 GB GTT allocation for RDNA 3.5 APU
ttm_pages_limit: 26214400       # 100 GB in 4KB pages (critical for >60GB models)
ttm_page_pool_size: 26214400
hsa_override_gfx: "11.5.0"      # Strix Halo RDNA 3.5 (Radeon 8060S / gfx1150) ISA target

After updating /etc/default/grub and rebooting, you can verify the active GTT pool and live GPU residency via sysfs:

awk '{printf "GTT total: %.1f GiB\n", $1/1073741824}' /sys/class/drm/card*/device/mem_info_gtt_total
awk '{printf "GTT used:  %.1f GiB\n", $1/1073741824}' /sys/class/drm/card*/device/mem_info_gtt_used

(Note: the DRM device index can flip between card0 and card1 across reboots, so always glob card* rather than hardcoding card0.)

Working Around Two ROCm 7.1 Gotchas on gfx1150

  1. libhsa-runtime64 Segfault & libxml2.so.2 Shim: On Ubuntu 26.04 (kernel 7.0.x), the upstream ROCm /opt/rocm/lib/libhsa-runtime64.so can segfault during HIP initialization on gfx1150, whereas the distro-packaged libhsa-runtime64.so.1 runs cleanly. Additionally, ROCm’s LLVM toolchain expects libxml2.so.2 while newer Ubuntu releases ship libxml2.so.16. The playbook creates /usr/local/lib/rocm-compat with a libxml2.so.2 symlink shim and orders LD_LIBRARY_PATH so the distro HSA runtime loads first.
  2. Isolated llama.cpp Build with patchelf: llama_ROCm.yaml compiles llama.cpp specifically for AMDGPU_TARGETS=gfx1150 with -DGGML_HIP=ON -DGGML_HIPBLAS=ON -DLLAMA_OPENSSL=ON -DBUILD_SHARED_LIBS=ON, installs the shared libraries into /usr/local/lib/llamacpp/bin, and sets the binary RUNPATH via patchelf so /usr/local/bin/llama-server never collides with system libraries.

Stage 2: On-Demand Multi-Model Routing & Speculative Decoding

Instead of running multiple llama-server processes that compete for memory, a single llama-server.service runs in router mode:

/usr/local/bin/llama-server \
  --models-dir /opt/llama_models/dir \
  --models-max 1 \
  --ctx-size 262144 \
  --n-gpu-layers 999 \
  --cont-batching \
  --metrics \
  --api-key-file /etc/llama-server/api_key

Each subdirectory in /opt/llama_models/dir/<ModelID>/ registers as a selectable model ID on /v1/chat/completions. With --models-max 1, llama-server loads the requested model into the 100 GiB GTT window on demand and evicts the previous one automatically.

llama_router.yaml provisions four models using the Hugging Face CLI (hf download):

  • Qwen3.6-35B-Q8 (Qwen3.6-35B-A3B-Q8_0.gguf + mtp-Qwen3.6-35B-A3B-Q8_0.gguf): Placing the Multi-Token Prediction (mtp-*) draft file in the same directory enables automatic speculative decoding (~50% faster token generation).
  • gpt-oss-120b (gpt-oss-120b-MXFP4.gguf): 120B parameters for heavy architectural reasoning and code review.
  • Devstral-Small-2507 (Devstral-Small-2507-Q5_K_M.gguf): Fast agentic tool-calling (unsloth build with working Jinja tool templates).
  • GLM-4.7-Flash (GLM-4.7-Flash-Q5_K_M.gguf): Low-latency completions and sub-agent queries.

Using --api-key-file /etc/llama-server/api_key (root:goobey, mode 0640) instead of --api-key ensures the secret never appears in ps aux output.

Stages 3 & 4: HTTPS & OpenCode Bootstrap

llama_https.yaml generates a local Root CA and SAN certificate (DNS:max, DNS:max.local, DNS:max.ocelot-company.ts.net, LAN IP, and Tailscale IP) so HTTPS clients can connect on :8443 while preserving local HTTP on :8080. Finally, opencode_bootstrap.yaml installs the OpenCode CLI, configures ~/.config/opencode/opencode.json against the local router, and deploys custom specialist agents (ansible-specialist) and 30 engineering skills.