The Local Vibe Coders Dream: DeepSeek V4 Flash on DGX Spark, Mac Studio, or Strix Halo

My wife’s Mac mini runs Hermes. My AI workstation is about to run it too. That gives us two Hermes agents and one dilemma: Can we find a model that serves both (concurrently) for both coding and agentic tasks? Models optimized for tool calls tend to suck at coding. Models optimized for coding tend to be large, and exceed consumer hardware. But as token costs climb, there are solutions on the horizon.

So the search became about the model, not the machine. DeepSeek shipped V4 Flash, a 284 billion parameter mixture-of-experts model with only 13 billion parameters active per token, a 1 million token context window, and an official 0731 build that beats its own preview and DeepSeek’s larger Pro preview on every published agentic benchmark. Paired with Hermes, that’s a model worth building a dedicated third machine around.

This research pits three candidate boxes against each other: NVIDIA’s GB10 (aka DGX Spark), a homelab darling in the Mac Studio, and a dark horse in AMD’s Ryzen AI Max+ 395, better known by its codename, Strix Halo. The question became which machine, and at what quantization, actually runs DeepSeek V4 Flash well.

DeepSeek V4 Flash 0731 earns the dedicated box

The 0731 release kept the same architecture as the preview and only changed the training. DeepSeek re-post-trained it for agentic work, added native Responses API support, and shipped explicit Codex adaptation. The result scores 82.7 on Terminal Bench 2.1 and 54.4 on DeepSWE, up sharply from the preview’s 61.8 and 7.3 on the same benchmarks, and it ships MIT licensed with weights available same day as the announcement.

The active-parameter count is the part that matters for hardware sizing. Decode speed tracks the 13 billion active parameters, not the full 284 billion, which is the same trick that makes MoE models like gpt-oss-120b feel faster than their total size suggests on bandwidth-limited boxes. Total weight size still has to sit in memory somewhere, though, and that’s where the three candidate machines start to diverge.

128GB Spark trades single-stream speed for batching headroom

The DGX Spark packs 128GB of unified memory at 273 GB/s around NVIDIA’s GB10 Grace Blackwell chip, running the real CUDA and vLLM stack, Linux only. Single-stream decode on a dense 70B model is genuinely slow on this hardware, and early reviewers wrote it off for exactly that reason. But four to six concurrent Hermes requests is the actual job here, not one long solo chat, and that’s where the Spark’s batching shows up. One owner logged a Qwen3-Next-80B server climbing from 43 tokens per second at one concurrent request to 136 tokens per second at sixteen. A separate concurrency benchmark measured 695 aggregate tokens per second across 256 concurrent streams on a model most single-chat reviews had already written off as too slow.

The tradeoffs run the other way too. There’s no ECC on the unified memory, so a silent bit flip is a real risk for an always-on appliance. Time to first token on very large prompts can stretch past a minute on the biggest models. And the CUDA stack, while genuinely native, is young enough that owners describe getting from a working prototype to a reliable production deployment as harder than it should be.

128GB Mac Studio wins bandwidth, loses the batching engine

A Mac Studio flips every one of those tradeoffs. The M4 Max reaches up to 546 GB/s, roughly double the Spark’s bandwidth, and bandwidth is the number that decides how fast tokens stream once generation starts. What it doesn’t have is a mature continuous-batching engine. MLX and llama.cpp serve requests close to sequentially, so six concurrent Hermes agents each get roughly a sixth of the single-stream speed, not the Spark’s better-than-linear scaling. For a quiet single-user chat box that tradeoff barely registers. For a shared brain fielding two Hermes agents at once, it’s the whole question. macOS only, of course.

A Mac mini never entered contention here. Its ceiling on the M4 Pro tops out well short of 128GB, too small to hold a model this size at any usable quantization.

128GB Strix Halo trades both extremes for efficiency and choice of OS

AMD’s answer borrows Apple’s own playbook. The Ryzen AI Max+ 395 pairs 16 Zen 5 cores with a 40-compute-unit RDNA 3.5 iGPU and up to 128GB of shared LPDDR5X memory, rated at roughly 256 GB/s theoretical and closer to 215 GB/s in real-world testing. The real story is stranger than the spec sheet: on MoE models, decode speed nearly matches NVIDIA’s dedicated box, but prefill does not. On the GPT-OSS 120B benchmark, one review clocked Strix Halo processing prompts at roughly 340 tokens per second against the Spark’s roughly 1,700, a five times gap, and an independent lab found the same pattern, slower time to first token that widens as prompts grow.

The software path is its own animal too. Most owners run llama.cpp on Vulkan for everyday use and switch to ROCm with Flash Attention once context gets long, since Vulkan degrades past roughly 4,000 tokens while ROCm holds flat well past 8,000. A community-maintained vLLM path exists through nightly ROCm builds, but it’s a toolbox project, not a first-party stack the way CUDA and vLLM are on the Spark. Where Strix Halo pulls ahead outright is flexibility and efficiency: it runs Windows or Linux, usable 70B-class inference draws somewhere in the 65 to 90 watt range, and it’s the quietest of the three by a wide margin. The Windows path has a real cost, though. ROCm has no official Windows build, so Windows users fall back to LM Studio’s Vulkan backend, running 20 to 30 percent behind the same chip on Linux.

None of the three 128GB boxes hold V4 Flash lossless

This is where the model’s own memory math rules out all three 128GB machines outright. The lossless 8-bit build of V4 Flash runs 162GB, with at least 169GB recommended once context and KV cache are counted. The practical shrink, a 3-bit quant, drops to 103GB and still wants a minimum of 110GB. Any of the three 128GB boxes can just barely load that 3-bit quant, with almost nothing left over for the model’s actual selling point, a 1 million token context window. Running V4 Flash on a 128GB box means giving up on lossless output and long context in the same breath.

Five scenarios reduce to one table

Here’s the full lineup side by side, the three boxes that can run the compressed model and the two paths that reach the memory tier the lossless build actually needs.

Strix Halo doesn’t have a seat at the lossless table the way the other two do. AMD’s own playbooks describe clustering two Ryzen AI Halo boxes over llama.cpp RPC for very large models, but it’s a community recipe, not an official pooled-memory path like the Spark’s NCCL cluster or a single Mac Studio’s unified memory. For this comparison, Strix Halo stays a 128GB, lossy-only contender.

256GB changes the quantization question, not the tradeoff

Both lossless paths land at 256GB, but they get there differently.

Apple’s newly announced M5 Ultra Mac Studio opens a 256GB configuration for the first time at this chip generation, backed by 1.2TB/s of memory bandwidth, up from 819 GB/s on the M3 Ultra. That’s one chip and one unified memory pool. The lossless 162GB build of V4 Flash fits with room to spare for context and KV cache, no partitioning required, and decode speed benefits from the full bandwidth jump on top of it.

The Spark reaches the same 256GB by clustering two units over a single cable, connected through MPI and NCCL rather than one shared memory pool. Combined capacity is generous. Forum math on this exact pairing puts a dual Spark cluster’s inference ceiling at roughly 470 billion parameters at 4-bit, nearly three times what V4 Flash’s lossless weights need. But the two 128GB nodes are joined by an interconnect running at roughly 25 GB/s, a small fraction of the 600 GB/s a single Spark uses internally between its own CPU and GPU. That gap matters far more for training, where gradients need constant synchronization, than for inference, where a pipeline split only has to pass a small activation tensor across the cable once per stage. It still adds real complexity: NCCL configuration, a cluster to keep healthy, and distributed serving support for V4 Flash’s specific hybrid attention design that is still young.

Compute wins the first token, bandwidth wins the sprint, batching wins the real job

This is where the paths split for good, and it maps onto what actually matters for two Hermes agents sharing one brain.

Time to first token favors the Spark, alone or clustered. Prefill is compute bound, and NVIDIA’s Blackwell tensor cores beat both Apple Silicon’s GPU compute and Strix Halo’s iGPU on that phase by a wide margin, even though the other two win on raw bandwidth or efficiency.

Tokens per second on a single request favors the Mac Studio. Its unified bandwidth with zero interconnect hop beats a pipeline-split pair of Sparks reading across a much narrower cable, and comfortably beats Strix Halo’s real-world bandwidth too.

Aggregate throughput at four to six concurrent agents, the actual daily job, favors the Spark again, for the same reason it did at 128GB. The batching advantage comes from the software stack, not the memory tier, and it’s the one place Strix Halo’s community-toolbox vLLM path can’t yet compete with a first-party CUDA stack.

Given the actual workload, low time to first token and a pair of agents making concurrent requests rather than one long solo chat, the dual Spark cluster is the better technical fit for running DeepSeek V4 Flash lossless. The honest caveat is that distributed serving of a month old model, built on an attention architecture nobody has run across two nodes for very long, is not the boring choice. The Mac Studio is. Same capacity, one box, no cluster to babysit. And if lossless isn’t the requirement, Strix Halo earns its spot in the lineup on efficiency and OS flexibility alone, running the same compressed quant as the other two 128GB boxes at a fraction of the power draw.

Given the choice between the better fit and the more boring one, which would you actually want answering pages at 2am?

By the Numbers

• 284B / 13B — total and active parameters in DeepSeek-V4-Flash
• 1M tokens — V4 Flash’s context window, the feature every quantization choice here either preserves or throws away
• 162GB — lossless 8-bit weight size, against 103GB at the practical 3-bit shrink
• 128GB — memory ceiling shared by all three lossy-tier boxes: Spark, Mac Studio M5 Max, and Strix Halo
• 256GB — memory reached either through an M5 Ultra Mac Studio or a two-node Spark cluster
• 1.2TB/s — M5 Ultra memory bandwidth, up from 819 GB/s on the M3 Ultra
• ~600 GB/s — bandwidth inside a single Spark between its own CPU and GPU
• ~25 GB/s — bandwidth across the cable connecting two clustered Sparks
• ~215 GB/s — Strix Halo’s real-world memory bandwidth, against a 256 GB/s theoretical ceiling
• 5x — the prefill gap between Strix Halo and a single Spark on GPT-OSS 120B
• 136 tok/s — aggregate throughput at sixteen concurrent requests on a single Spark, up from 43 at one
• 82.7 / 54.4 — V4 Flash 0731’s scores on Terminal Bench 2.1 and DeepSWE, both up sharply from the preview build