Seven local models on one RTX 5090 each wrote the same three posts, blind, so I could settle which one gets the writing lane of the homelab router. The house writer was not the fastest, and the judge did not agree with me on one of the three.
Tuning Hermes to run fast against local AI on a two-node DGX Spark cluster. One unrecognized reasoning_effort setting broke Hermes tool calls, and context compression cut the turn time about 4x. Covers the config fix, the context ceiling, and the latency floor tuning cannot remove.
Wiring the local model fleet into Coder Agents forced a realization. A policy router ignores the model name on the request and chooses a backend from the request itself, so Coder needs one generic default model instead of one entry per backend.
A two-node DGX Spark cluster carries the household model traffic, and the best tuning lever was one the deployment’s own A/B test had measured but production had reverted. Restoring it tightened the wait between concurrent requests, not raw speed.
Before I settle the local model policy for the homelab router, I ran GPT-5.5, Strix Qwen, RTX 5090 Qwen, and Spark DeepSeek through the same Coder Agents feature task. DeepSeek won the implementation quality round, while Strix taught me why policy needs speed data too.
After adding Strix Halo to the homelab, I wired Strix, Spark, and the RTX 5090 workstation into Coder Agents as explicit model choices, with the policy router still handling the default local path.
I wiped Windows from a Ryzen AI Max+ 395 box, made Ubuntu boot headless, joined it to the Tailnet, tuned Strix Halo unified memory, and got a fresh 512K-context Qwen coding model running for the homelab.
The local smart-home path moved from typed Hermes commands to real voice control, using a Pixel Tablet, Home Assistant Assist, Whisper, Piper, Android wake word detection, and Z-Wave lights on the ThinkCentre.
The local router prototype became useful when Hermes turned off a real Z-Wave light through Home Assistant, after USB passthrough, Z-Wave JS setup, entity mapping, and one stale MCP endpoint got fixed.
Turning a local AI policy-layer plan into a static router prototype, with FastAPI, Proxmox LXC, local DNS rewrites, Uptime Kuma monitoring, Spark DeepSeek, and 5090 Granite fallback routing.
A dual Spark DeepSeek endpoint solved the model problem, then exposed the next missing layer: local-first policy routing across household agents, specialist models, paid tools, PAIR, and future homelab nodes.
The dual Spark cluster moves from a working demo to durable household infrastructure: mDNS, reboot recovery, Hermes routing, remote dashboard access, benchmark results, and the first tuning lessons.
DeepSeek V4 Flash Vision-Exp on a dual DGX Spark cluster takes the broad local-agent benchmark lead, while Granite 4.2 no-think still wins the smart home slice that matters most day to day.
The dual DGX Spark cluster moved from a conservative DeepSeek V4 text endpoint to DeepSeek V4 Flash Vision-Exp after wrong turns through oversized models, generic runtimes, missing encoders, and RDMA failures.
A dual-node NVIDIA Spark cluster runs DeepSeek V4 Flash locally, extending the homelab from one RTX 5090 workstation toward a higher-fidelity local AI inference tier.
An architecture plan for folding a dual-node DGX Spark cluster and NVIDIA PAIR into the homelab, turning the RTX 5090 box and an M4 Mac mini into agentic workstations instead of the only inference engines in the house.
NVIDIA’s new Personal AI Router turns a pile of home GPUs into one shared endpoint for multi-agent inference. A look at what it actually does, why it fits a bottleneck already visible on this homelab, and what it rules out.
Round 2 of the local agent bakeoff saturated its coding domain completely, zero cracks across 18 fresh evals from two coding-focused models. That’s the moment to stop patching a broken test and split the question in two: what’s the best Home Assistant butler, and separately, what’s the best local coding model that fits a 32GB card. Here’s the research behind picking Terminal-Bench 2.1 over SWE-bench, and the plan for the actual run.
A random Tuesday sends an agent over SSH into the homelab to check every piece of software running there. Coder jumps two full versions, cloudflared and RustDesk get patched, and llama.cpp pulls in 317 commits, all with zero manual steps.
Three new contestants join the local agent bakeoff: IBM Granite 4.2, Poolside Laguna XS 2.1, and DeepReinforce Ornith 1.5. Granite 4.2 unseats Qwen 3.6 as the equal-weighted leader and dominates the one job that actually matters, Home Assistant tool-calling. Then a fourth run isolates a single variable, IBM’s default-on thinking mode, and finds it costs 10.8x the wall-clock time for no measurable accuracy gain on this battery. Plus: why the coding domain needs a harder replacement before round three.
Three consumer boxes claim they can run DeepSeek V4 Flash for a two-Hermes homelab: NVIDIA’s DGX Spark, Apple’s Mac Studio, and AMD’s Ryzen AI Max+ 395. Only two of them can reach the memory tier the model needs to run lossless.
A homelab experiment set out to find the optimal context window for a local Qwen 3.6 daily driver. It failed at that job, and revealed a more useful one: three real bugs, a clean speed-versus-depth curve, a prompt cache that hid 93% of the real token count, and a config change worth making in Hermes Agent’s compaction settings. Six tables. Zero cloud tokens spent.
The results already shipped. This is the part that took longer: forking an open-source Home Assistant benchmark, extending it into calendar, portfolio, and coding domains it never covered, and hitting two real bugs along the way — one hiding in code nobody had touched, one hiding in a floating-point boundary case we wrote ourselves.
Five local models, six evals, one real job: run Home Assistant, a calendar, an investment portfolio, and a to-do list — all self-hosted, no cloud. Qwen 3.6 holds the crown, but only barely, once three full runs replace one. Meta’s six-day-old Muse Glimmer nearly took it. Hermes 4.3 finishes last twice over. The scores, the failure transcripts, the run-to-run noise, and the one safety-relevant nuance the numbers alone don’t show.
An AI Transformation team at a Fortune 500 financial services firm told me they’ve shipped 19,000 skills across 175 unique Coder workspace types in six months. That’s not a headcount story. It’s proof that as AI intelligence becomes a commodity, the real competitive advantage shifts to how businesses codify and distribute their proprietary workflows.
A challenge to a coding agent: prove whether months of homegrown test harnesses and bakeoff scripts were reinventing wheels that already exist. A survey of what hobbyists, techtubers, and the AI benchmarking industry already publish, and an honest verdict on this blog’s own tooling habits.
I put Qwen 3.6 27B, Qwen 3.6 35B-A3B, Qwythos-9B, GLM-4.7-Flash, and Nemotron-3-Nano through the same real coding task on my homelab RTX 5090. Along the way I had to live-patch two separate llama.cpp bugs — and even after fixing them, I couldn’t fully prove one model’s failure wasn’t the harness’s fault.
Back in Q1 I read a headline about Google cutting AI memory use 6x and filed it under “watch and revisit.” Four months later, Google still hasn’t shipped official code, but a whole ecosystem of forks has, llama.cpp has an open PR, and at least one compatibility gotcha lands squarely on our daily driver. Here’s the honest state of TurboQuant heading into Q3, and the test I’d actually run against it.
A routine “update Coder” request turned into a full model bakeoff: fixing misconfigured thinking params so Sonnet 5 would stop calling itself 4.5, discovering Playwright MCP can’t be wired into Coder Agents at all, and watching three frontier models independently pause at the exact same step before finishing. Sonnet 5 won on score and on cost, by a mile.
A gloomy Cape Cod afternoon post-July 4th turns into a deep dive on ComfyUI, Lemonade Server, and LocalAI — plus llama-benchy and AMD’s AI Playbooks — and the case for a bakeoff against our hand-tuned llama.cpp stack.
GLM is suddenly everywhere in developer conversations. Before we run the bakeoff, we need to answer two questions: what is GLM, and is it suitable for a single RTX 5090 homelab?
A CEO panel at an AI event sparked a simple but powerful question every startup founder should ask themselves: does your business get better as AI models improve, or does it get worse?
I gave five local LLMs and one frontier cloud model the same coding task on my homelab: build a tag manager for the blog’s admin panel. Only two shipped anything. Here’s what happened.
Four frontier models, ten tasks, one government shutdown. We ran Claude Fable 5 through the homelab benchmark harness three hours before Anthropic pulled the plug — and it came in second. Here’s the full bakeoff.
Two Discord bots, one 14B model, five fitness-tracker tasks. Both agents failed on the first try. Getting them working required debugging context overflow, silent tool parameter drops, and a chat template flag that changes everything. The results reveal as much about the state of local AI agents as they do about which framework won.
A model refresh on the homelab (Qwen 3.6, new embeddings, 469 llama.cpp builds), a feature sprint on the vacation planning site (calendar sync, expense tracking, and three bugs that taught us more than the features did), and automating Substack syndication after discovering two more undocumented quirks. Three unrelated workstreams, one theme: maintenance is where the real learning happens.
I’ve been running OpenClaw on the homelab for a month. A recommendation sent me down the Hermes Agent rabbit hole — and the research before the first real test revealed my daily driver model was broken for tool calling all along.
DeepSeek V4-Pro, V4-Flash, and Zyphra ZAYA1 are three of the most exciting new models in local AI. None of them run on our RTX 5090 homelab — for completely different reasons. Here’s the research, the math, and what it means for anyone building a local inference rig.
Two AI models got the same prompt: review the blog fodder, check for redundancy, and draft a post. Opus chose a debugging war story. Qwen chose a data-driven redesign. Neither picked the same fodder. Here’s what the difference reveals about how models think about content.
We ripped out Ollama, migrated to llama.cpp, and benchmarked five local models across 12 tasks on an RTX 5090. The results surprised us — and the winner wasn’t who we expected.
Gemma 4 failed to build a single feature in our last test. This time we diagnosed the problem, switched from Ollama to llama.cpp, tuned the inference settings, and Gemma shipped a working search feature to production. Then Opus reviewed the code and made it better. Here’s what we learned about making local models actually work.
We pitted Gemma 4 against Opus 4.6 on a real feature build for vibescoder.dev. Gemma is the fastest model in our benchmark. It also couldn’t finish the job. Here’s what happened when we stopped testing toy apps and started building production code.
We added Google’s Gemma 4 and Moonshot’s 1-trillion-parameter Kimi K2 to the local model benchmark. Five out of six models scored perfect. Gemma 4 is the new speed king. And yes, we ran a 579 GB model off an NVMe drive — at 0.6 tokens per second.
We gave six LLM models the exact same coding prompt and measured everything: speed, tokens, and whether the code actually works. Three models scored perfect. Two built the wrong kind of app. One ran out of tokens mid-line.