vibescoder
cd ..

Homelab Router Part 1: Getting a Static Prototype Working with Hermes

·10 min read
homelabaiagentsllmself-hostedhome-automationproxmoxbuilding-in-public

After getting DeepSeek Vision running, I had a working heavyweight local model. After the benchmark where DeepSeek won overall but Granite won smart home, I had a different problem. The best local model depended on the request.

So I wrote up the missing layer in the router concept post. Then I did the dangerous thing vibe codes do. I YOLO’d the plan and let Coder Agents build and deploy the first version on my production home network.

The result is now live in the homelab. Hermes can keep thinking it is calling one OpenAI-compatible provider. The router decides whether the request belongs on Spark DeepSeek, 5090 Granite, or a fallback path.

One note before the diagrams and config snippets: I am using role-based hostnames and documentation IPs in this post. The real homelab names and addresses are different. Past audits caught me leaking a literal LAN IP once, and that was enough of a lesson. The architecture matters here. The exact address book does not.

The Router Belongs on the Boring Always-On Box

My first instinct was wrong. I initially thought the router could live on the RTX 5090 workstation. That machine is powerful. It is also the box running this Coder workspace. It already runs llama-swap, local models, Docker services, and enough expensive hardware to make every watt feel personal.

But it is not always on.

That is the same lesson I learned building the always-on dashboard. Control-plane services do not belong on the machine whose absence they need to report. A router that disappears when the 5090 is asleep cannot truthfully say, “Granite is unavailable, falling back to DeepSeek.” It can only vanish.

The always-on machine is the ThinkCentre running Proxmox VE. It already hosts Home Assistant OS, Uptime Kuma, AdGuard Home, and the little dashboard. So the router became one more small Proxmox LXC instead of another process on the GPU workstation.

The live placement now looks like this:

Hermes / OpenAI-compatible clients
        |
        v
router LXC on the always-on Proxmox box
        |
        +-- spark-cluster.internal.example:8888     DeepSeek V4 Flash Vision-Exp
        +-- gpu-workstation.internal.example:8080    Granite 4.2 no-think via llama-swap

The LXC is tiny. One vCPU. 512 MB of RAM. A reserved local address. It runs a FastAPI app under systemd from /opt/local-agent-router, with production config in /etc/local-agent-router/config.yaml.

That is the right shape for this service. The router does not run inference. It reads JSON, checks health, applies policy, forwards a request, and writes useful logs.

The First MVP Was Just Enough OpenAI Compatibility

I did not want a new agent framework. I wanted a boring OpenAI-compatible proxy.

The first build shipped four endpoints:

GET  /health
GET  /routes
GET  /v1/models
POST /v1/chat/completions

/health reports which nodes and models are currently alive. /routes shows whether each route is healthy or degraded. /v1/models only advertises models that are actually reachable. /v1/chat/completions forwards the request to the selected backend after replacing the public model alias with the backend’s real model name.

The router makes deterministic decisions from the request shape. No extra LLM classifier sits in the critical path.

RequestFacts(
    has_image=True or False,
    tools={"HassGetState", "HassTurnOn"},
    estimated_input_tokens=cheap_character_count // 4,
)

The YAML policy is deliberately readable:

routes:
  - name: vision
    when:
      has_image: true
    prefer:
      - deepseek-vision
 
  - name: home_assistant
    when:
      tools_include:
        - HassGetState
        - HassTurnOn
        - HassTurnOff
    prefer:
      - granite-home
    fallback:
      - deepseek-vision
 
  - name: long_context
    when:
      input_tokens_gt: 50000
    prefer:
      - deepseek-vision
 
  - name: default
    prefer:
      - deepseek-vision

That policy encodes the benchmark result directly. Home Assistant tool calls prefer Granite 4.2 no-think on the 5090. Vision, long context, and generic requests prefer DeepSeek on the Sparks. If Granite is unavailable, Home Assistant requests fall back to DeepSeek rather than failing at the first unreachable endpoint.

This is where NVIDIA PAIR and LiteLLM remain useful but not sufficient. PAIR can help discover and route infrastructure. LiteLLM can help with gateway mechanics. The missing bit for my house is the policy that says Home Assistant live state is different from a screenshot, and a side-effecting smart-home command is different from a coding question.

Health Checks Made Offline Nodes Disappear

The design criterion I cared about most was not raw routing. It was absence.

If the 5090 is off, Granite should not appear as a capability. If Spark is down, DeepSeek should not appear as a capability. Adding a machine later should feel like a node joining the network, not like editing every client by hand.

The MVP is not fully dynamic yet. It still starts from YAML. But the behavior is already correct from the client side. Static config says what a node is allowed to provide. Runtime health decides what the homelab advertises right now.

The important bit is that /v1/models is live inventory, not a wish list.

When the 5090 endpoint was unreachable, the router returned only DeepSeek:

{
  "ok": true,
  "healthy_nodes": ["spark-cluster"],
  "healthy_models": ["deepseek-vision"]
}

The Home Assistant route reported itself honestly:

{
  "name": "home_assistant",
  "preferred_available": [],
  "fallback_available": ["deepseek-vision"],
  "unavailable_preferred": ["granite-home"],
  "degraded": true,
  "reason": "preferred model unavailable; fallback available"
}

That is the behavior I wanted. The router does not pretend the smart-home specialist exists when the node is absent. It exposes the generic fallback and marks the route degraded.

The next development session will finish the idea with real node heartbeat registration. For now, health-aware static inventory was enough to put the router in front of live traffic.

Local Names Beat Hardcoded IP Addresses

The first working config used raw IPs. That was fine for a smoke test and bad as infrastructure.

The homelab now has four local names managed by AdGuard Home (note: example IP addresses provided):

gpu-workstation.internal.example -> 192.0.2.41
proxmox.internal.example         -> 192.0.2.50
router.internal.example          -> 192.0.2.32
spark-cluster.internal.example   -> 192.0.2.4

I originally reached for Avahi and mDNS everywhere. Individual machines already advertise themselves that way. The GPU workstation can answer under a role-based local name, and the router LXC can resolve it after installing avahi-daemon and libnss-mdns.

The Spark cluster exposed a cleaner distinction. spark-head.internal.example is a machine name. spark-cluster.internal.example is a service name. Today it points at the head node because vLLM listens there on port 8888. Tomorrow it might point somewhere else. That is local DNS territory, not pure mDNS.

So AdGuard owns the service aliases. The redacted router config looks like this:

nodes:
  spark-cluster:
    endpoint: http://spark-cluster.internal.example:8888/v1
    health:
      url: http://spark-cluster.internal.example:8888/health
 
  rtx-5090:
    endpoint: http://gpu-workstation.internal.example:8080/v1
    health:
      url: http://gpu-workstation.internal.example:8080/health

That still depends on stable LAN addresses underneath. The firewall on the 5090 has to allow the router LXC by IP. But the app config now speaks in roles instead of addresses, and that is the level of abstraction I want clients to see.

The 5090 Was Online but Its Model API Was Not Reachable

One useful correction came from a bad assumption.

I looked at the router and saw granite-home unavailable. I said the 5090 was offline. That was wrong. The 5090 was very much online. It was hosting the Coder workspace I was using.

The real problem was narrower and more interesting. llama-swap was listening on localhost only:

127.0.0.1:8080  llama-swap

That was perfect when every local client lived on the workstation. It was invisible to a router running in a ThinkCentre LXC.

The fix was not to throw Tailscale at it. Tailscale is great for remote access and machine identity, but the router and workstation already sit on the same LAN. The boring local path was better.

I changed llama-generate.service on the GPU workstation so llama-swap listens on 0.0.0.0:8080, then added a persistent llama-swap-firewall.service that allows only localhost and the router LXC to connect:

ALLOW 127.0.0.1 -> tcp/8080
ALLOW router LXC -> tcp/8080
DROP  tcp/8080 from everyone else

Once that was in place, the router saw both nodes:

{
  "ok": true,
  "healthy_nodes": ["rtx-5090", "spark-cluster"],
  "healthy_models": ["deepseek-vision", "granite-home"]
}

And a Home Assistant-shaped request selected Granite:

x-local-agent-route: home_assistant
x-local-agent-model-alias: granite-home
x-local-agent-fallback-used: false

That is the whole point of the router. Hermes does not need to learn that the 5090 exists. It sends a normal request. The router sees Home Assistant tools and sends it to the right specialist.

Monitoring Needed the Same Socket.io Detour as Last Time

Once the router was running, it needed a monitor. I used the existing Uptime Kuma instance on the ThinkCentre.

Kuma remains weird in the same way it was weird during the Home Assistant monitoring work. There is not a useful REST API for monitor management. The web UI talks over Socket.io, and the Python uptime-kuma-api package talks the same way.

So the monitor was created through that client, not by curl:

Name: Local Agent Router
URL:  http://router.internal.example:8088/health
Type: HTTP
Interval: 60 seconds
Retries: 2
Retry interval: 30 seconds

The Kuma LXC resolves the local DNS name and sees the live health payload:

{
  "ok": true,
  "healthy_nodes": ["rtx-5090", "spark-cluster"],
  "healthy_models": ["deepseek-vision", "granite-home"]
}

That gives the router the same status treatment as the rest of the house. If the control plane fails, I want the monitoring plane to complain before Hermes quietly starts feeling flaky.

Hermes Became a Client Instead of the Policy Layer

The Hermes side is intentionally boring.

I added one custom provider:

- name: Local Agent Router
  provider_key: local-agent-router
  base_url: http://router.internal.example:8088/v1
  model: deepseek-v4-flash-vision-exp
  api_mode: chat_completions

Then I tested a one-shot request through Hermes:

What is 17*19? Return only the final integer.

Hermes called the router. The router sent the request to Spark DeepSeek. The answer came back:

323

I did not flip Hermes’ default provider yet. That is deliberate. The router path works, but I want the next end-to-end Home Assistant test before I make it the default. The Z-Wave dongle arrives next, and that gives me the test I actually care about: can Hermes receive a normal smart-home command, route it to the right local model, call Home Assistant, and turn off the living room lights?

That test will exercise the whole chain:

Hermes
  -> local-agent-router
  -> Granite on the 5090
  -> Home Assistant tools
  -> Z-Wave device state change

If that works, the router stops being a neat infrastructure experiment and becomes part of the house.

This was the satisfying kind of homelab session. The code was small. The useful work was in the seams: where the service should live, what name clients should use, what should disappear when a machine turns off, and how little Hermes should have to know about any of it.

This is Part 1, not the finish line. Dynamic node registration is still next. But the first version is live, monitored, and already making the local AI fleet feel less like a pile of endpoints and more like one system.

By the Numbers

  • 1 dedicated Proxmox LXC now runs local-agent-router on the ThinkCentre
  • 4 OpenAI-compatible router endpoints shipped: /health, /routes, /v1/models, and /v1/chat/completions
  • 2 live model backends are advertised today: Spark DeepSeek and 5090 Granite
  • 4 AdGuard DNS rewrites replaced raw IPs in the router path
  • 1 persistent firewall service protects the 5090 model API from the rest of the LAN
  • 1 Uptime Kuma monitor watches the router health endpoint every 60 seconds
  • 7 automated tests currently cover routing, health-based model hiding, and fallback behavior
  • 0 changes required in Hermes’ tool logic for model selection. The router owns that policy

Comments