Ten Days with My AI Agent: Courses, Cooling Ducts, and 78 Million Tokens a Day

In my last post I described the hardware side of my AI@home setup, two DGX Spark boxes clustered over 200GbE. This one is about the other half: what my agent and I actually do with it. I kept a log of the last ten days, fifteen working sessions, and this is the summary. It came together the way most of the work happened: the agent read its own session histories and drafted, I edited.

The setup

The agent is Hermes, and it doesn’t live on just one machine. At my desk it runs on the Mac: it drives a real browser over the Chrome DevTools Protocol, uses the two Sparks (which serve GLM-5.3-Flash and occasionally other open models through vLLM). A second instance lives on my Windows laptop. While the Mac is more for business topics, the Windows Laptop is sole for my private matters. The third sits in a small VM under Unraid and is connected through Telegram. I text it ad-hoc tasks from my phone (look something up, check whether a machine is still up, prepare this, remind me of that) and it works through them and reports back into the chat. Each surface keeps its own session history, which turns out to matter later.

Wherever it runs, the deal is the same: it reads, researches, scripts and verifies. I click, approve and install anything privileged. It never sees my passwords and doesn’t run sudo.

Finishing a certification track, page by page

The biggest chunk of work was finishing OpenAI’s Cyber Deployment Practitioner program on PartnerU. The agent read every course page one by one over CDP (I insisted it skip nothing), summarized each in German, and during knowledge checks recommended an answer with reasoning for every option while I clicked and submitted. Five courses, one graded lab, one 20-question final exam. Four of the courses fell in a single morning.

Two moments stuck with me. In the graded lab the agent switched from tutor to ghostwriter: it drafted the customer recommendation, 398 words against a 250–400 limit, and rendered it to PDF for me to upload. After the exam it went through its own session history and rebuilt the exam questions from all three practitioner programs into a private archive. To its credit, it notes the same thing every time: it read and recommended, I clicked and submitted.

300 HTML files, three models, one benchmark

We turned the Fable-5.1 benchmark, 100 single-file HTML design prompts, into a repeatable local test for the models on the cluster. A small Python harness ran each prompt through the vLLM endpoint, two in parallel, while I swapped models behind a fixed alias between runs. All three passed everything: 100/100 valid files for GLM-5.3-Flash, 100/100 for Qwen 3.8, 100/100 for DeepSeek. The hard ones were a page with twelve clocks in a single file and a planetarium that drove one model into a repetition loop it only escaped on the third attempt.

My favorite bug of the week: a broken exclusion list meant the two prompts we meant to skip actually ran. Both succeeded. I’ll take that kind of bug.

What 78 million tokens a day actually means

I got curious how many tokens the local models burn and asked for a meter. The agent checked the live Prometheus counters instead of trusting the docs, wrote the PromQL and built a Grafana dashboard. The numbers surprised me. In 24 hours the cluster received about 78.8 million prompt tokens, but only 0.6 million of those were actually computed, a 96.5% prefix-cache hit rate. So “burned” is two different numbers, and the honest one is the smaller. Priced at frontier API rates, the same workload would have cost a three-digit amount per day.

The dashboard took fourteen iterations because I kept catching things in the screenshots. The best one: overlapping rolling windows counted every token about 330 times, which briefly put us in the billions per day. Pinning the interval to exactly one day fixed it.

When the memory backend wouldn’t boot

The best debugging story of the period started with a restart loop. I had self-hosted a long-term memory backend for the agent and its API container would not stay up. Exit code 132. That’s SIGILL, an illegal CPU instruction, so not a config problem. The agent reproduced the crash on demand, tested the dependency wheels against each other and ended up in gdb, which put the fault on a single SVE instruction in OpenSSL’s CPU feature probe inside the cryptography library. The same wheel ran fine outside Docker, directly on the Mac.

So the verdict was: not the app, not the library, not my Mac. Docker Desktop’s virtualization layer on Apple Silicon claims to support SVE2 and then mis-executes it, and an upstream issue confirmed we weren’t the first to trip over this. The workaround was one pinned dependency, the real fix a Docker upgrade. By the end of the session the memory backend was serving the agent from my server. The app, as it turned out, was only the messenger.

Teaching the agent about its own history

Two sessions dealt with the agent’s own plumbing, and they only exist because it now lives on three machines. First question: can we sync session history between machines through Nextcloud? No. The state is a single SQLite database in WAL mode, and file-syncing that is how you get conflicted copies and lost history. Along the way the agent was wrong twice about what its own import can do. I pushed back twice, and it eventually found the real mechanism and proved it live: 41 sessions and 5,680 messages round-tripped with zero errors, and a second import run skips everything it already has. Its admission, “not just partially but fundamentally wrong”, is exactly what I want from a collaborator.

Second question: a weekly review that writes itself, ten lines every Monday. The first attempt spent forty minutes and roughly eighty full-text searches getting nowhere. Then one SQL query against the session database answered everything. The draft it produced says “no session recorded that day” for the empty Wednesday instead of inventing something. The cron job doesn’t exist yet, because the rule was “show me before you schedule anything” and I never gave the final yes.

Making hardware behave

Around the model work there was the usual homelab plumbing. New boot orchestration: one dispatcher script that stops all model stacks and starts exactly one, wired up as a systemd unit. Which exposed that a unit file sitting in my home directory was never actually enabled, which is why nothing had been starting at boot. A second session worked out why the nightly auto-shutdown kept failing: CPU 99.5% idle, GPU at zero, but 39 GB of LAN traffic over four hours. The shutdown check now only looks at the GPU.

The thermal problem from the cluster post got its plan, at least. The agent compared five community cooling designs, checked heat-deflection temperatures, picked a PETG filament, worked out the power budget (a 12 V fan run at 5 V moves about half the air) and produced a slicer profile for an 883 gram, 23-hour duct print. The shopping list is done. The print is the next session.

And when a printer invoice went missing, the agent rebuilt it as a PDF. It refused to invent a single number, asked me for the real prices, then verified the output three ways, including actually looking at the render.

A workday assist

Not everything ran on GPUs. After an early-notification email about an upcoming RFP at work I used the agent as a bid-pursuit coach. It built the meeting agenda and a set of five mapping questions, enforced a discretion checklist without being asked (an early notification is not an invitation to pitch), re-planned the strategy twice as new context came in, briefed me on the platform through web research, and pulled a case study out of a slide deck by parsing the raw slide XML because no library was installed. After the meeting it turned my notes into structured records, a deadline table and a draft email to get colleagues moving, with careful hedging on the one commercially sensitive point. Two days of bid preparation, done in an afternoon.

The patterns that made this work

Reading back through fifteen transcripts, a few habits show up every time. For anything consequential (submitting an exam, enabling a systemd unit, scheduling a cron job) it prepares and recommends, and I do the clicking. It refuses to make things up: no invented invoice prices, no filler entries for days where nothing happened, no claiming a deliverable was verified when it wasn’t. It takes correction. I caught it on a stale product version and on the Grafana bug, and both times it changed course mid-session instead of defending the mistake. It stays read-only until I approve, and privileged steps arrive as copy-paste commands rather than surprises. And when it needs an enumeration, it queries the index instead of guessing at keywords, a lesson it paid for with eighty wasted searches.

Ten days in, the split of labor still feels right. The agent doesn’t get tired of reading course pages at eleven at night, and I still get to decide what’s true.