Securing an Always-On AI Agent at Home: A Journey Through Dead Ends

How I went about securing Hermes Agent, an always-on CLI-based AI assistant from Nous Research, running on a spare mini PC at home. This is not a "here's the perfect setup" post. It's the honest record of three approaches that failed and the one that triumphed.

---

The goal

I wanted an AI agent running 24/7 in my home that I could reach from anywhere: my phone, a laptop on someone else's network, a hotel room while traveling. Hermes can run shell commands, read and write files, browse the web, and call external APIs. That's what makes it useful. It's also what makes it dangerous. An always-on process with shell access, sitting inside your home network, is a serious thing to get wrong.

So the project became: how do I run this thing such that (a) I can reach it remotely, (b) it can't reach the rest of my home network, and (c) if it or the machine it runs on is ever compromised, the blast radius is as small as I can make it?

The host is a Dell OptiPlex 9020M running Linux: a cheap, low-power, almost-fanless mini PC. I installed the final stock version of Hermes in mid-June 2026.

One thing I'll skip past here: why Hermes, and not one of the other self-hosted agents. That was its own decision, mostly about security posture and the fact that it comes from an independent lab rather than a frontier vendor. I worked through that comparison in a separate post, Hermes vs. OpenClaw. This post picks up where that one left off: the agent is chosen and installed, and now I have to run it without it becoming the weakest point on my network.

---

Attempt 1: Harden it in place, on the home LAN

My first instinct was the textbook one: leave the machine on my home network and lock it down hard. Aggressive firewall rules, strict egress filtering, minimal services, the works. Harden the box until it's safe to live alongside everything else.

This failed, and it failed in a useful way. The hardening was so restrictive that the agent couldn't do its job. The most basic capability, reaching the web, broke. Outbound traffic the agent needed (API calls to model providers, web search, fetching pages) got caught in the same net I'd cast to keep bad traffic in. So I started poking holes. Every hole felt like it was undoing the security I'd just added. I was chasing a moving target: each new tool the agent picked up wanted a new outbound destination, and I was hand-maintaining an allowlist that was both too tight to be usable and too leaky to feel safe.

There was a second, independent problem. The hardening hadn't stayed at the operating-system level. Enforcing it properly meant extensive modifications to Hermes's own source code, and that made upgrades fragile. Hermes is under active development and ships frequent updates, and each one would clobber my patches and force me to re-apply every customization by hand. I was effectively maintaining a private fork of the agent just to keep my security rules in place, and those rules were already proving too brittle to do the job.

Two lessons, both obvious in retrospect. An agent whose entire value is taking actions in the world is fundamentally hostile to egress filtering. You can't really say "block everything outbound" to a process whose whole job is making outbound calls; you either neuter it or you maintain a brittle allowlist that lags behind reality. And hardening that's enforced by patching the agent's own source is a trap, because it pits your security directly against the project's release cadence.

Rather than keep fighting that fight, I changed the topology.

---

Attempt 2: Move the machine off the home network

So I stopped trying to make the agent safe inside my network and moved it outside instead.

My home internet comes in through the ISP router. The OptiPlex sits directly on that ISP router and has never touched my home LAN. Downstream, I put a second router that manages the home LAN, and everything I actually care about (laptops, phones, NAS) lives behind it, segmented away from the OptiPlex. The OptiPlex is on the ISP router's subnet, 192.168.2.x; the home LAN is on its own subnet, 192.168.100.x.

The effect: the agent's machine is on the internet side of a router boundary, but my home network is behind its own router, isolated from it. The OptiPlex has no route into the home LAN. Even if the agent or the machine were fully compromised, the attacker lands on a subnet that can reach the wider internet but cannot reach my personal devices. The blast radius for "the agent machine is owned" collapses from "my whole home network" to "one isolated box and its API keys."

                  ┌──────────────────────────────────────┐
   Internet ───►  │  ISP router                          │
                  └───────────────┬──────────────────────┘
                                  │
                  ┌───────────────┴──────────────────────┐
                  │                                      │
        ┌─────────▼──────────┐              ┌────────────▼──────────────┐
        │  Second router     │              │  OptiPlex (Hermes agent)  │
        │  (home LAN gateway)│              │  192.168.2.66             │
        └─────────┬──────────┘              │  no route to home LAN     │
                  │                         └───────────────────────────┘
        ┌─────────▼───────────┐
        │  Home LAN           │
        │  192.168.100.x      │
        │  (laptops, phones,  │
        │   NAS, etc.)        │
        └─────────────────────┘

This was the single biggest security win of the whole project, and it cost zero software. Network segmentation beat any firewall rule I could have written. It also solved both problems from Attempt 1 at once. The agent can reach the internet freely, because that's fine now: the thing I was worried about it reaching, my home network, sits on the other side of a router it can't cross. And because the isolation comes from the network rather than from edits to the agent's code, I could run stock Hermes again. No private fork, no patches to re-apply on every update.

A note on the topology choice. What matters for the security model is the router boundary between the agent machine and the home LAN, and in my setup that boundary is the second router's WAN firewall. I could have gotten a similar boundary without a second router, by putting the OptiPlex on its own VLAN or a separate subnet on the ISP router with an inter-zone firewall rule blocking traffic between the two segments. My ISP router is fully configurable and supports this; plenty of prosumer and business-grade routers do too. The isolation would be equivalent in the threat model that matters most (a compromised agent machine trying to reach home network devices): both approaches place a firewall boundary that the agent's traffic can't cross.

I went with the two-router approach for one reason: the isolation is structural rather than configuration-dependent. With two physical routers, there is no VLAN tag, no inter-zone firewall rule, and no switch port assignment that could be accidentally misconfigured or overwritten by a firmware update to silently open a path between the segments. The boundary exists because of where the cables are plugged in. That's a weaker argument than it sounds (a correctly configured VLAN on a well-maintained router is perfectly trustworthy), but I liked that the isolation survives configuration mistakes, firmware changes, and my own forgetfulness. If I didn't already have the second router, or were setting this up somewhere with only one router and no appetite for extra hardware, the single-router VLAN approach would be perfectly reasonable and I wouldn't hesitate to use it.

One honest caveat to the "no route to the home LAN" claim: the boundary is itself a device, and that device sits within reach of the box I'm treating as compromised. The OptiPlex shares the ISP router's subnet with the WAN side of the second router, so a compromised OptiPlex can't route through the second router, but it could attack the router directly, and if it got in it would be standing past the boundary. I've closed that path as far as I can. The second router's admin interface is disabled on its WAN side and protected by a strong password, so from the subnet the agent lives on there's no management surface to reach at all. The route into the home LAN doesn't exist, and the one device that could create such a route isn't administrable from where the agent sits.

But this created a new problem. If the machine is off in its own little subnet, how do I get to it?

---

Attempt 3: Tailscale, killed by a VPN conflict

For remote access, my first choice was Tailscale. It builds a WireGuard mesh overlay: install it on the target and on your client, and they find each other through Tailscale's coordination server and open a direct encrypted tunnel. Clean, end-to-end encrypted, with ACLs and device tags. I brought it up on the OptiPlex with an isolation tag (tag:isolated) and Tailscale's built-in SSH server:

sudo tailscale up --advertise-tags=tag:isolated --ssh

The plan was that the OptiPlex would be reachable only over the Tailnet, only from my authorized devices, with Tailscale handling both the encryption and the authentication. No exposed ports, no port forwarding on the router, no public IP.

I spent a while debugging it: inspecting the netmap, checking the SSH policy, tcpdumping on tailscale0 to watch the traffic. The setup itself was fine. What killed it was a conflict with my VPN.

This is a known and painful category of problem. Tailscale is itself a WireGuard client, so it wants to manage routes and a virtual interface. A commercial VPN client wants exactly the same thing, especially if it's also WireGuard-based or runs a kill switch that hijacks the default route. On the laptop or phone I was connecting from, running my personal VPN and Tailscale at once meant one of them kept losing. Connections were unstable, or just never came up.

I could have worked around it with policy-based routing, split tunneling, or turning the VPN off whenever I wanted to reach the agent. All of those defeated the point. I wanted something that "just worked" from any device, in any network state. Tailscale didn't, in my setup. So I ripped it out:

sudo apt-get remove --purge -y tailscale
sudo apt-get remove -y tailscale-archive-keyring

None of this is a knock on Tailscale. It's excellent, and if I weren't running a consumer VPN on my client devices it would have been the right answer. The conflict was specific to my habits, not the tool.

---

Attempt 4: A TCP tunnel via ngrok (what I use now)

I needed a remote-access method that didn't care about routing tables or VPN state on my client. Something where I connect to a fixed public address and it relays back to the machine. I settled on ngrok.

ngrok runs as a persistent service on the OptiPlex. It dials out to ngrok's relay infrastructure and holds a tunnel open. ngrok assigns a public TCP endpoint that forwards to a local port. I pointed it at the local SSH port:

ngrok tunnel:  tcp://<region>.ngrok.io:<port>  ──►  localhost:22

Now, from anywhere, I SSH to <region>.ngrok.io on port <port>, and ngrok relays the connection through to the OptiPlex's SSH daemon. The machine has no open inbound port on my home router. There's no port forwarding, no exposed public IP on my connection, no Dynamic DNS. The OptiPlex dialed out; the tunnel is held from the inside.

(A terminology note: people often call this a "reverse SSH tunnel," by analogy to ssh -R, where you open a port on a remote server that forwards back to your local machine. ngrok gets the same reach-in-from-outside effect, but the relay is ngrok's own infrastructure, not an SSH server's reverse-port-forwarding. Same concept, "a public address that tunnels to an internal service"; different mechanism.)

To keep it reliable, ngrok runs as a system-level systemd service that restarts on failure, so the tunnel comes back on its own after a reboot or a network blip. The SSH daemon behind it is locked to key-based authentication only: no passwords, no keyboard-interactive. So reaching the agent means having the right private key, knowing the ngrok TCP address and port, and connecting to a service that only accepts the two keys I've authorized.

This is the current access path, and it's been stable.

---

What I kept: defense in depth on the box itself

Getting the machine off the home network and making it reachable solved the network-level problems. But I still wanted the box itself hardened, so that even the "isolated subnet" worst case was as contained as possible. This is the hardening that survived. Unlike Attempt 1, none of it breaks the agent's ability to function, because it targets inbound traffic and the agent's own behavior rather than outbound connectivity. And none of it touches the agent's source code: it's all system-level configuration, filesystem layout, and built-in features switched on through config.

Host firewall (nftables, default-deny inbound). The machine runs nftables with a default-DROP policy on inbound traffic. Nothing reaches the box unless it's an established connection, loopback, DHCP, or ICMP. This sits on top of the NAT and the network segmentation, so inbound protection is layered three deep: the ISP router, the second router's boundary, and the host firewall.

SSH, key-only. Password and keyboard-interactive authentication are disabled. Two authorized keys, both mine. SSH is the one service the ngrok tunnel exposes, so this is the credential gate on the only realistic entry point.

The agent's own in-process defenses. Hermes ships with several layers that I left enabled:

• A pre-execution command scanner (Tirith) that inspects every shell command the agent wants to run for URL-based threats, homograph attacks, pipe-to-interpreter patterns, and terminal injection, before it executes.
• Secret redaction on all tool output, so that if an API key ever surfaces in a file or a command's stdout, it's masked before it enters the agent's context (and can't be exfiltrated through the model). This is snapshotted at startup and can't be turned off mid-session. That's deliberate: the agent can't disable its own redaction.
• Manual approval gating on destructive commands (rm -rf, force-pushes, etc.). The agent has to ask before it runs them. (Cron-scheduled jobs use an even stricter mode where destructive commands are auto-denied.)
• A threat scanner on project context files (the .hermes.md / AGENTS.md files that get injected into the system prompt), which neutralizes prompt-injection patterns.
• A caller allowlist on the Telegram gateway, which is the one channel the agent listens on.

Data separation. The agent's bulk data (sessions, logs, caches) lives on a dedicated external SSD, while the security-critical files (the .env with API keys, the config, the session database) stay on the system disk with 600 permissions.

A locked-down secondary profile. I keep a second Hermes profile with most of the dangerous toolsets disabled (coding, computer use, etc.) for research tasks where I don't need full capability.

---

Where this lands: an honest assessment

Here's what the final posture actually protects against, and what it doesn't.

Protected against:

• The agent machine being compromised and used to attack my home network (it can't route there).
• Unsolicited inbound connections from the internet (no open ports on my router; host firewall default-deny).
• Unauthorized people reaching the agent over Telegram (allowlist).
• The agent leaking my API keys into its own context or a chat (secret redaction).
• The agent running a destructive command by accident or via a naive prompt-injection (approval gate + command scanner).
• SSH password brute-force (key-only auth).

Not protected against (and I know it):

• A sufficiently determined prompt-injection that chains through to shell. The agent's terminal backend runs locally, as my user, with sudo. If an attacker could craft input that talked the agent into running arbitrary shell, there's no OS-level sandbox between the agent and the host. The in-process scanners and approval gate raise the bar, but they're heuristics, not a containment boundary.
• Physical theft of the box. The disks aren't encrypted. Whoever has the machine has the API keys and the session history.
• A malicious skill or plugin running inside the agent process with full privileges. I mitigate this by not loading third-party plugins, but it's a trust boundary, not a technical one.
• The ngrok endpoint itself. If ngrok were compromised, or if my ngrok credentials leaked, the SSH port would be exposed through their relay. I'm trusting ngrok as part of my trust chain, a conscious trade for the convenience of not running my own relay.
• Data exfiltration from the box itself. Free outbound access is the design; it's what makes the agent useful and what let me stop fighting egress filtering. The flip side is that a compromised host can read its own disk (the .env, the session database) and POST the contents anywhere it likes, and nothing on the box will stop it. Secret redaction guards against a key leaking into the model's context; it does nothing against a compromised host reading the file directly. I take this one on purpose: every API key the agent holds is scoped to a service with a hard spend limit, so the worst case for a leaked key is a capped bill, not unbounded abuse.

The honest summary: I traded OS-level isolation of the agent for the convenience of a local backend, and I compensated by isolating the machine on the network so that the agent's blast radius is bounded by a router instead of by a container. That's a reasonable trade for a personal, single-user, single-tenant agent. It would not be the right trade if the agent were serving untrusted users, ingesting untrusted content (like parsing arbitrary email), or running in a shared environment.

---

The takeaway

If there's one thing I'd tell someone setting up an always-on agent for the first time, it's this: don't try to make the agent safe with firewall rules alone. An agent's whole job is to reach out into the world, and egress filtering fights that at every step. Instead, put the agent on a network segment where the things it could hurt aren't reachable, and let it talk to the internet freely. Network segmentation is a one-time physical/configuration act that doesn't decay as the agent's behavior evolves; an egress allowlist is a maintenance burden that rots the moment you aren't looking.

A second principle, learned the hard way: when you do harden the agent itself, prefer configuration and built-in features over patching its source code. A customization that lives in the agent's source dies on the next update; one that lives in config or in the network outlives every upgrade.

The rest is defense-in-depth for the things segmentation doesn't catch: key-only SSH, a default-deny host firewall, the agent's own command scanning and secret redaction. But segmentation did more for this setup than every other measure combined, and it's the part I should have done first.

---

Host: Dell OptiPlex 9020M, Linux. Agent: Hermes (Nous Research). Remote access: ngrok TCP tunnel to key-only SSH. Written July 2026.