OpenClaw and Hermes: AI Agents Without Self-Hosting an LLM

You deploy OpenClaw, open it up, and it asks which model you want to use. So you go looking for the model. You spin up Ollama next to it, pull a 7B, ask your shiny new agent a question, and wait. And wait. Then you resize the VM, and it is still slow, and now you are reading about quantization at midnight when all you wanted was an assistant that reads your email.

This is the single most common wrong turn we see people make with self-hosted agents, and it comes from a reasonable assumption: I am self-hosting the agent, so I must self-host the brain too.

You do not. Those are two separate decisions, and treating them as one is what makes the whole thing feel broken.

The agent is a client, not an inference server

OpenClaw and Hermes do the same category of work: they hold your context, manage memory and tools, talk to your channels, run on a schedule, and decide what to do next. None of that is model inference. When they need a model to think, they make an API call.

You can see this in the deployment specs, which are the most honest documentation available:

Hermes asks for 2 GB. That is not the footprint of something running a language model. It is the footprint of a well-behaved application that makes HTTP requests.

OpenRouter is the piece nobody told you about

OpenRouter gives you one API endpoint that reaches hundreds of models across providers. It is OpenAI-compatible, so anything that speaks the OpenAI API speaks OpenRouter by changing a base URL. You top up credits and pay per token, with no GPU to rent and no model files to manage.

Switching models becomes a config change. Try a cheap fast model for routine work, move to a frontier model for the hard tasks, and if a provider has an outage you change one line instead of redeploying anything.

Both agents support it as a first-class provider.

Wiring up OpenClaw

Set the key and pick a model. OpenClaw uses provider/model references:

OPENROUTER_API_KEY=sk-or-your-key-here

{
  agents: {
    defaults: { model: { primary: "openrouter/auto" } },
  },
}

openrouter/auto lets OpenRouter choose a sensible model per request, which is a good default while you work out what you actually need. To pin one:

openclaw models list
openclaw models set openrouter/anthropic/claude-opus-4.6

There is also an OAuth path if you would rather not paste a key:

openclaw onboard --auth-choice openrouter-oauth

Wiring up Hermes

Hermes reads credentials from .env and keeps non-secret behavior in config.yaml:

OPENROUTER_API_KEY=sk-or-your-key-here

Then pick your model interactively:

hermes model

Useful extras: HERMES_OPENROUTER_CACHE turns on response caching and HERMES_OPENROUTER_CACHE_TTL sets the TTL in seconds. On repetitive agent work, caching is a real cost reduction rather than a rounding error.

Hermes also accepts any OpenAI-compatible endpoint, which is how you point it at your own inference later without touching anything else:

OPENAI_API_KEY=your-key
OPENAI_BASE_URL=https://your-endpoint/v1

Start on OpenRouter, and if you move to your own inference later it is two environment variables, not a migration.

The GPU math is worse than people expect

Here is where the do-it-yourself plan usually falls apart, and at first it is not about money. It is about VRAM.

A model you would actually want an agent to use, something in the 27B class, has to fit in GPU memory along with its context. Quantize aggressively and you are still looking for more than 16 GB once real conversations are in flight. That number matters because of what the entry-level GPU tiers actually are.

The cheapest GPU instances in most catalogs are built on the NVIDIA A16. That card carries 64 GB of VRAM, but split across four independent GPUs at 16 GB each, and it was designed for virtual desktop density rather than inference. Your model has to fit inside one 16 GB GPU. For a 27B-class model it does not, or it barely does with a context window too small to be useful. These entry tiers are also in constant demand and frequently out of stock, so the cheap option often is not an option.

Rates vary by provider and region, so check current pricing rather than quoting this post back at your finance team. The gap is not the twenty percent people imagine when they picture "just running it myself." It is roughly two orders of magnitude, paid every month, whether the agent is thinking or sitting idle overnight. Token billing charges you only when it works.

When self-hosting the model is the right call

This is not an argument against Ollama. There are good reasons to run your own inference:

Your data cannot leave your infrastructure, for legal or contractual reasons. That is the strongest one and it settles the question on its own.

You are running high, sustained volume, where a fixed GPU bill eventually beats per-token pricing. That crossover is a calculation worth doing rather than guessing.

You need to work offline, or you want zero dependency on a third party's uptime.

You are running a fine-tuned or unusual model that no provider hosts.

If that is you, do it on hardware built for it. The mistake is trying to squeeze inference onto a general-purpose cloud GPU tier. Rent from a specialist instead. An NVIDIA RTX PRO 6000 Blackwell carries 96 GB of GDDR7, which is a different universe from a 16 GB vGPU slice, and providers rent them by the hour.

Which specialist depends on why you are self-hosting. If compliance is the driver, you want certifications and known data residency: Verda runs European data centers with SOC 2 Type II and ISO 27001 and offers RTX PRO 6000 among others. If you are experimenting or running bursty jobs, Vast.ai is a marketplace of third-party hosts and is usually much cheaper, but a marketplace is a strange fit when the whole reason you left the API was control over where your data sits. Check host-level guarantees before assuming they satisfy an auditor.

Elestio runs the agent on a small CPU box either way. Where the model runs is a separate decision, and that is the entire point.

Troubleshooting

The agent starts but every request fails with a 401. The key never reached the process. Check your .env is actually loaded and that the value has no trailing whitespace, which is easy to introduce when pasting.

Model not found. OpenClaw needs the full provider/model reference, so openrouter/anthropic/claude-opus-4.6, not claude-opus-4.6. Run openclaw models list to see valid references.

Requests fail once you add credits but worked on a free model. Some OpenRouter models are restricted by account settings or region. Try openrouter/auto to confirm the plumbing works, then narrow down.

Costs are higher than expected. Look at how much context the agent resends each turn. Agents are chatty by design, and a long tool loop re-sends a lot of tokens. Turn on caching in Hermes and use a cheaper model for routine steps.

It works, but it is slow. If you are still pointing at a local Ollama on a CPU VM, that is your answer.

Get it running

Deploy OpenClaw or Hermes on Elestio, add one API key, pick a model, and you have a working agent in about the time it takes to read this. No GPU, no model downloads, no quantization rabbit hole.

Then, if you decide you want your own inference, you will be making that choice for a real reason instead of because you assumed you had to.

Thanks for reading ❤️ See you in the next one 👋