Self-Host Neo4j: Knowledge Graph Memory for Your AI Agent

Ask your agent "what breaks if we upgrade the auth service" and watch a pure vector store handle it.

It embeds the question and returns the five most similar chunks: the auth README, a paragraph about upgrades, a runbook using the same vocabulary. What you asked for was the list of services that depend on auth. That list exists in your data. Vector search has no way to walk it.

Vector search finds things that are similar. A graph finds things that are connected. Those are different questions, and most agent memory failures are somebody asking the second one with a tool that only answers the first.

We have written plenty about vector stores here and never once about Neo4j. Let us fix that.

Similar and connected are different problems

Similarity is a distance calculation, a decent proxy for meaning. Connection is a fact you stored on purpose: (auth)<-[:DEPENDS_ON]-(billing) is not an inference, it is a recorded relationship, and traversing it returns an exact answer rather than a plausible one.

The difference shows up in the failure mode. A vector store that does not know the answer returns the closest thing it has, and your agent summarizes it as though it were the answer. A graph traversal that finds nothing returns nothing, and for an agent an empty result beats a confident near-miss every time.

You no longer have to choose. Neo4j has native vector indexes and they work in the free Community Edition, so one database answers both kinds of question in one query.

Get one running

Neo4j on Elestio is a one-click deploy. The thing to know going in is the port: Neo4j speaks the Bolt protocol on 7687, and that is what your driver connects to, not the HTTP port serving the browser UI.

import os
from neo4j import GraphDatabase

driver = GraphDatabase.driver(
    "bolt://your-service.vm.elestio.app:7687",
    auth=("neo4j", os.environ["NEO4J_PASSWORD"]),
)

driver.verify_connectivity()

Budget around 4 GB. Neo4j is a JVM application wanting heap plus page cache, and vector indexes hold embeddings in memory on top of that. The NC-MEDIUM-2C-4G tier at $16/mo is a sane floor. Community Edition carries no license fees, but the VM is real infrastructure with a real bill, so check current pricing before sizing.

Model the thing you actually want to ask about

Resist dumping documents in as nodes and calling it a graph. A graph earns its keep when the relationships are real:

CREATE (auth:Service {name: 'auth', owner: 'platform'})
CREATE (billing:Service {name: 'billing', owner: 'payments'})
CREATE (billing)-[:DEPENDS_ON {since: '2025-03'}]->(auth)

Now the impact question is a traversal, and it is exact:

MATCH (s:Service {name: 'auth'})<-[:DEPENDS_ON*1..3]-(affected:Service)
RETURN DISTINCT affected.name, affected.owner

*1..3 follows the chain up to three hops, so you catch the services that depend on the services that depend on auth. Try expressing that as a similarity search.

Getting your data in

This is the step most graph articles skip, and it decides whether the whole thing works.

Nobody hands you a populated graph. If your relationships already live somewhere structured, a service catalog, a Terraform state file, a CI config, an org chart, import them directly. Your infrastructure already encodes most of the dependency graph you want.

LOAD CSV WITH HEADERS FROM 'file:///services.csv' AS row
MERGE (s:Service {name: row.name})
SET s.owner = row.owner

Use MERGE, not CREATE. MERGE matches an existing node or creates it if absent, so re-running your import updates the graph instead of duplicating every service.

Where relationships only exist in prose, extract them with an LLM: feed the document in, ask for entities and relationships as structured JSON, write those out as nodes and edges. That is how most GraphRAG pipelines work, and you trade exactness for coverage since an extracted edge is only as reliable as the model that read the paragraph. Store the source document on the edge so you can audit it later.

Most real systems do both: structured sources for a backbone you can trust, extraction for what nobody wrote down machine-readably.

Both halves in one query

Add a vector index over your document embeddings:

CREATE VECTOR INDEX docEmbeddings IF NOT EXISTS
FOR (d:Document)
ON d.embedding
OPTIONS { indexConfig: {
  `vector.dimensions`: 1536,
  `vector.similarity_function`: 'cosine'
}}

Then let the vector search pick the entry points and the graph do the reasoning:

MATCH (doc:Document)
  SEARCH doc IN (
    VECTOR INDEX docEmbeddings
    FOR $queryVector
    LIMIT 5
  ) SCORE AS score
MATCH (doc)-[:MENTIONS]->(svc:Service)<-[:DEPENDS_ON]-(dependent:Service)
RETURN doc.title AS source,
       svc.name AS service,
       collect(DISTINCT dependent.name) AS affected,
       score
ORDER BY score DESC

Semantic search picks the entry points, the graph expands them into concrete dependency facts. That combination is the whole argument for graph-backed agent memory.

How the agent actually calls this

Give the agent a few specific tools, not a Cypher prompt.

The tempting design is to let the model write its own Cypher. It demos well and it is a bad idea in production, for the same reason you would not hand an LLM a raw SQL connection. A model that can write Cypher can write DETACH DELETE, and prompt injection in a document your agent just read is a realistic path to that.

The safer shape is a handful of parameterized queries exposed as tools: find_dependents(service), who_owns(service), search_docs(question). The agent picks a tool and supplies arguments, you control the query text. Predictable performance, queries you can index for, and a blast radius of zero.

If you do want generated Cypher, connect with a dedicated read-only user so the worst case is a slow query rather than a deleted graph.

The part every tutorial gets wrong

If you search for Neo4j vector examples you will find this everywhere:

CALL db.index.vector.queryNodes('docEmbeddings', 5, $queryVector)
YIELD node, score

That procedure was deprecated in Neo4j 2026.04. The SEARCH clause replaced it as the preferred form in 2026.01, and it composes with the rest of your Cypher instead of forcing a CALL boundary.

It still runs and it emits deprecation warnings. Almost every tutorial predates the change, so the first result you copy is code that was correct eighteen months ago. Check the vector index manual instead.

What Community Edition will not do

Be clear-eyed before committing a production workload.

The one that bites people is online backup. On Community you are backing up a stopped database or working from filesystem snapshots, which is fine right up until it is not. Elestio's automated backups cover the VM, so confirm your restore path works before you need it. Neo4j publishes a full edition comparison, and the auth specifics are worth reading there rather than trusting any blog post, including this one.

Troubleshooting

Driver cannot connect but the browser UI loads. Different ports. The UI is HTTP, your driver needs Bolt on 7687. Make sure it is open and that your URI says bolt:// or neo4j://.

Authentication failure on a fresh instance. Neo4j forces a password change on first login. Do that in the browser UI before pointing a driver at it.

Your vector query returns nothing. Check the index dimension count matches your embedding model exactly. A 1536-dimension index will not serve 768-dimension vectors, and it fails silently.

Deprecation warnings in the logs. That is the db.index.vector.queryNodes procedure. Move to the SEARCH clause.

Queries slow down as the graph grows. Usually a missing index on the property you match on before traversing. Run EXPLAIN on the query and look for AllNodesScan, which means Neo4j is reading every node to find your starting point. A plain index on the property you filter by fixes it.

Worth building properly

The trap with graphs is modelling everything and ending up with a diagram nobody queries. Start narrow: pick the one question your agent keeps answering badly, model only the entities that question needs, grow from there. Four node types answering one question correctly beats an ontology describing your entire business that gets queried twice.

Keep your vector search. But when your agent needs to know what depends on what, who owns which thing, or what happened before what, it needs edges, and edges are something you have to store on purpose.

You can deploy Neo4j on Elestio and be running Cypher in minutes.

Thanks for reading ❤️ See you in the next one 👋