Why GraphRAG, and the architecture behind it

1. Why GraphRAG

Plain RAG retrieves chunks — passages of text ranked by how similar their embedding is to the question’s. That works well when the answer lives inside one or two passages. It works poorly for two kinds of question that come up constantly in real document sets:

  • "How are X and Y connected?" — the answer is spread across several documents, each mentioning only a piece of the relationship. No single chunk contains it, so similarity search never surfaces the connection as a whole.

  • "What are the main themes/risks/topics across all of this?" — a broad question has no single passage that answers it; the answer is a synthesis over the whole corpus, not a handful of matching chunks.

GraphRAG addresses both by building a second representation of your content alongside the chunks:

  • A knowledge graph — the entities documents talk about (people, products, systems, error codes, whatever your content is actually about) and the relationships between them, extracted by a language model as documents are ingested. A question about a connection can now be answered by traversing the graph outward from the entities the question mentions, instead of hoping one chunk happens to state the connection directly.

  • Community reports — the graph is periodically clustered into topical communities (groups of densely connected entities), and a language model writes a short summary of each one. A broad question is answered from these summaries directly, because they already are the synthesis a chunk-level search cannot produce.

The trade-off is real and worth stating up front: a GraphRAG Topic costs more to ingest (extraction is an LLM call per chunk, on top of embedding) and more to maintain (community reports need periodic recomputation as the graph changes), in exchange for being able to answer questions plain RAG structurally cannot. If your questions are always "what does the manual say about X" — single-passage, lookup-shaped — a RAG Topic is simpler, cheaper, and just as good. If your questions are relational ("what depends on what") or need a corpus-wide synthesis, GraphRAG is the tool that reaches them.

RAG Topic GraphRAG Topic

Retrieves

Chunks ranked by embedding similarity

Chunks and a subgraph of entities/relationships, or community summaries, depending on the question. Some messages need no search at all

Good at

"What does the document say about X"

The same, plus "how is X connected to Y" and "what are the main themes here"

Ingestion cost

One embedding call per chunk

One embedding call per chunk, plus one extraction call per chunk, plus periodic clustering and one report-writing call per community

Ongoing maintenance

None

Community reports must be recomputed as content changes, or GLOBAL questions answer from stale summaries

2. Architecture — what actually runs

Same shape as a RAG Topic: the management application does not answer chat itself — it generates and orchestrates a container. Pressing Start on a GraphRAG Topic provisions real infrastructure, just like a RAG Topic, but the container it starts is a different one (a dedicated GraphRAG container, not the RAG Topic container) and it talks to one more kind of external server: a graph database.

flowchart TD APP["Management application
(this web UI)"] APP -- "starts, per Topic" --> STACK subgraph STACK ["GraphRAG Topic stack"] direction TB GTOPIC["GraphRAG service
(always on)"] KEYWORD["keyword index
(only with keyword search)"] end STACK -- "reaches out to" --> DEPS subgraph DEPS ["External dependencies"] direction LR AI["AI servers"] DOCLING["Docling servers"] VDB["Vector database"] GDB["Graph database"] end

Everything true of a RAG Topic’s operating model is equally true here:

  1. Configuration is delivered when the container starts. Changing a setting on a running Topic means stopping and starting it. The wizard offers Restart Now when you save such a change. Unlike a RAG Topic, a GraphRAG Topic has no setting you can change live.

  2. Each Topic gets its own host port, chosen in the wizard.

  3. Docker (or Kubernetes) must be available. Without it, a GraphRAG Topic goes STARTING → ERROR exactly like a RAG Topic.

What is genuinely new compared to a RAG Topic

  • A graph database and a vector database are hard requirements, not options. There is no built-in store for either. You register them under Admin → Graph Databases and Admin → Vector Databases, and the wizard asks you to pick both in step 1.

  • Two storage backends are in play at once. The vector database holds the embeddings of text pieces, entities and community summaries. The graph database holds the entities and relationships themselves, as real graph structure.

  • Deleting a Topic does not delete its graph. The graph stays in your graph database, like the data in a registered vector database, and you remove it there.

  • Keyword search is optional. By default, searches work by meaning only (dense). If you switch LOCAL or GLOBAL search to meaning and keywords in wizard step 3, the Topic keeps a keyword (BM25) index in its own storage volume. See the glossary for why it exists next to the vector store.

  • More model roles than a RAG Topic. A RAG Topic needs a chat model and an embedding model. A GraphRAG Topic needs the same two, and can add more: extraction, deduplication, community report writing, routing and follow-up questions, plus models for pictures. Each can be set separately, or tick use the Language Model to reuse the chat model. See Models and their roles.

  • Communities are always on. Grouping the graph into communities (with the Leiden method) is part of every GraphRAG Topic. You can tune how it runs, but you cannot switch it off.

  • Pictures in documents can be ignored, read like normal text, or stored as pictures in the graph so that answers can show them. See File types, Docling and pictures.

Lifecycle states

Same as a RAG Topic’s. A Topic that is STARTING or ERROR is checked against its container’s health, so a Topic whose container is up but cannot reach a store shows ERROR, and shows RUNNING again when the problem goes away.

Status Meaning

CREATED / CONFIGURED

Saved in the database, nothing running yet

STARTING

Container is being pulled and started

RUNNING

Ready for uploads and chat

STOPPED

Container stopped. Configuration and indexed data are retained

ERROR

Start failed — most often Docker unreachable, a misconfigured server, or a required model role left unset

Related