Models and Their Roles

A RAG Topic needs two models: one to embed, one to answer. A GraphRAG Topic needs the same two. It also uses five more roles for the graph parts of the work: extraction, deduplication, community reports, routing and HYBRID’s follow-up reasoning. That makes seven. An eighth role, the Image Describer, reads pictures during upload. This page says what each role does. Use it to decide where a model of its own is worth the effort and where your answer model is enough.

Every optional role works the same way in the wizard. Pick a model for it, or tick Use the Topic’s Language Model and let it use your answer model. A role needs one of the two. The form does not let you leave it empty. When you tick the box, the role simply uses the answer model’s server and model, so it costs nothing extra to set up. See Creating a GraphRAG Topic for where each picker lives.

The two required roles

Language Model (the "answer model")

The model that writes what you read in the chat. It is also the model every optional role uses when you tick Use the Topic’s Language Model. The wizard makes you choose it, because there is no working default.

Your system prompt goes only to this model, and only for the final answer of each question: the LOCAL or PATH answer, the last step of GLOBAL, the HYBRID final answer, or a DIRECT or ATTACHMENT answer. The router, extraction and deduplication never see it. Their replies must keep a fixed format, and a persona instruction would break it.

The Language Model also reads images that chat users attach to a question (if you allow image attachments). The form does not check that the model can see pictures. Choose one that can, or those questions will fail.

Embedding Model

Turns text into numbers (vectors) so the search can compare it with a question. It does this for chunks, as in a RAG Topic. It also does it for entities and community reports.

Changing the Embedding Model later lowers search quality. Old and new vectors can no longer be compared. Search quality drops until everything is uploaded again and the communities are rebuilt. Choose the model before your first upload.

The model must have a known vector size ("dimension"). The picker asks the model for it. If the model does not answer, you must type the size yourself. The Topic cannot start without it.

The five optional pipeline roles

Graph Extraction Model

Reads every chunk during upload and finds the entities and relationships in it. This step turns text into a graph. It runs once per chunk, so the cost grows with the amount you upload. A stronger model gives a better graph, but uploads are slower and cost more. See how a document becomes graph.

Deduplication Model

Also runs during upload, right after extraction. It decides whether a new entity is the same real thing as one already in the graph, for example "ECB" and "European Central Bank", and merges them. Without it, the same thing mentioned in three documents can become three separate entities. A merged entity also links your documents together, so a later search can reach all of them from one point.

Community Report Model

Writes the summary for each community. GLOBAL answers are built only from these summaries, so this model sets the quality of every overview answer. It runs once per community on every refresh, so on a large graph a stronger model costs noticeably more time and money. Communities are always on, so this role always matters.

Router Model

Decides for every chat message how it is answered: LOCAL, GLOBAL, HYBRID, PATH, DIRECT or ATTACHMENT (see Retrieval strategies). It only chooses and does not write the answer, so a small, fast model is usually enough. It runs on every message, so a slow model makes every answer slower. The confidence thresholds are separate settings. They apply even when the router uses the Language Model.

Follow-up Model

Used in HYBRID only. It writes the follow-up questions that fill gaps between what the LOCAL and GLOBAL parts found, and it writes the final HYBRID answer. That last step reads everything that was found, so choose a model that can handle long input.

The eighth role: Image Describer

Describes each picture found during upload as text, so the picture’s content can be found later. It is set under Image Handling in step 2. That section appears only when pictures can arrive: when binary documents (such as PDF) or image files are among the upload types. You can turn it off with the picture mode Ignore. The standard mode, Picture in the graph, needs this role. The better the model, the more facts you get from pictures.

This role is different from the others in one way. When you tick Use the Topic’s Language Model, the Topic sends picture descriptions to the answer model’s own server. Other roles just share the answer model. Here the answer model itself must be able to read images. The wizard asks you to choose the Language Model in step 1 first.

  • The model for this role must be able to read pictures (a "vision" model). Check this yourself.

  • Servers of every provider can be used for this role.

  • Chat attachments are a separate matter. A picture that a user attaches in the chat goes to the Language Model, never to the Image Describer.

See File types, Docling, and image handling for how this works with Docling and with chat attachments.

Where each role is set

Role Wizard step Required, or can use the Language Model?

Language Model

1 — Basic Information

Required

Embedding Model

1 — Basic Information

Required

Graph Extraction

2 — Data Ingestion

Own model or tick "Use the Topic’s Language Model"

Deduplication

2 — Data Ingestion

Own model or tick

Image Describer

2 — Data Ingestion (Image Handling)

Needed in every picture mode except Ignore. Own model or tick; a ticked role uses the Language Model, which must read images

Community Report

3 — Vector Settings

Own model or tick

Router

4 — Retrieval

Own model or tick

Follow-up

4 — Retrieval

Own model or tick

Related