Retrieval Strategies — What the Router Chooses and What You Can Tune

How a prompt is answered explains what each strategy does. This page explains what you can tune about each one. Most of these settings are on the wizard’s Retrieval step (step 4). The search-type choice and the communities are on step 3. You can change all of them later by editing the Topic. A running Topic must be stopped and started again before a change takes effect.

Many of these settings sit in expert sections: a tick box in the section title unlocks them. Each has a working standard value. Change them only when you have a reason, for example when the logs show that answers miss facts or cite too much.

The six ways a question can be answered

For every chat message, the router picks one way. Users do not choose.

Way Used for Searches?

LOCAL

Facts about named things: a product, a person, an error code

Yes, follows connections in the graph

GLOBAL

Overview questions: "What are the main themes?"

Yes, over community summaries

HYBRID

Questions that need both detail and overview, or when the router is unsure

Yes, LOCAL and GLOBAL, plus follow-up questions

PATH

"How is A related to B?"

Yes, looks for the chain of connections between two entities

DIRECT

Thanks, small talk, "shorter", "as a table"

No. Uses only the conversation so far

ATTACHMENT

A question that a file attached to it answers alone

No. Uses only the attached file

You can switch DIRECT off. ATTACHMENT has no switch. The chat window does not show which way was used. If you need to know, look at the Topic’s Logs or DEV Reports tab.

Both LOCAL and GLOBAL can search by meaning only (the standard, "dense") or by meaning and keywords. You choose separately for LOCAL and for GLOBAL on step 3. Both start with meaning only.

Keyword search finds exact codes, IDs and names that search by meaning can miss ("what does error E204 mean"). Switch it on for LOCAL if users ask about named things or codes. The cost is a little more work per question and an extra index in the Topic’s own storage. GLOBAL gains less from it, because it searches summaries, not exact terms.

LOCAL

Setting What it does If you change it

Top-K Nodes (standard 6)

How many entities the search starts from. It does not limit how much is found from there

Too low: a relevant entity may never be reached. Higher: more starting points, slower

Min. Similarity Score (empty = no limit)

How well an entity must match the question to be a starting point, from 0 to 1

Higher: fewer, better matches. Too high: nothing is found

Hop Depth (standard 2)

How many connections the search follows from each starting entity

Higher: reaches more distant facts, but is slower and less focused

Max Subgraph Nodes (0 = 7 × Top-K Nodes)

The most entities one search may collect in total

Higher: more facts for the model, but slower, and the model needs room for longer input

Max Cited Chunks (standard 5, up to 50)

How many text passages the answer is written from, and so how many sources it can show

Lower: shorter source lists and more focused answers. Higher: more complete, but longer source lists

Seed Strategy (standard FUSED)

How the starting entities are found (see below)

See below

Top-K Seed Chunks (standard 8)

How many matching text passages are used to find starting entities. CHUNKS and FUSED only

Higher: more starting points, broader but slower

Top-K Unlinked Chunks (standard 2, 0 = off)

Extra text passages that have no entity in the graph, for example tables the extraction missed. CHUNKS and FUSED only

Higher: more chance that such passages help. 0: they are never used

Path Max Hops (standard 4)

For PATH questions: the longest chain of connections searched

Higher: finds distant links, but is slower

If lowering Top-K Nodes does not shorten the source list, this is why: Top-K Nodes only affects the first step, the search for starting entities. Everything after that has its own limit. Max Cited Chunks is the setting that bounds the sources. Lower it when answers cite more files than are useful.

Seed Strategy in plain words:

  • NODES starts from entity names. It suits questions that name something.

  • CHUNKS starts from text passages that match the question. It suits questions worded like your documents, for example steps or values.

  • FUSED does both. It is a little slower and is the standard.

HYBRID follows the same seed strategy for its LOCAL half.

GLOBAL

Setting What it does If you change it

Top-K Reports (standard 10)

How many community summaries the answer is built from

Higher: broader answers, but slower. It is the only size limit GLOBAL has

Min. Similarity Score (empty = no limit)

How well a summary must match the question, from 0 to 1

Higher: fewer, more relevant summaries

GLOBAL answers show no document sources, because they come from summaries. The minimum scores of LOCAL and GLOBAL are separate settings. A good value for one is not automatically good for the other.

HYBRID

Setting What it does If you change it

Max Follow-up Rounds (standard 2)

How many times HYBRID may ask follow-up questions and search again

Higher: more complete answers, but each round costs time and model calls

Max Follow-up Questions (standard 3)

Follow-up questions per round

Rounds × questions is the most extra searches one chat message can cause

Follow-up Min. Score (empty = no limit)

How useful a follow-up question must be rated to be searched

Higher: fewer follow-ups, faster

Top-K Chunks (0 = standard value)

Extra text passages found directly, next to what the graph gives

Higher: more context, but longer input

Min. Similarity Score (empty = no limit)

How well those passages must match the question

Higher: fewer, more relevant passages

HYBRID is the slowest and most complete way. Its LOCAL half uses the same settings as LOCAL above.

The router’s settings

These decide which way runs. They are in the Router section of step 4.

Setting What it does If you change it

Confidence Threshold (standard 0.6)

How sure the router must be for its choice between the searching ways to be used. If it is less sure, the question goes to HYBRID

Higher: more questions go to HYBRID, which misses the least but is slowest. Lower: the router’s choice is used more often, so answers are faster

Answer without searching (DIRECT) (standard on)

Lets the router answer thanks, small talk and requests like "shorter" without a search

Off: every message is searched

DIRECT Confidence Threshold (standard 0.9)

How sure the router must be before it answers without a search

Lower: more messages are answered directly and faster, but a factual question may be answered from the conversation only. Higher: more searches, safer but slower

Unsure questions always go to HYBRID. You cannot change this. The Confidence Threshold applies even when the router uses the Topic’s Language Model. Small router models often say they are very sure about everything. Then the threshold changes nothing. Look at the Topic’s logs before you change it.

Communities — the summaries behind GLOBAL

Set on step 3, next to the search-type choice.

Setting What it does If you change it

Community Resolution (standard 1.0, useful range 0.5 to 2.0)

How finely the graph is split into communities

Higher: more and smaller communities, more detailed summaries, more model calls. Lower: fewer and larger communities, broader summaries

Refresh Cycle

When all communities and summaries are rebuilt (a schedule with six fields, with quick choices)

More often: fresher GLOBAL answers, but more model calls. New documents appear in GLOBAL answers only after the next rebuild

You cannot switch communities off. A Topic without summaries would still get overview questions and have nothing to answer them from. To rebuild right away instead of waiting, use the Actions tab (see Configuration and functions).

Related