REST Interface — File Upload and Chat

A Topic can be driven programmatically: upload documents and ask questions from a script, without a browser session. This is how you automate ingestion or embed a Topic into another application.

To create, start, stop or delete Topics (and GraphRAG Topics and Agents) programmatically, use the Management REST API instead — it runs on the management app and is authenticated with personal access tokens.

Two hosts are involved, and confusing them is the main source of trouble:

Host Port Used for

Management application

8080 (or APP_PORT)

Getting a token — POST /api/auth/token

The Topic itself

Its own address (see section 8)

Chat, upload, file list, downloads

The token comes from the management app. Every other call goes directly to the Topic. This works because each Topic container gets the same signing secret, so the Topic can check the token on its own. It does not ask the management app. A token therefore keeps working for its whole lifetime, even while the management app is down.


1. Prerequisites

Requirement Notes

A user with the API_ACCESS permission

Grant it under Admin → Roles & Permissions. Without it the token endpoint returns 403

The permission for each call

TOPIC_CHAT for chat, TOPIC_UPLOAD for upload, TOPIC_VIEW for the lists, TOPIC_EDIT for deleting a file. See the table in section 3

That user’s role has access to the Topic

Ticked under Roles & Permissions → Resource Access. The token lists the Topics the user may use, and the Topic refuses everyone else with 403

The Topic is RUNNING

A stopped Topic has nothing listening


2. Obtain a token

With user name and password

curl -s -X POST http://localhost:8080/api/auth/token \
  -H 'Content-Type: application/json' \
  -d '{"username": "api-user", "password": "your-password"}'

With a token from your login provider (SSO users)

Users who sign in with SSO have no password in this app. They send a token from the login provider instead. The answer is the same. Setup and rules: OIDC Settings — API access for SSO users.

curl -s -X POST http://localhost:8080/api/auth/token \
  -H 'Content-Type: application/json' \
  -d '{"idpToken": "<token from your login provider>"}'

Response

{
  "token": "eyJhbGciOiJSUzI1NiIsImtpZCI6Ii4uLiJ9...",
  "tokenType": "Bearer",
  "expiresIn": 3600,
  "username": "api-user",
  "roles": ["API"],
  "permissions": ["API_ACCESS", "TOPIC_CHAT", "TOPIC_UPLOAD", "TOPIC_VIEW"],
  "topics": [
    { "id": "3f8b2c1a-...", "name": "HR_Assistant", "status": "RUNNING",
      "publicUrl": "https://hr-assistant.kf.example.com" }
  ],
  "graphRagTopics": [],
  "agents": [],
  "pipelines": []
}

The response is also a discovery call. topics, graphRagTopics, agents and pipelines list exactly what this user may use, with IDs, names and the current status (pipelines have no status). A script can find its Topic here instead of hard-coding a UUID.

publicUrl is the address where the Topic is published for callers outside the server or cluster. Use it as the base URL for all Topic calls from outside. It is null when the administrator has not set a public address for this Topic — then use one of the addresses in section 8. See Public Endpoints.

The same token works for Topics, GraphRAG Topics and Agents.

expiresIn is in seconds (1 hour by default, JWT_EXPIRY_HOURS). Keep the token and request a new one when it expires. The token also stores the user’s permissions and Topic access at the moment it was issued. After a change to the user’s roles, request a new token.

Code Meaning

200

Token issued

400

Malformed body, or user name or password missing. For idpToken: the token exchange is not set up

401

Unknown user or wrong password (also for SSO users, who have no password here). For idpToken: the token was refused

403

Account disabled or locked, or the user lacks API_ACCESS

429

Too many failed attempts for this user or from this address (5 per user, 50 per address within 15 minutes by default). Wait 15 minutes; the credentials are not checked meanwhile

503

Only for idpToken: the login provider could not be reached

TOKEN=$(curl -s -X POST http://localhost:8080/api/auth/token \
  -H 'Content-Type: application/json' \
  -d '{"username":"api-user","password":"your-password"}' | jq -r .token)

3. The Topic’s endpoints

All calls go to the Topic’s base URL with Authorization: Bearer <token>. The examples below assume a Topic on the Docker host with RAG Service Port 8081, so the base URL is localhost:8081.

Method Path Permission Purpose

POST

/chatbot/stream

TOPIC_CHAT

Ask a question, get the answer word by word (multipart, Server-Sent Events)

POST

/chatbot/chat

TOPIC_CHAT

Ask a question, get the whole answer at once (multipart)

POST

/chatbot/chat/abort?memoryId=…

TOPIC_CHAT

Try to cancel a running /chatbot/chat call

POST

/embeddings/embed

TOPIC_UPLOAD

Upload and index a document (multipart)

GET

/embeddings/file/list

TOPIC_VIEW

List indexed file names

GET

/embeddings/chunk/list

TOPIC_VIEW

List all stored chunks

GET

/embeddings/file/download?fileName=…

— (token or downloadToken)

Download an indexed file or a stored picture

DELETE

/embeddings/file?fileName=…

TOPIC_EDIT

Remove a file and its chunks

Status codes that every call can return:

Code Meaning

401

No token, or the token is invalid or expired. Request a new one

403

The token lacks the permission for this call, or the user was not granted this Topic

503

A service the Topic needs is unreachable (model server, Docling, …​), or the Topic is misconfigured. The body says which. Retrying the same call is safe


4. Chat

Both chat endpoints take multipart/form-data with the same fields:

Field Type Notes

memoryId

text, required

The conversation. Reuse it to keep context; use a new one to start over

prompt

text, required

The question

files

file, repeatable, optional

Attachments for this one question (see Attachments)

The fields are form fields, not JSON. A JSON body is refused.

4a. Streaming — POST /chatbot/stream

This is what the chat window uses. The answer arrives piece by piece, so a user sees text after a moment instead of waiting for the whole answer.

curl -N -s -X POST http://localhost:8081/chatbot/stream \
  -H "Authorization: Bearer $TOKEN" \
  -F 'memoryId=session-42' \
  -F 'prompt=How many vacation days do I get after five years?'

The response is a stream of Server-Sent Events (text/event-stream). Each event has a name (event:) and data (data:):

Event Data What to do with it

retrieved

{"sourceFiles": […​]}

The documents found for the question. Can be an empty list. Only a preview — use the list from done

token

Plain text, not JSON

The next piece of the answer. Append it to what you have

tool_start

{"toolName": …​, "args": …​}

The Topic is asking another connected Topic. Can be ignored

tool_result

{"toolName": …​, "result": …​}

That Topic answered. Can be ignored

done

{"inputTokens": …​, "outputTokens": …​, "sourceFiles": […​], "notices": […​]}

The answer is complete. sourceFiles is the final list of sources, including pictures. notices see below. The two counts can be missing

error

{"message": "…​"}

The answer failed after the stream had started. The stream ends here

How token data is framed. A text piece can contain line breaks. The Topic sends each line of the piece as its own data: line. To rebuild the piece, join all data: lines of one event with a line break (\n). Take the text after data: exactly as it is — do not remove a leading space. A space there is part of the answer, for example the gap between two words.

event:token
data:Vacation

event:token
data: days:

event:token
data:
data:- 30 days after five years

The third event is a line break followed by - 30 days after five years. The answer so far is Vacation days:\n- 30 days after five years.

Status codes. Problems found before the answer starts come back as a normal HTTP status: 400 (empty memoryId or prompt), 413, 422 (see Attachments) and 503. Once the stream has started, the status is already 200. A later failure arrives as the error event.

4b. Whole answer — POST /chatbot/chat

curl -s -X POST http://localhost:8081/chatbot/chat \
  -H "Authorization: Bearer $TOKEN" \
  -F 'memoryId=session-42' \
  -F 'prompt=How many vacation days do I get after five years?'
{
  "answer": "After five years of service you are entitled to 30 days.",
  "sourceFiles": [
    {
      "fileName": "hr-policy.pdf",
      "origin": "DOCUMENT",
      "downloadToken": "eyJhbGciOiJIUzI1NiJ9...",
      "caption": null,
      "page": null,
      "documentFileName": null,
      "downloadPath": "/embeddings/file/download?fileName=hr-policy.pdf&token=eyJhbGciOiJIUzI1NiJ9...",
      "downloadUrl": "https://hr-assistant.kf.example.com/embeddings/file/download?fileName=hr-policy.pdf&token=eyJhbGciOiJIUzI1NiJ9...",
      "resourceId": "3f8b2c1a-..."
    }
  ],
  "notices": []
}

The call returns only when the whole answer is ready. This can take a while, so set a generous client timeout. Status codes: 200 answer · 400 empty memoryId or prompt · 413 / 422 see Attachments · 503 the chat model’s server is unreachable.

Sources (sourceFiles)

sourceFiles lists what the answer was built from — the same as the Sources in the chat window. It is the same list in the done event and in the /chatbot/chat response. It is never null.

Field Meaning

fileName

The uploaded file name. For a picture: the picture’s ID (img_ followed by 16 hex characters)

origin

DOCUMENT — an uploaded file. IMAGE — a stored picture. GRAPH_SYNC — a record from a connected GraphRAG Topic; there is nothing to download

downloadToken

A key for downloading this one file, valid for about 10 minutes. See 6a

caption, page, documentFileName

Pictures only: the caption, the page (counted from 1) and the document the picture comes from. null for documents

downloadPath

The download link on the Topic, without host, token included. null when there is no downloadToken

downloadUrl

The complete download link from outside: the Topic’s public address plus downloadPath. null when the Topic has no public address

resourceId

The ID of the Topic that found this source

Documents come first, pictures after them.

Sources from other Topics. A Topic can ask other Topics it is connected to (MCP Interfaces). Their sources appear in the same list, with their resourceId, and their links and tokens work only on that other Topic. Use downloadUrl, or match resourceId against the topics list from the token response.

Pictures in the answer

When the Topic stores pictures from documents, the answer can show them. The answer text then contains a marker on its own line:

{{image:img_0123456789abcdef}}

The ID matches a sourceFiles entry with origin IMAGE. Download the picture like any source file (see 6a) and show it at that place. If you do not show pictures, remove the markers, for example with the pattern \{\{image:[A-Za-z0-9_\-]+\}\}. sourceFiles lists only pictures that the answer places with a marker. While streaming, a marker always arrives in one token event, never split.

Notices

notices explains gaps in an answer. Example: the Topic asked another connected Topic, but this user may not use it. That Topic is then left out, and the answer is still returned.

{ "kind": "ACCESS_DENIED", "source": "Finance", "message": "You have no permission to use the topic \"Finance\", so it was left out of this answer." }

kind is ACCESS_DENIED (no permission) or UNAUTHENTICATED (the other Topic could not check the sign-in). message is an English sentence you can show to the user as it is.

Attachments

curl -s -X POST http://localhost:8081/chatbot/chat \
  -H "Authorization: Bearer $TOKEN" \
  -F 'memoryId=session-42' \
  -F 'prompt=Summarise the attached report.' \
  -F 'files=@quarterly-report.pdf'

Attachments count for this one question only. They are never added to the knowledge base. Repeat files to attach several files.

Code Meaning

413

A file is larger than the Topic’s Largest Chat Attachment setting (10 MB unless changed). Also when the model server refuses the question as too long for the model

422

A picture was attached, but the Topic’s language model cannot read pictures. Or a binary file (PDF, Word, …​) was attached, but the Topic has no Docling server

The Chat Attachments setting of the Topic (which file types are allowed) is checked only by the chat window. The Topic itself does not know it. A script can attach other types, and only the rules in the table above apply.

memoryId is the conversation

There is no session endpoint: the memoryId is the conversation. Reuse the same value and the Topic keeps the context (limited by the Context Memory setting). Change it and you start fresh.

The Topic keeps a separate history for each user. Two users with the same memoryId do not share a conversation. Do not use the value -1: it is reserved for Topic-to-Topic calls and keeps no history.

Aborting

curl -s -X POST "http://localhost:8081/chatbot/chat/abort?memoryId=session-42" \
  -H "Authorization: Bearer $TOKEN"

Returns 204, also when nothing was running. Only your own calls are affected.

The abort is a best effort, and a Topic usually cannot really stop an answer:

  • A waiting /chatbot/chat call is interrupted only if the Topic happens to be at a point where it can stop. It then answers 499. Otherwise the answer is still produced.

  • A streamed answer (/chatbot/stream) is not affected at all. Close the connection to stop reading. The Topic may still finish the answer in the background.


5. Upload a document

POST /embeddings/embed as multipart/form-data. Unlike chat attachments, this permanently indexes the document.

curl -s -X POST http://localhost:8081/embeddings/embed \
  -H "Authorization: Bearer $TOKEN" \
  -F 'file=@hr-policy.pdf'

Returns 200 with an empty body. The call is synchronous and can take a long time: conversion, chunking, embedding and optional enrichment all happen before it returns. Set a generous client timeout. A large PDF with a local embedding model can take minutes.

The file name identifies the document. Uploading the same name again replaces the old version.

Optional fields:

Field Sent as Default Meaning

strategy

query parameter

RECURSIVE

RECURSIVE or SEMANTIC. SEMANTIC adds one language-model call per chunk to add context to it — much slower

doclingOptions

multipart text field

—

Docling settings for this one document, as a JSON object. See below

renamedFrom

query parameter

—

The previous file name. The old chunks are deleted before the new file is indexed

url

query parameter

—

Where the file lives. Required when the Topic stores files externally (EXTERNAL mode); then the call returns 400 without it

The management app sends strategy from the Topic’s chunking strategy with every upload.

Docling options

curl -s -X POST "http://localhost:8081/embeddings/embed?strategy=SEMANTIC" \
  -H "Authorization: Bearer $TOKEN" \
  -F 'file=@hr-policy.pdf' \
  --form-string 'doclingOptions={"do_ocr": false, "ocr_lang": ["de", "en"]}'

doclingOptions changes how Docling reads this one document, for example whether it runs text recognition (OCR) on scanned pages. The keys are Docling’s own option names. The Topic passes them on and does not remember them: send them with every upload, or a re-upload is read with Docling’s defaults. The management app sends the Topic’s Docling options with every upload for this reason.

  • It must be a plain JSON object, without comments. Otherwise the call returns 400.

  • Some options are set by the Topic itself and are refused with 400, for example to_formats, image_export_mode, include_images and the picture-description options.

  • Pictures and text files are not read by Docling, so the options have no effect on them.

Use --form-string with curl. Plain -F gives special meaning to some characters in the value.

Upload status codes

Code Meaning

200

Indexed

400

File missing, url missing in EXTERNAL mode, or invalid doclingOptions

503

A service the upload needs is unreachable — Docling, the embedding model, or an enrichment model. Retrying the same file is safe

Ingesting a directory

for f in ./docs/*.pdf; do
  echo "Ingesting $f"
  curl -s -f -X POST http://localhost:8081/embeddings/embed \
    -H "Authorization: Bearer $TOKEN" \
    -F "file=@$f" \
    || echo "FAILED: $f"
done

Upload one file after the other, not in parallel: each file keeps the embedding model busy, and parallel requests mostly end in timeouts. The UI does the same.


6. Inspecting and removing content

# Which files are indexed?
curl -s http://localhost:8081/embeddings/file/list -H "Authorization: Bearer $TOKEN"
["hr-policy.pdf", "onboarding.md"]
# What can actually be found? (chunk text + metadata)
curl -s http://localhost:8081/embeddings/chunk/list -H "Authorization: Bearer $TOKEN" | jq '.[0]'

# Remove a file and all its chunks (needs TOPIC_EDIT)
curl -s -X DELETE "http://localhost:8081/embeddings/file?fileName=hr-policy.pdf" \
  -H "Authorization: Bearer $TOKEN"

The delete returns true if the file was found and deleted, false if there was nothing for that name. The chunk list shows the same as the Chunks tab on the Topic’s Data Upload tab. It is the fastest way to check from a script that a document can really be found.

A file removed here is gone from the Topic, but it may still be listed on the Database tab of the Topic’s Data Upload tab. That tab shows what was submitted, not what is indexed now.

6a. Downloading a source file

# With your normal token — the usual case for a script
curl -s -o hr-policy.pdf \
  "http://localhost:8081/embeddings/file/download?fileName=hr-policy.pdf" \
  -H "Authorization: Bearer $TOKEN"

# With the downloadToken from a chat answer — no header, so this link also works in a browser
curl -s -o hr-policy.pdf \
  "http://localhost:8081/embeddings/file/download?fileName=hr-policy.pdf&token=$DOWNLOAD_TOKEN"

URL-encode both parameters. downloadPath and downloadUrl in sourceFiles already contain the encoded link with the token.

For a picture, use its ID as fileName, for example fileName=img_0123456789abcdef.

Code Meaning

200

The file or picture

302

Redirect to the file’s external location (EXTERNAL mode)

400

fileName is empty

401

No token at all, or the downloadToken is expired (after about 10 minutes) or belongs to another file. Ask the question again to get a new one, or send your normal token

404

No file or picture with this name

A downloadToken works only on the Topic that issued it.


7. A complete example

#!/usr/bin/env bash
set -euo pipefail

APP="http://localhost:8080"
TOPIC="http://localhost:8081"

TOKEN=$(curl -s -X POST "$APP/api/auth/token" \
  -H 'Content-Type: application/json' \
  -d '{"username":"api-user","password":"your-password"}' | jq -r .token)

# Ingest
curl -s -f -X POST "$TOPIC/embeddings/embed" \
  -H "Authorization: Bearer $TOKEN" \
  -F 'file=@hr-policy.pdf'

# Verify it is indexed
curl -s "$TOPIC/embeddings/file/list" -H "Authorization: Bearer $TOKEN" | jq .

# Ask
curl -s -X POST "$TOPIC/chatbot/chat" \
  -H "Authorization: Bearer $TOKEN" \
  -F 'memoryId=script-run-1' \
  -F 'prompt=How many vacation days after five years?' | jq -r '.answer, .sourceFiles[].fileName'

Python

import json
import requests

APP, TOPIC = "http://localhost:8080", "http://localhost:8081"

token = requests.post(f"{APP}/api/auth/token",
                      json={"username": "api-user", "password": "your-password"},
                      timeout=30).json()["token"]
headers = {"Authorization": f"Bearer {token}"}

# Upload — allow plenty of time; indexing is synchronous
with open("hr-policy.pdf", "rb") as fh:
    requests.post(f"{TOPIC}/embeddings/embed", headers=headers,
                  files={"file": ("hr-policy.pdf", fh)},
                  data={"doclingOptions": json.dumps({"do_ocr": False})},  # optional
                  timeout=600).raise_for_status()

# Ask — whole answer at once
form = {"memoryId": (None, "py-1"),
        "prompt": (None, "How many vacation days after five years?")}
# To attach a file, add: form["files"] = ("report.pdf", open("report.pdf", "rb"))
r = requests.post(f"{TOPIC}/chatbot/chat", headers=headers, files=form, timeout=300)
r.raise_for_status()
answer = r.json()
print(answer["answer"], [s["fileName"] for s in answer["sourceFiles"]])

The chat endpoints accept only multipart/form-data. With requests, data= alone sends a plain form, and json= sends JSON — the Topic refuses both with 415. Passing the fields through files= as (None, value) sends them as multipart form fields.

Python — streaming

import json
import requests

def sse_events(response):
    """Yields (event, data) pairs. Joins multi-line data with \n and keeps leading spaces."""
    event, data = None, None
    for line in response.iter_lines(decode_unicode=True):
        if line == "":                       # a blank line ends an event
            if data is not None:
                yield event or "message", "\n".join(data)
            event, data = None, None
        elif line.startswith(":"):           # comment line
            continue
        else:
            field, _, value = line.partition(":")
            if field == "event":
                event = value.strip()
            elif field == "data":
                data = (data or []) + [value]  # value exactly as sent

form = {"memoryId": (None, "py-2"),
        "prompt": (None, "How many vacation days after five years?")}
with requests.post(f"{TOPIC}/chatbot/stream", headers=headers, files=form,
                   stream=True, timeout=300) as r:
    r.raise_for_status()
    r.encoding = "utf-8"                     # the stream has no charset; requests would guess wrong
    answer = ""
    for event, data in sse_events(r):
        if event == "token":
            answer += data
            print(data, end="", flush=True)
        elif event == "done":
            done = json.loads(data)
            print("\nSources:", [s["fileName"] for s in done["sourceFiles"]])
            for notice in done["notices"]:
                print("Note:", notice["message"])
        elif event == "error":
            raise RuntimeError(json.loads(data)["message"])

8. Reaching the Topic

If the token response shows a publicUrl for the Topic, use it from outside. Otherwise:

Caller Base URL

Script on the Docker host

localhost:<RAG Service Port>

Another container, Docker Desktop

host.docker.internal:<RAG Service Port>

Another container, plain Linux Docker

172.17.0.1:<RAG Service Port>

Elsewhere on the network (Docker)

<host>:<RAG Service Port> — the port must be open in the firewall

Inside the Kubernetes cluster

topic-<topicId>.<namespace>.svc.cluster.local:8080

On Docker, the Topic’s RAG Service Port is the port on the host. On Kubernetes this port is not used. There the Topic is reachable inside the cluster only, unless the administrator publishes it (see Public Endpoints).


9. Troubleshooting

Symptom Cause and fix

403 from /api/auth/token

The user lacks API_ACCESS, or is disabled or locked

401 from the token endpoint

Unknown user name or wrong password. SSO users have no password here — use idpToken

Token works, Topic calls return 401

The token expired (8 h by default). Request a new one

Topic calls return 403

The user lacks the permission for this call, or was not granted this Topic. After changing roles, request a new token

Every Topic call returns 503 with "misconfigured"

The Topic container was started without its Topic ID, so it cannot check access grants. Stop and start the Topic from the management app, and check the Topic’s log

401 from /embeddings/file/download

The downloadToken is older than about 10 minutes or belongs to another file. Ask again, or send your normal Authorization header instead

topics list is empty in the token response

The user’s role has no access to any Topic. Grant it under Roles & Permissions → Resource Access

publicUrl is null

No public address is set for this Topic. See Public Endpoints

Connection refused on the Topic port

The Topic is not RUNNING, or you used the wrong host or port

415 on chat

The body was JSON. Send form fields (multipart)

413 on chat with an attachment

The file is larger than the Topic’s Largest Chat Attachment setting (10 MB unless changed), or the question with the file is too long for the model

422 on chat with an attachment

A picture without a picture-capable language model, or a binary file without Docling

Streamed answer loses text after a blank line, or words stick together

The client does not join multi-line data: with \n, or it strips the space after data:. See 4a

{{image:...}} shows up in the answer text

A picture marker. Show the picture from sourceFiles, or remove the marker. See Pictures in the answer

400 on upload mentioning doclingOptions

Not a JSON object, contains comments, or sets an option the Topic controls itself

Upload times out on the client side

Indexing is synchronous. Raise the client timeout (minutes for large files with local models)

Parallel uploads mostly fail

Upload one file after the other

Answers ignore an uploaded file

Check /embeddings/chunk/list. If the file is not there, indexing failed

Every answer is the no-answer text

Nothing indexed, or the minimum similarity score filters everything out

Context is not kept between calls

Send the same memoryId, with the same user. A new value starts a new conversation