Knowledge Bases (RAG)
Knowledge bases let your applications answer from your own documents. You upload files once; the platform extracts their text, splits it into chunks, indexes them for semantic and keyword search, and keeps the index current as you replace or delete files. You then either search a knowledge base directly, or chat with one or more knowledge bases through the normal POST /v1/chat/completions endpoint and get an answer with numbered citations.
Everything is available through the API and in the portal (Knowledge bases in the sidebar). The portal uses the same API and permissions; it has no extra powers.
| Page | What it covers |
|---|---|
| Quickstart | API key, create, upload, poll, search, chat — curl, Python and JavaScript |
| Website Sources | Not available yet. Index a website: preview, scope, InfusBot and robots.txt, sync runs, the deletion hold, limits, pricing and responsibility |
| Metadata and Search Filters | Metadata rules, filter operators, search parameters |
| Chat with Citations | knowledge_base_ids, retrieval, the citations field and the streaming first chunk |
| Webhooks | The knowledge events and their payload. Setup, signing and retries are on the platform Webhooks page |
| Data Lifecycle and Privacy | Export, deletion, retention schedules, what is and is not stored |
| Pricing and Billing | Free trial, then pay-as-you-go for every organization; units, prices, spend cap |
| Error Codes | Every reason code with status and retry advice |
| Knowledge API Reference | Every endpoint and schema |
Knowledge bases are a platform module. GET /v1/knowledge_module tells you whether it is on, your effective limits, your free-trial allowance and your prices. When your platform operator switches the module off:
- An organization without knowledge bases gets
404 module_disabledon every knowledge route (and on chat withknowledge_base_ids), and the portal hides the pages. - An organization that still has knowledge bases keeps its data read and export only: listing, reading, search, grounded chat, downloads, exports (new ones too), metadata edits, cancelling and deleting keep working. Creating knowledge bases, uploads, file replacements and retries are refused with
403 module_disabled.GET /v1/knowledge_modulethen answers200with"enabled": false, "read_only": true. - Billing does not change: search and grounded chat are billed as usual, and stored documents stay billed (GB-month) until you delete them. Exports stay free — export your data, then delete what you no longer need.
Concepts
Knowledge base
A named collection of documents that belongs to your organization. Knowledge bases are organization-wide:
- Every member of the organization can list, open, search and chat with every knowledge base.
- Only admins and owners can create, rename and delete knowledge bases, and upload, replace, re-tag or delete documents.
- API keys need an explicit knowledge scope, and each key can be limited to a list of knowledge bases — see API keys and access.
Creating a knowledge base is free; it counts towards your knowledge-base limit.
| Status | Meaning | Search / upload | List, download, export, delete |
|---|---|---|---|
active | Normal | Open | Open |
frozen | A free-trial knowledge base was idle too long (status_reason: trial_idle) | Closed (409 trial_kb_frozen) | Open |
suspended | Paid work stopped because of a payment problem (payment_suspended) | Closed (409 kb_suspended) | Open |
deleting | Deletion accepted; it no longer appears in lists or search | Closed | — |
deletion_scheduled_at is set while a retention schedule will delete the knowledge base. See Data Lifecycle and Privacy.
Embedding model
Every knowledge base uses one embedding model: the model that turns its documents, and the queries you search it with, into vectors. You choose it when you create the knowledge base:
GET /v1/knowledge_embedding_modelslists the models your platform offers for knowledge bases, with each model's name, vector size (dimensions) and processing price. Every model in the list can be used by your organization, whether or not you also use that model for inference.- Exactly one model is recommended. It is used when you create a knowledge base without choosing one, and the portal preselects it.
- Send the chosen entry's
idasembedding_modelonPOST /v1/knowledge_bases. Every knowledge base response shows its model underembedding_model. - The model cannot be changed later. A
PATCHthat namesembedding_modelis refused with422 validation_failed(details.reason: immutable). To use another model, create a new knowledge base and upload the documents there. - Processing (embedding your documents) is billed at the price of the knowledge base's model — see Processing.
In the portal, the Create a knowledge base dialog has an Embedding model field with the recommended model preselected and marked Recommended; each option shows its dimensions and price per 1M tokens. When only one model is offered, it is shown on its own. The knowledge base page shows the model in its facts line (Embedding model: …).
Document
One uploaded file (or one page of a website source, origin: web) plus its metadata. Each document moves through these statuses:
uploaded → validating → queued → processing → ready
| Status | Portal label | Meaning |
|---|---|---|
uploaded | Uploading | The file is stored; checks have not started. |
validating | Validating | Structural checks run (type, signature, encryption, macros, archive size). |
queued | Queued | Waiting for a processing slot. |
processing | Processing | Text extraction, chunking, embedding and indexing. |
ready | Ready | Searchable. |
awaiting_budget | Awaiting budget | Neither the free trial nor pay-as-you-go (balance or credit) could cover processing yet. The document waits up to 24 hours and is retried automatically (and on POST …/retry); after that it fails with awaiting_budget_timeout. |
failed | Failed | failure holds {code, message, retryable}. Retryable failures can be retried with POST …/retry; platform retries are never charged. |
cancelled | Cancelled | You cancelled processing before the first version became ready. |
deleting | Deleting | Deletion accepted; the document is already out of search. |
202 Accepted on upload means the platform has durably accepted the file, not that it is processed. Poll the document, list documents with ?status= / ?updated_since=, or subscribe to webhooks.
Versions are internal
Replacing a file (PUT …/documents/{document_id}/file) creates a new internal version. The current ready version stays searchable until the new one is ready, then the switch is atomic — search never mixes old and new text. While the replacement is processed the document shows it under pending_version. If the new file fails, the old one stays ready. Only one replacement can run at a time (409 version_conflict), and version history is never exposed.
The temporary second copy is not billed as storage; processing the new file is billed like any upload.
Chunks and locations
Each ready document is split into chunks. GET …/documents/{document_id}/chunks returns a read-only preview of the extracted text, in order, so you can check extraction quality (tables, multi-column pages) yourself. Every chunk and search hit carries a location that fits the file type:
| File type | Location fields |
|---|---|
page | |
| PPTX | slide, slide_title |
| XLSX | sheet, row_start, row_end |
| CSV | row_start, row_end |
| TXT | line_start, line_end |
| DOCX, MD, HTML | heading_path |
Supported files
Version 1 accepts PDF, DOCX, PPTX, XLSX, CSV, HTML, TXT and MD.
A file is rejected — with a reason code, without affecting the other files of the same upload — when:
| Code | Reason |
|---|---|
unsupported_file_type | Any other type (legacy .doc/.xls/.ppt, images, archives, …). |
file_type_mismatch | Extension, declared MIME type and file signature disagree. |
corrupt_file | The file cannot be read structurally. |
encrypted_file | Password-protected or encrypted PDF or Office file — including PDFs that only have an owner (permissions) password. Remove the protection and upload again. |
macro_file | Office file containing macros, including macro-enabled files renamed to .docx/.xlsx/.pptx. |
archive_bomb | A DOCX/PPTX/XLSX expands beyond the unpacked size or entry-count ceiling. |
no_text_content | No extractable text, typically a scanned PDF. OCR is not part of version 1. |
extraction_limit_exceeded | Extracted text, cells, rows or chunks exceed the processing ceilings — including a PDF with more than 200 pages and a spreadsheet or CSV with more data rows than limits.max_rows_per_document (default 50,000). The document fails; it is never half-indexed. |
file_too_large | Larger than 50 MB. |
Some content is removed rather than rejected, and the document gets a warnings entry:
| Warning | Meaning |
|---|---|
content_removed_active_elements | HTML scripts, iframes and external resources, or PDF JavaScript and embedded attachments, were dropped from the extracted text. |
tables_flattened | Tables were converted to text with their column headers. |
low_extraction_quality | Extraction looks poor (for example complex multi-column layouts). Check the chunk preview. |
The rows_truncated warning code still exists in the schema for compatibility but is no longer emitted: an oversized spreadsheet fails with extraction_limit_exceeded instead.
Uploading the same content (same SHA-256) twice into one knowledge base is refused per file with 409 duplicate_content unless you send on_duplicate=allow. The error carries details.existing_document_id. Identical content in different knowledge bases is always allowed.
Uploaded files are not scanned for viruses or malware, and their content is not checked for personal data. Only the structural checks above run. The original file is stored as uploaded and returned as-is on download. Upload only files you trust and are allowed to process. See Data Lifecycle and Privacy.
Limits
Platform defaults are below. Your effective values come from GET /v1/knowledge_module → limits; they are the same for every organization (a plan does not change them).
| Limit | Default | Error when exceeded |
|---|---|---|
| File size | 50 MB (50,000,000 bytes) | 413 file_too_large (per file) |
| Files per upload request | 50 | 413 too_many_files |
| Upload request size | 500 MB | 413 request_too_large |
| Knowledge bases per organization | 20 | 409 resource_limit_reached |
| Uploaded documents per knowledge base | 1,000 (website pages count against their source's page cap instead: 100 per source on an unverified domain) | 409 resource_limit_reached |
| Query length | 2,000 characters | 422 query_too_long (search); chat truncates the retrieval query instead |
Results per search (top_k) | 1–20, default 5 | 422 validation_failed |
| Knowledge bases per chat request | 5 | 422 too_many_knowledge_bases |
| Metadata keys per document | 16 | 422 metadata_too_many_keys |
| Ids per batch operation | 100 | 422 batch_too_large |
| Running exports per knowledge base | 1 | 409 export_in_progress |
| Processing time per document | 1 hour | document fails with processing_limit_exceeded |
| Pages per PDF document | 200 | document fails with extraction_limit_exceeded |
Data rows per XLSX/CSV document (max_rows_per_document) | 50,000 | document fails with extraction_limit_exceeded |
Embedding tokens per document (max_tokens_per_document) | 3,000,000 | document fails with processing_limit_exceeded |
Query and upload rates are limited per organization; a burst above them gets 429 rate_limited or 429 concurrency_limited with a Retry-After header. The free trial also caps the number of knowledge bases (402 trial_exhausted) — see Pricing and Billing.
Maintenance mode
During platform maintenance, uploads, file replacements, retries and exports are refused with 503 maintenance_mode and Retry-After: 300. Search, chat, downloads and deletions keep working. GET /v1/knowledge_module reports "maintenance": true while it lasts.