Skip to main content

Knowledge Bases (RAG)

Knowledge bases let your applications answer from your own documents. You upload files once; the platform extracts their text, splits it into chunks, indexes them for semantic and keyword search, and keeps the index current as you replace or delete files. You then either search a knowledge base directly, or chat with one or more knowledge bases through the normal POST /v1/chat/completions endpoint and get an answer with numbered citations.

Everything is available through the API and in the portal (Knowledge bases in the sidebar). The portal uses the same API and permissions; it has no extra powers.

PageWhat it covers
QuickstartAPI key, create, upload, poll, search, chat — curl, Python and JavaScript
Website SourcesNot available yet. Index a website: preview, scope, InfusBot and robots.txt, sync runs, the deletion hold, limits, pricing and responsibility
Metadata and Search FiltersMetadata rules, filter operators, search parameters
Chat with Citationsknowledge_base_ids, retrieval, the citations field and the streaming first chunk
WebhooksThe knowledge events and their payload. Setup, signing and retries are on the platform Webhooks page
Data Lifecycle and PrivacyExport, deletion, retention schedules, what is and is not stored
Pricing and BillingFree trial, then pay-as-you-go for every organization; units, prices, spend cap
Error CodesEvery reason code with status and retry advice
Knowledge API ReferenceEvery endpoint and schema
Availability

Knowledge bases are a platform module. GET /v1/knowledge_module tells you whether it is on, your effective limits, your free-trial allowance and your prices. When your platform operator switches the module off:

  • An organization without knowledge bases gets 404 module_disabled on every knowledge route (and on chat with knowledge_base_ids), and the portal hides the pages.
  • An organization that still has knowledge bases keeps its data read and export only: listing, reading, search, grounded chat, downloads, exports (new ones too), metadata edits, cancelling and deleting keep working. Creating knowledge bases, uploads, file replacements and retries are refused with 403 module_disabled. GET /v1/knowledge_module then answers 200 with "enabled": false, "read_only": true.
  • Billing does not change: search and grounded chat are billed as usual, and stored documents stay billed (GB-month) until you delete them. Exports stay free — export your data, then delete what you no longer need.

Concepts​

Knowledge base​

A named collection of documents that belongs to your organization. Knowledge bases are organization-wide:

  • Every member of the organization can list, open, search and chat with every knowledge base.
  • Only admins and owners can create, rename and delete knowledge bases, and upload, replace, re-tag or delete documents.
  • API keys need an explicit knowledge scope, and each key can be limited to a list of knowledge bases — see API keys and access.

Creating a knowledge base is free; it counts towards your knowledge-base limit.

StatusMeaningSearch / uploadList, download, export, delete
activeNormalOpenOpen
frozenA free-trial knowledge base was idle too long (status_reason: trial_idle)Closed (409 trial_kb_frozen)Open
suspendedPaid work stopped because of a payment problem (payment_suspended)Closed (409 kb_suspended)Open
deletingDeletion accepted; it no longer appears in lists or searchClosed—

deletion_scheduled_at is set while a retention schedule will delete the knowledge base. See Data Lifecycle and Privacy.

Embedding model​

Every knowledge base uses one embedding model: the model that turns its documents, and the queries you search it with, into vectors. You choose it when you create the knowledge base:

  • GET /v1/knowledge_embedding_models lists the models your platform offers for knowledge bases, with each model's name, vector size (dimensions) and processing price. Every model in the list can be used by your organization, whether or not you also use that model for inference.
  • Exactly one model is recommended. It is used when you create a knowledge base without choosing one, and the portal preselects it.
  • Send the chosen entry's id as embedding_model on POST /v1/knowledge_bases. Every knowledge base response shows its model under embedding_model.
  • The model cannot be changed later. A PATCH that names embedding_model is refused with 422 validation_failed (details.reason: immutable). To use another model, create a new knowledge base and upload the documents there.
  • Processing (embedding your documents) is billed at the price of the knowledge base's model — see Processing.

In the portal, the Create a knowledge base dialog has an Embedding model field with the recommended model preselected and marked Recommended; each option shows its dimensions and price per 1M tokens. When only one model is offered, it is shown on its own. The knowledge base page shows the model in its facts line (Embedding model: …).

Document​

One uploaded file (or one page of a website source, origin: web) plus its metadata. Each document moves through these statuses:

uploaded → validating → queued → processing → ready
StatusPortal labelMeaning
uploadedUploadingThe file is stored; checks have not started.
validatingValidatingStructural checks run (type, signature, encryption, macros, archive size).
queuedQueuedWaiting for a processing slot.
processingProcessingText extraction, chunking, embedding and indexing.
readyReadySearchable.
awaiting_budgetAwaiting budgetNeither the free trial nor pay-as-you-go (balance or credit) could cover processing yet. The document waits up to 24 hours and is retried automatically (and on POST …/retry); after that it fails with awaiting_budget_timeout.
failedFailedfailure holds {code, message, retryable}. Retryable failures can be retried with POST …/retry; platform retries are never charged.
cancelledCancelledYou cancelled processing before the first version became ready.
deletingDeletingDeletion accepted; the document is already out of search.

202 Accepted on upload means the platform has durably accepted the file, not that it is processed. Poll the document, list documents with ?status= / ?updated_since=, or subscribe to webhooks.

Versions are internal​

Replacing a file (PUT …/documents/{document_id}/file) creates a new internal version. The current ready version stays searchable until the new one is ready, then the switch is atomic — search never mixes old and new text. While the replacement is processed the document shows it under pending_version. If the new file fails, the old one stays ready. Only one replacement can run at a time (409 version_conflict), and version history is never exposed.

The temporary second copy is not billed as storage; processing the new file is billed like any upload.

Chunks and locations​

Each ready document is split into chunks. GET …/documents/{document_id}/chunks returns a read-only preview of the extracted text, in order, so you can check extraction quality (tables, multi-column pages) yourself. Every chunk and search hit carries a location that fits the file type:

File typeLocation fields
PDFpage
PPTXslide, slide_title
XLSXsheet, row_start, row_end
CSVrow_start, row_end
TXTline_start, line_end
DOCX, MD, HTMLheading_path

Supported files​

Version 1 accepts PDF, DOCX, PPTX, XLSX, CSV, HTML, TXT and MD.

A file is rejected — with a reason code, without affecting the other files of the same upload — when:

CodeReason
unsupported_file_typeAny other type (legacy .doc/.xls/.ppt, images, archives, …).
file_type_mismatchExtension, declared MIME type and file signature disagree.
corrupt_fileThe file cannot be read structurally.
encrypted_filePassword-protected or encrypted PDF or Office file — including PDFs that only have an owner (permissions) password. Remove the protection and upload again.
macro_fileOffice file containing macros, including macro-enabled files renamed to .docx/.xlsx/.pptx.
archive_bombA DOCX/PPTX/XLSX expands beyond the unpacked size or entry-count ceiling.
no_text_contentNo extractable text, typically a scanned PDF. OCR is not part of version 1.
extraction_limit_exceededExtracted text, cells, rows or chunks exceed the processing ceilings — including a PDF with more than 200 pages and a spreadsheet or CSV with more data rows than limits.max_rows_per_document (default 50,000). The document fails; it is never half-indexed.
file_too_largeLarger than 50 MB.

Some content is removed rather than rejected, and the document gets a warnings entry:

WarningMeaning
content_removed_active_elementsHTML scripts, iframes and external resources, or PDF JavaScript and embedded attachments, were dropped from the extracted text.
tables_flattenedTables were converted to text with their column headers.
low_extraction_qualityExtraction looks poor (for example complex multi-column layouts). Check the chunk preview.

The rows_truncated warning code still exists in the schema for compatibility but is no longer emitted: an oversized spreadsheet fails with extraction_limit_exceeded instead.

Uploading the same content (same SHA-256) twice into one knowledge base is refused per file with 409 duplicate_content unless you send on_duplicate=allow. The error carries details.existing_document_id. Identical content in different knowledge bases is always allowed.

No malware scanning, no PII detection

Uploaded files are not scanned for viruses or malware, and their content is not checked for personal data. Only the structural checks above run. The original file is stored as uploaded and returned as-is on download. Upload only files you trust and are allowed to process. See Data Lifecycle and Privacy.

Limits​

Platform defaults are below. Your effective values come from GET /v1/knowledge_module → limits; they are the same for every organization (a plan does not change them).

LimitDefaultError when exceeded
File size50 MB (50,000,000 bytes)413 file_too_large (per file)
Files per upload request50413 too_many_files
Upload request size500 MB413 request_too_large
Knowledge bases per organization20409 resource_limit_reached
Uploaded documents per knowledge base1,000 (website pages count against their source's page cap instead: 100 per source on an unverified domain)409 resource_limit_reached
Query length2,000 characters422 query_too_long (search); chat truncates the retrieval query instead
Results per search (top_k)1–20, default 5422 validation_failed
Knowledge bases per chat request5422 too_many_knowledge_bases
Metadata keys per document16422 metadata_too_many_keys
Ids per batch operation100422 batch_too_large
Running exports per knowledge base1409 export_in_progress
Processing time per document1 hourdocument fails with processing_limit_exceeded
Pages per PDF document200document fails with extraction_limit_exceeded
Data rows per XLSX/CSV document (max_rows_per_document)50,000document fails with extraction_limit_exceeded
Embedding tokens per document (max_tokens_per_document)3,000,000document fails with processing_limit_exceeded

Query and upload rates are limited per organization; a burst above them gets 429 rate_limited or 429 concurrency_limited with a Retry-After header. The free trial also caps the number of knowledge bases (402 trial_exhausted) — see Pricing and Billing.

Maintenance mode​

During platform maintenance, uploads, file replacements, retries and exports are refused with 503 maintenance_mode and Retry-After: 300. Search, chat, downloads and deletions keep working. GET /v1/knowledge_module reports "maintenance": true while it lasts.