Files¶
Files are the documents the ingestion will read. You upload them once, under a
scope, and the ingestion finds them there. A scope is an ordered list of up to
five labels and works like a storage path: ["acme", "legal"] places a file in
the legal part of acme. A file is identified by its database, its scope
and its name together, so uploading the same name under the same scope again
stores a new version of that file, never a duplicate.
The files area is the files namespace on the client:
uploaded = client.files.upload_file("./contracts/nda.pdf", scope=["acme", "legal"])
print(uploaded.id, uploaded.version)
Available operations¶
Operation |
Status |
Section |
|---|---|---|
Upload one file |
Available |
|
Upload several files |
Available |
|
Upload a folder |
Available |
|
List, download or delete files |
Not yet available |
Arrives in a later release. |
Scopes¶
Every upload takes a scope. The platform accepts at most five labels, each
1 to 128 characters of letters, digits, underscores and hyphens, and at most
256 characters combined. The SDK checks these rules before sending anything, so
a bad scope fails immediately rather than after the transfer, with an
InvalidFileScopeError.
Labels are part of how the platform stores a file, so shorter labels leave more room for long file names. Reordering the labels addresses a different file.
Versions and deletion¶
Uploading a file whose name and scope already exist stores version 2, 3 and so
on of the same file; the returned id
stays the same and version increases. Re-running an upload is therefore safe.
A new upload is marked safe_to_delete. Once the ingestion has built a graph
from a file it locks it, so a file in use cannot be removed by accident.
Uploading one file¶
uploaded = client.files.upload_file(
"./contracts/nda.pdf",
scope=["acme", "legal"],
name="nda-2026.pdf", # optional, defaults to the file's name
custom_metadata={"owner": "legal"}, # optional
)
The file is streamed from disk, never read into memory as a whole. The call
returns a RagInputUploadResponse describing the
stored version.
custom_metadata is a flat string-to-string mapping kept with this version
and returned on later reads. Each version carries its own; the next upload
replaces it rather than merging.
Errors: a path that does not exist or is a folder raises Python’s
FileNotFoundError or IsADirectoryError; an empty file raises
EmptyFileError; a bad scope raises
InvalidFileScopeError. All three are checked
before anything is sent; the platform applies the same rules on its side, so
the last two can also come back from it. Rejected metadata raises
InvalidFileMetadataError, and a platform that
could not store the file raises
FileUploadFailedError after rolling the upload
back.
Uploading several files¶
# All under one scope
results = client.files.upload_files(
["./contracts/nda.pdf", "./contracts/msa.pdf"],
scope=["acme", "legal"],
)
# Each under its own scope
results = client.files.upload_files({
"./contracts/nda.pdf": ["acme", "legal"],
"./reports/q1.pdf": ["acme", "finance", "2026"],
})
The files go to the platform in one request and are stored one by one, each
under its own file name. A file the platform rejects, for example an empty one,
is reported in the result and does not stop the others. With a mapping, each
file’s scope comes from the mapping and the scope argument is ignored.
One request carries at most 100 files and 2 GiB in total. A larger set is
refused with FileBatchTooLargeError, normally
before anything is sent; split it across several calls. To store a file under
a different name, use Uploading one file with name.
The return value¶
A list with one RagInputBatchItemResult per
file, in the order given. A file can be rejected while the others are stored,
so check status first: "ok" means the storage fields (id,
version, size, …) are set; "error" means they are None and
detail says why.
for result in results:
if result.status == "ok":
print(result.id, result.version)
else:
print(result.name, result.detail)
Two files with the same name under the same scope in one call are stored as version 1 and version 2 of that file, in order. The same happens to a path listed twice: it is uploaded twice.
Errors: ValueError for an empty list;
FileNotFoundError or IsADirectoryError for a
bad path; InvalidFileScopeError and
FileBatchTooLargeError as above. All are checked
before anything is sent. InvalidFileMetadataError
comes from the platform.
Uploading a folder¶
results = client.files.upload_folder("./contracts", scope=["acme", "legal"])
Uploads the files directly inside the folder, all under scope and each by
its own name, in alphabetical order. Subfolders are ignored, together with
everything in them: to upload a nested layout, call upload_folder once per
folder, or Uploading several files with a mapping.
Everything else, limits, return value and errors, is as in
Uploading several files. In addition, a folder that does not exist or is
not a folder raises Python’s NotADirectoryError, and a folder with no
files raises ValueError.
Waiting for the platform¶
The platform stores the files of a request one after another and answers when
the last one is written, so a large upload can take minutes to return. Uploads
therefore wait up to five minutes for the answer instead of the connection’s
usual timeout. Pass timeout (in seconds) to any upload method for a very
large file or a slow deployment. The platform’s own gateway stops waiting after
fifteen minutes and answers with an error of its own, so values above that
have no effect.