LLM completion

Handles client LLM completion requests, applies rate limits based on tokens and requests, supports streaming or normal responses, and records usage.

Headers

AuthorizationstringRequired
API key with Bearer prefix
X-Delegated-ForstringOptional

Optional header to indicate the original client’s user id for delegated requests. (Only for management keys)

x-session-idstringOptional
A unique session identifier. This is only required when interacting with agentic models to persist the session across multiple requests. Can be any random string.

Request

LLM Completion Payload
messageslist of objectsRequired
modelstringRequired
agent_addressstringOptional
max_tokensintegerOptional
planner_modebooleanOptional
streambooleanOptional
study_modebooleanOptional
temperaturedoubleOptional
toolslist of objectsOptional

Response

Successful completion or stream
choiceslist of objects

A list of generated choices/messages.

idstring
usageobject
Information about token usage.
executable_datalist of objects or null
metadataobject or null

Errors