LLM completion
Handles client LLM completion requests, applies rate limits based on tokens and requests, supports streaming or normal responses, and records usage.
Headers
Authorization
API key with Bearer prefix
X-Delegated-For
Optional header to indicate the original client’s user id for delegated requests. (Only for management keys)
x-session-id
A unique session identifier. This is only required when interacting with agentic models to persist the session across multiple requests. Can be any random string.
Request
LLM Completion Payload
messages
model
agent_address
max_tokens
planner_mode
stream
study_mode
temperature
tools
web_search
Response
Successful completion or stream
choices
A list of generated choices/messages.
id
usage
Information about token usage.
executable_data
metadata