Providers¶
Cloud providers, local OpenAI-compatible servers and LiteLLM.
providers ¶
Cloud providers: vendors, keys, budgets and spend, connection tests and price lookup.
Provider ¶
Provider(data: dict[str, Any] | str)
Bases: Handle
A cloud provider account: vendor, endpoint, API key (kept encrypted in the data folder, never returned), a hard budget and what has been spent against it.
test ¶
test() -> dict[str, Any]
Check the key and endpoint by listing models: {"ok", "error", "models", "latency_ms"}.
models ¶
models(refresh: bool = False) -> list[Any]
Model ids this provider offers (from the last connection test; refresh=True tests again).
update ¶
update(*, name: str | None = None, budget_usd: float | None = None, api_style: str | None = None, base_url: str | None = None) -> 'Provider'
Rename, change the budget (USD, a hard cap), the API style ("openai-chat", "openai-responses", "anthropic", …) or the base URL.
list_providers ¶
list_providers(vendor: str | None = None) -> list[Any]
Your cloud providers (optionally of one vendor).
provider ¶
A provider by id, name, or vendor id when you have only one of that vendor.
add_provider ¶
add_provider(vendor: str, *, api_key: str | None = None, name: str | None = None, base_url: str | None = None, api_style: str | None = None, budget_usd: float = 5.0) -> Provider
Add a provider account. vendor is one of vendors() ("custom" for any OpenAI-compatible endpoint, which needs base_url). cloud_model() does this for you when no provider of that vendor exists yet.
vendors ¶
vendors() -> list[Any]
Supported vendors with their API styles, endpoints, where to get a key and notes.
lookup_price ¶
lookup_price(vendor: str, model: str) -> dict[str, Any] | None
Price per 1M tokens ({"input", "output", "cached"}, plus context length and the matched model) from OpenRouter's catalog, or None when the model is not found.
local_servers ¶
local_servers() -> list[Any]
OpenAI-compatible servers running on the machine LLMDyno runs on (Ollama, LM Studio, vLLM, SGLang, a llama.cpp server, a LiteLLM proxy), each with its URL, the URL the sandbox uses, and its models.
ollama_model ¶
ollama_model(model: str, *, name: str | None = None, **params: Any)
A model served by Ollama on this machine, as a profile: ollama_model("qwen3-coder:30b", ctx=65536). Ollama must be running (it is found on its usual port, 11434).
openai_compatible ¶
openai_compatible(base_url: str, model: str | None = None, *, api_key: str | None = None, name: str | None = None, api_style: str | None = None, **params: Any)
Any OpenAI-compatible endpoint as a profile: LM Studio, vLLM, SGLang, a llama.cpp server, a gateway. localhost URLs are rewritten so the sandbox can reach them. Without model=, the endpoint must serve exactly one (it is looked up among local_servers()).
litellm_model ¶
litellm_model(model: str, *, base_url: str | None = None, api_key: str | None = None, name: str | None = None, budget_usd: float | None = None, pricing: Any = 'auto', **params: Any)
A model behind a LiteLLM proxy (Azure OpenAI, Bedrock, Vertex AI, … through one OpenAI-compatible endpoint):
litellm_model("bedrock/anthropic.claude-sonnet-4-5", reasoning_effort="low"). Start the proxy yourself, e.g.
litellm --model bedrock/... or litellm --config config.yaml (pip install "litellm[proxy]"); base_url defaults
to it on this machine's port 4000. Prices come from LiteLLM's table when the litellm package is installed
("llmdyno[litellm]"); budgets cap spend as for any cloud provider.