Skip to content

Providers

Cloud providers, local OpenAI-compatible servers and LiteLLM.

providers

Cloud providers: vendors, keys, budgets and spend, connection tests and price lookup.

Provider

Provider(data: dict[str, Any] | str)

Bases: Handle

A cloud provider account: vendor, endpoint, API key (kept encrypted in the data folder, never returned), a hard budget and what has been spent against it.

spent_usd property

spent_usd: float

Spend recorded against the budget (fetched fresh).

test

test() -> dict[str, Any]

Check the key and endpoint by listing models: {"ok", "error", "models", "latency_ms"}.

models

models(refresh: bool = False) -> list[Any]

Model ids this provider offers (from the last connection test; refresh=True tests again).

update

update(*, name: str | None = None, budget_usd: float | None = None, api_style: str | None = None, base_url: str | None = None) -> 'Provider'

Rename, change the budget (USD, a hard cap), the API style ("openai-chat", "openai-responses", "anthropic", …) or the base URL.

reset_spend

reset_spend() -> 'Provider'

Start counting spend against the budget from zero again.

delete

delete() -> None

Remove the provider and its key (refused while profiles use it).

list_providers

list_providers(vendor: str | None = None) -> list[Any]

Your cloud providers (optionally of one vendor).

provider

provider(name_or_id: Union[str, Provider]) -> Provider

A provider by id, name, or vendor id when you have only one of that vendor.

add_provider

add_provider(vendor: str, *, api_key: str | None = None, name: str | None = None, base_url: str | None = None, api_style: str | None = None, budget_usd: float = 5.0) -> Provider

Add a provider account. vendor is one of vendors() ("custom" for any OpenAI-compatible endpoint, which needs base_url). cloud_model() does this for you when no provider of that vendor exists yet.

vendors

vendors() -> list[Any]

Supported vendors with their API styles, endpoints, where to get a key and notes.

lookup_price

lookup_price(vendor: str, model: str) -> dict[str, Any] | None

Price per 1M tokens ({"input", "output", "cached"}, plus context length and the matched model) from OpenRouter's catalog, or None when the model is not found.

local_servers

local_servers() -> list[Any]

OpenAI-compatible servers running on the machine LLMDyno runs on (Ollama, LM Studio, vLLM, SGLang, a llama.cpp server, a LiteLLM proxy), each with its URL, the URL the sandbox uses, and its models.

ollama_model

ollama_model(model: str, *, name: str | None = None, **params: Any)

A model served by Ollama on this machine, as a profile: ollama_model("qwen3-coder:30b", ctx=65536). Ollama must be running (it is found on its usual port, 11434).

openai_compatible

openai_compatible(base_url: str, model: str | None = None, *, api_key: str | None = None, name: str | None = None, api_style: str | None = None, **params: Any)

Any OpenAI-compatible endpoint as a profile: LM Studio, vLLM, SGLang, a llama.cpp server, a gateway. localhost URLs are rewritten so the sandbox can reach them. Without model=, the endpoint must serve exactly one (it is looked up among local_servers()).

litellm_model

litellm_model(model: str, *, base_url: str | None = None, api_key: str | None = None, name: str | None = None, budget_usd: float | None = None, pricing: Any = 'auto', **params: Any)

A model behind a LiteLLM proxy (Azure OpenAI, Bedrock, Vertex AI, … through one OpenAI-compatible endpoint): litellm_model("bedrock/anthropic.claude-sonnet-4-5", reasoning_effort="low"). Start the proxy yourself, e.g. litellm --model bedrock/... or litellm --config config.yaml (pip install "litellm[proxy]"); base_url defaults to it on this machine's port 4000. Prices come from LiteLLM's table when the litellm package is installed ("llmdyno[litellm]"); budgets cap spend as for any cloud provider.