Skip to content

Remote mode

Driving a LLMDyno server on another machine.

remote

Drive LLMDyno on another machine: every library call goes to its web API, so the same code works on a laptop against the GPU box down the hall.

llmdyno.connect("http://gpu-box:8777", token="...")      # or LLMDYNO_URL / LLMDYNO_TOKEN
qwen = llmdyno.profile("Qwen 27B 32k")
results = llmdyno.bench(qwen, "suite_quick")              # runs there; progress and results here
llmdyno.disconnect()                                     # back to this machine

On the server: llmdyno serve --host 0.0.0.0 (it prints the access token the first time) and keep it on a trusted network, behind HTTPS, or reach it through an SSH tunnel (ssh -L 8777:localhost:8777 gpu-box). API keys you pass (cloud_model(api_key=...)) are sent to the server once and stored there; they never come back over the API.

RemoteBackend

RemoteBackend(url: str, token: str | None = None, *, client: Any = None, timeout: float = 120.0, verify: bool = True)

Bases: Backend

Runs every call on a LLMDyno server through its web API (httpx), and follows live events over its WebSocket.

route_table

route_table() -> dict[str, Any]

service operation -> (method, path, {parameter: "path" | "query" | "body"}), read from the API's own routes so it always matches this version (the server's routes call services with the same parameter names).

connect

connect(url: str | None = None, token: str | None = None, *, timeout: float = 120.0, verify: bool = True, client: Any = None) -> RemoteBackend

Send every later llmdyno call to the LLMDyno server at url (default LLMDYNO_URL), authenticating with its access token (default LLMDYNO_TOKEN; not needed on the server's own machine). Returns the connection.

disconnect

disconnect() -> None

Go back to running everything on this machine.

connection

connection() -> Backend

Where calls go now: a LocalBackend, or the RemoteBackend from connect().