Remote mode¶
Driving a LLMDyno server on another machine.
remote ¶
Drive LLMDyno on another machine: every library call goes to its web API, so the same code works on a laptop against the GPU box down the hall.
llmdyno.connect("http://gpu-box:8777", token="...") # or LLMDYNO_URL / LLMDYNO_TOKEN
qwen = llmdyno.profile("Qwen 27B 32k")
results = llmdyno.bench(qwen, "suite_quick") # runs there; progress and results here
llmdyno.disconnect() # back to this machine
On the server: llmdyno serve --host 0.0.0.0 (it prints the access token the first time) and keep it on a trusted
network, behind HTTPS, or reach it through an SSH tunnel (ssh -L 8777:localhost:8777 gpu-box). API keys you pass
(cloud_model(api_key=...)) are sent to the server once and stored there; they never come back over the API.
RemoteBackend ¶
RemoteBackend(url: str, token: str | None = None, *, client: Any = None, timeout: float = 120.0, verify: bool = True)
Bases: Backend
Runs every call on a LLMDyno server through its web API (httpx), and follows live events over its WebSocket.
route_table ¶
route_table() -> dict[str, Any]
service operation -> (method, path, {parameter: "path" | "query" | "body"}), read from the API's own routes so it always matches this version (the server's routes call services with the same parameter names).
connect ¶
connect(url: str | None = None, token: str | None = None, *, timeout: float = 120.0, verify: bool = True, client: Any = None) -> RemoteBackend
Send every later llmdyno call to the LLMDyno server at url (default LLMDYNO_URL), authenticating with its access token (default LLMDYNO_TOKEN; not needed on the server's own machine). Returns the connection.
connection ¶
connection() -> Backend
Where calls go now: a LocalBackend, or the RemoteBackend from connect().