Remote mode¶
Run LLMDyno on the machine with the GPU and Docker, and drive it from anywhere: from Python, from the command line, or in a browser.
On the server¶
llmdyno serve --host 0.0.0.0
The first time, this prints an access token. It is shown once, so keep it somewhere safe. llmdyno token --rotate makes
a new one, and clients using the old one stop working at once.
Requests from the server's own machine need nothing, as always. Every other request must present the token:
- in an Authorization: Bearer <token> header;
- or as ?token= for downloads and the WebSocket, which cannot send headers.
A request that came through a reverse proxy counts as remote even if the proxy runs on the same machine. So putting
nginx or Caddy in front for HTTPS does not open a hole. Changes made by remote clients are logged to
<data>/logs/remote-requests.jsonl, with the time, client address, method, path and status.
LLMDyno speaks plain HTTP. Across a network you do not trust, put it behind HTTPS (a reverse proxy), or reach it through an SSH tunnel:
ssh -L 8777:localhost:8777 gpu-box # then use http://localhost:8777 on your laptop: no token needed
From Python¶
import llmdyno
llmdyno.connect("http://gpu-box:8777", token="...") # or set LLMDYNO_URL and LLMDYNO_TOKEN
qwen = llmdyno.profile("Qwen 27B 32k")
plan = llmdyno.bench(qwen, "suite_quick", wait=False) # runs on the server
for ev in llmdyno.watch(plan, transcript=True): # live events over the WebSocket
print(ev.kind, ev.status)
results = plan.results()
results.open() # opens the server's dashboard, already signed in
llmdyno.disconnect() # back to this machine
Every call in the library works the same against a server. Tests run each library area both ways to keep that true.
Calls that only make sense on one machine say so: init() and serve() refuse while connected, and data_dir()
reports the server's folder.
API keys you pass, such as cloud_model(api_key=...) or set_hf_token(), are sent to the server once and stored
there encrypted. No API call ever returns a key, and the access token is never returned either.
From the command line¶
Every command takes --url and --api-token, or reads LLMDYNO_URL and LLMDYNO_TOKEN:
export LLMDYNO_URL=http://gpu-box:8777 LLMDYNO_TOKEN=...
llmdyno list-plans
llmdyno bench "Qwen 27B 32k" --benchmarks suite_quick
In a browser¶
Open http://gpu-box:8777. The dashboard asks for the access token once and keeps it in that browser.