API reference¶
Everything here is available as llmdyno.<name> after import llmdyno, and as a llmdyno command (see
the command line). The pages follow the areas of the dashboard.
| Page | What is in it |
|---|---|
| Setup, system and jobs | Opening a data folder, checking and installing dependencies, the machine's specs and GPU, background jobs, cleanup and the panic stop, and the dashboard. |
| Profiles | A model plus the exact settings it is benchmarked with. |
| Models | GGUF models: Hugging Face search, fit estimates, downloads and imports. |
| Providers | Cloud providers, local OpenAI-compatible servers and LiteLLM. |
| Harnesses and agents | Built-in and custom harnesses, and your own Python agent as a harness. |
| Benchmarks and suites | Built-in, your own and imported benchmarks (SWE-bench, Aider polyglot), and suites. |
| Plans | Running profiles × harnesses × benchmarks × iterations. |
| Runs and transcripts | One run's details, transcript, workspace, logs and telemetry. |
| Results | Scores, every view of the Results page, exports and charts. |
| Judging | Blind code review by LLM judges. |
| Live events and async | Following work as it happens. |
| Remote mode | Driving a LLMDyno server on another machine. |
| Settings and references | Settings, tokens and Artificial Analysis reference numbers. |
| Interop and CI | Inspect AI, regression checks for CI and the pytest plugin. |
| Errors | One error family for the library and the web API. |
Integrations with other libraries are methods of these objects (Results.to_pandas(), Results.plot,
Results.log_to_mlflow(), …) and are described in Integrations.