Harnesses and agents¶
Built-in and custom harnesses, and your own Python or TypeScript agent as a harness.
harnesses ¶
Agent harnesses: built-in and custom, images, Dockerfiles, previews and portable runtimes.
Harness ¶
Harness(data: dict[str, Any] | str)
Bases: Handle
A coding-agent harness (Pi, OpenCode, Codex CLI, Qwen Code, Crush, Aider, mini-SWE-agent, or your own) and the Docker image it runs in.
build ¶
build(wait: bool = True, quiet: bool = False) -> Union['Harness', Job]
Build (or rebuild) its image on top of the sandbox image, then smoke-test it. wait=False returns the Job.
preview ¶
preview(api_style: str = 'openai-chat') -> dict[str, Any]
Exactly what a run would execute, for a sample endpoint: config files written into the agent's HOME, environment, the first command and the command that continues a multi-stage task.
runtime ¶
runtime(platform: str = '') -> dict[str, Any]
Its portable runtime (used inside benchmarks that bring their own image, such as SWE-bench) for a platform ("linux/amd64", "linux/arm64"; default this machine's): whether it is built, plus its Dockerfile.
build_runtime ¶
build_runtime(platform: str = '', wait: bool = True, quiet: bool = False) -> Union[dict[str, Any], Job]
Build the portable runtime for a platform. Returns its status, or the Job with wait=False.
update ¶
update(manifest: Union[dict[str, Any], str, Path]) -> 'Harness'
Replace a custom harness's manifest (rebuild afterwards for image changes to take effect).
list_harnesses ¶
list_harnesses(ready: bool | None = None) -> list[Any]
Built-in and custom harnesses with their image status; ready=True keeps the ones whose image is built.
harness ¶
A harness by id ("pi", "opencode", …) or name.
add_harness ¶
add_harness(manifest: Union[dict[str, Any], str, Path], build: bool = False, quiet: bool = False) -> Harness
Add your own harness from a manifest (a dict or a JSON file; see harness_schema() and docs/HARNESSES.md): id, name, install lines (Dockerfile instructions), the command with $PROMPT, config files, API styles. build=True builds its image right away.
agents ¶
Benchmark your own agent: python_harness() turns a Python function into a harness, so an agent built with LangGraph, the OpenAI Agents SDK, smolagents, PydanticAI or plain code runs in the same offline sandbox, behind the same proxy, with the same transcripts, budgets and grading as the built-in harnesses. node_harness() does the same for a JavaScript or TypeScript function: an agent built with the Vercel AI SDK, Mastra, LangChain.js or the OpenAI Agents SDK for JS.
h = llmdyno.python_harness("my_agent:run", requirements=["langgraph==1.0.2", "langchain-openai==1.0.1"])
llmdyno.bench(qwen, "suite_quick", harnesses=h.id)
The Python entry point is called once per stage, in the task's workspace, with any of these keyword arguments it declares: prompt, model, base_url, api_key, workspace, stage, ctx, max_tokens, reasoning. base_url is LLMDyno's OpenAI-compatible proxy (api_key is a placeholder; the proxy adds the real key). It may be async. Whatever it returns is printed. The JavaScript entry point gets one object with the same fields in camelCase (baseUrl, apiKey, maxTokens). Ready-made agents for each framework are in examples/agents/ (Python) and examples/agents/ts/ (TypeScript).
python_harness ¶
python_harness(entrypoint: str, *, requirements: Sequence[str] = (), source: Union[str, Path, None] = None, id: str | None = None, name: str | None = None, version: str = '1', apis: Sequence[str] = ('openai-chat',), multi_stage: str = 'fresh', build: bool = False, quiet: bool = False, replace: bool = True) -> Harness
Make a harness from a Python function ("module:function"). requirements are pip packages installed into the
image (pin versions for repeatable results); source is the module's file or package folder (found on your path
when omitted). id defaults to "py-
node_harness ¶
node_harness(entrypoint: str, *, packages: Sequence[str] | None = None, source: Union[str, Path, None] = None, id: str | None = None, name: str | None = None, version: str = '1', apis: Sequence[str] = ('openai-chat',), multi_stage: str = 'fresh', build: bool = False, quiet: bool = False, replace: bool = True) -> Harness
Make a harness from a JavaScript or TypeScript function ("agent.ts:run"; run, or the default export, when no
name is given). It runs on Node 24 in the sandbox, which runs TypeScript directly (type annotations only: no enums
or namespaces). packages are npm packages installed for it (pin versions, e.g. "ai@7.0.179"); when omitted, the
entry file's export const PACKAGES = [...] list is used, and a package.json in the source folder is installed
as well. source is the agent's file, or a folder that the entry point's path is
relative to (node_modules, .git and build output are left out). The function gets one object: prompt, model,
baseUrl, apiKey, workspace, stage, ctx, maxTokens, reasoning. id defaults to "js-