Skip to content

Models

GGUF models: Hugging Face search, fit estimates, downloads and imports.

models

GGUF models: Hugging Face search, fit estimates before download, downloads, imports and the recommended list.

Model

Model(data: dict[str, Any] | str)

Bases: Handle

A GGUF model in the data folder, downloaded from Hugging Face or registered from disk.

status property

status: str

ready, queued, downloading, paused, failed, … (as last fetched; refresh() to update).

paths property

paths: list[Any]

The model's file(s) on disk.

contexts

contexts() -> list[Any]

Projected memory and verdict at each context size with the default settings for this machine (what the model page's context table shows). Empty until the model is downloaded.

profile

profile(name: str | None = None, **params: Any)

A local profile for this model with these settings (same keywords as local_model()).

pause

pause() -> None

Pause its download (the partial file is kept; resume() carries on).

resume

resume(wait: bool = True, quiet: bool = False) -> Union['Model', Job]

Carry on with an interrupted or paused download. wait=False returns the download Job.

delete

delete(files: bool = False) -> dict[str, Any]

Forget the model. files=True also deletes downloaded files (files you imported are never deleted). Profiles using it stay, without a model. Returns what was removed.

list_models

list_models(status: str | None = None) -> list[Any]

Models in the data folder, newest first; status="ready" keeps the usable ones.

model

model(name_or_id: Union[str, Model]) -> Model

A model by id, by name (the file name without .gguf), or as "repo:quant".

search_models

search_models(query: str, sort: str = 'downloads', limit: int = 30) -> list[Any]

Search Hugging Face for GGUF repositories (sort: downloads, likes or lastModified).

repo_info

repo_info(repo: str) -> dict[str, Any]

A Hugging Face repository's GGUF variants (quant, size, files, a first fit hint for this GPU) plus its license, downloads, likes, tags and whether it is gated.

estimate

estimate(repo: str, quant: str, **params: Any) -> dict[str, Any]

Will it fit, before downloading anything: reads the GGUF header over HTTP and projects GPU and system memory for these settings (same keywords as local_model(): ctx, kv_cache, n_cpu_moe, …). Returns {"check": verdict, projected memory and warnings, "contexts": the table for other context sizes, "params", "meta"}.

download_model

download_model(repo: str, quant: str, *, wait: bool = True, quiet: bool = False) -> Union[Model, Job]

Download one quant of a Hugging Face GGUF repository into the models folder (resumable, checksum-verified). Returns the Model when done, or the download Job with wait=False. An interrupted download resumes.

import_models

import_models(folder: Union[str, Path]) -> list[Any]

Register the GGUF files in a folder where they are (nothing is copied); returns the models added.

recommended_models

recommended_models() -> list[Any]

LLMDyno's suggested local models for coding agents, each with why and whether it is already downloaded.