Skip to content

HuggingFace Inference API provider #484

Description

@jeremymanning

Context

The provider set is Dartmouth Chat + HuggingFace only (decided 2026-08-21; ADR 0001). HuggingFace support means the hosted Inference API, not the local transformers inference in integrations/huggingface_model.py / lazy_huggingface_model.py — those stay in the frozen layer and retire under #430.

What to build

A HuggingFaceProvider in the canonical layer (models/providers/), speaking the Inference API over HTTP with aiohttp (already a core dependency — same shape as DartmouthProvider, no new core deps, no provider extra required unless the SDK is used).

  • Text-generation against a small set of served models, selected from the live catalog rather than a hard-coded list (two rounds of hard-coded Anthropic ids were wrong; do not repeat that here).
  • HF_TOKEN from the environment or ~/.orchestrator/.env, same as other credentials.
  • Free-tier vs paid distinction must fail closed: a model the token cannot serve must raise, not silently route elsewhere. Model fallback across providers does not exist (ADR 0001).
  • Registered in the normal model registry when a credential is present, so pipelines can select the models without touching the provider directly.

Acceptance criteria

  • Contract tests (hermetic, recorded/mocked HTTP boundary is fine at this layer) covering: request shape, auth header, error mapping (401/403/402/429/5xx), model-unavailable, and the registry integration.
  • A live-huggingface job in live-tests.yml, mirroring live-dartmouth: hard-fails when HF_TOKEN is missing under ORCHESTRATOR_REQUIRE_LIVE=1, and fails when every test skips.
  • Live tests exercise at least one real served model end-to-end: generate() returns non-empty content, models.list()/discovery works, health_check() works.
  • Reasoning-model behaviour handled: a response that spends its token budget on reasoning and returns empty content must not be reported as a successful empty answer (learned from the Dartmouth work — see Verify bare model-family resolution against the live API (blocked: no Anthropic credit) #432 comments).
  • README provider table flips HuggingFace to supported only after the live job passes remotely.

Out of scope

Related: #429 (adapter-layer consolidation — this provider lands in models/providers/), #430 (retirement of the frozen layer), ADR 0001 provider policy.

Metadata

Metadata

Assignees

No one assigned

    Labels

    architectureRelated to toolbox design/architectureenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions