Top score of 20 teams: 98.00.
Every agent faced an unseen MCP server with hidden tools and no docs, live. Ours checked every call against its permission before it ran.
sha256sum -c proofable-submission.tar.gz.sha256The expected hash is 7b00e595ac4dbe4ff81bdb972af6f0d5b2473c966a126a587f5c4d79c3d5f601.
tar -xzf proofable-submission.tar.gz && cd gauntlet-submission| File | What it does |
|---|---|
proofable_gate.py |
Decides every tool call before it runs, and writes each decision and result into a hash-chained receipt |
proofable_authority.py |
Proofable's permission engine, ported to Python, that the gate asks about each call |
proofable_authority_contract.json |
The public contract those rules follow, pinned by hash |
proofable_agent.py |
The agent: explores the server, solves the challenge, submits |
agent_tools.py |
Turns the server's tools, resources and prompts into calls the model can make |
my_strategy.py |
Model choice, prompts and time budgets |
arena_clients/, base_strategy.py, model_selector.py |
NVIDIA's starter kit, unchanged |
Reading a clue, sending an answer and running code were allowed. Anything destructive was refused before it ran. The tools the challenge needed were granted as one named group, so the agent could not widen its own reach mid-run.
- How it went, in our words: Proofable won NVIDIA's Live Agent Gauntlet
- Every benchmark, beside the tools teams use today: proofable.me/benchmarks
- Put the same limits on your own agent: github.com/proofable/agent