Skip to content

Commit c373532

Browse files
committed
Polish TRACE SDK release documentation
Turn the SDK README into a developer-facing landing page, add linked docs and examples, include runtime proof points, and complete optional integration extras for the latence package.
1 parent 305987c commit c373532

6 files changed

Lines changed: 493 additions & 51 deletions

File tree

‎README.md‎

Lines changed: 290 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,27 @@
1-
# Latence Python SDK
1+
<p align="center">
2+
<img src="https://raw.githubusercontent.com/latenceainew/latence-trace-python/main/docs/assets/latence-logo.svg" alt="Latence" width="180" />
3+
</p>
24

3-
Thin Python SDK for Latence TRACE.
5+
<h1 align="center">Latence TRACE Python SDK</h1>
6+
7+
<p align="center">
8+
Real-time protection for AI agents. Verify answers, redact private data,
9+
reduce wasted context, and log decisions without replacing your stack.
10+
</p>
11+
12+
<p align="center">
13+
<a href="https://pypi.org/project/latence/">PyPI</a>
14+
·
15+
<a href="docs/quickstart.md">Quickstart</a>
16+
·
17+
<a href="docs/sessions.md">Sessions</a>
18+
·
19+
<a href="docs/integrations.md">Integrations</a>
20+
·
21+
<a href="examples/">Examples</a>
22+
·
23+
<a href="https://latence.ai">Website</a>
24+
</p>
425

526
```bash
627
pip install latence
@@ -9,41 +30,193 @@ pip install latence
930
```python
1031
from latence import Latence
1132

12-
client = Latence(api_key="lat_...")
33+
trace = Latence(api_key="lat_...")
34+
35+
score = trace.grounding.rag(
36+
query="Can I promise this customer a refund?",
37+
response_text="Yes, the refund will arrive within 48 hours.",
38+
raw_context="Refunds require manual finance approval before timelines are promised.",
39+
)
40+
41+
print(score.risk_band)
42+
print(score.runtime_decision)
43+
```
44+
45+
## Why This Exists
46+
47+
Agents are moving from demos into real workflows. That means private data,
48+
unsupported answers, prompt attacks, tool drift, and wasted memory are no longer
49+
abstract research problems. They become support escalations, broken automations,
50+
audit gaps, token waste, and user trust issues.
51+
52+
TRACE is the protection layer that sits next to your RAG pipeline, coding agent,
53+
or tool-using workflow. Your agent keeps running. TRACE checks the turn and
54+
returns evidence plus a decision your application can route.
55+
56+
## What TRACE Does
57+
58+
TRACE is intentionally small at the SDK layer. The heavy work lives in your
59+
TRACE runtime deployment; this package is the thin Python interface.
60+
61+
- Verify RAG answers against retrieved context.
62+
- Score coding-agent output against codebase context.
63+
- Redact private data before it spreads through tools, logs, or prompts.
64+
- Compress and repair long-running context with InfiniMem.
65+
- Roll up an agent session into review-ready signals.
66+
- Persist caller-carried state without forcing sticky server sessions.
67+
68+
## Proof Points
69+
70+
These are runtime proof points from the TRACE freeze evidence, not SDK-only
71+
microbenchmarks. See the linked artifacts for full context.
72+
73+
- Grounding: local managed-runtime 360 reported `1.00` AUROC for grounded vs.
74+
ungrounded RAG cases.
75+
- Coding agents: local 360 reported `1.00` AUROC for code phantom detection.
76+
- Wasted context: held-out unused-context classification reported `1.00`
77+
precision and `1.00` recall.
78+
- Latency: local concurrency burst reported about `368 ms` RAG p95 and
79+
`334 ms` code p95 in the managed-runtime proof.
80+
- Privacy: redaction returns labels, offsets, scores, redacted output, entity
81+
counts, and timings for logging-ready GDPR workflows.
82+
- Memory: InfiniMem is designed for up to `90%` context reduction while keeping
83+
hot context available to the agent.
84+
- Guard checks: Prompt Guard warmup proved `torch.compile` enabled on CUDA in
85+
the runtime proof.
86+
87+
Evidence:
88+
89+
- [SDK runtime freeze audit](https://github.com/latenceainew/latence-trace/blob/main/docs/core_freeze/sdk_runtime_freeze_audit.md)
90+
- [Local 360 proof JSON](https://github.com/latenceainew/latence-trace/blob/main/docs/core_freeze/local_sdk_runtime_freeze_360.json)
91+
- [Endpoint observability audit](https://github.com/latenceainew/latence-trace/blob/main/docs/core_freeze/endpoint_observability_360.md)
92+
93+
## How It Works
94+
95+
1. Deploy or access a TRACE runtime: RunPod, FastAPI, VPC, or on-prem.
96+
2. Install `latence`.
97+
3. Send the agent turn to the product path that matches the workflow.
98+
4. Route on `risk_band`, `runtime_decision`, scores, spans, and evidence.
99+
5. Store only the audit evidence your policy allows.
100+
101+
```python
102+
from latence import Latence
103+
104+
trace = Latence(
105+
api_key="lat_...",
106+
base_url="https://your-trace-endpoint.example.com",
107+
)
108+
```
109+
110+
Environment variables are supported:
111+
112+
```bash
113+
export LATENCE_TRACE_API_KEY="lat_..."
114+
export LATENCE_TRACE_URL="https://your-trace-endpoint.example.com"
115+
```
116+
117+
## The SDK Surface
118+
119+
The SDK mirrors the TRACE product API directly:
120+
121+
- Privacy: `client.privacy.redact(...)`
122+
- RAG grounding: `client.grounding.rag(...)`
123+
- Code grounding: `client.grounding.code(...)`
124+
- Text compression: `client.compression.text(...)`
125+
- Message compression: `client.compression.messages(...)`
126+
- Memory update: `client.memory.step(...)`
127+
- Stateless rollup: `client.rollup(...)`
128+
- Caller-carried sessions: `client.session(...)`
129+
130+
`Latence` is synchronous. `AsyncLatence` exposes the same surface for asyncio
131+
services.
132+
133+
```python
134+
from latence import AsyncLatence, Latence
135+
```
136+
137+
Base dependencies are only `httpx` and `pydantic`. Runtime and model packages
138+
such as `torch`, `transformers`, `triton`, FastAPI, and vLLM are not SDK
139+
dependencies.
140+
141+
## Start With The Path You Need
142+
143+
### RAG Agents
144+
145+
Use TRACE when your answer must be grounded in retrieved context.
146+
147+
```python
148+
score = trace.grounding.rag(
149+
query="What is the refund policy?",
150+
response_text=agent_answer,
151+
raw_context=retrieved_context,
152+
)
153+
154+
if score.risk_band.value != "green":
155+
send_to_review(score)
156+
```
157+
158+
Example: [RAG grounding with guard checks](examples/rag_grounding_guard.py)
159+
160+
### Coding Agents
161+
162+
Use TRACE when an agent explains, edits, or reasons over code.
13163

14-
result = client.grounding.rag(
15-
query="When will this customer be refunded?",
16-
response_text="The customer will be refunded within 48 hours.",
17-
raw_context="Refunds are reviewed manually after finance approval.",
164+
```python
165+
score = trace.grounding.code(
166+
query="Does this patch add retry handling?",
167+
response_text=agent_answer,
168+
raw_context=code_context,
169+
extra={"response_language_hint": "python"},
170+
)
171+
```
172+
173+
Example: [Code grounding](examples/code_grounding.py)
174+
175+
### Privacy
176+
177+
Use TRACE before customer data enters prompts, tools, traces, or logs.
178+
179+
```python
180+
redacted = trace.privacy.redact(
181+
text="Email jane@example.com and charge IBAN DE89370400440532013000.",
18182
)
19183

20-
print(result.risk_band)
184+
print(redacted.redacted_text)
185+
print(redacted.unique_labels)
21186
```
22187

23-
## Surface
188+
Example: [Privacy redaction](examples/privacy_redact.py)
189+
190+
### Compression And Memory
24191

25-
The SDK maps 1:1 to the TRACE product API:
192+
Use TRACE when long-running workflows start dragging dead context forward.
193+
194+
```python
195+
compressed = trace.compression.text(
196+
"Long retrieved context...",
197+
compression_rate=0.4,
198+
)
199+
```
26200

27-
- `client.privacy.redact(...)`
28-
- `client.grounding.rag(...)`
29-
- `client.grounding.code(...)`
30-
- `client.compression.text(...)`
31-
- `client.compression.messages(...)`
32-
- `client.memory.step(...)`
33-
- `client.rollup(...)`
34-
- `client.session(...)`
201+
```python
202+
memory = trace.memory.step(
203+
turn_text="User asked for manual refund approval.",
204+
prior_memory_state=current_state,
205+
)
206+
current_state = memory.next_memory_state
207+
```
35208

36-
`Latence` is the sync client. `AsyncLatence` exposes the same product namespaces for asyncio apps.
209+
Examples: [Compression](examples/compression.py), [Memory step](examples/memory_step.py)
37210

38-
## Sessions
211+
### Sessions
39212

40-
TRACE deployments are stateless by default. The SDK owns caller-carried session state for agent workflows that need memory continuity, rollups, idempotency, and local persistence.
213+
TRACE runtimes can stay stateless while the SDK carries state for your agent.
41214

42215
```python
43216
from latence import FileSessionStorage, Latence
44217

45-
client = Latence(api_key="lat_...")
46-
session = client.session(
218+
trace = Latence()
219+
session = trace.session(
47220
session_id="support-run-42",
48221
storage=FileSessionStorage(".trace-sessions"),
49222
)
@@ -58,24 +231,112 @@ score = session.rag(
58231
session.save()
59232
```
60233

61-
## Compatibility
234+
Docs: [Sessions](docs/sessions.md)
235+
Example: [Session facade](examples/session_facade.py)
236+
237+
## For Whom
238+
239+
TRACE is for teams building or operating:
240+
241+
- RAG products where unsupported answers are expensive.
242+
- Coding agents that need codebase-grounded reasoning over many steps.
243+
- Support agents that touch customer records and policies.
244+
- Legal, finance, healthcare, or regulated workflows that need evidence.
245+
- Internal agent platforms where observability, retries, and human review matter.
246+
247+
It is also useful for framework authors and platform teams that need one
248+
consistent protection API across LangGraph, LangChain, LlamaIndex, n8n, Cursor,
249+
Claude Code, Codex, and custom agent runners.
250+
251+
## Integrations
252+
253+
Direct calls are the recommended path. Optional helpers live under
254+
`latence.integrations`.
255+
256+
```bash
257+
pip install "latence[langchain]"
258+
pip install "latence[llama_index]"
259+
pip install "latence[openai]"
260+
pip install "latence[haystack]"
261+
```
262+
263+
Docs: [Integrations](docs/integrations.md)
264+
Example: [Async batch](examples/async_batch.py)
62265

63-
The primary import is:
266+
## Async
64267

65268
```python
66-
from latence import Latence, AsyncLatence
269+
from latence import AsyncLatence
270+
271+
async with AsyncLatence() as trace:
272+
score = await trace.grounding.rag(
273+
query="What changed?",
274+
response_text="The policy now allows refunds.",
275+
raw_context="The policy still requires manual approval.",
276+
)
67277
```
68278

69-
For migration from the TRACE preview SDK, `LatenceTraceClient` and `AsyncLatenceTraceClient` remain aliases.
279+
## Now What
280+
281+
If you are integrating TRACE into an agent:
282+
283+
1. Run the [quickstart](docs/quickstart.md).
284+
2. Pick one product path: RAG, code, privacy, compression, memory, or session.
285+
3. Add one route in your app for `green`, `amber`, and `red`.
286+
4. Log request ID, risk band, runtime decision, and redacted evidence.
287+
5. Replay a few real failures and tune your thresholds.
70288

71-
## Development
289+
If you are publishing or validating this SDK:
72290

73291
```bash
74292
python -m pip install -e ".[dev]"
75293
python -m pytest
76294
python -m ruff check .
295+
python scripts/check_contract.py --manifest ../latence-trace/docs/core_freeze/api_surface_manifest.json
77296
python -m build
78297
python -m twine check dist/*
79298
```
80299

81-
Set `LATENCE_TRACE_RUNTIME_REPO=/path/to/latence-trace` when running contract tests from a non-sibling checkout.
300+
Clean-wheel smoke testing should run outside the repo root:
301+
302+
```bash
303+
python -m venv /tmp/latence-sdk-smoke
304+
/tmp/latence-sdk-smoke/bin/pip install dist/*.whl
305+
cd /tmp && /tmp/latence-sdk-smoke/bin/python - <<'PY'
306+
from importlib.metadata import distribution
307+
from latence import Latence
308+
309+
requires = distribution("latence").requires or []
310+
for forbidden in ("torch", "transformers", "triton", "fastapi", "vllm"):
311+
assert not any(req.lower().startswith(forbidden) for req in requires), requires
312+
313+
assert Latence(base_url="http://localhost:8090")
314+
print("latence SDK smoke passed")
315+
PY
316+
```
317+
318+
## Migration
319+
320+
Primary imports:
321+
322+
```python
323+
from latence import Latence, AsyncLatence
324+
```
325+
326+
Preview aliases remain available so existing TRACE preview code can move first
327+
and clean up names later:
328+
329+
```python
330+
from latence import LatenceTraceClient, AsyncLatenceTraceClient
331+
```
332+
333+
## More
334+
335+
- [Quickstart](docs/quickstart.md)
336+
- [Sessions](docs/sessions.md)
337+
- [Integrations](docs/integrations.md)
338+
- [Errors](docs/errors.md)
339+
- [Publishing](docs/publishing.md)
340+
- [Examples](examples/)
341+
- [TRACE runtime repo](https://github.com/latenceainew/latence-trace)
342+
- [Website](https://latence.ai)

‎docs/assets/latence-logo.svg‎

Lines changed: 25 additions & 0 deletions
Loading

0 commit comments

Comments
 (0)