Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
7480ae2
feat(session): add the livekit-session crate manifest and module layout
MaxHeimbrock Nov 3, 2026
3272184
chore(workspace): register livekit-session with cargo, knope and CODE…
MaxHeimbrock Nov 3, 2026
265ca80
IMPORTANT(session): add the agent interop constants
MaxHeimbrock Nov 3, 2026
0d24a4c
feat(session): add the start and send error types
MaxHeimbrock Nov 3, 2026
961d4d4
feat(session): add the Message model and the MessageStore feed
MaxHeimbrock Nov 3, 2026
55d8892
test(session): cover MessageStore ordering, upsert and restore
MaxHeimbrock Nov 3, 2026
0063ead
IMPORTANT(session): aggregate lk.transcription streams into transcrip…
MaxHeimbrock Nov 3, 2026
d2a8d06
test(session): cover transcript aggregation
MaxHeimbrock Nov 3, 2026
2002a79
feat(session): add AgentState and AgentFailure
MaxHeimbrock Nov 3, 2026
5ff81dc
feat(session): find the agent, its avatar worker and their tracks
MaxHeimbrock Nov 3, 2026
de78368
IMPORTANT(session): project AgentState with timeout and leave detection
MaxHeimbrock Nov 3, 2026
4f1a9cd
test(session): cover agent discovery and state parsing
MaxHeimbrock Nov 3, 2026
5f10295
test(session): cover agent projection across reconnects, leaves and t…
MaxHeimbrock Nov 3, 2026
551210f
feat(session): add the Dispatcher fan-out
MaxHeimbrock Nov 3, 2026
b0f2d7b
test(session): cover Dispatcher pruning
MaxHeimbrock Nov 3, 2026
2556a3d
feat(session): add the state shared between a session and its actor
MaxHeimbrock Nov 3, 2026
0612af1
feat(session): add SessionOptions, SessionState and SessionEvent
MaxHeimbrock Nov 3, 2026
4c940e5
feat(session): add the Session handle and its constructors
MaxHeimbrock Nov 3, 2026
2b6e720
IMPORTANT(session): add the session actor and its agent-state projection
MaxHeimbrock Nov 3, 2026
fbf4f9c
IMPORTANT(session): read chat and transcript streams in the actor
MaxHeimbrock Nov 3, 2026
9460fab
IMPORTANT(session): start and end a session
MaxHeimbrock Nov 3, 2026
8b38587
feat(session): expose state, room, agent tracks and the message feed
MaxHeimbrock Nov 3, 2026
c1d40c0
test(session): assert with_agent fetches with the agent name
MaxHeimbrock Nov 3, 2026
f3b41f2
test(session): add the e2e harness with a stand-in agent
MaxHeimbrock Nov 3, 2026
af1e271
test(session): e2e agent state follows joins, leaves and reconnects
MaxHeimbrock Nov 3, 2026
0676e26
test(session): e2e agent connect timeout
MaxHeimbrock Nov 3, 2026
8e5af8e
test(session): e2e start publishes the platform microphone
MaxHeimbrock Nov 3, 2026
dace62c
test(session): e2e the feed merges chat and transcripts
MaxHeimbrock Nov 3, 2026
4f1b186
test(session): e2e agent audio and avatar video tracks
MaxHeimbrock Nov 3, 2026
72685f5
test(session): e2e restart after a disconnect and a rejected token
MaxHeimbrock Nov 3, 2026
67a3c6e
test(session): e2e end and drop cancel a pending start
MaxHeimbrock Nov 3, 2026
1635457
test(session): e2e a start cancelled during the handshake leaves noth…
MaxHeimbrock Nov 3, 2026
72f8df7
test(session): e2e memory lifecycle
MaxHeimbrock Nov 3, 2026
e6e0a5b
docs(session): add the crate README
MaxHeimbrock Nov 3, 2026
caaab57
feat(example): add the session voice and text chat example
MaxHeimbrock Nov 3, 2026
8895b1d
chore(session): add the changeset
MaxHeimbrock Nov 3, 2026
1765cc5
AgentState is now nested like Swift
MaxHeimbrock Nov 4, 2026
f7ea775
Moving waiting for publish microphone into the actor
MaxHeimbrock Nov 4, 2026
2992cf9
agent tracks can be tracked without touching room
MaxHeimbrock Nov 9, 2026
84a662c
Session options are now private
MaxHeimbrock Nov 9, 2026
3697a81
Documented that sids should be compared on state changeds
MaxHeimbrock Nov 9, 2026
caca86a
Room connection state Disconnected maps to session connection state D…
MaxHeimbrock Nov 16, 2026
1d7e337
Making actor enums non exhaustive
MaxHeimbrock Nov 16, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/add_livekit_session_crate.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
livekit-session: patch
---

Add the `livekit-session` crate, whose `Session` fetches credentials from a token source, connects a room, publishes the platform microphone and plays the agent's audio by default (`SessionOptions::with_platform_audio`), projects the agent's state (`AgentState`) into a `watch` channel, and merges chat and transcripts into one message feed, with a README and an `examples/session` voice and text chat.
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,5 @@
/livekit-wakeword/ @pham-tuan-binh
/livekit-net/ @jhugman
/livekit-token-source/ @MaxHeimbrock
/livekit-session/ @MaxHeimbrock
/livekit-signaling/ @lukasIO
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,13 +115,13 @@ check = [
]
```

A crate with no such table is not feature-checked at all. Today eleven crates are `"powerset"`, `livekit` and `livekit-api` are `"curated"`, and `livekit-ffi` deliberately declares nothing.
A crate with no such table is not feature-checked at all. Today twelve crates are `"powerset"`, `livekit`, `livekit-api` and `livekit-session` are `"curated"`, and `livekit-ffi` deliberately declares nothing.

- **When adding a workspace crate**, give it `mode = "powerset"` and stop there
- Its features are then picked up automatically as they are added, and no CI file changes
- Reach for `"curated"` only when the powerset is genuinely too expensive — that is a real loss of coverage, so it needs a reason recorded next to the table
- cargo-hack only varies the features of packages it is given. A crate with no table still gets built as a dependency, which makes it easy to assume it is covered when it is not
- **When adding a feature to a `"curated"` crate** (`livekit`, `livekit-api`), add the configurations a user would plausibly select to that crate's `check` list
Comment thread
1egoman marked this conversation as resolved.
Resolved
- **When adding a feature to a `"curated"` crate** (`livekit`, `livekit-api`, `livekit-session`), add the configurations a user would plausibly select to that crate's `check` list
- A curated crate's new feature is invisible to this job until it is listed there — this is the standing cost of `"curated"`
- Each entry is a complete feature set, run as `--no-default-features --features <entry joined by commas>`. Spell defaults as `"default"`, since a bare `[]` already means "no features at all"
- Pair the feature with what it realistically ships alongside (a TLS backend together with `native`, say) rather than listing it on its own
Expand Down
28 changes: 28 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ members = [
"livekit-telemetry",
"livekit-token",
"livekit-token-source",
"livekit-session",
"livekit-ffi-node-bindings",
"livekit-capture",
"livekit-net",
Expand Down Expand Up @@ -41,6 +42,7 @@ members = [
"examples/rpc",
"examples/save_to_disk",
"examples/screensharing",
"examples/session",
"examples/token_source",
"examples/webhooks",
]
Expand All @@ -64,6 +66,7 @@ livekit-telemetry = { version = "0.1.1", path = "livekit-telemetry" }
livekit-signaling = { version = "0.1.9", path = "livekit-signaling" }
livekit-token = { version = "0.4.0", path = "livekit-token" }
livekit-token-source = { version = "0.1.4", path = "livekit-token-source" }
livekit-session = { version = "0.1.0", path = "livekit-session" }
livekit-common = { version = "0.1.6", path = "livekit-common" }
livekit-data-stream = { version = "0.2.0", path = "livekit-data-stream" }
livekit-net = { version = "0.1.4", path = "livekit-net" }
Expand Down
11 changes: 11 additions & 0 deletions examples/session/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
[package]
name = "session"
version = "0.1.0"
edition.workspace = true
publish = false

[dependencies]
env_logger = { workspace = true }
livekit-session = { workspace = true, features = ["rustls-tls-native-roots"] }
livekit-token-source = { workspace = true }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: At present, users must manually depend on livekit and livekit-token-source at the correct version to use types from those crates that appear in the public API of livekit-session. I think it's worth thinking about re-exporting them from this crate to avoid that requirement:

// lib.rs
pub use livekit; 
pub use livekit_token_source;

@1egoman, curious your thoughts here.

@1egoman 1egoman Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good point to consider. I am a bit conflicted though, in other places to solve this type of problem (ie, a "peer dependency") I've seen places where standard practice seems to dictate that keeping multiple packages in sync via version to be what people recommend. I've never really liked this pattern though and think it is really easy to cause things to break.

I think if we went down this path, then there would be a natural push for people going forward to do livekit_session::livekit::Room instead of livekit::Room. Would the idea be that we'd point people at livekit_session::livekit::Room generally though going forward in all docs surfaces? Or a mixture of livekit::Room / livekit_session::livekit::Room?

Other idea: expose a session feature on livekit which inverts the dependency, would mean livekit-session stays internal, and that all users would always access livekit objects under the same namespace:

# Cargo.toml
livekit = { version = "0.10", features = ["session"] } 
// A user's main.rs
use livekit::session::Session;

tokio = { workspace = true, features = ["full"] }
28 changes: 28 additions & 0 deletions examples/session/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Session

A conversation with a LiveKit agent, built on `livekit-session`. The microphone is published and
the agent's audio is played through platform audio, with echo cancellation, so no headphones are
needed. Lines typed on stdin are sent to the agent as well; the agent's replies, what you say once
transcribed, and state changes are printed as they arrive.

Credentials come from a [development token server](https://docs.livekit.io/frontends/build/authentication/development-token-server/),
which also dispatches the agent:

```bash
LIVEKIT_TOKEN_SERVER_ID=<token server id> LIVEKIT_AGENT_NAME=<agent name> cargo run -p session
```

Or from a pre-generated token, whose room configuration dispatches the agent:

```bash
LIVEKIT_URL=wss://<project>.livekit.cloud LIVEKIT_TOKEN=<token> cargo run -p session
```

Pass `--text` to leave the audio devices alone and talk by typing only, e.g. on a machine without
a microphone. The agent then needs to accept text input.

```bash
cargo run -p session -- --text
```

Close stdin (Ctrl-D) to end the session.
99 changes: 99 additions & 0 deletions examples/session/src/main.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
use std::error::Error;
use std::{env, time::Duration};

use livekit_session::{
Message, MessageOrigin, Session, SessionEvent, SessionOptions, SessionState,
};

use livekit_token_source::{development_token_server, literal, TokenSourceFetchOptions};
use tokio::io::{AsyncBufReadExt, BufReader};

/// Creates a session with credentials from a pre-generated token if one is
/// given, and from a development token server otherwise.
fn create_session(options: SessionOptions) -> Result<Session, Box<dyn Error>> {
if let (Ok(url), Ok(token)) = (env::var("LIVEKIT_URL"), env::var("LIVEKIT_TOKEN")) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: Consider leveraging clap for argument parsing (and reading environment variables) for consistency with other examples. clap is already a workspace dependency.

return Ok(Session::new(literal(url, token), options));
}
let token_server_id = env::var("LIVEKIT_TOKEN_SERVER_ID")
.map_err(|_| "set LIVEKIT_TOKEN_SERVER_ID, or LIVEKIT_URL and LIVEKIT_TOKEN")?;
let mut fetch_options = TokenSourceFetchOptions::new();
if let Ok(agent_name) = env::var("LIVEKIT_AGENT_NAME") {
fetch_options = fetch_options.with_agent_name(agent_name);
}
Ok(Session::with_fetch_options(
development_token_server(token_server_id),
fetch_options,
options,
))
}

/// Prints a message once it is complete. The user's own input is skipped, as
/// it is already on screen.
fn print_message(message: &Message) {
if !message.is_final {
return;
}
let speaker = match message.origin {
MessageOrigin::AgentTranscript => "agent",
MessageOrigin::UserTranscript => "you (transcribed)",
MessageOrigin::UserInput => return,
_ => message.participant_identity.as_str(),
};
println!("{speaker}: {}", message.text);
}

fn print_state(state: &SessionState) {
println!("[{:?}, agent {:?}]", state.connection, state.agent);
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
env_logger::init();

// By default the session captures the microphone and plays the agent's
// audio, with echo cancellation between the two. `--text` keeps the
// conversation to typed lines, e.g. on a machine without audio devices.
let text_only = env::args().any(|arg| arg == "--text");
let session_options = SessionOptions::default()
.with_agent_connect_timeout(Duration::from_secs(15))
.with_platform_audio(!text_only);
let session = create_session(session_options)?;
let mut events = session.subscribe();
session.start().await?;

// `changed` reports only later updates, so print where the session starts.
let mut state = session.state();
print_state(&state.borrow_and_update());
let mut lines = BufReader::new(tokio::io::stdin()).lines();
loop {
tokio::select! {
changed = state.changed() => {
if changed.is_err() {
break;
}
let current = *state.borrow_and_update();
print_state(&current);
if !current.connection.is_connected() {
break;
}
}
Some(event) = events.recv() => {
if let SessionEvent::MessageUpserted(message) = event {
print_message(&message);
}
}
line = lines.next_line() => match line? {
Some(line) if line.trim().is_empty() => {}
Some(line) => {
if let Err(err) = session.send_text(line.trim()).await {
eprintln!("failed to send: {err}");
}
}
None => break,
},
}
}

session.end().await;
Ok(())
}
8 changes: 8 additions & 0 deletions knope.toml
Original file line number Diff line number Diff line change
Expand Up @@ -211,6 +211,14 @@ versioned_files = [
]
changelog = "livekit-token-source/CHANGELOG.md"

[packages.livekit-session]
versioned_files = [
"livekit-session/Cargo.toml",
"Cargo.lock",
{ path = "Cargo.toml", dependency = "livekit-session" },
]
changelog = "livekit-session/CHANGELOG.md"

[packages.livekit-capture]
versioned_files = [
"livekit-capture/Cargo.toml",
Expand Down
63 changes: 63 additions & 0 deletions livekit-session/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
[package]
name = "livekit-session"
description = "Agent sessions for the LiveKit Rust SDK"
version = "0.1.0"
license.workspace = true
edition.workspace = true
repository.workspace = true
readme = "README.md"

[features]
default = ["native"]

# Backend and TLS pass-throughs, forwarded to both the room connection
# (livekit) and the token fetch (livekit-token-source).
native = ["livekit/native", "livekit-token-source/native"]
native-tls = ["livekit/native-tls", "livekit-token-source/native-tls"]
native-tls-vendored = ["livekit/native-tls-vendored", "livekit-token-source/native-tls-vendored"]
rustls-tls-native-roots = [
"livekit/rustls-tls-native-roots",
"livekit-token-source/rustls-tls-native-roots",
]
rustls-tls-webpki-roots = [
"livekit/rustls-tls-webpki-roots",
"livekit-token-source/rustls-tls-webpki-roots",
]

# End-to-end tests against a running `livekit-server --dev`.
__lk-e2e-test = []

# How CI checks this crate's features, read by
# `.github/workflows/feature-combinations-curated.yml` via `cargo metadata`.
# See the equivalent table in `livekit/Cargo.toml` for the encoding.
#
# Curated rather than a powerset: every TLS feature is forwarded to `livekit`,
# so most powerset entries rebuild `livekit` underneath, and depth 2 would
# spend 22 checks mostly on two TLS backends at once or `__lk-e2e-test` paired
# with something unrelated, which nobody can select. The TLS forwards
# themselves are covered by `livekit`'s own list.
[package.metadata.feature-combinations]
mode = "curated"
check = [
["default"], # a plain `cargo check`
[], # --no-default-features
["native", "rustls-tls-webpki-roots"],
["default", "rustls-tls-native-roots"], # defaults *plus* one
]

[dependencies]
chrono = { workspace = true }
futures-util = { workspace = true, features = ["std"] }
livekit = { workspace = true }
livekit-token-source = { workspace = true }
Comment on lines +51 to +52

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Disabling defaults still enables native backends

Building livekit-session without default features still enables native networking. Both dependencies activate native by default, so the crate's native switch cannot disable either backend.

Suggested change
livekit = { workspace = true }
livekit-token-source = { workspace = true }
livekit = { workspace = true, default-features = false }
livekit-token-source = { workspace = true, default-features = false }

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

log = { workspace = true }
thiserror = { workspace = true }
tokio = { workspace = true, features = ["macros", "rt", "sync", "time"] }

[dev-dependencies]
anyhow = { workspace = true }
async-trait = "0.1"
livekit-api = { workspace = true }
livekit-protocol = { workspace = true }
livekit-token = { workspace = true }
tokio = { workspace = true, features = ["io-util", "macros", "net", "rt-multi-thread", "sync", "time"] }
Loading