README.md

local-llm — tshell + Koog agent sample

A CLI chat application that connects a local LLM (via any OpenAI-compatible API) to tshell through the Koog agent framework. The LLM gets a single tshell tool and can execute tshell code to answer questions, transform data, and explore files.

Prerequisites

• JDK 21+
• A local LLM server exposing an OpenAI-compatible API with tool-call support. llama.cpp llama-server works well:

# Example: start llama-server with a model
llama-server -m model.gguf --port 8080

Build

From the repository root:

./gradlew :local-llm:installDist

This produces the runnable distribution at local-llm/build/install/local-llm/.

Usage

Interactive chat

./local-llm/build/install/local-llm/bin/local-llm

Opens a REPL. The LLM can call the tshell tool to compute answers.

Single prompt

./local-llm/build/install/local-llm/bin/local-llm -p "What is 2^10?"

Runs one prompt and exits.

Custom server URL

./local-llm/build/install/local-llm/bin/local-llm --url http://localhost:1234

Default is http://localhost:8080.

File access

./local-llm/build/install/local-llm/bin/local-llm -d /path/to/project

Gives the LLM read-only access to the directory via tshell's FileToolkit. The LLM can then use tree(), read("file"), glob("**/*.kt"), and grep("path", {match: "pattern"}) to explore files.

CLI options

Option	Description
--url URL	Base URL for the OpenAI-compatible API (default: http://localhost:8080)
-p, --prompt TEXT	Run a single prompt and exit
-d, --dir PATH	Root directory for read-only file access

How it works

1. On startup, fetches /v1/models from the server and selects a model.
2. Sets up a TShell instance with CoreToolkit (and FileToolkit if --dir is given).
3. Wraps the shell in a TShellTools Koog ToolSet — a single tshell(code) tool.
4. Creates a Koog AIAgent with a system prompt that includes tshell syntax reference (auto-generated by shell.toPrompt()).
5. Each user message is sent to the agent, which may call tshell(...) one or more times before producing a final answer.

Console tracing (ConsoleTracingFeature) prints LLM calls, tool invocations, and results as they happen, so you can see the agent's reasoning.

Example session

$ ./local-llm/build/install/local-llm/bin/local-llm -d .
Connecting to http://localhost:8080 ...
Using model: my-model
File access: /home/user/project (read-only)

tshell local-llm chat (type 'quit' to exit)
──────────────────────────────────────────────────

you> What files are in this project?
  ⟶ LLM call...
  ⚙ Tool call: tshell(tree())
  ✓ Tool result: { ... }
  ⟶ LLM call...

llm> The project contains src/, build.gradle.kts, README.md, ...

you> How many Kotlin files?
  ⟶ LLM call...
  ⚙ Tool call: tshell(glob("**/*.kt") |> len())
  ✓ Tool result: 42
  ⟶ LLM call...

llm> There are 42 Kotlin files in the project.

you> quit
Goodbye.