Skip to content

feat(providers): add Basic Memory provider - #47

Open
groksrc wants to merge 1 commit into
supermemoryai:mainfrom
groksrc:add-basic-memory-provider
Open

feat(providers): add Basic Memory provider#47
groksrc wants to merge 1 commit into
supermemoryai:mainfrom
groksrc:add-basic-memory-provider

Conversation

@groksrc

@groksrc groksrc commented Jun 11, 2026

Copy link
Copy Markdown

Summary

Adds a Basic Memory provider to memorybench (GitHub) — an open-source, local-first knowledge graph that stores information as Markdown files and exposes full-text + semantic (vector) search via a CLI and MCP server.

This PR implements the MemoryProvider interface for Basic Memory, closing basicmachines-co/basic-memory#672.

What changed

  • src/providers/basic-memory.ts — new provider that drives the local bm CLI via node:child_process.spawn (matching the pattern used by the filesystem and rag providers).
  • Isolation: each benchmark containerTag maps to a throwaway BM project rooted under data/providers/basic-memory/ (gitignored). The provider redirects BASIC_MEMORY_CONFIG_DIR / BASIC_MEMORY_HOME so it never touches the user's real config or projects.
  • Ingest: sessions are written as Markdown notes with speaker-attributed dialogue via bm tool write-note.
  • Indexing: polls bm status until sync settles, then runs bm reindex --full --embeddings to ensure vector embeddings are up to date (lazy indexing otherwise causes semantic search misses).
  • Search: uses bm tool search-notes --hybrid (FTS + semantic). BM's default FTS uses implicit-AND which fails on natural-language questions; hybrid mode correctly handles them.
  • Registration: provider wired into providers/index.ts, types/provider.ts (ProviderName), utils/config.ts (no API key required).
  • Docs: provider table in src/providers/README.md updated with a Basic Memory Setup section.

Setup (for reviewers)

# Install Basic Memory CLI
pip install basic-memory   # or: brew install basic-memory

# Run benchmark
bun run bench --provider basic-memory

The CLI path is auto-resolved at init; override with BASIC_MEMORY_CLI env var or config.baseUrl.


🤖 Generated with Claude Code

Add a provider that drives a local Basic Memory install (the `bm` CLI's
`bm tool ... --local` JSON interface) so Basic Memory can be benchmarked
alongside Supermemory, Mem0, Zep, filesystem, and rag.

- Maps each containerTag to a dedicated, throwaway BM project, isolated via
  BASIC_MEMORY_CONFIG_DIR/BASIC_MEMORY_HOME so it never touches the user's
  real config or projects.
- ingest: writes each session as a Markdown note via `bm tool write-note`.
- awaitIndexing: polls `bm status` until file/db sync settles, then runs
  `bm reindex --embeddings` (vector embeddings lag the FTS index and are
  required for semantic recall).
- search: `bm tool search-notes --hybrid` (full-text + semantic).
- clear: removes the BM project and its on-disk data.
- Resolves the CLI to an absolute path (Bun's spawn does no PATH lookup);
  overridable via BASIC_MEMORY_CLI.

Registers the provider, adds custom answer prompt, documents setup in the
providers README, and adds unit tests for the pure helpers. Verified with a
local smoke test (ingest -> reindex -> hybrid search -> clear) against a
throwaway project.

Co-Authored-By: Claude <noreply@anthropic.com>
@phernandez

Copy link
Copy Markdown

Checked this against current Basic Memory main (368e607622e9af3d982d0429cb48cfc6c83521f1) in a temporary config/project. Project creation and the exact stdin tool write-note command work and return JSON as expected.

One compatibility fix is needed in awaitIndexing: settled bm status now prints indexed and settled, not no changes. The current regex therefore waits out the full 120-second loop even when indexing is already idle. Prefer bm status --project PROJECT --local --json --wait --timeout 120 and check its exit status, or parse readiness.phase (idle) from JSON. The structured result also exposes files, relations and embeddings stages.

Please also propagate a failed embeddings rebuild rather than logging it and reporting every document completed. Similarly, returning [] after a failed search turns a provider/configuration failure into a retrieval miss in the benchmark score. Those should fail the run or be explicitly recorded as provider failures.

I have verified CLI creation/write/status compatibility only; I have not run a scored benchmark or validated hybrid quality. Keeping basicmachines-co/basic-memory#672 open until the provider lands and an end-to-end smoke run verifies ingest, indexing, search and cleanup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants