Skip to content

Add normal chat support for creating fact and instruction memories #1153

Description

@paullizer

Summary

Normal chat should support both explicit and inferred durable memory creation for fact and instruction memories. Explicit requests can save directly when the user clearly asks to remember something. Inferred memories must be shown as separate reviewable conversation cards and must never be persisted until the user approves or edits and saves them.

User Value

Users can naturally build useful memory while chatting without switching to Profile or launching an agent, while still retaining control over inferred persistence. The assistant response stays focused on the conversation, and memory suggestions appear as separate cards that can be approved, denied, or edited.

Proposed Behavior

  • When fact memory is enabled by admins, normal chat should support explicit user requests to save a memory and persist them through the existing FactMemoryStore using the authorized chat scope.
  • Normal chat may infer candidate memories after the assistant response is available, but inferred candidates are saved only as pending proposal metadata until the user acts.
  • Inferred candidates should render as a separate card attached to the assistant message, not as inline assistant prose or prompt text.
  • Pending cards should support Approve, Deny, and Edit & Save actions.
  • Approve/Edit & Save should persist the final value through FactMemoryStore.set_fact(...); Deny should record a dismissed state only.
  • After Approve, Deny, or Edit & Save, the card should collapse and show the action result, such as "Saved fact memory: ..." or "Memory suggestion dismissed."
  • Saved entries should support both memory_type="instruction" and memory_type="fact".
  • Instruction memories should continue to apply to every future prompt; fact memories should continue to use relevance-based recall.
  • The implementation should not expose arbitrary plugin execution or bypass existing profile/agent authorization boundaries.

Acceptance Criteria

  • Normal non-agent chat can create a user-scoped fact memory from an explicit memory-save request.
  • Normal non-agent chat can create a user-scoped instruction memory from an explicit memory-save request.
  • Normal chat can infer candidate memories without saving them automatically.
  • Inferred memory candidates render in a separate conversation card attached to the assistant message.
  • Pending cards support approve, deny, and edit-save actions.
  • Cards collapse after action and persist their saved/dismissed result across conversation reloads.
  • Group chat uses the same authorized scope behavior already used for memory recall and agent plugin calls.
  • Standard and streaming chat paths behave consistently.
  • Created memories appear in Profile > Fact Memory and are available to existing instruction/fact recall flows.
  • Functional regression coverage validates write behavior, inference proposal behavior, admin toggle gating, memory type handling, authorization scope, standard/streaming parity, and persisted card state.
  • Documentation and release notes are updated with the configured app version.

Notes

Relevant areas discovered during planning:

  • application/single_app/semantic_kernel_fact_memory_store.py
  • application/single_app/semantic_kernel_plugins/fact_memory_plugin.py
  • application/single_app/route_backend_chats.py
  • application/single_app/route_frontend_profile.py
  • application/single_app/static/js/chat/chat-messages.js
  • application/single_app/static/js/chat/chat-streaming.js
  • functional_tests/test_fact_memory_profile_and_mini_sk.py
  • functional_tests/test_fact_memory_streaming_context_fix.py

Implementation direction:

  • Store pending proposals on assistant message metadata, for example metadata.fact_memory_proposals, so standard and streaming final payloads can reuse the existing message metadata flow.
  • Add review APIs that validate message/conversation access before saving approved memories or recording dismissals.
  • Use local static JavaScript only for the proposal card UI.

Metadata

Metadata

Labels

enhancementNew feature or request

Type

No type

Projects

Status
Pending Evaluation

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions