nexusAI/packages/memory-service/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

See the root CLAUDE.md for overall architecture, service roles, and the dual-store memory model.

Running This Service

npm run memory             # From repo root (node src/index.js)
npm -w packages/memory-service run dev   # With --watch

Default port: 3002. Requires Qdrant and the embedding-service to be reachable on startup.

SQLite Schema

src/db/schema.js is the source of truth for the data model. Key schema facts:

• sessions and episodes are linked by FK with cascade delete — deleting a session removes all its episodes automatically.
• episodes_fts is an FTS5 virtual table that mirrors user_message and ai_response. It is kept in sync via SQL triggers on INSERT/UPDATE/DELETE. On service startup, the FTS index is fully rebuilt from live episode data.
• Several columns (sessions.name, sessions.project_id, entities.mention_count, etc.) were added as migrations using ALTER TABLE wrapped in individual try-catch blocks. Failures are silently swallowed — if a column already exists, the alter fails and the service continues. The idx_summaries_project index is defined twice (benign duplicate).
• summaries rows with session_id IS NULL and a project_id represent project-level overviews, not session summaries. This distinction is how GET /projects/:id/overview works.
• entity_episodes is a join table linking entities to the episodes where they were first extracted. Used for provenance tracking and future orphan cleanup. Defined in schema.js (not a migration), so it exists on all installs.

New columns on entities (added via migration):

• mention_count INTEGER DEFAULT 1 — incremented every time this entity is re-extracted
• confidence REAL DEFAULT 1.0 — reserved for future confidence scoring
• source TEXT DEFAULT 'extraction' — 'extraction' or 'manual'
• last_seen_at INTEGER — Unix timestamp of most recent extraction hit

New columns on relationships (added via migration):

• mention_count INTEGER DEFAULT 1 — incremented every time this edge is re-extracted
• notes TEXT — relationship context sentence from extraction

Async Pipeline: Episode Creation

POST /episodes returns a 201 as soon as the SQLite insert succeeds. Two background tasks run after without blocking the response:

1. Embedding — Fetches a vector from embedding-service, stores to Qdrant with {sessionId, createdAt} as payload metadata.
2. Entity + relationship extraction — Sends the episode text to Ollama (qwen2.5:3b, temp 0.1, 1500 tokens) and upserts any recognized entities and relationships to both SQLite and Qdrant. Also links each entity to the episode via entity_episodes.

Both tasks catch and log errors silently. An episode can exist in SQLite with no corresponding Qdrant point if either step fails.

Entity Extraction Details

src/entities/extraction.js:

• Fetches the last 20 known entities from SQLite before prompting the model, so the prompt can ask for name/type consistency with existing entries.
• Recognized entity types: person, place, project, technology, concept, organization — anything else is discarded.
• Ignores a hardcoded list of low-value names (hello, thanks, good morning, etc.).
• Extracts JSON using a regex ({...}) applied to raw model output, so surrounding prose doesn't break parsing.
• The model is asked to return both entities and relationships in a single JSON response: { "entities": [...], "relationships": [...] }.
• Entity upsert uses ON CONFLICT(name, type) DO UPDATE — preserves existing notes if the new extraction returns null, increments mention_count, updates last_seen_at.
• Relationship upsert uses ON CONFLICT(from_id, to_id, label) DO UPDATE — increments mention_count, preserves existing notes if new is null.
• Relationships are resolved by looking up both endpoints in the entityMap built during entity processing — if either entity wasn't saved (filtered out or invalid type), the relationship is silently dropped.
• After upsert, embeds each entity as "${name} (${type}): ${notes}" and stores to Qdrant with projectId in the payload for project-scoped filtering.

For full details see docs/services/entity-extraction.md and docs/services/knowledge-graph.md.

Knowledge Graph

src/graph/index.js provides two SQLite traversal functions:

• getNeighborhood(entityId, depth) — Single-entity recursive CTE traversal. Bidirectional (follows edges in both directions). Returns { nodes: [...entities], edges: [...relationships] }. Depth defaults to ENTITIES.GRAPH_HOP_DEPTH (1), max enforced to 3 at the HTTP layer.

• getEntityNeighbors(entityIds[]) — Bulk 1-hop version for orchestration. Given a set of seed entity IDs, returns their immediate neighbors plus all edges within the combined node set.

The recursive CTE uses UNION (not UNION ALL) to eliminate cycles and duplicate visits automatically.

For full design rationale and usage see docs/services/knowledge-graph.md.

Summarization Strategy

src/summarization/project.js:

• Preferred path: generate a project overview from existing session-level summaries (higher-level abstraction, shorter input).
• Fallback path: if no session summaries exist, summarize raw episodes directly (up to SUMMARIES.MAX_PROJECT_EPISODE_LIMIT).
• Both paths truncate input at SUMMARIES.MAX_SUMMARY_CHARS (8,000 chars) by slicing from the end (most recent content wins).
• Strips ChatML tokens from the Ollama response (<|im_start|>, <|im_end|>).
• Uses temp 0.2 and num_predict 1200.

Qdrant Client

src/semantic/index.js creates the Qdrant client lazily on first use and reuses it. All three collections (episodes, entities, summaries) are created at startup if missing. There is no connection health check — if Qdrant is unreachable, semantic operations throw at call time.

API Endpoints Quick Reference

Method	Path	Notes
GET	/health	Static response, no dependency checks
GET/POST	/sessions	POST requires externalId; duplicate → 409
GET/PATCH	/sessions/by-external/:externalId	PATCH accepts name, projectId
DELETE	/sessions/by-external/:externalId	Cascades to episodes, summaries, relationships
GET/POST	/episodes	POST triggers async embedding + entity/relationship extraction
GET	/episodes/search	FTS5 search; route must precede /:id
GET	/sessions/:id/episodes	Paginated, ordered created_at DESC
DELETE	/episodes/:id	Removes from SQLite + async Qdrant delete
POST	/entities	Upsert by (name, type); increments mention_count on conflict
GET	/entities/by-type/:type	All entities of given type
GET/DELETE	/entities/:id
POST	/relationships	Upsert by (fromId, toId, label); increments mention_count on conflict. Body: fromId, toId, label, notes (optional)
GET	/entities/:id/relationships	Outbound only
DELETE	/relationships	Body: fromId, toId, label
GET	/graph/neighborhood/:entityId	Single-entity neighborhood; ?depth= (default 1, max 3)
POST	/graph/neighbors	Bulk 1-hop neighborhood; body: { entityIds: [...] }
GET/POST	/projects	POST requires non-empty name
GET/PATCH/DELETE	/projects/:id
POST	/projects/:id/summarize	On-demand overview generation; 422 if no data
GET	/projects/:id/overview	Returns null (not 404) if no overview exists
GET	/projects/:id/summaries	All summaries for project
POST	/summaries	Requires content + at least one of sessionId/projectId
GET	/sessions/:id/summaries
PATCH/DELETE	/summaries/:id