documentation updates

docs/services/memory-service.md

@@ -36,8 +36,9 @@ relationship extraction and embeds results into Qdrant.
 ```
 src/
 ├── db/
-│   ├── index.js       # SQLite connection + initialization + migrations
-│   ├── schema.js      # Table definitions, indexes, FTS5, triggers
+│   ├── index.js       # SQLite connection + init + migrate() + one-time FTS backfill
+│   ├── migrations.js  # Forward-only versioned migration runner (PRAGMA user_version)
+│   ├── schema.js      # Complete current shape: tables, indexes, FTS5, triggers
 │   ├── projects.js    # Project CRUD functions
 │   └── summaries.js   # Summary CRUD functions
 ├── episodic/
@@ -64,36 +65,56 @@ Eight core tables:
 - **summaries** — condensed episode groups for efficient context retrieval
 - **projects** — named groupings of sessions with `name`, `description`, `colour`, `icon`, `isolated`, `notes`, `system_prompt`
 
-### Migrations
+### Schema & Migrations
 
-Schema changes that cannot use `CREATE TABLE IF NOT EXISTS` are applied as
-idempotent migrations in `db/index.js` at startup:
+`schema.js` holds the **complete current shape** — every table, column, index,
+the FTS5 virtual table, and its triggers — as the single source of truth for a
+fresh database. It uses `CREATE TABLE IF NOT EXISTS`, so on a fresh DB it builds
+everything; on an existing DB it skips tables that already exist (and therefore
+does **not** reconcile columns on old tables — that's what migrations are for).
+
+`db/migrations.js` is a forward-only versioned runner keyed on
+`PRAGMA user_version`:
 
 ```js
-try { db.exec(`ALTER TABLE sessions ADD COLUMN name TEXT`); } catch {}
-try { db.exec(`ALTER TABLE sessions ADD COLUMN project_id INTEGER REFERENCES projects(id)`); } catch {}
-try { db.exec(`CREATE INDEX IF NOT EXISTS idx_sessions_project ON sessions(project_id)`); } catch {}
-try { db.exec(`ALTER TABLE projects ADD COLUMN isolated INTEGER NOT NULL DEFAULT 0`); } catch {}
-try { db.exec(`ALTER TABLE projects ADD COLUMN notes TEXT`); } catch {}
-try { db.exec(`ALTER TABLE projects ADD COLUMN system_prompt TEXT`); } catch {}
-// Knowledge graph columns:
-try { db.exec(`ALTER TABLE entities ADD COLUMN mention_count INTEGER NOT NULL DEFAULT 1`) } catch {}
-try { db.exec(`ALTER TABLE entities ADD COLUMN confidence REAL NOT NULL DEFAULT 1.0`) } catch {}
-try { db.exec(`ALTER TABLE entities ADD COLUMN source TEXT NOT NULL DEFAULT 'extraction'`) } catch {}
-try { db.exec(`ALTER TABLE entities ADD COLUMN last_seen_at INTEGER`) } catch {}
-try { db.exec(`ALTER TABLE relationships ADD COLUMN mention_count INTEGER NOT NULL DEFAULT 1`) } catch {}
-try { db.exec(`ALTER TABLE relationships ADD COLUMN notes TEXT`) } catch {}
+const migrations = [
+  (_db) => {},   // v0 → v1: consolidated baseline (historical ALTERs folded into schema.js)
+];
+const LATEST_VERSION = migrations.length;  // derived, never hand-maintained
 ```
 
-`entity_episodes` is defined in `schema.js` itself (not a migration) since it is a new table.
+`migrate(db)` reads `user_version`, applies every entry newer than it (each in a
+transaction alongside its version bump), and stamps the result. A fresh DB is
+built whole by `schema.js` and simply stamped to `LATEST_VERSION`; the baseline
+entry is a no-op.
 
-New migrations are always appended — never modify the schema file for existing tables since `ALTER TABLE` cannot use `IF NOT EXISTS`.
+**Adding a schema change:** append a new function to the `migrations` array
+(which bumps `LATEST_VERSION` automatically). Never edit an already-shipped
+entry, and never edit a table in `schema.js` expecting existing DBs to pick it
+up — they won't. This replaces the previous pattern of stacking silent
+`try/catch ALTER TABLE` statements in `db/index.js` on every boot.
+
+> **Consolidation note:** the historical ALTERs were folded into `schema.js`
+> rather than preserved as replayable migrations, so this assumes a fresh
+> database (which is the case post-wipe). An older, pre-consolidation database
+> would **not** auto-upgrade — `schema.js` skips its existing tables and the
+> baseline migration is a no-op. To support upgrading old DBs, the v1 baseline
+> would instead perform guarded (`ADD COLUMN if missing`) catch-up.
 
 ### FTS5 Full-Text Search
 
-An `episodes_fts` virtual table enables keyword search across all episodes.
-Three triggers (`episodes_fts_insert`, `episodes_fts_update`, `episodes_fts_delete`)
-keep the FTS index automatically in sync with the episodes table.
+An `episodes_fts` external-content virtual table enables keyword search across
+episodes. Three triggers (`episodes_fts_insert`, `episodes_fts_update`,
+`episodes_fts_delete`) keep the index in sync with the `episodes` table
+automatically during normal operation.
+
+A one-time backfill in `db/index.js` handles the case where the FTS table is
+created on a DB that already holds episodes (e.g. episodes predating FTS). It is
+gated on "did `episodes_fts` not exist before this boot," checked via
+`sqlite_master` **before** running the schema — not on a row-count comparison,
+because `COUNT(*)` on an external-content FTS5 table proxies the content table
+and cannot detect a desync. This replaced an unconditional full FTS rebuild that
+previously ran on every startup.
 
 ### SQLite Configuration