The pgvector adapter does not create tables or generate embeddings. It only defines the adapter contract for a future semantic memory backend and requires explicit `enabled: true`, an injected PostgreSQL pool, and either an input embedding or an injected `embedQuery(...)` function.
The server runtime uses `createMemoryV2Runtime(...)`. It keeps pgvector dormant unless all of these are true:
-`MEMORY_VECTOR_ENABLED=1`
-`MEMORY_PGVECTOR_DATABASE_URL` is set
-`MEMORY_PGVECTOR_EMBEDDING_MODULE` points to a module exporting `embedQuery(query, input)` or a default function
If any requirement is missing, pgvector reports `available=false` and Memory V2 falls back to the legacy conversation-memory backend. The runtime never reads the app's MySQL `DATABASE_URL` for pgvector.
When pgvector is available and `MEMORY_BACKEND=pgvector`, only `resolve(...)` uses pgvector. `write(...)` and `compact(...)` continue to use the legacy backend unless a future backend explicitly supports those operations.
Local development may call `ensurePgvectorMemorySchema(...)` manually to create an empty table. Production must treat the same schema as a controlled migration with backup, approval, and a separate backfill plan.
## API Contract
### `resolve(input)`
Reads memory for the current request.
Expected input:
```json
{
"userId":"h5 user id",
"sessionId":"goosed session id",
"query":"current user prompt",
"limit":40
}
```
Returns a normalized payload:
```json
{
"ok":true,
"enabled":true,
"skipped":false,
"source":"legacy-conversation-memory",
"profile":null,
"semanticMemories":[],
"behaviorSummary":null,
"activeGoals":[],
"memories":[]
}
```
### `write(input)`
Writes or queues memory-related evidence. In the legacy adapter this maps to `saveAndAnalyze(...)`.
Expected input:
```json
{
"userId":"h5 user id",
"sessionId":"goosed session id",
"messages":[]
}
```
### `compact(input)`
Compacts or analyzes pending user memory. In the legacy adapter this maps to `analyzeUser(...)`.
Expected input:
```json
{
"userId":"h5 user id",
"sessionId":"goosed session id"
}
```
### `getStatus()`
Returns the facade policy and backend contract status for read-only observability.
Example:
```json
{
"enabled":true,
"backend":"legacy",
"selectedBackend":"legacy-conversation-memory",
"profileEnabled":true,
"eventLogEnabled":true,
"vectorEnabled":false,
"failOpen":true,
"backends":[
{
"name":"legacy-conversation-memory",
"available":true,
"supports":{
"resolve":true,
"write":true,
"compact":true
}
}
]
}
```
## Backend Adapter Contract
Every backend adapter must be optional and fail-open. A backend may implement any subset of the API, but it must not create a new execution path.
Required field:
-`name`: stable backend name used by `MEMORY_BACKEND`
Optional fields:
-`isAvailable()`: returns `false` when the backend is configured but not usable
-`resolve(input)`: returns profile, semantic memories, behavior summary, active goals, or legacy memories
-`write(input)`: records or queues memory evidence
-`compact(input)`: performs pending extraction, summarization, or lifecycle maintenance
Backend implementations must not:
- call goosed directly
- mutate PG session state
- change SSE payloads
- block chat when unavailable
- assume they are the only memory backend
## Plugin Registry
Memory V2 exposes future plugin slots through read-only backend status. This is a contract registry, not a service integration.
The default facade includes unavailable placeholders for:
- backend smoke probes for every configured non-legacy backend
Unconfigured external backends are reported explicitly as `not_configured` instead of failing the whole check.
To see exactly which env vars are still missing for any backend, run:
```bash
npm run check:memory-v2-config -- --backend qdrant
npm run check:memory-v2-config -- --backend letta --format shell
```
`json` mode reports missing keys and readiness. `shell` mode prints a copy-paste export template for the selected backend.
When replacing the pgvector placeholder with the real disabled adapter:
```bash
npm run check:memory-v2-contracts -- --include-pgvector-adapter
```
Future adapters for Qdrant, Weaviate, Mem0, Letta, Neo4j, Redis Streams, or LangGraph must pass this contract before being wired into `createMemoryV2Runtime(...)`.
## Current Validation Baseline
As of the current Memory V2 rollout branch, local validation has already proven:
-`memoryV2.resolve(...)` can select `pgvector`
-`memoryV2.write(...)` and `memoryV2.compact(...)` remain legacy-first
-`/user-memory/v1/remember-recent` and `/user-memory/v1/sync` still reconcile back into the active session
- the live Portal path `agent/start -> agent/runs -> SSE Finish -> session detail` continues to work under Memory V2
That baseline is now captured by `npm run check:memory-v2-session` and included in `npm run check:memory-v2-stack`.
## Backend Scaffold
`memory-v2-adapter-scaffold.mjs` renders disabled-by-default backend adapter templates that satisfy the contract gate.
Dry-run a Qdrant adapter:
```bash
npm run scaffold:memory-v2-backend -- \
--name qdrant \
--category semantic \
--role scale-out-vector-store \
--capability resolve
```
Write a Mem0 adapter scaffold:
```bash
npm run scaffold:memory-v2-backend -- \
--name mem0 \
--category extraction \
--role automatic-memory-generation \
--capability write \
--capability compact \
--write
```
The scaffold command refuses to overwrite existing files. After generating a real adapter, run:
```bash
npm run check:memory-v2-contracts
```
Then add focused adapter tests before wiring the adapter into `createMemoryV2Runtime(...)`.
## Feature Flags
`MEMORY_ENABLED`
Global Memory V2 switch. If unset, it follows the existing `USER_CONVERSATION_MEMORY_ENABLED` behavior for backward compatibility.
`MEMORY_PROFILE_ENABLED`
Controls whether resolved profile payloads may be returned.
`MEMORY_EVENT_LOG_ENABLED`
Controls memory write/event capture. When disabled, `write(...)` returns a skipped result without touching the backend.
`MEMORY_VECTOR_ENABLED`
Reserved for future vector backends. It is disabled by default.
`MEMORY_BACKEND`
Preferred backend name. The current default is `legacy`.
`MEMORY_FAIL_OPEN`
Defaults to enabled. Backend failures return degraded empty payloads rather than blocking chat.
`MEMORY_PGVECTOR_DATABASE_URL`
Dedicated PostgreSQL connection string for Memory V2 semantic memory. It must not point at the MySQL business database and is ignored unless `MEMORY_VECTOR_ENABLED=1`.
`MEMORY_PGVECTOR_EMBEDDING_MODULE`
Optional module path for pgvector runtime retrieval. The module must export `embedQuery(query, input)` or a default async function. Without this module, runtime pgvector retrieval stays unavailable even when a PostgreSQL URL is present.
`MEMORY_PGVECTOR_TABLE`
Optional table name for pgvector retrieval. Defaults to `memory_embeddings`.
`MEMORY_PGVECTOR_POOL_MAX`
Optional PostgreSQL pool size for Memory V2 pgvector retrieval. Defaults to `5`.
## Current Integration Points
Memory V2 is only allowed to sit above existing memory code.
Read path:
```text
tkmind-proxy
-> memoryV2.resolve
-> legacy conversation-memory backend
-> existing reconcileAgentSession
-> existing buildSessionMemoryEntries
-> existing harness remember/bootstrap
```
Write path:
```text
/user-memory/v1/remember-recent
-> memoryV2.write
-> legacy conversation-memory saveAndAnalyze
```
Compact path:
```text
/user-memory/v1/sync
-> memoryV2.compact
-> legacy conversation-memory analyzeUser
```
If `MEMORY_ENABLED=0`, Memory V2 returns skipped/empty results and does not touch the legacy backend. The session reconcile path still runs for non-memory context such as sandbox guidance and time anchors.
Runtime observability:
```text
/api/runtime/status
-> memory: memoryV2.getStatus()
```
This is read-only and exists only to make rollout/debugging visible.
Local health gate:
```bash
npm run check:memory-v2-contracts
npm run check:memory-v2 -- --require-enabled --expect-backend legacy
```
Local app canary against a running Portal service:
```bash
npm run canary:memory-v2-app -- \
--base-url http://127.0.0.1:8081 \
--require-enabled \
--expect-backend legacy
```
For pgvector canary, run the app with explicit Memory V2 env and assert both configured and selected backends:
```bash
npm run canary:memory-v2-app -- \
--base-url http://127.0.0.1:18081 \
--require-enabled \
--require-target-healthy \
--expect-backend pgvector \
--expect-selected-backend pgvector
```
Production rollout details live in [production-rollout-runbook.md](production-rollout-runbook.md).
## Backend Plugin Slots
These are optional future backends. None is required for Memory V2 phase one.
Semantic memory:
- pgvector as the primary vector option
- Qdrant as a scale-out option
- Weaviate for knowledge graph fusion
Memory extraction:
- Mem0 for automatic memory generation
Memory lifecycle management:
- Letta for long-term and short-term memory operating semantics
Behavior and user model:
- Neo4j for behavior graph modeling
- Redis Streams for event tracking
Memory policy engine:
- LangGraph for routing and reasoning policy
## Integration Rule
Future integration must use existing Memind injection points. The preferred path is:
```text
tkmind-proxy / agent run
-> Memory V2 resolve
-> existing session reconcile
-> existing buildSessionMemoryEntries
-> existing harness remember/bootstrap
```
Do not add direct goosed changes, PG session changes, SSE protocol changes, or new blocking calls in the reply path.
## Rollout Checklist
Phase 1 rollout:
1. Keep `MEMORY_BACKEND=legacy`.
2. Enable `MEMORY_ENABLED=1`.
3. Keep `MEMORY_VECTOR_ENABLED=0`.
4. Verify `/api/runtime/status` reports `memory.enabled=true` and `selectedBackend=legacy-conversation-memory`.
5. Run targeted tests before release.
Rollback:
```bash
MEMORY_ENABLED=0
```
Rollback must stop Memory V2 reads/writes while preserving non-memory session reconcile behavior.
Future backend rollout:
1. Add the backend adapter behind `MEMORY_BACKEND=<name>`.
2. Replace the matching unavailable placeholder with the real adapter only inside Memory V2 wiring.
The CLI defaults to dry-run and never reads the existing MySQL `DATABASE_URL`. `--apply` requires an explicit PostgreSQL URL environment variable.
Expected table shape:
```sql
CREATETABLEmemory_embeddings(
idBIGSERIALPRIMARYKEY,
user_idTEXTNOTNULL,
contentTEXTNOTNULL,
embeddingVECTOR(1536)NOTNULL,
typeTEXTNOTNULLDEFAULT'fact',
source_memory_idTEXT,
source_session_idTEXT,
source_message_idTEXT,
metadataJSONBNOTNULLDEFAULT'{}'::jsonb,
created_atTIMESTAMPTZNOTNULLDEFAULTNOW(),
updated_atTIMESTAMPTZNOTNULLDEFAULTNOW()
);
```
Rollout rule:
Keep `MEMORY_BACKEND=legacy` until pgvector schema, embedding generation, backfill, and canary verification are designed separately.
Production migration considerations:
1. Confirm pgvector is installed on the target PostgreSQL server.
2. Run `CREATE EXTENSION IF NOT EXISTS vector` only through the approved DB migration path.
3. Create the empty `memory_embeddings` table first; do not backfill in the same release.
4. Keep `MEMORY_BACKEND=legacy` after table creation.
5. Build a separate backfill job from `h5_user_memory_items` into pgvector with idempotent checkpoints.
6. Verify row counts, embedding dimensions, and retrieval quality before canarying `MEMORY_BACKEND=pgvector`.
7. Rollback for the app remains `MEMORY_BACKEND=legacy` or `MEMORY_ENABLED=0`; database rollback should not delete populated memory rows without an explicit retention decision.
## pgvector Backfill Skeleton
`memory-v2-pgvector-backfill.mjs` defines the future MySQL-to-pgvector backfill contract.
The `source_memory_id` unique index makes backfill idempotent. Re-running the same batch updates content, embedding, type, source pointers, metadata, and `updated_at`.
Current constraints:
- Defaults to dry-run.
- Dry-run reads MySQL only and does not call embedding or PostgreSQL.
- Apply requires `pgPool.query(...)`.
- Apply requires an explicit `embedMemory(memory) => number[]`.
- Backfill is paginated with `{ updatedAt, id }` checkpoints.
- Backfill does not change `MEMORY_BACKEND`.
- Backfill does not create schema; run schema setup separately first.
The apply command requires the embedding module to export `embedMemory(memory)` or a default function. The CLI intentionally uses `MEMORY_BACKFILL_MYSQL_URL` instead of the app's normal MySQL env so production backfills are explicit and auditable.
Production backfill sequence:
1. Run schema migration and keep `MEMORY_BACKEND=legacy`.
2. Run dry-run batches and inspect counts/checkpoints.
3. Run apply batches with a fixed embedding model and recorded dimensions.
4. Store the last checkpoint externally after each successful batch.
5. Compare MySQL active memory count against pgvector distinct `source_memory_id` count.
6. Validate retrieval quality on a canary user.
7. Only then consider `MEMORY_BACKEND=pgvector` canary.
## pgvector Smoke Test
`memory-v2-pgvector-smoke.mjs` and `scripts/smoke-memory-v2-pgvector.mjs` provide a local pgvector read/write smoke test.
The smoke test:
- uses synthetic `memory-v2-smoke-*` rows only
- inserts deterministic vectors
- verifies the nearest vector is returned first through the Memory V2 pgvector adapter
- cleans synthetic rows by default
- never runs from app startup
- never reads the MySQL `DATABASE_URL`
Existing-schema smoke:
```bash
MEMORY_PGVECTOR_DATABASE_URL='postgresql://...'\
npm run smoke:memory-v2-pgvector
```
Local setup smoke that also creates an empty schema:
```bash
MEMORY_PGVECTOR_DATABASE_URL='postgresql://...'\
npm run smoke:memory-v2-pgvector -- \
--create-schema \
--create-extension \
--table memory_embeddings \
--dimensions 3
```
Production usage is limited to post-migration verification. Do not use the smoke script as a migration mechanism unless the migration plan explicitly approves `--create-schema`.
Expected with embedding module and reachable Qdrant:
```text
selectedBackend=qdrant
write_uses_legacy=true
compact_uses_legacy=true
```
## Mem0 Extraction Adapter
`memory-v2-mem0.mjs` defines the Memory Extraction backend boundary.
Current constraints:
- No Mem0 SDK is imported.
- Runtime uses a lightweight HTTP client only when `MEMORY_MEM0_ENABLED=1` and `MEMORY_MEM0_API_KEY` are configured.
- HTTP paths are configurable with `MEMORY_MEM0_WRITE_PATH` and `MEMORY_MEM0_COMPACT_PATH`.
- Without a full configuration, status reports `mem0.available=false` with a concrete reason and Memory V2 falls back to legacy.
-`MEMORY_BACKEND=mem0` does not affect `resolve(...)`.
-`write(...)` and `compact(...)` can use Mem0 when explicitly selected; legacy remains the default and rollback path.
Runtime configuration:
```bash
MEMORY_ENABLED=1
MEMORY_BACKEND=mem0
MEMORY_MEM0_ENABLED=1
MEMORY_MEM0_API_KEY='...'
MEMORY_MEM0_PROJECT_ID='memind_project'
MEMORY_MEM0_BASE_URL='https://api.mem0.ai'
```
Smoke:
```bash
npm run smoke:memory-v2-external -- --backend mem0 --operation write
```
## Letta Lifecycle Adapter
`memory-v2-letta.mjs` defines the Memory Lifecycle Management backend boundary.
Current constraints:
- No Letta SDK is imported.
- Runtime uses a lightweight HTTP client only when `MEMORY_LETTA_ENABLED=1`, `MEMORY_LETTA_API_KEY`, and `MEMORY_LETTA_AGENT_ID` are configured.
- HTTP paths are configurable with `MEMORY_LETTA_RESOLVE_PATH`, `MEMORY_LETTA_WRITE_PATH`, and `MEMORY_LETTA_COMPACT_PATH`.
- Without a full configuration, status reports `letta.available=false` with a concrete reason and Memory V2 falls back to legacy.
-`resolve(...)`, `write(...)`, and `compact(...)` can use Letta when explicitly selected.
Runtime configuration:
```bash
MEMORY_ENABLED=1
MEMORY_BACKEND=letta
MEMORY_LETTA_ENABLED=1
MEMORY_LETTA_API_KEY='...'
MEMORY_LETTA_PROJECT_ID='memind_project'
MEMORY_LETTA_AGENT_ID='agent_1'
MEMORY_LETTA_BASE_URL='https://api.letta.com'
```
Smoke:
```bash
npm run smoke:memory-v2-external -- --backend letta --operation resolve
```
Do not enable Letta as the selected backend in production until sampling, retention, conflict resolution, and rollback rules are defined for long/short-term lifecycle ownership.
## Remaining External Adapters
The remaining optional backends now have runtime-wired clients:
