# Orchestration Service

**Package:** `@nexusai/orchestration-service`  
**Location:** `packages/orchestration-service`  
**Deployed on:** Mini PC 2 (192.168.0.205)  
**Port:** 4000

## Purpose

The main entry point for all clients. Assembles context packages from
memory, routes prompts to inference, and writes new episodes back to
memory after each interaction. Clients never talk directly to the memory
or inference services — all traffic flows through orchestration.

## Dependencies

- `express` — HTTP API
- `cors` — cross-origin resource sharing middleware
- `dotenv` — environment variable loading
- `@nexusai/shared` — shared utilities

## Environment Variables

| Variable | Required | Default | Description |
|---|---|---|---|
| PORT | No | 4000 | Port to listen on |
| MEMORY_SERVICE_URL | No | http://localhost:3002 | Memory service URL |
| EMBEDDING_SERVICE_URL | No | http://localhost:3003 | Embedding service URL |
| INFERENCE_SERVICE_URL | No | http://localhost:3001 | Inference service URL |
| LLAMA_SERVER_URL | No | http://localhost:8080 | Direct llama-server URL for /models/props |
| QDRANT_URL | No | http://localhost:6333 | Qdrant URL for semantic search |
| CORS_ORIGIN | No | http://localhost:5173 | Allowed origin for CORS requests |
| EXTRACTION_URL | No | http://localhost:11434 | Ollama URL for summarisation |
| EXTRACTION_MODEL | No | qwen2.5:3b | Ollama model used for summarisation |

## Internal Structure

```
src/
├── services/
│   ├── memory.js         # HTTP client for memory service
│   ├── inference.js      # HTTP client for inference service
│   ├── embedding.js      # HTTP client for embedding service
│   ├── qdrant.js         # HTTP client for Qdrant (direct vector search)
│   └── summarization.js  # Session summarisation — triggers after each episode
├── chat/
│   └── index.js          # Core pipeline — context assembly, isolation, auto-naming
├── config/
│   └── settings.js       # Settings load/save — reads/writes data/settings.json
├── routes/
│   ├── chat.js           # POST /chat and POST /chat/stream
│   ├── sessions.js       # Session CRUD proxy
│   ├── projects.js       # Project CRUD proxy
│   ├── episodes.js       # Episode list and delete proxy
│   ├── summaries.js      # GET /summaries/session/:id and /summaries/project/:id
│   ├── settings.js       # GET /settings and PATCH /settings
│   ├── health.js         # GET /health/services — pings all four services
│   └── models.js         # GET /models and GET /models/props
└── index.js              # Express app entry point
```

The `services/` layer wraps all downstream HTTP calls in named functions.
URL or endpoint changes have a single place to be updated.

## Settings

Settings are persisted to `data/settings.json` and loaded on every request
via `appSettings.load()` — changes apply immediately without a service restart.

| Setting | Default | Description |
|---|---|---|
| `recentEpisodeLimit` | 5 | Recent episodes injected into prompt |
| `semanticLimit` | 5 | Semantic search results injected into prompt |
| `scoreThreshold` | 0.75 | Minimum similarity score for semantic results |
| `modelsFolderPath` | `/mnt/nexus-models` | Path to folder containing .gguf files |
| `temperature` | 0.7 | Inference temperature |
| `repeatPenalty` | 1.1 | Repeat token penalty |
| `topP` | 0.9 | Nucleus sampling probability mass |
| `topK` | 40 | Top-K token candidates per step |
| `systemPrompt` | *(ORCHESTRATION.SYSTEM_PROMPT)* | Global system prompt. `null` reverts to hardcoded constant. |

## Chat Pipeline

Both `POST /chat` and `POST /chat/stream` share the same steps. The only
difference is how the inference response is delivered to the client.

### Steps

1. **Session resolution** — look up session by `externalId`. Auto-create if
   not found.

2. **Project context resolution** — if the session has a `project_id`, fetch
   the project and all its session IDs. Used to scope semantic search. The
   project's `system_prompt` is also read at this step if set.

3. **System prompt resolution** — three-tier hierarchy:
   - `project.system_prompt` — highest priority
   - `settings.systemPrompt` — global setting from `settings.json`
   - `ORCHESTRATION.SYSTEM_PROMPT` — hardcoded constant (last resort)

4. **Recent episode retrieval** — fetch most recent episodes (`recentEpisodeLimit`).

5. **Semantic search** — embed user message, query Qdrant for similar past
   episodes. Deduplicated against recent episodes. Non-critical.

6. **Entity search** — query `entities` Qdrant collection filtered by
   `projectId`. Non-project sessions receive no entity context. Non-critical.

7. **Prompt assembly** — combine system prompt, entity context, semantic
   episodes, recent episodes, and user message.

8. **Inference** — send to inference service. `/chat` awaits full response;
   `/chat/stream` pipes SSE chunks to the client.

9. **Episode write** — write exchange back to memory with `projectId`.

10. **Summarisation trigger** — `triggerSummary(session, allEpisodes)` called
    fire-and-forget. See `summarization.md` for full details.

11. **Auto-naming** — on first message with no session name, fires a secondary
    inference call (max 20 tokens, temperature 0.3) to generate a session name.

### Prompt Structure

```
[Resolved system prompt]

Here is what you know about entities relevant to this conversation:
- {name} ({type}): {notes}
---
Here are some relevant memories from earlier conversations:
User: {past user message}
Assistant: {past ai response}
---
Here are some relevant memories from your past conversations:
User: {past user message}
Assistant: {past ai response}
--- End of recent memories ---

User: {current message}
Assistant:
```

## Summarisation

After each episode write, `triggerSummary` is called fire-and-forget. It
checks token thresholds and episode counts before generating, then stores
the result in the memory service.

> For full details on trigger conditions, prompt format, cumulative updates,
> ChatML token stripping, and episode range tracking, see `summarization.md`.

## SSE Stream Format

Inference service → orchestration:
```
data: {"response":"Hello","done":false}
data: {"done":true,"model":"gemma-4-26B...gguf","tokenCount":42}
data: [DONE]
```

Orchestration → client:
```
data: {"text":"Hello"}
data: {"done":true,"model":"gemma-4-26B...gguf","tokenCount":42}
```

The `[DONE]` sentinel is consumed internally and not forwarded.

## Models Route

`GET /models` scans `.gguf` files live from `modelsFolderPath` and merges
with `models.json` for metadata. Returns file size in GB.

`GET /models/props` fetches directly from llama-server. Returns
`{ contextWindow, modelAlias }`. Returns `503` if unreachable.

## Sessions Route Behaviour

`PATCH /sessions/:sessionId` accepts `name`, `projectId`, or both.
Rejects only when neither is provided — allows `useChat` to write project
assignment separately from rename operations.

## Caddy Configuration

Each route prefix needs a handle block in the Caddyfile on Mini PC 2.
**Any new top-level route must be added here AND in `vite.config.js`.**

```
handle /chat*      { reverse_proxy localhost:4000 }
handle /sessions*  { reverse_proxy localhost:4000 }
handle /models*    { reverse_proxy localhost:4000 }
handle /projects*  { reverse_proxy localhost:4000 }
handle /episodes*  { reverse_proxy localhost:4000 }
handle /settings*  { reverse_proxy localhost:4000 }
handle /summaries* { reverse_proxy localhost:4000 }
handle /health*    { reverse_proxy localhost:4000 }
```

After updating: `caddy reload --config /path/to/Caddyfile`

For all HTTP endpoints, see `api-routes.md`.