# Orchestration Service

**Package:** `@nexusai/orchestration-service`  
**Location:** `packages/orchestration-service`  
**Deployed on:** Mini PC 2 (192.168.0.205)  
**Port:** 4000

## Purpose

The main entry point for all clients. Assembles context packages from
memory, routes prompts to inference, and writes new episodes back to
memory after each interaction. Clients never talk directly to the memory
or inference services — all traffic flows through orchestration.

## Dependencies

- `express` — HTTP API
- `node-fetch` — inter-service HTTP communication (memory service client only)
- `dotenv` — environment variable loading
- `@nexusai/shared` — shared utilities

> `memory.js` uses `node-fetch` v2 (pinned) because it is CommonJS. All other
> service clients use Node.js built-in `fetch`.

## Environment Variables

| Variable | Required | Default | Description |
|---|---|---|---|
| PORT | No | 4000 | Port to listen on |
| MEMORY_SERVICE_URL | No | http://localhost:3002 | Memory service URL |
| EMBEDDING_SERVICE_URL | No | http://localhost:3003 | Embedding service URL |
| INFERENCE_SERVICE_URL | No | http://localhost:3001 | Inference service URL |
| QDRANT_URL | No | http://localhost:6333 | Qdrant URL for semantic search |

## Internal Structure
```
src/
├── services/
│   ├── memory.js      # HTTP client for memory service
│   ├── inference.js   # HTTP client for inference service
│   ├── embedding.js   # HTTP client for embedding service
│   └── qdrant.js      # HTTP client for Qdrant vector search
├── chat/
│   └── index.js       # Core pipeline logic — context assembly and coordination
├── routes/
│   ├── chat.js        # POST /chat and POST /chat/stream route handlers
│   └── sessions.js    # GET /sessions/:sessionId/history route handler
└── index.js           # Express app entry point
```

The `services/` layer wraps all downstream HTTP calls in named functions,
keeping the pipeline logic in `chat/index.js` readable and ensuring that
URL or endpoint changes have a single place to be updated.

## Chat Pipeline

Both `POST /chat` and `POST /chat/stream` share the same context assembly
steps. The only difference is how the inference response is delivered to
the client.

1. **Session resolution** — looks up the session by `externalId` in the memory
   service. If not found, auto-creates a new session. Clients can generate a
   UUID for new conversations and pass it directly — no pre-creation step needed.

2. **Recent episode retrieval** — fetches the most recent episodes for the session
   (default: 10) from the memory service.

3. **Semantic search** — embeds the user message via the embedding service, then
   queries Qdrant for the top-5 most similar past episodes (score threshold: 0.75).
   Results are deduplicated against the recent episode set using a `Set` of IDs.
   Full episode content is fetched from the memory service by ID. This step is
   non-critical — if it fails, a warning is logged and the pipeline continues with
   recency-only context.

4. **Prompt assembly** — combines the system prompt, semantic episodes (if any),
   recent episodes, and the current user message into a single prompt string.

5. **Inference** — sends the assembled prompt to the inference service. `/chat`
   awaits the full response; `/chat/stream` opens an SSE connection and pipes
   chunks to the client as they arrive.

6. **Episode write** — writes the new exchange (user message + AI response)
   back to the memory service as a fire-and-forget operation. For streaming,
   the full response text is accumulated across chunks before writing.

7. **Response** — returns the AI response, model name, session ID, and token
   count to the client.

## Prompt Structure
[System prompt]
Here are some relevant memories from earlier conversations:
User: {past user message}
Assistant: {past ai response}
... (up to 5 semantic episodes)
Here is the recent conversation history:
User: {past user message}
Assistant: {past ai response}
... (up to 10 recent episodes)
--- End of memories ---
User: {current message}
Assistant:

Semantic episodes appear before recent episodes so the model encounters
long-range relevant context before the immediate conversation flow.

## SSE Stream Format

The inference service emits chunks in this format:
data: {"model":"companion:latest","response":"Hello","done":false}
data: {"model":"companion:latest","response":"!","done":true,"eval_count":3,...}
data: [DONE]

The orchestration service re-emits to the client as:
data: {"text":"Hello"}
data: {"text":"!"}
data: {"done":true}

The `[DONE]` sentinel from the inference service is consumed internally
and not forwarded. The client stream is terminated by `res.end()` after
the `{"done":true}` event.

## Endpoints

### Health

| Method | Path | Description |
|---|---|---|
| GET | /health | Service health check — reports downstream service URLs |

### Chat

| Method | Path | Description |
|---|---|---|
| POST | /chat | Send a message and receive a complete response |
| POST | /chat/stream | Send a message and receive a streaming SSE response |

### Sessions

| Method | Path | Description |
|---|---|---|
| GET | /sessions/:sessionId/history | Get paginated episode history for a session |

---

**POST /chat**

Request body:
```json
{
  "sessionId": "your-session-uuid",
  "message": "Hello, my name is Tim.",
  "model": "companion:latest",
  "temperature": 0.7
}
```

`model` and `temperature` are optional — fall back to inference service defaults
if omitted.

Response:
```json
{
  "sessionId": "your-session-uuid",
  "response": "Hello Tim! How can I help you today?",
  "model": "companion:latest",
  "tokenCount": 87
}
```

---

**POST /chat/stream**

Same request body as `POST /chat`.

Response is a stream of Server-Sent Events. Each event contains a text
delta. The stream ends with a `done` event.
data: {"text":"Hello"}
data: {"text":" Tim"}
data: {"text":"!"}
data: {"done":true}

Clients should read the `text` field from each chunk and accumulate them
to build the full response string. The connection is closed by the server
after the `{"done":true}` event.

---

**GET /sessions/:sessionId/history**

Returns paginated episode history for a session identified by its external ID.

Query parameters:

| Parameter | Default | Description |
|---|---|---|
| limit | 20 | Maximum number of episodes to return |
| offset | 0 | Number of episodes to skip (for pagination) |

Response:
```json
{
  "sessionId": "your-session-uuid",
  "episodes": [
    {
      "id": 42,
      "session_id": 1,
      "user_message": "Hello, my name is Tim.",
      "ai_response": "Hello Tim! How can I help you today?",
      "token_count": 87,
      "created_at": 1712345678,
      "metadata": null
    }
  ]
}
```

Episodes are ordered newest first. Returns `404` if the session does not exist.