# HTTP API guide

The SDK for capturing; the API for one-off calls and your backend. Most integrations combine both.

Next to the SDK, the API is for reading results later, polling, the catalogue, participants and contexts, your backend's tasks and any language. The full capture by HTTP stays here, for those who capture outside the browser or already have the frames.

## With the SDK

These are the usual uses of the API next to the SDK, each with its call:

- Read the results later, from your backend: `GET /results/{userProtocolId}/{protocolMechanicId}`.
- Follow a capture as it happens: `GET /stream/{userProtocolId}`.
- List the protocols you can enrol people in: `GET /catalog/protocols`.
- Check where a person stands: `GET /users/{externalId}/status` and `GET /users/{externalId}/user-protocols`.
- Read or ask again for the interpretation: `POST /analyze`.

From Node, `@neurologyca/capture-server` wraps these calls: see [Results in your backend](/docs/sdk/results-backend/). The key goes in your backend as `process.env.ENGINE_API_KEY`, never in the page.

## The full capture by HTTP

Use it from your server, in any language, when you capture outside the browser or already have the frames. This guide goes through a protocol of **one of your contexts with a taxonomy**, from the context to the interpretation. To measure without interpretation, see [Measurement only](/docs/guides/measurement/).

The commands are `bash` with `curl` and `jq`, as in the [quickstart](/docs/start/quickstart/). Your context, protocol and ids will differ: the guide reads them from your catalogue.

## Before you start

A token and a user, as in steps 1 and 2 of the quickstart:

```bash
export ENGINE="<engine base URL>"
export ENGINE_API_KEY="<your-api-key>"

TOKEN=$(curl -s -X POST "$ENGINE/token" -H "Content-Type: application/json" \
  -d "{\"apiKey\":\"$ENGINE_API_KEY\"}" | jq -r .token)
EXT="guide-$(date +%s)"
curl -s -X POST "$ENGINE/users" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d "{\"externalId\":\"$EXT\",\"name\":\"guide-user\"}" > /dev/null
```

Engine recommends a [baseline](/docs/concepts/baseline/) before any protocol; this guide skips it.

## 1. Pick a protocol with a taxonomy

Each protocol belongs to a **group**, and the group carries the taxonomy that the interpretation needs: the context id and, optionally, the target indicator. List the groups of your contexts that have one, and your published protocols:

```bash
curl -s "$ENGINE/catalog/protocol_group" -H "Authorization: Bearer $TOKEN" \
  | jq '.[] | select(.constellationContextId != null) | {pid, name, constellationContextId, targetConstellationHciId}'
curl -s "$ENGINE/catalog/protocols" -H "Authorization: Bearer $TOKEN" \
  | jq '.[] | select(.status == "published") | {pid, name, protocolGroupPid}'
```

Choose a protocol whose `protocolGroupPid` is one of those groups, and keep the taxonomy of its group:

```bash
PROTOCOL="<pid of a published protocol of your context>"
GROUP=$(curl -s "$ENGINE/catalog/protocols" -H "Authorization: Bearer $TOKEN" \
  | jq -r --arg p "$PROTOCOL" '.[] | select(.pid == $p) | .protocolGroupPid')
TAXONOMY=$(curl -s "$ENGINE/catalog/protocol_group" -H "Authorization: Bearer $TOKEN" \
  | jq -c --arg g "$GROUP" '.[] | select(.pid == $g)
      | {constellationContextId, constellationHciId: .targetConstellationHciId} | with_entries(select(.value != null))')
echo "$TAXONOMY"
```

Some contexts also define **subcontexts** (`taxonomy.subcontexts` in `GET /catalog/contexts`). Which subcontext each mechanic uses is part of your context's guide; add it as `subcontextId`. It is optional: without it, Engine uses the mechanic's own subcontext (`constellationSubcontextId` in the catalogue and the script) and, if the mechanic declares none, the default subcontext of the context; if you send one, yours wins. A misspelled one falls back to the default without an error. Today no mechanic declares its own yet.

## 2. Enrol and read the plan

```bash
UP=$(curl -s -X POST "$ENGINE/user-protocols" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d "{\"user\":\"$EXT\",\"protocol\":\"$PROTOCOL\"}" | jq -r .userProtocolId)
PM=$(curl -s "$ENGINE/user-protocols/$UP" -H "Authorization: Bearer $TOKEN" \
  | jq -r .nextProtocolMechanic.protocolMechanicId)
curl -s "$ENGINE/user-protocols/$UP/mechanics/$PM" -H "Authorization: Bearer $TOKEN" \
  | jq '.mechanic | {flowMode, steps: [.steps[] | {order, type, media: [.media[]?.pid]}]}'
```

The plan lists the steps of the position in order, with their media: what the person sees or hears in each step.

```json
{
  "mechanic": {
    "flowMode": "fixed",
    "steps": [
      {
        "order": 1,
        "type": "<step type>",
        "media": [
          {
            "pid": "<media pid>"
          }
        ]
      },
      {
        "order": 2,
        "type": "<step type>"
      }
    ]
  }
}
```

## 3. Open the live stream

Open the stream **before** the first batch and keep it open: it tells you when the interpretation is ready.

```bash
curl -sN "$ENGINE/stream/$UP" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Accept: text/event-stream" > stream.log &
STREAM=$!
```

The first event is `connected`; a `: ping` comment arrives every 30 seconds. See [HCIs and results](/docs/concepts/results/).

## 4. Set the context

Send the taxonomy **before the first frame** of the position:

```bash
curl -s -X POST "$ENGINE/user-protocols/$UP/$PM/context" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d "$TAXONOMY" | jq '{contextStored, taxonomyWarnings}'
```

The body you send looks like this:

```json
{
  "constellationContextId": "<your context id>",
  "constellationHciId": "<your target HCI id>"
}
```

Engine answers `201` and stores it:

```json
{
  "status": 201,
  "body": {
    "contextStored": true,
    "constellationContextId": "<your-constellationContext-id>",
    "constellationHciId": "<your-constellationHci-id>",
    "subcontextId": null,
    "situationId": null
  }
}
```

- Fields you omit in a later call keep their previous value: they cannot be emptied.
- An id Engine does not recognise is **not** rejected today: the answer is still `201` and lists it in `taxonomyWarnings` (`field`, `value`, `code`, `message`). Read that list. Engine checks the ids against its local copy of the taxonomy, which in the test environment's Engine is refreshed at start-up and every 6 hours and can still lag behind: a warning on the ids your kit gives you does not stop the analysis, and what confirms the context works is that `POST /analyze` answers.
- `409` if the position is already completed, or if the enrolment is retired.

## 5. Send each step

Every step of the plan needs its own batches and its own `Final`:

- `step` is the step's `order`, and `kpnfile.order` starts at 1 **in every step**.
- `mediaId` is the `pid` of the media you showed, in every batch of that step. With one media Engine resolves it; with several, the interpretation needs to know which one the person saw.
- On the `Final` of the **last** step, `triggerAnalysis: true` asks Engine to interpret when the position completes.
- In a mechanic with `flowMode: "free"`, also send `mechanicComplete: true` on that last `Final`.
- Leave a few seconds before each `Final`: Engine aggregates the step when the `Final` arrives, and the batch just before it may still be in process. The SDK waits 2.5 seconds.

### The frame format

Each item of `frames` describes one video frame as MediaPipe FaceLandmarker gives it. The SDK computes it for you; the [sample capture](/docs/samples/sample-capture.json) is a valid one. If you compute your own frames, match this:

| Field | What it carries |
| --- | --- |
| `clientTsMs` | When the frame was captured, in epoch milliseconds, increasing (about 33 ms apart at 30 fps) |
| `landmarks` | 478 points `[x, y, z]`: the 468 of the face and the 10 of the iris (FaceLandmarker with iris). `x` and `y` normalised from 0 to 1 over the image |
| `blendshapes` | The 52 categories of FaceLandmarker, in its order: `_neutral` at index 0, then the ARKit names (`jawOpen` 25, `mouthSmileLeft` 44, `mouthSmileRight` 45), with no `tongueOut`. From an ARKit dictionary (iOS), put each value at its MediaPipe index |
| `headPose` | `pitch`, `yaw` and `roll` in radians |
| `featuresMeta` | Without it, no valence or arousal for that frame. The sample's keys: `space: "frame"`, `z: "mediapipe_width"`, `image_width`, `image_height` (pixels) and `client_ts_ms` (the frame's `clientTsMs`) |

A frame with no face comes out of the codec with `landmarks: []` and `blendshapes: []`; the SDK does not send it (it drops every frame without a head pose), and neither should you. 478 are expected (468 face + 10 iris). With 468, measured on 2026-08-09, Engine accepted every batch and the session never closed; a frame with no face (`landmarks: []`) has not been measured.

This guide reuses the [sample capture](/docs/samples/sample-capture.json) of the quickstart for every step (download it from your browser into this folder). It is **synthetic**, with no voice, so the interpretation you get is not meaningful: it shows the flow, not a reading.

```bash
send_step() {  # $1 = step order, $2 = media pid or "", $3 = true on the last step
  LAST=$(jq 'length' sample-capture.json); i=0
  jq -c '.[]' sample-capture.json | while read -r packet; do
    i=$((i + 1)); [ "$i" -eq "$LAST" ] && sleep 3
    echo "$packet" | jq --arg up "$UP" --arg pm "$PM" --argjson step "$1" --arg media "$2" --argjson last "$3" \
      '. + {userProtocolId: $up, protocolMechanicId: $pm, step: $step}
         + (if $media != "" then {mediaId: $media} else {} end)
         + (if $last and .kpnfile.status == "Final" then {triggerAnalysis: true} else {} end)' \
    | curl -s -o /dev/null -w "step $1: %{http_code}\n" \
      -X POST "$ENGINE/process-kpn-emo" \
      -H "Authorization: Bearer $TOKEN" \
      -H "Content-Type: application/json" \
      --data-binary @-
  done
}

STEPS=$(curl -s "$ENGINE/user-protocols/$UP/mechanics/$PM" \
  -H "Authorization: Bearer $TOKEN" \
  | jq -c '[.mechanic.steps[] | {order, media: (.media[0].pid // "")}]')
N=$(echo "$STEPS" | jq 'length')
for k in $(seq 0 $((N - 1))); do
  ORDER=$(echo "$STEPS" | jq ".[$k].order"); MEDIA=$(echo "$STEPS" | jq -r ".[$k].media")
  send_step "$ORDER" "$MEDIA" "$([ "$k" -eq $((N - 1)) ] && echo true || echo false)"
done
```

Each batch answers `202`. The `Final` of the last step looks like this (frames left out):

```json
{
  "userProtocolId": "<UP>",
  "protocolMechanicId": "<PM>",
  "step": 2,
  "mediaId": "<media pid>",
  "kpnfile": {
    "order": 10,
    "status": "Final"
  },
  "triggerAnalysis": true,
  "frames": []
}
```

### Send the transcript

If you transcribe the session, send the text once, as top-level keys of the `Final` of the last step, next to `triggerAnalysis` (the SDK does the same):

- `speech`: the whole session, with `language`, `accuracy` (`null`), `text`, `list` (`[]`) and `spokenMs` (milliseconds with voice), plus `startMs` and `endMs` when you know them.
- `sttByStep`: the same speech split by step, `[{ step, text, startMs, endMs, spokenMs }]`, where `step` is the step's order (the SDK sends `null` for speech it could not place in a step; when you compute it, always send the order) and `startMs` and `endMs` are epoch milliseconds (the clock of `clientTsMs`) or `null`.
- `sttCutBy`, only when the transcription was cut.

The public spec does not type `sttByStep` yet, and its `speech` schema lists only `language`, `accuracy`, `text` and `list`: this is the shape the SDK sends and Engine accepts, and Engine stores it as sent. The interpretation's language is the person's `narrative.language`, not the transcript's.

## 6. Wait for the interpretation

When the position completes and the interpretation is stored, the stream emits `constellation.complete`. Use it as a signal, then read the interpretation with `POST /analyze`. A position that asked for the interpretation does not normally send `results.ready`: that event is for positions with no interpretation to wait for (see [Live events](/docs/concepts/results/#live-events)).

```bash
for _ in $(seq 1 60); do grep -q 'event: *constellation.complete' stream.log && break; sleep 5; done
kill "$STREAM"
curl -s -X POST "$ENGINE/analyze" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d "{\"userProtocolId\":\"$UP\",\"protocolMechanicId\":\"$PM\"}" \
  | jq '{from_cache, audience_applied, headline: .structured_data.headline}'
```

With a protocol of six steps that asks for the interpretation in its last `Final`, the stream carried this (`resultsReady` lists the `results.ready` events: none, as expected for a position that asks for the interpretation):

```json
{
  "events": [
    "connected",
    "hci.warning",
    "constellation.complete"
  ],
  "constellationComplete": true,
  "steps": 6,
  "resultsReady": []
}
```

`hci.warning` is a warning about an indicator your context includes: it arrives once per batch and it is not an alert; DevLab does not show the level that triggers it (`threshold`), so do not build on it; whether it is sent depends on the environment, so do not wait for it. The list above shows each event name once.

`POST /analyze` then answers from the stored interpretation (`from_cache: true`). `audience_applied` is the audience you asked for (in the request or in the user's profile), or `null` when there is none; it does not prove that the reading changed. The shape, with the texts replaced:

```json
{
  "status": 200,
  "body": {
    "interpretation": "<text>",
    "from_cache": true,
    "audience_applied": "<audience>",
    "structured_data": {
      "headline": "<text>",
      "sections": [
        {
          "title": "<text>",
          "body": "<text>"
        }
      ]
    },
    "metadata": {
      "status": "complete"
    }
  }
}
```

The indicators are in `GET /results/{userProtocolId}/{protocolMechanicId}`, one entry per step, as in the quickstart. The same response carries `speech`, the question and the transcript of each step, when there are any ([`speech`](/docs/concepts/results/#speech); only in the test environment's Engine).

## 7. Ask for another audience

The interpretation is written for an **audience**. Without `audience`, Engine uses the one in the user's profile (`audience` in `PATCH /users/{external_id}`), or the default. Neurologyca gives you the audiences available in your contexts: the kit lists those of each context and, when Neurologyca publishes it, marks the default one.

```bash
curl -s -X POST "$ENGINE/analyze" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d "{\"userProtocolId\":\"$UP\",\"protocolMechanicId\":\"$PM\",\"audience\":\"<audience>\"}" \
  | jq '{from_cache, audience_applied}'
```

```json
{ "userProtocolId": "<UP>", "protocolMechanicId": "<PM>", "audience": "<audience>" }
```

- Each audience is generated once per session and then served from the cache: the first request for an audience takes as long as an interpretation.
- Engine does not check the label: an audience it does not know is accepted **without an error**, echoed in `audience_applied` and generated as one more interpretation, with the default reading. Use only the audiences Neurologyca gives you.
- `POST /analyze` uses the last completed session of the position.

## Without the event

If you cannot keep a stream open, leave out `triggerAnalysis` and call `POST /analyze` when the results are ready. It waits for the interpretation, which can take about two minutes, so give your HTTP client a long timeout: a blocking call has also been seen to end in `504` while the interpretation was still being generated. See [Timeouts](/docs/guides/timeouts/).

## Streaming the interpretation

`POST /analyze/stream` takes the same body and answers with a stream of progress events that ends with the interpretation. It always generates a new interpretation and Engine does not store it. The event names come from the interpretation service and are **not** part of the contract: use it to show progress, and `POST /analyze` to keep the result.

## Your users, from your backend

Two reads help your backend keep track of the people it created. Both go with your key, never from the browser.

| Route | Returns |
| --- | --- |
| `GET /users` | The people of your organization, newest first, with who registered each one, its baseline and how many protocols it has. Optional filters: `externalId`, `developerName`, `role`, `hasBaseline`, `baselineIsValid` and `limit` |
| `GET /users/{external_id}/status` | Every protocol of that person (except the baseline one) with its state, computed on request (`capturing`, `awaiting_processing`, `completed`, `abandoned`…), its progress and the sessions still to do |

A `read_all` key also lists every person of the organization with `GET /users` (see [Access model](/docs/access/model/)). `@neurologyca/capture-server` offers `listUsers()`.

## What can go wrong

| What you see | Why |
| --- | --- |
| `201` with `taxonomyWarnings` | An id Engine does not recognise; today it warns instead of rejecting |
| `202` for every batch, but the position never completes | A step without its `Final`, or a free-flow mechanic without `mechanicComplete` |
| `400` with `KPN-EMO-MISSING-PACKETS` on a `Final` | A gap in `kpnfile.order`: resend the batches in `missingOrders`, then the `Final` |
| An aggregate that misses the last batches | The `Final` arrived right after the batch before it: leave a few seconds |
| `400` mentioning the media | A `mediaId` that does not exist |
| An interpretation that does not fit the audience you asked for | A label Engine does not know is accepted without an error; use the audiences Neurologyca gives you |
| `404 { "code": 404, "message": "Not found." }` from `/analyze` | An id that does not exist or is not yours |
| `404 { "code": 404, "message": "No results found." }` from `/analyze` | The position has no completed capture yet |
| `409` with `reason: PROTOCOL_NOT_ANALYSED` from `/analyze` | A protocol that is not interpreted, such as Measurement: read its indicators with `GET /results` |
| `504` from a blocking `/analyze` | The call waited too long for the interpretation; use the event, or ask again later |

The two `404` answers, as Engine sends them:

```json
{ "status": 404, "body": { "code": 404, "message": "Not found." } }
```

```json
{ "status": 404, "body": { "code": 404, "message": "No results found." } }
```

> The interpretation of each context is designed by Neurologyca with you: which taxonomy, subcontexts and audiences apply is part of the guide of your context.
