Skip to content
DevLab

The SDK for capturing; the API for one-off calls and your backend. Most integrations combine both.

Next to the SDK, the API is for reading results later, polling, the catalogue, participants and contexts, your backend’s tasks and any language. The full capture by HTTP stays here, for those who capture outside the browser or already have the frames.

These are the usual uses of the API next to the SDK, each with its call:

  • Read the results later, from your backend: GET /results/{userProtocolId}/{protocolMechanicId}.
  • Follow a capture as it happens: GET /stream/{userProtocolId}.
  • List the protocols you can enrol people in: GET /catalog/protocols.
  • Check where a person stands: GET /users/{externalId}/status and GET /users/{externalId}/user-protocols.
  • Read or ask again for the interpretation: POST /analyze.

From Node, @neurologyca/capture-server wraps these calls: see Results in your backend. The key goes in your backend as process.env.ENGINE_API_KEY, never in the page.

Use it from your server, in any language, when you capture outside the browser or already have the frames. This guide goes through a protocol of one of your contexts with a taxonomy, from the context to the interpretation. To measure without interpretation, see Measurement only.

The commands are bash with curl and jq, as in the quickstart. Your context, protocol and ids will differ: the guide reads them from your catalogue.

A token and a user, as in steps 1 and 2 of the quickstart:

Shell
export ENGINE="<engine base URL>"
export ENGINE_API_KEY="<your-api-key>"
TOKEN=$(curl -s -X POST "$ENGINE/token" -H "Content-Type: application/json" \
-d "{\"apiKey\":\"$ENGINE_API_KEY\"}" | jq -r .token)
EXT="guide-$(date +%s)"
curl -s -X POST "$ENGINE/users" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d "{\"externalId\":\"$EXT\",\"name\":\"guide-user\"}" > /dev/null

Engine recommends a baseline before any protocol; this guide skips it.

Each protocol belongs to a group, and the group carries the taxonomy that the interpretation needs: the context id and, optionally, the target indicator. List the groups of your contexts that have one, and your published protocols:

Shell
curl -s "$ENGINE/catalog/protocol_group" -H "Authorization: Bearer $TOKEN" \
| jq '.[] | select(.constellationContextId != null) | {pid, name, constellationContextId, targetConstellationHciId}'
curl -s "$ENGINE/catalog/protocols" -H "Authorization: Bearer $TOKEN" \
| jq '.[] | select(.status == "published") | {pid, name, protocolGroupPid}'

Choose a protocol whose protocolGroupPid is one of those groups, and keep the taxonomy of its group:

Shell
PROTOCOL="<pid of a published protocol of your context>"
GROUP=$(curl -s "$ENGINE/catalog/protocols" -H "Authorization: Bearer $TOKEN" \
| jq -r --arg p "$PROTOCOL" '.[] | select(.pid == $p) | .protocolGroupPid')
TAXONOMY=$(curl -s "$ENGINE/catalog/protocol_group" -H "Authorization: Bearer $TOKEN" \
| jq -c --arg g "$GROUP" '.[] | select(.pid == $g)
| {constellationContextId, constellationHciId: .targetConstellationHciId} | with_entries(select(.value != null))')
echo "$TAXONOMY"

Some contexts also define subcontexts (taxonomy.subcontexts in GET /catalog/contexts). Which subcontext each mechanic uses is part of your context’s guide; add it as subcontextId. It is optional: without it, Engine uses the mechanic’s own subcontext (constellationSubcontextId in the catalogue and the script) and, if the mechanic declares none, the default subcontext of the context; if you send one, yours wins. A misspelled one falls back to the default without an error. Today no mechanic declares its own yet.

Shell
UP=$(curl -s -X POST "$ENGINE/user-protocols" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d "{\"user\":\"$EXT\",\"protocol\":\"$PROTOCOL\"}" | jq -r .userProtocolId)
PM=$(curl -s "$ENGINE/user-protocols/$UP" -H "Authorization: Bearer $TOKEN" \
| jq -r .nextProtocolMechanic.protocolMechanicId)
curl -s "$ENGINE/user-protocols/$UP/mechanics/$PM" -H "Authorization: Bearer $TOKEN" \
| jq '.mechanic | {flowMode, steps: [.steps[] | {order, type, media: [.media[]?.pid]}]}'

The plan lists the steps of the position in order, with their media: what the person sees or hears in each step.

JSON
{
"mechanic": {
"flowMode": "fixed",
"steps": [
{
"order": 1,
"type": "<step type>",
"media": [
{
"pid": "<media pid>"
}
]
},
{
"order": 2,
"type": "<step type>"
}
]
}
}

Open the stream before the first batch and keep it open: it tells you when the interpretation is ready.

Shell
curl -sN "$ENGINE/stream/$UP" \
-H "Authorization: Bearer $TOKEN" \
-H "Accept: text/event-stream" > stream.log &
STREAM=$!

The first event is connected; a : ping comment arrives every 30 seconds. See HCIs and results.

Send the taxonomy before the first frame of the position:

Shell
curl -s -X POST "$ENGINE/user-protocols/$UP/$PM/context" \
-H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
-d "$TAXONOMY" | jq '{contextStored, taxonomyWarnings}'

The body you send looks like this:

JSON
{
"constellationContextId": "<your context id>",
"constellationHciId": "<your target HCI id>"
}

Engine answers 201 and stores it:

JSON
{
"status": 201,
"body": {
"contextStored": true,
"constellationContextId": "<your-constellationContext-id>",
"constellationHciId": "<your-constellationHci-id>",
"subcontextId": null,
"situationId": null
}
}
  • Fields you omit in a later call keep their previous value: they cannot be emptied.
  • An id Engine does not recognise is not rejected today: the answer is still 201 and lists it in taxonomyWarnings (field, value, code, message). Read that list. Engine checks the ids against its local copy of the taxonomy, which in the test environment’s Engine is refreshed at start-up and every 6 hours and can still lag behind: a warning on the ids your kit gives you does not stop the analysis, and what confirms the context works is that POST /analyze answers.
  • 409 if the position is already completed, or if the enrolment is retired.

Every step of the plan needs its own batches and its own Final:

  • step is the step’s order, and kpnfile.order starts at 1 in every step.
  • mediaId is the pid of the media you showed, in every batch of that step. With one media Engine resolves it; with several, the interpretation needs to know which one the person saw.
  • On the Final of the last step, triggerAnalysis: true asks Engine to interpret when the position completes.
  • In a mechanic with flowMode: "free", also send mechanicComplete: true on that last Final.
  • Leave a few seconds before each Final: Engine aggregates the step when the Final arrives, and the batch just before it may still be in process. The SDK waits 2.5 seconds.

Each item of frames describes one video frame as MediaPipe FaceLandmarker gives it. The SDK computes it for you; the sample capture is a valid one. If you compute your own frames, match this:

Field What it carries
clientTsMs When the frame was captured, in epoch milliseconds, increasing (about 33 ms apart at 30 fps)
landmarks 478 points [x, y, z]: the 468 of the face and the 10 of the iris (FaceLandmarker with iris). x and y normalised from 0 to 1 over the image
blendshapes The 52 categories of FaceLandmarker, in its order: _neutral at index 0, then the ARKit names (jawOpen 25, mouthSmileLeft 44, mouthSmileRight 45), with no tongueOut. From an ARKit dictionary (iOS), put each value at its MediaPipe index
headPose pitch, yaw and roll in radians
featuresMeta Without it, no valence or arousal for that frame. The sample’s keys: space: "frame", z: "mediapipe_width", image_width, image_height (pixels) and client_ts_ms (the frame’s clientTsMs)

A frame with no face comes out of the codec with landmarks: [] and blendshapes: []; the SDK does not send it (it drops every frame without a head pose), and neither should you. 478 are expected (468 face + 10 iris). With 468, measured on 2026-08-09, Engine accepted every batch and the session never closed; a frame with no face (landmarks: []) has not been measured.

This guide reuses the sample capture of the quickstart for every step (download it from your browser into this folder). It is synthetic, with no voice, so the interpretation you get is not meaningful: it shows the flow, not a reading.

Shell
send_step() { # $1 = step order, $2 = media pid or "", $3 = true on the last step
LAST=$(jq 'length' sample-capture.json); i=0
jq -c '.[]' sample-capture.json | while read -r packet; do
i=$((i + 1)); [ "$i" -eq "$LAST" ] && sleep 3
echo "$packet" | jq --arg up "$UP" --arg pm "$PM" --argjson step "$1" --arg media "$2" --argjson last "$3" \
'. + {userProtocolId: $up, protocolMechanicId: $pm, step: $step}
+ (if $media != "" then {mediaId: $media} else {} end)
+ (if $last and .kpnfile.status == "Final" then {triggerAnalysis: true} else {} end)' \
| curl -s -o /dev/null -w "step $1: %{http_code}\n" \
-X POST "$ENGINE/process-kpn-emo" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
--data-binary @-
done
}
STEPS=$(curl -s "$ENGINE/user-protocols/$UP/mechanics/$PM" \
-H "Authorization: Bearer $TOKEN" \
| jq -c '[.mechanic.steps[] | {order, media: (.media[0].pid // "")}]')
N=$(echo "$STEPS" | jq 'length')
for k in $(seq 0 $((N - 1))); do
ORDER=$(echo "$STEPS" | jq ".[$k].order"); MEDIA=$(echo "$STEPS" | jq -r ".[$k].media")
send_step "$ORDER" "$MEDIA" "$([ "$k" -eq $((N - 1)) ] && echo true || echo false)"
done

Each batch answers 202. The Final of the last step looks like this (frames left out):

JSON
{
"userProtocolId": "<UP>",
"protocolMechanicId": "<PM>",
"step": 2,
"mediaId": "<media pid>",
"kpnfile": {
"order": 10,
"status": "Final"
},
"triggerAnalysis": true,
"frames": []
}

If you transcribe the session, send the text once, as top-level keys of the Final of the last step, next to triggerAnalysis (the SDK does the same):

  • speech: the whole session, with language, accuracy (null), text, list ([]) and spokenMs (milliseconds with voice), plus startMs and endMs when you know them.
  • sttByStep: the same speech split by step, [{ step, text, startMs, endMs, spokenMs }], where step is the step’s order (the SDK sends null for speech it could not place in a step; when you compute it, always send the order) and startMs and endMs are epoch milliseconds (the clock of clientTsMs) or null.
  • sttCutBy, only when the transcription was cut.

The public spec does not type sttByStep yet, and its speech schema lists only language, accuracy, text and list: this is the shape the SDK sends and Engine accepts, and Engine stores it as sent. The interpretation’s language is the person’s narrative.language, not the transcript’s.

When the position completes and the interpretation is stored, the stream emits constellation.complete. Use it as a signal, then read the interpretation with POST /analyze. A position that asked for the interpretation does not normally send results.ready: that event is for positions with no interpretation to wait for (see Live events).

Shell
for _ in $(seq 1 60); do grep -q 'event: *constellation.complete' stream.log && break; sleep 5; done
kill "$STREAM"
curl -s -X POST "$ENGINE/analyze" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d "{\"userProtocolId\":\"$UP\",\"protocolMechanicId\":\"$PM\"}" \
| jq '{from_cache, audience_applied, headline: .structured_data.headline}'

With a protocol of six steps that asks for the interpretation in its last Final, the stream carried this (resultsReady lists the results.ready events: none, as expected for a position that asks for the interpretation):

JSON
{
"events": [
"connected",
"hci.warning",
"constellation.complete"
],
"constellationComplete": true,
"steps": 6,
"resultsReady": []
}

hci.warning is a warning about an indicator your context includes: it arrives once per batch and it is not an alert; DevLab does not show the level that triggers it (threshold), so do not build on it; whether it is sent depends on the environment, so do not wait for it. The list above shows each event name once.

POST /analyze then answers from the stored interpretation (from_cache: true). audience_applied is the audience you asked for (in the request or in the user’s profile), or null when there is none; it does not prove that the reading changed. The shape, with the texts replaced:

JSON
{
"status": 200,
"body": {
"interpretation": "<text>",
"from_cache": true,
"audience_applied": "<audience>",
"structured_data": {
"headline": "<text>",
"sections": [
{
"title": "<text>",
"body": "<text>"
}
]
},
"metadata": {
"status": "complete"
}
}
}

The indicators are in GET /results/{userProtocolId}/{protocolMechanicId}, one entry per step, as in the quickstart. The same response carries speech, the question and the transcript of each step, when there are any (speech; only in the test environment’s Engine).

The interpretation is written for an audience. Without audience, Engine uses the one in the user’s profile (audience in PATCH /users/{external_id}), or the default. Neurologyca gives you the audiences available in your contexts: the kit lists those of each context and, when Neurologyca publishes it, marks the default one.

Shell
curl -s -X POST "$ENGINE/analyze" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d "{\"userProtocolId\":\"$UP\",\"protocolMechanicId\":\"$PM\",\"audience\":\"<audience>\"}" \
| jq '{from_cache, audience_applied}'
JSON
{ "userProtocolId": "<UP>", "protocolMechanicId": "<PM>", "audience": "<audience>" }
  • Each audience is generated once per session and then served from the cache: the first request for an audience takes as long as an interpretation.
  • Engine does not check the label: an audience it does not know is accepted without an error, echoed in audience_applied and generated as one more interpretation, with the default reading. Use only the audiences Neurologyca gives you.
  • POST /analyze uses the last completed session of the position.

If you cannot keep a stream open, leave out triggerAnalysis and call POST /analyze when the results are ready. It waits for the interpretation, which can take about two minutes, so give your HTTP client a long timeout: a blocking call has also been seen to end in 504 while the interpretation was still being generated. See Timeouts.

POST /analyze/stream takes the same body and answers with a stream of progress events that ends with the interpretation. It always generates a new interpretation and Engine does not store it. The event names come from the interpretation service and are not part of the contract: use it to show progress, and POST /analyze to keep the result.

Two reads help your backend keep track of the people it created. Both go with your key, never from the browser.

Route Returns
GET /users The people of your organization, newest first, with who registered each one, its baseline and how many protocols it has. Optional filters: externalId, developerName, role, hasBaseline, baselineIsValid and limit
GET /users/{external_id}/status Every protocol of that person (except the baseline one) with its state, computed on request (capturing, awaiting_processing, completed, abandoned…), its progress and the sessions still to do

A read_all key also lists every person of the organization with GET /users (see Access model). @neurologyca/capture-server offers listUsers().

What you see Why
201 with taxonomyWarnings An id Engine does not recognise; today it warns instead of rejecting
202 for every batch, but the position never completes A step without its Final, or a free-flow mechanic without mechanicComplete
400 with KPN-EMO-MISSING-PACKETS on a Final A gap in kpnfile.order: resend the batches in missingOrders, then the Final
An aggregate that misses the last batches The Final arrived right after the batch before it: leave a few seconds
400 mentioning the media A mediaId that does not exist
An interpretation that does not fit the audience you asked for A label Engine does not know is accepted without an error; use the audiences Neurologyca gives you
404 { "code": 404, "message": "Not found." } from /analyze An id that does not exist or is not yours
404 { "code": 404, "message": "No results found." } from /analyze The position has no completed capture yet
409 with reason: PROTOCOL_NOT_ANALYSED from /analyze A protocol that is not interpreted, such as Measurement: read its indicators with GET /results
504 from a blocking /analyze The call waited too long for the interpretation; use the event, or ask again later

The two 404 answers, as Engine sends them:

JSON
{ "status": 404, "body": { "code": 404, "message": "Not found." } }
JSON
{ "status": 404, "body": { "code": 404, "message": "No results found." } }