Processing service
What our processing service does with a recording, what you expose to it, the token URL, the webhook, its error codes and Measurement in deferred.
What the service does
Section titled “What the service does”It downloads the video from your backend, replays the capture against Engine with the same SDK (replayRecording from @neurologyca/capture/replay), transcribes the recording’s audio (all of it, when the job asks for any step) and passes to Engine only the text of the steps each job asks for (transcribeSteps), and tells your backend with a signed webhook. It does not compute indicators: Engine does. You never run /replay yourself: the service does.
With our service, or self-hosted
Section titled “With our service, or self-hosted”| Aspect | With our service (available today) | Self-hosted (coming later) |
|---|---|---|
| What you get | Nothing to deploy. One credential per organization: the service URL (PROCESSOR_URL) and its outbound IP, in your kit; the processor key (PROCESSOR_API_KEY) and the webhook secret (PROCESSOR_SECRET), one pair shared by every team: any active team gets it in its recording starter’s zip with Include my key and the processor credentials, and your organization’s admin team reveals and rotates it in Your account |
The processor (a container image) and its hosting guide, to run it on your own servers. Later, without dates |
| What you expose | Three public HTTPS routes under PUBLIC_BASE_URL: the signed video, the token URL and the webhook |
Nothing public: an internal listener, reachable only from the processor’s network |
| What you keep | Different secrets for the signed URLs and the webhook, one nonce store per purpose and a table of jobs | The same |
| Context, language, transcription | Per job: every submit carries that capture’s taxonomy, language and transcribeSteps |
The same; a processor whose client fixes them accepts the same values or none |
- Limiting the three routes to the service’s outbound IP is defense in depth, not authentication. That IP is the same for every organization: another one can make the service call your routes from it. What protects you is the HMAC signature with your
PROCESSOR_SECRET, whichcreateWebhookHandlerandcreateProcessorTokenHandlerverify, and the video URL’s signature. Never skip either, whether you limit by IP or not. PUBLIC_BASE_URLis your backend’s public HTTPS origin. The service accepts any publichttps:origin and refuseshttp:,localhost, private and loopback addresses, redirects and URLs with credentials (E_DESTINATION_NOT_ALLOWED). To test on your machine, the service cannot calllocalhost: open a public HTTPS tunnel (for example cloudflared or ngrok), put its origin inPUBLIC_BASE_URL, and close it when you finish; or deploy. The recording starter explains both.- If a credential leaks, Neurologyca disables your organization’s access to the service (
E_CLIENT_DISABLED; jobs in flight finish), your admin team rotates in two steps —the new values stay pending until you deploy them and activate them— and the access is re-enabled. No webhook in flight is signed with a secret your backend does not have yet: If a processor credential leaks.
Submitting a recording
Section titled “Submitting a recording”From your receiver’s onComplete, sign a URL for the video and submit the job with that capture’s parameters. Save the jobId.
// Your backend, handing a recording to Neurologyca's processing service (Node 20.3+). `url`, `apiKey` and the webhook// secret come from your DevLab account; `publicBase` is your backend's public HTTPS origin (PUBLIC_BASE_URL), where the// service calls the video, the token URL and the webhook. Never localhost: to test on your machine, use a public tunnel.import { createProcessorClient } from '@neurologyca/capture-server'
/** * @param {object} o * @param {string} o.url the processing service's URL (PROCESSOR_URL, from your kit) * @param {string} o.apiKey your key for the service (PROCESSOR_API_KEY; never the Engine key) * @param {Pick<import('@neurologyca/capture-server').UploadReceiver, 'signedUrl' | 'manifest'>} o.receiver your upload receiver * @param {string} o.publicBase your backend's public origin for the service (PUBLIC_BASE_URL, HTTPS) * @param {string} o.engineBase your environment's Engine URL (ENGINE_BASE) */export function processing(o) { const processor = createProcessorClient({ url: o.url, apiKey: o.apiKey }) return { // Call it from your receiver's onComplete. Idempotent per captureId: the service answers with the same job. // `params` are THIS capture's: taxonomy.contextId (your page's) and language (the session's) are required by the // service; with a non-empty transcribeSteps the service transcribes the whole audio of the recording and passes to Engine only // the text of the listed step types (absent or [] = nothing is transcribed). async submit( /** @type {string} */ captureId, /** @type {{ taxonomy: import('@neurologyca/capture-server').ProcessorJobTaxonomy, language: string, transcribeSteps?: string[] }} */ params, ) { const videoUrl = await o.receiver.signedUrl(captureId, { baseUrl: `${o.publicBase}/media` }) // valid 24 h by default const manifest = /** @type {{ userProtocolId: string, protocolMechanicId: string }} */ (await o.receiver.manifest(captureId)) const { jobId, status } = await processor.submit({ videoUrl, manifest, engineBaseUrl: o.engineBase, tokenUrl: `${o.publicBase}/processor/token`, webhookUrl: `${o.publicBase}/processor/webhook`, taxonomy: params.taxonomy, language: params.language, transcribeSteps: params.transcribeSteps ?? [], }) // Save all of it next to the captureId: the token URL compares the pair with the JOB's, because the upload // (manifest included) is deleted at `interpreting` while the job still runs. return { jobId, status, userProtocolId: manifest.userProtocolId, protocolMechanicId: manifest.protocolMechanicId } }, job: (/** @type {string} */ jobId) => processor.getJob(jobId), // poll when a webhook does not arrive }}With our service, engineBaseUrl must be an Engine the service admits (your account’s): any other is refused at submit. Every submit carries THAT capture’s parameters: taxonomy (contextId required, hciId and subcontextId optional; the same taxonomy.contextId as your page’s configuration), language (the session’s language, for example 'es'), transcribeSteps (the step types to transcribe; [] is always none; absent is none with the hosted service; with a self-hosted service, the steps that its own configuration sets for your client) and, optionally, interimWebhook. The service requires taxonomy.contextId and language: without them, 400 E_BAD_REQUEST. createProcessorClient validates them before sending (a malformed value is a TypeError at the call) and only sends the ones you pass. They need capture-server 0.13.0 or later.
The service only accepts MP4 with H.264, which is what the recording client produces. If it refuses the call, submit rejects with a ProcessorError (code, status, retryAfterMs); every call gives up after 30 s.
The token URL
Section titled “The token URL”Mount the token URL and the webhook before any body parser: both verify the signature over the raw body. The service asks your backend for an Engine token, signed, with the capture’s pair and its jobId. createProcessorTokenHandler verifies the signature; your authorize decides whether to hand a token: only if the jobId is the one submit returned, the job is still running (interpreting included) and the pair is the one you stored for that job at submit. Never compare it with receiver.manifest(): after the interpreting notice the upload, manifest included, is already deleted, the service still renews its token while it waits for the interpretation, and a denial ends that wait in a false timeout. By default, the token is your organization’s developer token, not scoped to that capture: authorize limits when it is issued, never what it can do. This is a condition for production.
Since capture-server 0.10.0 you can hand a token scoped to the capture instead: captureToken: { baseUrl } in createProcessorTokenHandler, with baseUrl your environment’s Engine URL (ENGINE_BASE). The handler asks Engine for it with POST /token/capture, for exactly the signed pair and captureId, authenticated with your tokens (the developer token never leaves your backend), and hands out only that token. It lasts 30 minutes (the service asks again when it expires) and reaches only that capture’s ingestion, its plan and its stream. It is off by default: today only the test environment’s Engine has that route; against an Engine without it, the handler answers a 502 that your onError sees, never a silent failure. With it on, the replay calls no /analyze* (the token may not, and the service replays with awaitInterpretation: false), so it never warns analysis.out-of-token-scope: Engine runs the interpretation the closing Final asked for, and you read it later with your own credential. Engine also limits that token’s requests per capture and per minute; since @neurologyca/capture 0.37.0 the replay waits the Retry-After and goes on, slower, with the warning transport.rate-limited (a processor on an earlier version fails the job). What that token can and cannot do: A token for one capture. DevLab’s own deferred mode can hand it too: an option of its server, off by default, only against the test environment’s Engine.
The webhook
Section titled “The webhook”The service tells your backend when the job ends (succeeded or failed). The contract (signed fields, nonces, at-least-once, idempotence, getJob as the fallback): Results in your backend.
Since capture-server 0.11.0 there is also an interim notice, status: 'interpreting', signed like the others, with report and processor set to null and no failure fields. The service sends it while interimWebhook is on: with our service, by default (turn it off per job with interimWebhook: false in submit). A 0.10.0 receiver answers it with a 400. It is not terminal:
- Delete the video at
interpretingor atsucceeded, whichever comes first, and keep the job open untilsucceeded. Without the notice (off, or lost) the video stays in your receiver for the whole wait, up to 300 s more. - Branch on
interpretingfirst. AnonEventthat treats every status butsucceededas a failure closes the job: the token renewal is then denied, the wait ends in a falsetimeout, and the video is never deleted. Update the handler before you deploy against our service, whereinterimWebhookis on by default. - An
interpretingthat arrives after the terminal event (a retried delivery) reopens nothing.
The interpretation
Section titled “The interpretation”Since capture-server 0.11.0 the job waits for Engine’s interpretation. If the protocol is interpreted, the last Final asks for it and Engine produces it on its own (it saves it and announces it with constellation.complete); meanwhile the job is interpreting, up to 300 s by default. It counts as in flight, getJob returns it as such, and it never ends in failed. succeeded arrives once the interpretation is ready or once it is known it will not come, and its interpretation (in the webhook and in getJob) says how the analysis ended. The job ends succeeded whatever that value is: the indicators are in /results. The text never travels in the webhook: read it with your credential (client.analyze() or POST /analyze).
interpretation |
What it means | What to do |
|---|---|---|
ready |
The analysis came out complete | Read it now: it comes from the cache (from_cache: true) and costs no tokens again. Measured on 2026-10-13: 726 ms |
degraded |
It arrived, but the analysis came out degraded | No call reads the stored analysis without redoing it: every /analyze regenerates it (tens of seconds, up to about two minutes, and LLM tokens) and returns the new one or, if it comes out worse, the stored one. Ask once if you need it, never in a loop |
failed |
It arrived, but the analysis failed | The same as degraded |
timeout |
No event within the wait | Not a failure. You can ask for it, at the cost of LLM tokens if it is not stored: Engine generates it then |
unknown |
It was asked for and how it ended is not known | You can ask for it, at the cost of LLM tokens. When processor.interpretation.reconnects is a number and the outcome is not already-processed, the event arrived without a readable status: the analysis is stored and /analyze serves it from the cache, as with ready. reconnects: null is a service restart during the wait; outcome: 'already-processed', a session processed by an earlier job |
not-requested |
It was not asked for (triggerAnalysis: false, a protocol that is not interpreted), or Engine skipped it because an input was missing (for example, the transcript of an answer that ended without speech) |
Nothing to ask for; in the second case, record again with speech in every answer |
The job carries the wait in processor.interpretation (e.processor in the webhook, job.processor in getJob; not in report): { outcome, waitedMs, reconnects, lastClose }. Why waiting matters, measured by DevLab on 2026-10-13: asked when the replay ended, from_cache: false (another one was generated and paid twice); asked 300 s later, from_cache: true both times.
With the capture token (captureToken) nothing changes here: the service replays with awaitInterpretation: false and calls no /analyze* (out of that token’s scope), so analysis.out-of-token-scope does not appear. The session ends complete, and the interpretation is the one Engine makes on its own.
Error codes
Section titled “Error codes”From the service’s error catalogue. “/retry” says whether the job can be retried; “Reservation”, whether the service releases the person’s reservation in Engine.
| Code | When | /retry | Reservation | What your backend does |
|---|---|---|---|---|
Call errors (ProcessorError, submit or retry rejects with them) |
E_UNAUTHORIZED (401): no credential or a wrong one. E_CLIENT_DISABLED (403): your organization’s access to the service is disabled; submit and retry refuse, getJob still answers, and jobs in flight finish. E_BAD_REQUEST (400; 413 over 4 MiB; with our service, also a job without taxonomy.contextId or language), E_DESTINATION_NOT_ALLOWED (400, the job’s URLs are outside the service’s rules, for example a PUBLIC_BASE_URL on http:), E_JOB_PARAMS_CONFLICT (400, self-hosted: the processor’s client fixes taxonomy, language, transcribeSteps or interimWebhook and the job brings another value), E_CAPTURE_ID_CONFLICT (409, the captureId belongs to another client), E_NOT_RETRYABLE (409, retry on a job that is not failed or not retryable). E_STARTING and E_STOPPING (503). E_PURGE_PENDING (503, Retry-After: 5, retry while the job’s files are deleted). E_PROCESSOR_HTTP (an error response without code) and E_PROCESSOR_PROTOCOL (a 2xx the client does not understand), from the client |
Not applicable | Not applicable | Check your service key (E_UNAUTHORIZED). After a leak, rotate in two steps and wait for it to be re-enabled; otherwise, talk to Neurologyca (E_CLIENT_DISABLED). A defect of yours, do not retry (E_BAD_REQUEST: send taxonomy and language on every submit; E_JOB_PARAMS_CONFLICT: send the client’s values, or none; E_DESTINATION_NOT_ALLOWED: fix the configuration, then call submit again; E_CAPTURE_ID_CONFLICT, E_NOT_RETRYABLE). Retry later (E_STARTING, E_STOPPING, E_PROCESSOR_HTTP, E_PROCESSOR_PROTOCOL); retry after 5 s (E_PURGE_PENDING) |
E_REDIRECT_REFUSED, E_DESTINATION_NOT_ALLOWED, E_VIDEO_TOO_LARGE |
The download or the token URL was refused: a redirect, an address outside the service’s rules (with our service, anything but a public HTTPS origin), or a video over the service’s size limit | Yes, when the download or the token URL fails (on submit, E_DESTINATION_NOT_ALLOWED is a call error: fix it and call submit again) |
No | Fix the configuration (a public HTTPS origin in PUBLIC_BASE_URL, never localhost) or send a new videoUrl, then retry. It is not a blind retry |
E_VIDEO_URL_EXPIRED |
The download answered 401 or 403 | Yes, with a new videoUrl |
No | Retry with a new URL |
E_DOWNLOAD_FAILED |
Network, 5xx, 408 or 429 after 3 attempts | Yes | No | Retry later |
E_DOWNLOAD_FAILED (another 4xx) |
Your video route answered another 4xx, for example a 404 because receiver.mediaHandler is not mounted at /media/<captureId>/video |
No | Yes | Fix the video route for the next recordings; this capture cannot be recovered (submitting it again returns the same failed job) |
E_QUEUE_FULL |
More than 50 unfinished jobs for the client | Not applicable (the call is refused) | No | Retry submit later with a growing wait: retryAfterMs is empty (the service sends no Retry-After) |
E_MANIFEST_INVALID |
The manifest is malformed or does not match the video | No | Yes | Do not retry |
E_MEDIA_UNSUPPORTED |
The video is not MP4/H.264 | No | Yes | Do not retry |
E_MEDIA_INVALID, E_ROTATION_UNRESOLVED |
The file cannot be read as a video, or its rotation cannot be resolved | No | Yes | Do not retry |
E_NO_FACE_PREROLL, E_EMPTY_STEP |
No face in the first seconds, or a step without frames with a face | No | Yes | Do not retry; record again with the person |
E_TOKEN_FAILED |
The token URL did not answer properly after 3 attempts, or on the first attempt when it answers a 4xx | Yes | No | Check your token URL; retry |
E_POSITION_BUSY, E_OWN_ATTEMPT_PROCESSING |
The position is busy, or the person’s own attempt is still processing, after the service’s own waits | Yes | No | Retry later |
E_ALREADY_PROCESSED |
Not a failure: the attempt was already completed in Engine and the job ends succeeded |
Not applicable | No | Not a failure: treat as success |
E_CAPTURE_DISCARDED, E_CAPTURE_OUT_OF_ORDER |
The person discarded the recording, or a later session is already computed | No | Yes | Do not retry; delete the video |
E_USER_BLOCKED, E_USER_GONE |
Engine answered 403 or 404 for the user or the position | No | Yes | Do not retry |
E_POSITION_COMPLETED, E_CONFLICT |
Another capture completed the position, or another window conflict | No | Yes | Do not retry |
E_SESSION_ABANDONED, E_KODEC, E_STT_FAILED |
Engine abandoned the session, or the service’s analysis or transcription failed | Yes | No | Retry |
E_GPU_UNAVAILABLE, E_CHROME_SANDBOX |
The service’s host cannot run the job | Yes | No | Retry later; it is the service’s side |
E_PROCESSOR_HANG, E_JOB_TIMEOUT, E_PROCESSOR_RESTARTED |
No heartbeat for 60 s, 120 minutes running, or the service restarted | Yes | No | Retry |
E_INTERNAL |
An unexpected failure | Yes | No | Retry; report it if it repeats |
| Any other code | The service has more codes than this table (errorCode is a string) |
The job’s retryable |
Retry only if the job says retryable: true; otherwise close it |
If you see E_PROCESSOR_AUTH, E_PROCESSOR_UNAVAILABLE or E_PROCESSOR_REJECTED in DevLab’s playground, those are DevLab’s own way of showing “uploaded but not delivered to the service”: on the upload side it is E_UPLOAD_NOT_NOTIFIED (Deferred), and on the submit side one of the ProcessorError codes above.
A failure you will not retry closes the job: stop giving tokens for it on your token URL, and keep or delete the recording according to your retention. A job that fails with E_REDIRECT_REFUSED, E_DESTINATION_NOT_ALLOWED or E_VIDEO_TOO_LARGE (the three configuration errors) keeps the person’s reservation until Engine expires it (72 hours after the end notice).
Measurement in deferred
Section titled “Measurement in deferred”Works since 0.27.0: triggerAnalysis: false travels in the manifest and the service does not call /analyze. End to end, the job ends succeeded and Engine stores every frame the replay sends. Whether it transcribes depends on the job: with transcription off (transcribeSteps: []) the service transcribes nothing; with transcription on, ask for measurement_free in transcribeSteps (Measurement’s only step is not a speech step, so no other list includes it) and its text reaches Engine. Since 0.29.0, when Engine says interpreted: false, the recording client reserves without a taxonomy and writes interpreted: false in the manifest, and the service’s replay sends no taxonomy and asks for no analysis without you passing anything. Production does not return interpreted yet: there, keep triggerAnalysis: false on the capture.