Skip to content
DevLab

Processing service

What our processing service does with a recording, what you expose to it, the token URL, the webhook, its error codes and Measurement in deferred.

On this page
  1. What the service does
  2. With our service, or self-hosted
  3. Submitting a recording
  4. The token URL
  5. The webhook
  6. The interpretation
  7. Error codes
  8. Measurement in deferred
Preview

It downloads the video from your backend, replays the capture against Engine with the same SDK (replayRecording from @neurologyca/capture/replay), transcribes the recording’s audio (all of it, when the job asks for any step) and passes to Engine only the text of the steps each job asks for (transcribeSteps), and tells your backend with a signed webhook. It does not compute indicators: Engine does. You never run /replay yourself: the service does.

Aspect With our service (available today) Self-hosted (coming later)
What you get Nothing to deploy. One credential per organization: the service URL (PROCESSOR_URL) and its outbound IP, in your kit; the processor key (PROCESSOR_API_KEY) and the webhook secret (PROCESSOR_SECRET), one pair shared by every team: any active team gets it in its recording starter’s zip with Include my key and the processor credentials, and your organization’s admin team reveals and rotates it in Your account The processor (a container image) and its hosting guide, to run it on your own servers. Later, without dates
What you expose Three public HTTPS routes under PUBLIC_BASE_URL: the signed video, the token URL and the webhook Nothing public: an internal listener, reachable only from the processor’s network
What you keep Different secrets for the signed URLs and the webhook, one nonce store per purpose and a table of jobs The same
Context, language, transcription Per job: every submit carries that capture’s taxonomy, language and transcribeSteps The same; a processor whose client fixes them accepts the same values or none
  • Limiting the three routes to the service’s outbound IP is defense in depth, not authentication. That IP is the same for every organization: another one can make the service call your routes from it. What protects you is the HMAC signature with your PROCESSOR_SECRET, which createWebhookHandler and createProcessorTokenHandler verify, and the video URL’s signature. Never skip either, whether you limit by IP or not.
  • PUBLIC_BASE_URL is your backend’s public HTTPS origin. The service accepts any public https: origin and refuses http:, localhost, private and loopback addresses, redirects and URLs with credentials (E_DESTINATION_NOT_ALLOWED). To test on your machine, the service cannot call localhost: open a public HTTPS tunnel (for example cloudflared or ngrok), put its origin in PUBLIC_BASE_URL, and close it when you finish; or deploy. The recording starter explains both.
  • If a credential leaks, Neurologyca disables your organization’s access to the service (E_CLIENT_DISABLED; jobs in flight finish), your admin team rotates in two steps —the new values stay pending until you deploy them and activate them— and the access is re-enabled. No webhook in flight is signed with a secret your backend does not have yet: If a processor credential leaks.

From your receiver’s onComplete, sign a URL for the video and submit the job with that capture’s parameters. Save the jobId.

JavaScript
// Your backend, handing a recording to Neurologyca's processing service (Node 20.3+). `url`, `apiKey` and the webhook
// secret come from your DevLab account; `publicBase` is your backend's public HTTPS origin (PUBLIC_BASE_URL), where the
// service calls the video, the token URL and the webhook. Never localhost: to test on your machine, use a public tunnel.
import { createProcessorClient } from '@neurologyca/capture-server'
/**
* @param {object} o
* @param {string} o.url the processing service's URL (PROCESSOR_URL, from your kit)
* @param {string} o.apiKey your key for the service (PROCESSOR_API_KEY; never the Engine key)
* @param {Pick<import('@neurologyca/capture-server').UploadReceiver, 'signedUrl' | 'manifest'>} o.receiver your upload receiver
* @param {string} o.publicBase your backend's public origin for the service (PUBLIC_BASE_URL, HTTPS)
* @param {string} o.engineBase your environment's Engine URL (ENGINE_BASE)
*/
export function processing(o) {
const processor = createProcessorClient({ url: o.url, apiKey: o.apiKey })
return {
// Call it from your receiver's onComplete. Idempotent per captureId: the service answers with the same job.
// `params` are THIS capture's: taxonomy.contextId (your page's) and language (the session's) are required by the
// service; with a non-empty transcribeSteps the service transcribes the whole audio of the recording and passes to Engine only
// the text of the listed step types (absent or [] = nothing is transcribed).
async submit(
/** @type {string} */ captureId,
/** @type {{ taxonomy: import('@neurologyca/capture-server').ProcessorJobTaxonomy, language: string, transcribeSteps?: string[] }} */ params,
) {
const videoUrl = await o.receiver.signedUrl(captureId, { baseUrl: `${o.publicBase}/media` }) // valid 24 h by default
const manifest = /** @type {{ userProtocolId: string, protocolMechanicId: string }} */ (await o.receiver.manifest(captureId))
const { jobId, status } = await processor.submit({
videoUrl, manifest, engineBaseUrl: o.engineBase,
tokenUrl: `${o.publicBase}/processor/token`, webhookUrl: `${o.publicBase}/processor/webhook`,
taxonomy: params.taxonomy, language: params.language, transcribeSteps: params.transcribeSteps ?? [],
})
// Save all of it next to the captureId: the token URL compares the pair with the JOB's, because the upload
// (manifest included) is deleted at `interpreting` while the job still runs.
return { jobId, status, userProtocolId: manifest.userProtocolId, protocolMechanicId: manifest.protocolMechanicId }
},
job: (/** @type {string} */ jobId) => processor.getJob(jobId), // poll when a webhook does not arrive
}
}

With our service, engineBaseUrl must be an Engine the service admits (your account’s): any other is refused at submit. Every submit carries THAT capture’s parameters: taxonomy (contextId required, hciId and subcontextId optional; the same taxonomy.contextId as your page’s configuration), language (the session’s language, for example 'es'), transcribeSteps (the step types to transcribe; [] is always none; absent is none with the hosted service; with a self-hosted service, the steps that its own configuration sets for your client) and, optionally, interimWebhook. The service requires taxonomy.contextId and language: without them, 400 E_BAD_REQUEST. createProcessorClient validates them before sending (a malformed value is a TypeError at the call) and only sends the ones you pass. They need capture-server 0.13.0 or later.

The service only accepts MP4 with H.264, which is what the recording client produces. If it refuses the call, submit rejects with a ProcessorError (code, status, retryAfterMs); every call gives up after 30 s.

Mount the token URL and the webhook before any body parser: both verify the signature over the raw body. The service asks your backend for an Engine token, signed, with the capture’s pair and its jobId. createProcessorTokenHandler verifies the signature; your authorize decides whether to hand a token: only if the jobId is the one submit returned, the job is still running (interpreting included) and the pair is the one you stored for that job at submit. Never compare it with receiver.manifest(): after the interpreting notice the upload, manifest included, is already deleted, the service still renews its token while it waits for the interpretation, and a denial ends that wait in a false timeout. By default, the token is your organization’s developer token, not scoped to that capture: authorize limits when it is issued, never what it can do. This is a condition for production.

Since capture-server 0.10.0 you can hand a token scoped to the capture instead: captureToken: { baseUrl } in createProcessorTokenHandler, with baseUrl your environment’s Engine URL (ENGINE_BASE). The handler asks Engine for it with POST /token/capture, for exactly the signed pair and captureId, authenticated with your tokens (the developer token never leaves your backend), and hands out only that token. It lasts 30 minutes (the service asks again when it expires) and reaches only that capture’s ingestion, its plan and its stream. It is off by default: today only the test environment’s Engine has that route; against an Engine without it, the handler answers a 502 that your onError sees, never a silent failure. With it on, the replay calls no /analyze* (the token may not, and the service replays with awaitInterpretation: false), so it never warns analysis.out-of-token-scope: Engine runs the interpretation the closing Final asked for, and you read it later with your own credential. Engine also limits that token’s requests per capture and per minute; since @neurologyca/capture 0.37.0 the replay waits the Retry-After and goes on, slower, with the warning transport.rate-limited (a processor on an earlier version fails the job). What that token can and cannot do: A token for one capture. DevLab’s own deferred mode can hand it too: an option of its server, off by default, only against the test environment’s Engine.

The service tells your backend when the job ends (succeeded or failed). The contract (signed fields, nonces, at-least-once, idempotence, getJob as the fallback): Results in your backend.

Since capture-server 0.11.0 there is also an interim notice, status: 'interpreting', signed like the others, with report and processor set to null and no failure fields. The service sends it while interimWebhook is on: with our service, by default (turn it off per job with interimWebhook: false in submit). A 0.10.0 receiver answers it with a 400. It is not terminal:

  • Delete the video at interpreting or at succeeded, whichever comes first, and keep the job open until succeeded. Without the notice (off, or lost) the video stays in your receiver for the whole wait, up to 300 s more.
  • Branch on interpreting first. An onEvent that treats every status but succeeded as a failure closes the job: the token renewal is then denied, the wait ends in a false timeout, and the video is never deleted. Update the handler before you deploy against our service, where interimWebhook is on by default.
  • An interpreting that arrives after the terminal event (a retried delivery) reopens nothing.

Since capture-server 0.11.0 the job waits for Engine’s interpretation. If the protocol is interpreted, the last Final asks for it and Engine produces it on its own (it saves it and announces it with constellation.complete); meanwhile the job is interpreting, up to 300 s by default. It counts as in flight, getJob returns it as such, and it never ends in failed. succeeded arrives once the interpretation is ready or once it is known it will not come, and its interpretation (in the webhook and in getJob) says how the analysis ended. The job ends succeeded whatever that value is: the indicators are in /results. The text never travels in the webhook: read it with your credential (client.analyze() or POST /analyze).

interpretation What it means What to do
ready The analysis came out complete Read it now: it comes from the cache (from_cache: true) and costs no tokens again. Measured on 2026-10-13: 726 ms
degraded It arrived, but the analysis came out degraded No call reads the stored analysis without redoing it: every /analyze regenerates it (tens of seconds, up to about two minutes, and LLM tokens) and returns the new one or, if it comes out worse, the stored one. Ask once if you need it, never in a loop
failed It arrived, but the analysis failed The same as degraded
timeout No event within the wait Not a failure. You can ask for it, at the cost of LLM tokens if it is not stored: Engine generates it then
unknown It was asked for and how it ended is not known You can ask for it, at the cost of LLM tokens. When processor.interpretation.reconnects is a number and the outcome is not already-processed, the event arrived without a readable status: the analysis is stored and /analyze serves it from the cache, as with ready. reconnects: null is a service restart during the wait; outcome: 'already-processed', a session processed by an earlier job
not-requested It was not asked for (triggerAnalysis: false, a protocol that is not interpreted), or Engine skipped it because an input was missing (for example, the transcript of an answer that ended without speech) Nothing to ask for; in the second case, record again with speech in every answer

The job carries the wait in processor.interpretation (e.processor in the webhook, job.processor in getJob; not in report): { outcome, waitedMs, reconnects, lastClose }. Why waiting matters, measured by DevLab on 2026-10-13: asked when the replay ended, from_cache: false (another one was generated and paid twice); asked 300 s later, from_cache: true both times.

With the capture token (captureToken) nothing changes here: the service replays with awaitInterpretation: false and calls no /analyze* (out of that token’s scope), so analysis.out-of-token-scope does not appear. The session ends complete, and the interpretation is the one Engine makes on its own.

From the service’s error catalogue. “/retry” says whether the job can be retried; “Reservation”, whether the service releases the person’s reservation in Engine.

Code When /retry Reservation What your backend does
Call errors (ProcessorError, submit or retry rejects with them) E_UNAUTHORIZED (401): no credential or a wrong one. E_CLIENT_DISABLED (403): your organization’s access to the service is disabled; submit and retry refuse, getJob still answers, and jobs in flight finish. E_BAD_REQUEST (400; 413 over 4 MiB; with our service, also a job without taxonomy.contextId or language), E_DESTINATION_NOT_ALLOWED (400, the job’s URLs are outside the service’s rules, for example a PUBLIC_BASE_URL on http:), E_JOB_PARAMS_CONFLICT (400, self-hosted: the processor’s client fixes taxonomy, language, transcribeSteps or interimWebhook and the job brings another value), E_CAPTURE_ID_CONFLICT (409, the captureId belongs to another client), E_NOT_RETRYABLE (409, retry on a job that is not failed or not retryable). E_STARTING and E_STOPPING (503). E_PURGE_PENDING (503, Retry-After: 5, retry while the job’s files are deleted). E_PROCESSOR_HTTP (an error response without code) and E_PROCESSOR_PROTOCOL (a 2xx the client does not understand), from the client Not applicable Not applicable Check your service key (E_UNAUTHORIZED). After a leak, rotate in two steps and wait for it to be re-enabled; otherwise, talk to Neurologyca (E_CLIENT_DISABLED). A defect of yours, do not retry (E_BAD_REQUEST: send taxonomy and language on every submit; E_JOB_PARAMS_CONFLICT: send the client’s values, or none; E_DESTINATION_NOT_ALLOWED: fix the configuration, then call submit again; E_CAPTURE_ID_CONFLICT, E_NOT_RETRYABLE). Retry later (E_STARTING, E_STOPPING, E_PROCESSOR_HTTP, E_PROCESSOR_PROTOCOL); retry after 5 s (E_PURGE_PENDING)
E_REDIRECT_REFUSED, E_DESTINATION_NOT_ALLOWED, E_VIDEO_TOO_LARGE The download or the token URL was refused: a redirect, an address outside the service’s rules (with our service, anything but a public HTTPS origin), or a video over the service’s size limit Yes, when the download or the token URL fails (on submit, E_DESTINATION_NOT_ALLOWED is a call error: fix it and call submit again) No Fix the configuration (a public HTTPS origin in PUBLIC_BASE_URL, never localhost) or send a new videoUrl, then retry. It is not a blind retry
E_VIDEO_URL_EXPIRED The download answered 401 or 403 Yes, with a new videoUrl No Retry with a new URL
E_DOWNLOAD_FAILED Network, 5xx, 408 or 429 after 3 attempts Yes No Retry later
E_DOWNLOAD_FAILED (another 4xx) Your video route answered another 4xx, for example a 404 because receiver.mediaHandler is not mounted at /media/<captureId>/video No Yes Fix the video route for the next recordings; this capture cannot be recovered (submitting it again returns the same failed job)
E_QUEUE_FULL More than 50 unfinished jobs for the client Not applicable (the call is refused) No Retry submit later with a growing wait: retryAfterMs is empty (the service sends no Retry-After)
E_MANIFEST_INVALID The manifest is malformed or does not match the video No Yes Do not retry
E_MEDIA_UNSUPPORTED The video is not MP4/H.264 No Yes Do not retry
E_MEDIA_INVALID, E_ROTATION_UNRESOLVED The file cannot be read as a video, or its rotation cannot be resolved No Yes Do not retry
E_NO_FACE_PREROLL, E_EMPTY_STEP No face in the first seconds, or a step without frames with a face No Yes Do not retry; record again with the person
E_TOKEN_FAILED The token URL did not answer properly after 3 attempts, or on the first attempt when it answers a 4xx Yes No Check your token URL; retry
E_POSITION_BUSY, E_OWN_ATTEMPT_PROCESSING The position is busy, or the person’s own attempt is still processing, after the service’s own waits Yes No Retry later
E_ALREADY_PROCESSED Not a failure: the attempt was already completed in Engine and the job ends succeeded Not applicable No Not a failure: treat as success
E_CAPTURE_DISCARDED, E_CAPTURE_OUT_OF_ORDER The person discarded the recording, or a later session is already computed No Yes Do not retry; delete the video
E_USER_BLOCKED, E_USER_GONE Engine answered 403 or 404 for the user or the position No Yes Do not retry
E_POSITION_COMPLETED, E_CONFLICT Another capture completed the position, or another window conflict No Yes Do not retry
E_SESSION_ABANDONED, E_KODEC, E_STT_FAILED Engine abandoned the session, or the service’s analysis or transcription failed Yes No Retry
E_GPU_UNAVAILABLE, E_CHROME_SANDBOX The service’s host cannot run the job Yes No Retry later; it is the service’s side
E_PROCESSOR_HANG, E_JOB_TIMEOUT, E_PROCESSOR_RESTARTED No heartbeat for 60 s, 120 minutes running, or the service restarted Yes No Retry
E_INTERNAL An unexpected failure Yes No Retry; report it if it repeats
Any other code The service has more codes than this table (errorCode is a string) The job’s retryable Retry only if the job says retryable: true; otherwise close it

If you see E_PROCESSOR_AUTH, E_PROCESSOR_UNAVAILABLE or E_PROCESSOR_REJECTED in DevLab’s playground, those are DevLab’s own way of showing “uploaded but not delivered to the service”: on the upload side it is E_UPLOAD_NOT_NOTIFIED (Deferred), and on the submit side one of the ProcessorError codes above.

A failure you will not retry closes the job: stop giving tokens for it on your token URL, and keep or delete the recording according to your retention. A job that fails with E_REDIRECT_REFUSED, E_DESTINATION_NOT_ALLOWED or E_VIDEO_TOO_LARGE (the three configuration errors) keeps the person’s reservation until Engine expires it (72 hours after the end notice).

Works since 0.27.0: triggerAnalysis: false travels in the manifest and the service does not call /analyze. End to end, the job ends succeeded and Engine stores every frame the replay sends. Whether it transcribes depends on the job: with transcription off (transcribeSteps: []) the service transcribes nothing; with transcription on, ask for measurement_free in transcribeSteps (Measurement’s only step is not a speech step, so no other list includes it) and its text reaches Engine. Since 0.29.0, when Engine says interpreted: false, the recording client reserves without a taxonomy and writes interpreted: false in the manifest, and the service’s replay sends no taxonomy and asks for no analysis without you passing anything. Production does not return interpreted yet: there, keep triggerAnalysis: false on the capture.