Skip to content
DevLab

Deferred (server-side)

The recording client: record the video and a step manifest, upload them to your receiver, and follow the job of the processing service.

On this page
  1. The loop at a glance
  2. The mode at a glance
  3. Record
  4. What is recorded
  5. Upload
  6. Pending recordings and shared devices
  7. If something fails halfway
  8. Your receiver
  9. Transcription
  10. The job: webhook and polling
  11. What you can do today
  12. Try it on your phone
Preview

The SDK records the video and a step manifest and uploads them; our processing service handles them shortly after. It is not hours later: it just doesn’t run on the device. See the example in the playground.

  1. BrowsercreateRecordingClient → uploadRecording
  2. Your receivercreateUploadReceiver.onComplete
  3. Your backendsubmit to the service
  4. Processing servicePreview
  5. Back to your backendtoken URL while it runs · webhook / getJob
  6. interpreting → succeededread /results + interpretation
Processing is Preview. Answer the webhook 2xx fast and do the work after; stop polling at succeeded or failed.
  1. Page — createRecordingClient, the same step loop as on-device; session.stop() gives the recording.
  2. Page — uploadRecording(recording, { url }) uploads it in chunks and deletes the local copy only after your receiver confirms.
  3. Your backend — your receiver’s onComplete signs a URL for the video and submits the job to the processing service.
  4. Your backend — the service replays the capture against Engine (transcribing the recording and passing to Engine only the text of the steps the job asks for) and tells your webhook; poll getJob if it does not arrive. Answer the webhook 2xx fast and do the work after, and stop polling once the job is succeeded or failed.
Deferred (server-side)
TopicDeferred (server-side) · Preview
TodayYou can record, upload and process with Neurologyca's processing service: nothing to deploy.
Where it runsOn our processing service, shortly after the upload. It is not hours later: it just does not run on the device.
When the results arriveMinutes after the upload, adding three stretches: the upload (about 7 min for 3 min of video at 1.75 Mbps, a calculation), the queue (one job at a time) and the processing, somewhat longer than the video itself.
What leaves the deviceVideo and audio (with the camera, the microphone is always on) to your receiver, and from there to the processing service, which, with transcription on, transcribes the whole audio of the recording and passes to Engine only the text of the steps the job asks for (transcribeSteps). The raw audio identifies a person more than numbers do, and the manifest that travels with the video holds the full userAgent of the browser.
What limits youUpload bandwidth. At 4 Mbps, 3 min of video are about 93 MB (about 310 MB per 10 min). The upload resumes after a cut. The quality depends on the recording's bitrate and the light, not on the device's power: the processing runs on the server.
Browsers and phonesRecords MP4/H.264: desktop Chrome and Edge, Chrome on Android and Safari on iOS. An iPhone records MP4 too: its file can end about 1.1 s before the stop, which is why the SDK keeps recording 2 s more. On a phone or a tablet it records in portrait: it waits until the device is upright.
MeasurementYes, since 0.27.0: triggerAnalysis: false travels in the manifest and the processor does not call /analyze.
Typical casesA ten-minute interview on a modest phone; a field session on a weak device, uploaded later.
Moving between themAdd the receiver to your backend (createUploadReceiver); record from the camera, a <video> or a stream.

The recording client comes from @neurologyca/capture/record, which does not download kodec. Its loop is the live one, with the steps marked in the video (Steps and marking).

JavaScript
import { createRecordingClient } from '@neurologyca/capture/record'
import { keepScreenOn, guardUnload } from '@neurologyca/capture/helpers'
const recorder = createRecordingClient({ baseUrl: '/engine', tenant })
const session = await recorder.captureProtocol({ userProtocolId, protocolMechanicId, source: { type: 'webcam' } })

Before recording on a phone: ask for persistent storage (navigator.storage.persist()), keep the screen on (keepScreenOn) and guard the tab (guardUnload) until the upload ends; on a phone held upright, waitForPortraitOrientation. You can also record a <video> ({ type: 'video', el }, with its src set before) or a MediaStream ({ type: 'stream', stream }): when it ends, the SDK warns source.ended.

Since 0.42.0, before asking for any permission, recorder.prepare() tells you whether this device can record: it loads no model, does not open the camera and never contacts Engine. Besides the recording’s checks it adds indexeddb (without it nothing can be recorded) and storage-quota (it warns below 150 MB free), and resolves ready, failed (with the message prepare.record-unsupported; device and capture say which check failed) or aborted (by its signal). The recording client also takes startup (timeouts.gateMs, locale, messages, report) and onStartupStage, and warns recording.option-ignored for startup.timeouts.vision and startup.timeouts.speech, which a recording does not have. keepPrepared (since 0.43.0) is live only: createProtocolClient({ mode: 'record', keepPrepared }) ignores it with the same recording.option-ignored warning (a recording loads no model, and captureBaseline keeps nothing in record mode), and createRecordingClient throws E_UNKNOWN_OPTION for it. In record mode no SessionReport is emitted, so the startup report reaches you only through startup.report.url (Your backend). The signal of captureProtocol works until it resolves.

MP4 with H.264 video and AAC audio, about 4 Mbps by default (recording.videoBitsPerSecond). The camera is asked for 1280×720 (ideal: a preference, not a guarantee), and RecordingManifest.recorder says the size it got. With the camera, the microphone is always recorded. A recording carries at most 30 frames per second, whatever the source (recording.maxFps, from 1 to 120): the processing service replays every frame to Engine, and its time grows with them. If a stream cannot be capped, the recording goes on as it comes and the SDK warns recording.fps-high once. The recording is kept in this browser (IndexedDB) until it is uploaded.

The quality of a deferred capture depends on the recording’s bitrate and the light, not on the device’s power: the processing runs on the server. On a phone or a tablet the recording is in portrait: if the device is in landscape, the camera does not open and recording.rotate-to-portrait comes out until it is turned upright (waitForPortraitOrientation waits for it before you start); recording.handheld corrects a device detected wrongly (false records in landscape, for a kiosk taken for a phone). Choosing the mode by device: Devices, quality and mode.

session.stop() gives a RecordingResult (blob, manifest, release(), storage). uploadRecording(recording, { url }) uploads it in chunks to your receiver, resumes where it stopped and retries network errors (6 times by default, with Retry-After); it only deletes the local copy after your receiver confirms (200 of complete) or answers 410. If it cannot, it rejects with UploadRecordingError (code, status, retryable):

Code When Retryable What you do
E_UPLOAD_NETWORK Network, 5xx, 408 or 429 after the retries Yes Call again later: the local copy is still there
E_UPLOAD_NOT_NOTIFIED Your receiver stored it but its onComplete failed (for example, the processing service is down or refuses your key) Yes Tell the user it is uploaded; your backend retries onComplete itself (receiver.notify()), or call again later
E_UPLOAD_AUTH 401: no session Yes, after signing in Sign the user in and call again
E_UPLOAD_REJECTED 403, 400, 413, 415 or too many open uploads No It is not this user’s, or it is malformed
E_UPLOAD_CONFLICT 409 without a way to resync No Tell the user
E_UPLOAD_TOO_LARGE Over maxBytes No Record shorter or at a lower bitrate
E_UPLOAD_LOCAL_UNREADABLE The local copy can no longer be read No abandonUpload
E_UPLOAD_PROTOCOL Your receiver answers something impossible No Check the receiver
E_UPLOAD_ABORTED Your own signal Yes Call again when you want to resume

listPendingRecordings() returns what was recorded and not uploaded in this browser: filter it by the signed-in user before offering it. abandonUpload(captureId, { url }) gives one up; recording.release() deletes the local copy.

abandonUpload only gives up an unfinished upload: once complete has stored it, your receiver answers 409 E_ALREADY_COMPLETE, because a complete upload belongs to your backend. To cancel it then, call a route of your own backend that closes its job, stops handing tokens for it and deletes the video (receiver.remove()), and tell the user it was discarded only when your backend confirms it.

The camera stops or the tab is closed: E_RECORDING_LOST, nothing is kept; record again. No space: E_STORAGE_FULL. The browser cannot record MP4/H.264: E_RECORDER_UNSUPPORTED (Firefox is not a target: kodec has not been tested there, and desktop Firefox does not record MP4/H.264). A captureId that already has a recording here: E_CAPTURE_ID_IN_USE. An upload cut by the network resumes on the next call.

Your backend receives the chunks with createUploadReceiver, from @neurologyca/capture-server: it authenticates your user, checks that the enrolment is theirs before the first byte (checkManifest), and calls onComplete once per recording. How it fits with the proxy and the retention: Your backend.

JavaScript
// Your backend for the deferred mode (Node 20.3+): the upload receiver. Mount both BEFORE any body parser:
// if (req.url.startsWith('/uploads/')) return listener(req, res)
// /media/<captureId>/video, the signed video the processing service downloads: return media(req, res)
import { createUploadReceiver } from '@neurologyca/capture-server'
import { localDiskStorage, toNodeListener } from '@neurologyca/capture-server/node'
/**
* @param {object} o
* @param {string} o.dir a private directory for the recordings
* @param {string} o.urlSecret 32+ characters, for the signed URLs of the videos
* @param {(req: Request) => Promise<string | null>} o.signedIn your session: a subject (your user id) or null
* @param {(subject: string, userProtocolId: string, protocolMechanicId: string) => Promise<boolean>} o.ownsPair is this enrolment the user's?
* @param {(captureId: string) => Promise<void>} o.onRecording a recording is complete: submit it to the processing service (or keep it, if you process it yourself)
*/
export function uploadsHandler(o) {
const receiver = createUploadReceiver({
mountPath: '/uploads',
storage: localDiskStorage({ dir: o.dir }),
urlSecret: o.urlSecret,
authorize: (req) => o.signedIn(req), // null answers 401; never throw here
checkManifest: (subject, m) => o.ownsPair(subject, m.userProtocolId, m.protocolMechanicId), // refused before the first byte
onComplete: ({ captureId }) => o.onRecording(captureId), // must be idempotent per captureId
onError: (_error, ctx) => console.error(`upload receiver: ${ctx.piece} ${ctx.route ?? ''} ${ctx.captureId ?? ''}`), // never the body or a URL
})
return { receiver, listener: toNodeListener(receiver.handler), media: toNodeListener(receiver.mediaHandler) }
}

The recording client has no transcribe option: in deferred, the processing service transcribes. Which steps’ text reaches Engine is decided per job: every submit from your backend carries that capture’s transcribeSteps and language (Submitting a recording); [] is always nothing; absent is nothing with the hosted service. Keep in mind that the audio goes to the service, which transcribes the whole audio on the server, also what is said between steps; on-device, the audio never leaves the browser (On-device (real time)).

The transcript’s language is the job’s language; the interpretation’s language is the person’s, narrative.language, fixed when you create or update them (Your backend). If the user switches language between creating the person and recording, the transcript follows the new language and the interpretation stays in the old one: update the person first (updateUser with narrative). In DevLab’s playground the test person is created with the interface language the first time you measure, record or capture the baseline; if you switch the interface language after that, the transcript changes and the interpretation keeps the person’s language until you use a new person.

When your receiver submits a recording, the processing service answers with a jobId. The signed webhook is the notice; processor.getJob(jobId) is the way to poll the job (status, outcome, errorCode, retryable, attempt and, since capture-server 0.11.0, interpretation). The job waits for the interpretation in interpreting, which counts as in flight, and ends succeeded even when the interpretation goes wrong: what to do with each value, in The interpretation. The contract of the webhook: Results in your backend; the service: Processing service.

JavaScript
const job = await processor.getJob(jobId) // status: queued, downloading, preparing, transcribing, replaying, waiting, interpreting, succeeded or failed; interpretation on a succeeded

How often to poll. The SDK sets no interval for getJob and never polls by itself: you choose. A job takes minutes (the replay, and up to 300 s more in interpreting by default), longer if it is retried. Every few seconds is enough while someone waits on screen, less often when no one does, and stop at succeeded or failed. If getJob rejects with a ProcessorError that carries retryAfterMs, wait at least that long. For reference, DevLab’s playground asks its own server every 3 s, its server calls getJob at most once every 2 s per recording, and the page waits up to 40 minutes.

Record, upload and process with Neurologyca’s processing service, the only option today, turned on for your organization: the service URL and its outbound IP are in your kit, and the processor key and the webhook secret, one pair shared by every team, come in your recording starter’s zip with Include my key and the processor credentials (your organization’s admin team rotates them in Your account). Your backend exposes three public HTTPS routes under PUBLIC_BASE_URL (Processing service). Without those values, record and upload against your own receiver without processing: onComplete keeps the recording and does not submit it, which is how the SDK’s own recording example runs with processing off. Running the processor yourself comes later. Measurement in deferred was measured end to end on 2026-10-12.

HTTPS to the phone, as on-device (Try it on your phone), plus: hold the phone upright, keep the screen on and keep the tab in front until the upload ends (Chrome on Android does not show the leave warning). An iPhone records MP4 too: its file can end about 1.1 s before the stop, which is why the SDK keeps recording 2 s more after the last step. Upload time depends on your network: at 4 Mbps, 3 minutes of video are about 93 MB.