CSP and transcription
The external origins Capture loads, a Content Security Policy that works with them, how to host them yourself, and how to set up transcription.
Your page loads the analysis models from public CDNs; if your site has a Content-Security-Policy, it must allow them, or you host the files yourself.
What Capture loads, and from where
Section titled “What Capture loads, and from where”Capture analyses the camera (and, with transcription, the microphone) in the page. By default it loads its models and runtimes from public CDNs:
| When | Origin | What | Host it yourself with |
|---|---|---|---|
| Always | cdn.jsdelivr.net |
The vision runtime (MediaPipe Tasks Vision, a pinned version) and its WebAssembly | kodec.visionUrl, kodec.wasmUrl |
| Always | storage.googleapis.com |
The face model | kodec.faceModel |
| With hands or body models | storage.googleapis.com |
The hand and pose models | kodec.handModel, kodec.poseModel |
| With transcription | cdn.jsdelivr.net |
The speech runtime (Transformers.js) | transcribe.selfHost.transformersUrl |
| With transcription | huggingface.co and *.hf.co |
The speech model (Whisper), about 136 MB | transcribe.selfHost.modelHost |
| With transcription | the speech runtime’s own path | The ONNX Runtime WebAssembly | transcribe.selfHost.wasmPaths |
Those URLs load code: set them in your build, never from a query string or a remote configuration.
Content Security Policy
Section titled “Content Security Policy”This policy lets your page capture with the SDK, transcribe and talk to your proxy, with everything loaded from the CDNs above. We tested it with the live client, transcription on, through the SDK’s proxy (Your backend).
Content-Security-Policy: default-src 'self'; script-src 'self' 'wasm-unsafe-eval' blob: https://cdn.jsdelivr.net; worker-src 'self' blob:; connect-src 'self' https://cdn.jsdelivr.net https://storage.googleapis.com https://huggingface.co https://*.hf.co; style-src 'self'; img-src 'self' data: blob:; media-src 'self' blob:'wasm-unsafe-eval'is enough:'unsafe-eval'is not needed.style-src 'self'is enough: Capture does not need inline styles ('unsafe-inline').blob:inscript-srcis required: kodec’s audio processing (its AudioWorklet) loads from a Blob. Without it, the whole transcription is lost, and it shows:codec.errorwithaudio-start-failed, thesttwarning, and the report’stranscriptionisnull.connect-src 'self'covers your proxy when it lives in your origin, and the startup report URL (startup.report.url) when it is yours; on another origin, add it.- If you host everything yourself, the CDN origins can go:
script-src 'self' 'wasm-unsafe-eval' blob:andconnect-src 'self'.
When the policy blocks Capture, the camera never gets ready (a download is blocked), or the transcription is lost as above. workerBlockedByCsp in the report only says whether the speech worker was blocked: workerBlockedByCsp does not cover the AudioWorklet.
Hosting the files yourself
Section titled “Hosting the files yourself”Pass your own URLs to the client:
import { createProtocolClient } from '@neurologyca/capture'
const client = createProtocolClient({ baseUrl: '/engine', tenant, kodec: { visionUrl: '/vendor/tasks-vision/', wasmUrl: '/vendor/tasks-vision/wasm', faceModel: '/vendor/face_landmarker.task' }, transcribe: { language: 'es', selfHost: { modelHost: '/vendor/models/', transformersUrl: '/vendor/transformers.js', wasmPaths: '/vendor/ort/' } },})wasmPaths since 0.42.0 (breaking). As a string it is a prefix that must end in /: kodec puts it in front of the file name and asks for the ONNX Runtime pair of the executor it runs on: ort-wasm-simd-threaded.jspi.mjs and ort-wasm-simd-threaded.jspi.wasm on WebGPU with JSPI; ort-wasm-simd-threaded.mjs and ort-wasm-simd-threaded.wasm on WASM and on WebKit (Safari, and every browser on iOS). On WebGPU without JSPI the prefix goes to ONNX Runtime as before. The four files come from onnxruntime-web/dist/: serve both pairs if your people arrive with both executors. Before loading, kodec checks both files of the pair with a HEAD; if one answers 404 or 410, speech does not start and the capture goes on without it, with the stt warning carrying subStage: 'stt-runtime', reason: 'network' and, in cut, code: 'stt-runtime-missing' with the missing url (in prepare(), partial with speech-error). A host that serves only the Asyncify pair (ort-wasm-simd-threaded.asyncify.*), which used to work on WebGPU with JSPI, now fails that way. As an object, wasmPaths: { wasm, mjs } points at the exact files and reaches kodec as it is; on WebKit, point it at the plain build.
Transcription
Section titled “Transcription”Transcription turns what the person says into text on the device. It is off until something turns it on: transcribe: { … } in the client, or a transcribe object in the kit (tenant), which turns transcription on even if your code does not pass transcribe. Only transcribe: false in the client keeps it off whatever the kit says; transcribe: false in the kit keeps it off unless the client passes an object. With the camera it opens the microphone; with a <video> source it transcribes the file’s own audio and does not open the microphone. In Safari, with the <video> muted, the SDK adds no audio track and transcription fails with an explicit error.
The first time, the browser downloads about 160 MB for speech (136 MB of them the Whisper weights, plus the runtime and the library) and keeps it for that origin; on a slow mobile connection that is minutes. client.prepare({ models: { speech: true } }) can download it before Start, except on Safari and iOS (Prepare before Start).
Language
Section titled “Language”The language is declared, not detected: it is the language the person will speak, not your page’s. Set it as a code without a region:
{ "transcribe": { "language": "es" } }- If you pass a code with a region, such as
es-ES, Capture uses its language part (es) and notes it in the console. - Without
language, Capture uses the language of the page (<html lang>, then the browser’s). language: nulldetects the language from the audio. It is not reliable with short answers: it can return the text translated into English, with no error. Capture warnsstt.autodetect.- A code the model does not know falls back to detection, with a warning.
- The language can also come from the kit (
transcribe.language); the client’s fields win over the kit’s. - With a source that is not a microphone (a
<video>captured withcaptureStream(), like DevLab’s sample video), the track does not reportautoGainControl: kodec warns once that energy and shimmer are untrusted (the message names WebKit/iOS, but any such track triggers it). It is expected; those two features are not comparable with a microphone’s.
In the client:
const client = createProtocolClient({ baseUrl: '/engine', tenant, transcribe: { language: 'es' } })If the language is not the one spoken
Section titled “If the language is not the one spoken”kodec forces the model to the declared language. If the person speaks another one, one of two things happens, depending on the pair of languages: the model translates into the declared language, fluently and with good confidence, so the text reaches Engine translated and with no warning; or it repeats itself or scores low, and kodec discards those utterances on purpose, so the voice steps reach Engine with no text or very little (only the stt.silent warning, when no utterance is left). In one test, with English declared over Spanish speech the first happened; with Spanish declared over English speech, the second. Pinning the wrong language is worse than not pinning it. Deferred, the job’s language works the same way: the transcript arrives labelled with the language asked for, and its text is not reliable (Deferred).
Ways to get it right:
- Keep it with the person or the interview in your backend (the language the candidate answers in, for instance) and pass it to every capture. Do not infer it from the interface language if the two can differ.
- If you do not know it, ask before the capture, on the consent screen.
- One language per capture. Whoever switches language mid-capture is transcribed in the declared one: split it into captures.
- Catch it afterwards with
onReport: inSessionReport.transcription,vad.utterancesabove 0 with few or noemittedand highdiscardedLowConfidenceordiscardedRepetitionLooppoints to the wrong language (with none emitted, Capture warnsstt.silent). Offer to repeat the capture in the right one. A translation passes every counter: only declaring the right language prevents it. - Autodetection is not the way out. In one test with
language: null, the longer answers in Spanish also came back translated into English. When you know the person’s language, declare it. - The interpretation’s language is separate:
narrative.languageon the person.
What leaves the device
Section titled “What leaves the device”Only text. The audio is processed in the browser and never sent. The transcript travels once, in the Final of the last step: the whole text (speech) and the text of each step with its times (sttByStep). Engine adds it to the context of the interpretation.
When the device cannot transcribe
Section titled “When the device cannot transcribe”Some devices cannot run the speech model. Capture tells you before it starts, with a warning whose code is stt and a blocker, or during the capture, with stt.cut. The blockers are in STT_BLOCKERS, and sttBlockerOf(error) reads one from an error:
| Blocker | Meaning |
|---|---|
no-webgpu |
No usable WebGPU adapter in this browser (it may have WebGPU and still offer none) |
ladder-exhausted |
Every speech configuration closed the tab while loading on this device, so it is not tried again |
video-cost-veto |
Transcribing made the camera analysis too slow on this device |
no-memory-to-learn |
The browser stores nothing (private mode or blocked storage) and would run speech without WebGPU, so it is not risked |
dtype-required |
The model you chose needs an explicit dtype on this device |
The veto is kept in that browser’s storage and outlives the page. When it was measured while the video ran below 8 fps, typically with the tab hidden, it is stored as provisional. Since SDK 0.41.0 the first capture that shows video above 8 fps deletes it and transcribes: only that capture waits, up to about 3 s (about 1 s on a healthy device), and SessionReport.transcriptionSetup.provisionalVetoLifted reports the deleted veto, its only trace. A capture that stays at 8 fps or less, such as a hidden tab or a capture fps of 8 or less, keeps it. Up to 0.40.0 it never lapsed inside a capture (transcription started before the rate was measured), and 0.40.1 transcribed but kept it, waiting in every capture. Show every warning, not only the last one, because stt is the only sign that a capture is not transcribing.
video-cost-veto is the only one the SDK lets you lift, with forgetTranscriptionVeto behind a press of the person, never automatically. transcribe.allowWasmWorker: true lets devices without a usable WebGPU adapter try the speech model in WebAssembly, more slowly. Since 0.43.0 that path needs its transcription Worker: if the Worker does not start (for instance a Content-Security-Policy that blocks workers), there is no voice (a codec.error with wasm-needs-worker, and transcriptionLoad.reason: 'needs-worker' in the report) instead of falling back to the main thread, which froze the video on every utterance. With WebGPU nothing changes. The capture of the indicators goes on either way.
Pass both functions the kodec the SDK itself exports (kodec, next to readTranscriptionVeto and forgetTranscriptionVeto, in @neurologyca/capture): you do not install kodec, the SDK brings it pinned. readTranscriptionVeto(kodec) only reads, and returns { veto, learned }: call it first to know whether there is a veto to lift. forgetTranscriptionVeto(kodec) lifts that veto and nothing else; it never erases the rest of what kodec learned on the device, such as the memory limits (do not use kodec’s forgetMemoryLimit for this). The veto belongs to the browser, not to the person: lifting it only changes the next capture in this browser, and if transcribing slows the video down again, the SDK vetoes it again.
onReport in the client receives the SessionReport when the capture ends, also when it ends with endedBy: 'abort' or 'error'; CaptureOutcome does not carry it. For transcription, read transcriptionSetup (model, language, languageSource, backend and provisionalVetoLifted) and, per step, transcription.stepsTranscription[] (order, transcribed, emitted, spokenMs): a step with transcribed: false or emitted: 0 heard no speech, which explains an empty transcript in /results. DevLab’s playground keeps only those fields, with provisionalVetoLifted as a yes or no (never its content), and shows them in «Technical data of the response». When a session is not transcribed, the playground says so under the stage and in the result card with the sentence of its blocker from the table above, and, only with video-cost-veto and once the session has ended, offers «Retry transcription», which does exactly this: reads the veto and, if there is one, lifts it.
Bundlers
Section titled “Bundlers”- The SDK brings kodec 5.0.0-rc.14, pinned: you do not install kodec yourself. Minification does not break transcription (fixed in kodec 4.0.2). With webpack (Next.js), kodec 4.2.2 or later, which the SDK brings since 0.34.0; Turbopack is not tested.
- Do not install the speech runtime or the vision runtime “just in case”: Capture loads them when needed. Passing the speech runtime you import as
transcribe.transformersonly saves its download: the model still comes frommodelHostand ONNX Runtime fromwasmPaths, and withouttransformersUrlthere is no speech worker (speech runs on the main thread).