Load the models
The five ways to load the face and speech models in the browser, which one to choose, what is downloaded and from where, and how to check it in your app.
By default there is nothing to do: the capture loads the models when it starts. Preparing only moves that wait to a moment when the person is doing something else. This is the on-device mode; recording for the deferred mode downloads no model.
In short
Section titled “In short”- Without preparing, the wait is visible between Start and the first frame; that is fine for a single session.
- Preparing loads the face model ahead, on the same client that will capture, and never calls Engine or asks for a permission.
- Speech is a large download: ask for it only once the person has agreed.
- On Safari and iOS only the face is fetched ahead; speech loads when the capture starts.
- With several sessions,
keepPreparedkeeps the models from one to the next until you release them.
Which way to choose
Section titled “Which way to choose”| Your situation | Way | Why |
|---|---|---|
| One session | A or B | A needs nothing; B hides the wait behind a screen you have |
| With transcription | C | Speech, only after the person agrees |
| Several sessions | E | From the second session on, the models are reused: no download and no model load again (the camera and the first frame still start) |
| With a baseline | E | The baseline adopts the prepared face and leaves speech for the measurement |
| A phone, or data saver on | B | The face ahead; speech when the person presses Start, not before |
| Safari or iOS | any | Only the face gets ready ahead; speech loads when the capture starts |
| No WebGPU | B or E, face only | Transcription is refused unless you allow transcribe.allowWasmWorker (slower); where it is refused, with E, do not prepare between sessions |
| One device for several people | E | client.release() between one person and the next (a kiosk, a shared tablet) |
| Two tabs of your app | one client per page | Each tab is a page that loads its own models; they share only the browser’s cache |
| A page before, on the same origin | D | It warms the browser’s cache; nothing stays in memory |
The five ways
Section titled “The five ways”From least to most work. In every block, /engine is your backend’s proxy (the browser never sees the key) and your_context_id is the contextId of your kit.
A · Without preparing
Section titled “A · Without preparing”When: one session, with no hurry about the startup. Gains: no call at all. Costs: the wait, visible, until the first frame.
// A · Without preparing: the capture loads the models when it starts, and the person waits until the first frame.import { createProtocolClient } from '@neurologyca/capture'
// /engine is your backend's proxy: the browser never sees the key. Replace your_context_id with the contextId of your kit.const client = createProtocolClient({ baseUrl: '/engine', taxonomy: { contextId: 'your_context_id' } })
/** @param {{ userProtocolId: string, protocolMechanicId: string }} pair from your backend */export async function measure(pair) { const session = await client.captureProtocol({ ...pair, source: { type: 'webcam' } }) // … the session's steps … return session.stop() // CaptureOutcome}B · Prepare on the previous screen
Section titled “B · Prepare on the previous screen”When: a form or instructions before measuring. Gains: the face model is ready when the person presses Start; the capture adopts it with no option. Costs: the model comes from a third-party CDN, which sees the IP before anything is measured, even if the person never measures. If they leave without measuring, client.release().
// B · Prepare on the previous screen: the face model loads while the person fills in a form or reads the instructions.import { createProtocolClient } from '@neurologyca/capture'
const client = createProtocolClient({ baseUrl: '/engine', taxonomy: { contextId: 'your_context_id' } })
// When the form or the instructions show. The face only: no permission and no call to Engine.// Note: the model comes from a third-party CDN, which sees the IP, even if the person never measures.export function onInstructionsShown() { void client.prepare({ models: { face: true } })}
/** On Measure, the capture adopts the face model already loaded, with no option. @param {{ userProtocolId: string, protocolMechanicId: string }} pair */export async function onMeasure(pair) { const session = await client.captureProtocol({ ...pair, source: { type: 'webcam' } }) // … the session's steps … return session.stop()}
// If the person leaves this view without measuring: what was prepared stays in memory until you release it.export function onLeave() { client.release()}C · Speech after consent
Section titled “C · Speech after consent”When: with transcription. Gains: speech loads after your consent screen, while the person reads what comes next. Costs: about 160 MB the first time; after that, from the cache. prepare() does not ask for the microphone, so nothing in the browser warns about that download: your yes is the only one. On Safari and iOS speech is not prepared and loads at the start: estimatedSpeechMs (a guide that tends to come out high) lets you warn about that wait; where the device will not transcribe ('unavailable'), there is none.
// C · Speech after consent: a large download on the person's data plan, only once they have said yes.import { createProtocolClient } from '@neurologyca/capture'
// Speech needs transcription configured on the client.const client = createProtocolClient({ baseUrl: '/engine', taxonomy: { contextId: 'your_context_id' }, transcribe: { language: 'en' },})
/** * Called by the «I agree» of your consent screen, never before. prepare() does not ask for the microphone: the capture * does, when it starts. * @param {(ms: number) => void} showWaitingNote your screen: a note that speech will still take a while at the start */export async function onConsentAccepted(showWaitingNote) { const prepared = await client.prepare({ models: { face: true, speech: true } }) // Not ready ahead, but the capture will load it at the start (Safari and iOS, a slow network); never with 'unavailable'. // The estimate tends to come out high; null when it cannot be estimated. const speechLater = prepared.transcription === 'not-prepared' || prepared.transcription === 'failed' if (speechLater && prepared.estimatedSpeechMs !== null) showWaitingNote(prepared.estimatedSpeechMs) return prepared}D · Warm the cache on an earlier page
Section titled “D · Warm the cache on an earlier page”When: a landing page or a waiting room on the same origin. Gains: the files reach the browser’s cache before the capture page; it pays off above all for speech. Costs: nothing stays in memory, so the capture still creates the models when it starts. Both pages must read the same configuration, or the capture downloads again what does not match.
// D · Warm the cache on an earlier page of the same origin: the files reach the browser's cache, nothing stays in memory.import { captureCacheStatus, prepareCapture } from '@neurologyca/capture'
// capture-config: one module both pages import, so the files warmed are the ones the capture asks for.// No baseUrl or taxonomy: there is no client on this page (they would throw E_UNKNOWN_OPTION)./** @type {import('@neurologyca/capture').CaptureCacheStatusOptions} */export const captureConfig = { transcribe: { language: 'en' } }
// On the landing page or the waiting room.export async function warmTheCache() { const status = await captureCacheStatus(captureConfig) if (status.speechWeights !== 'cached') { await prepareCapture({ ...captureConfig, level: 'cache', prepare: { face: true, speech: true } }) }}
// On the capture page: the same config, plus what only a client has.// createProtocolClient({ ...captureConfig, baseUrl: '/engine', taxonomy: { contextId: 'your_context_id' } })E · Several sessions
Section titled “E · Several sessions”When: baseline, then measure, then measure. Gains: from the second session on, the models are reused: no download and no model load again (on Safari and iOS, only the face); the camera and the first frame still start. One prepare() before the first session, none between sessions. Costs: they take memory until you release them. The face model may carry state from the last frame into the next session, so call client.release() between one person and the next, and before you drop the client (a view that unmounts). If the person already agreed to speech, ask for it in that one call (prepare({ models: { face: true, speech: true } })); otherwise the first measurement loads it.
// E · Several sessions (baseline, measure, measure): what each session loads stays for the next one.import { createProtocolClient } from '@neurologyca/capture'
const client = createProtocolClient({ baseUrl: '/engine', taxonomy: { contextId: 'your_context_id' }, transcribe: { language: 'en' }, keepPrepared: true, // what each session loads stays for the next one})/** @type {import('@neurologyca/capture').CaptureSource} */const source = { type: 'webcam' }
/** The same person: a baseline and two measurements, with the ids from your backend. * @param {string} userId @param {string} userProtocolId @param {string} firstPm @param {string} secondPm */export async function baselineAndTwoSessions(userId, userProtocolId, firstPm, secondPm) { // Once, before the baseline. await client.prepare() await client.captureBaseline({ userId, source })
// No prepare() between sessions: each one adopts what the previous one left. const first = await client.captureProtocol({ userProtocolId, protocolMechanicId: firstPm, source }) await first.stop() const second = await client.captureProtocol({ userProtocolId, protocolMechanicId: secondPm, source }) await second.stop()}
// Between one person and the next, and before you drop the client (a view that unmounts): what is kept stays in// memory until then. When the page goes away, the SDK releases it on its own.export function releaseModels() { client.release()}What is downloaded
Section titled “What is downloaded”| What | How much | From where | How long it stays in the cache |
|---|---|---|---|
| The face: the vision runtime and the face model | about 13.5 MB | cdn.jsdelivr.net, storage.googleapis.com |
The browser’s HTTP cache; the face model is revalidated after an hour |
| Speech, only with transcription | about 160 MB the first time | cdn.jsdelivr.net, huggingface.co, *.hf.co |
The browser’s Cache Storage, until the browser or the person clears the site’s data |
The cache is the browser’s: it may clear it whenever it likes, so no period can be taken for granted. Those servers see the device’s IP and receive nothing captured; you can host the files yourself (CSP and transcription). What the person should know: Privacy and data.
What the person sees
Section titled “What the person sees”The SDK draws nothing. onStartupStage gives each stage with userMessage, a ready-made text (STARTUP_MESSAGES); a preparation’s events carry kind: 'prepare', and the capture’s screen paints its own. More: The startup screen and Timeouts.
Check it in your app
Section titled “Check it in your app”There is no reference figure here: check it with your own sessions on your devices. In onReport, report.startup.prepared says what each session did: face and speech read 'adopted' when it used what was prepared or kept, prepared.origin says where it came from ('prepare' or 'previous-session'; absent the first time a preparation is lent; it appears from the second session on), and faceReloaded appears, true, only when it had to load the face again. startup.totalMs is how long that startup took; compare it between your first and second sessions. Without anything prepared, prepared is null.
What can go wrong
Section titled “What can go wrong”- The capture downloads everything again: you prepared on another client. Only the next capture of the same client adopts (pitfall).
prepare()during a session rejects withPrepareErrorandreason: 'capture-in-progress': prepare before, or once it ends.- A narrower call releases:
prepare()with the face only after asking for speech releases the speech. Decide once what to prepare; in a component that mounts twice, pass the samesignal, or none. - Where transcription is refused (no WebGPU and no
allowWasmWorker), preparing speech between sessions releases the kept face and loads it again every time: with E, prepare once, or the face only. - Speech does not arrive in time:
prepare()resolvespartialwithspeech-timeoutorspeech-error. Start anyway (the capture tries speech again), go on without it withcaptureProtocol({ …, transcribe: false }), which keeps the prepared face, or offer the deferred capture. - No WebGPU (many Android phones):
transcription: 'unavailable'andtranscriptionGate.blocker: 'no-webgpu'; nothing was downloaded. Go on without speech or offer the deferred capture (pitfall). - The face fails and speech loaded (
failed,face: 'failed',transcription: 'ready'): start anyway; the capture loads the face and adopts the speech. If nobody is going to start, callclient.release(), or about 160 MB stay loaded. - The tab closes on a modest phone while speech loads (memory): the page reloads. Prepare speech just before Start, or only the face.
More: Prepare before Start and Keep the models between captures. Whether this device can measure and transcribe: Devices, quality and mode.