# Steps and marking

There is no capture component: your page draws each step and decides when it ends. The SDK gives you the data and checks the durations.

## The data of a step

`describeStep(step, { lang })` reads a plan step without rules per type.

| Field | What it is |
| --- | --- |
| `StepFacts.minSeconds` | The minimum duration of the step, or `null` if the catalogue does not say |
| `StepFacts.maxSeconds` | The maximum duration of the step, or `null` |
| `StepFacts.responseClock` | The catalogue value, which the SDK does not interpret (`'paused'`, `'running'`, another value or `null`; it is `null` in many step types). Decide what it means per protocol |
| `StepFacts.expectsSpeech` | Whether the step expects the person to speak: `'yes'`, `'no'`, `'any'` or `null`; `'yes'` marks an answer step (below) |
| `StepFacts.text` | The text to show, `{ value, lang, mediaPid }`, or `null` |
| `StepFacts.texts` | Every text of the step, with its `mediaPid` and one body per language |
| `StepFacts.phases` | The phases of the step, when it has them |
| `StepFacts.rounds` | The number of rounds, or `null` |
| `StepFacts.bands` | The time bands of the step, when it has them |
| `StepFacts.unknown` | What the catalogue did not say: `'duration'`, `'text'`, `'responseClock'`, `'bands'` |

The language: the one you ask for, then the plan's default.

## Who closes a step

`run({ durationMs })` closes it after that time (a cap). `start()` and `end()` let you close it yourself: start the step, then call `end()` on whichever happens first, your button, `maxSeconds` or the end of your video (the `source.ended` warning). The order is strict: one step at a time, in the plan's order (`E_STEP_SEQUENCE`). `start()` returns a promise: await it. Live it settles at once (resolves, or rejects when called); recording, it can also reject later. Only a wrong order (`StepSequenceError`) is thrown synchronously; anything else arrives as a rejected promise, which a `try`/`catch` without `await` does not catch.

## The clock

`session.stepTiming()` returns `null` outside a step, or a `StepTiming`:

| Field | What it is |
| --- | --- |
| `StepTiming.elapsedMs` | Milliseconds since the step started |
| `StepTiming.minMs` | The minimum, in milliseconds, or `null` |
| `StepTiming.maxMs` | The maximum, in milliseconds, or `null` |
| `StepTiming.minReached` | Whether the minimum has passed; `null` when there is no minimum |
| `StepTiming.maxReached` | Whether the maximum has passed; `null` when there is no maximum |
| `StepTiming.remainingMs` | Milliseconds left until the maximum, or `null` |

Enable your button when `minReached` is not `false`, and draw `remainingMs` as a countdown.

## Checking the durations

`stepDurations` in the client: `'warn'` (the default) warns when a step ends outside its window (`step.duration-below-min`, `step.duration-above-max`); `'enforce'` refuses with `E_STEP_DURATION`; `'off'` checks nothing. With `'enforce'`:

- a `run({ durationMs })` out of range is rejected before the step opens;
- an `end()` before the minimum is rejected and the step stays open: call `end()` again when `stepTiming().minReached` is `true`;
- past the maximum, `end()` only warns: closing the step is yours;
- with a `<video>` or stream source that has already ended, `'enforce'` drops to `'warn'`, because there is nothing left to wait for.

Warnings have 250 ms of tolerance. With no minimum and no maximum nothing is checked.

## The media you showed

Declare the text you showed with `mediaPid`, once per step: `start({ mediaPid })` or `run({ durationMs, mediaPid })`. When the step has one text, `facts.text.mediaPid` is it. When it has several (alternative questions), choose one from `describeStep(step).texts`, which lists every text media as `{ mediaPid, body }`: `facts.text` is only the first that matches the language, so taking it blindly is a guess. A step with several media and no `mediaPid` warns `step.media-undeclared`, and the interpretation then lacks the question (`metadata.missing_inputs` includes `question_text`; see [the pitfall](/docs/resources/pitfalls/#pf-media-undeclared)). The warning only fires in steps without phases: in a step with phases the text media are the cues of each phase, not alternative questions, and there is nothing to choose. A `mediaPid` that is not the step's throws `E_MEDIA_UNKNOWN`.

## Steps without a clock or without text

A step without a clock or without text of its own is not an answer step by default. Never decide its role by the name of the step type or by `responseClock`: the catalogue says it. What each step shows:

- **Its own media**: the text you declare with `mediaPid` (above).
- **No media of its own**: `delivery.media_from` says where its text comes from. `'previous_step'`: keep the previous step's question on screen. `'next_step'`: a silent reading step shows the question chosen for the next step; it lends the text, never the media, so declare no `mediaPid` there.
- **An answer step** (`category: 'answer'` or `expectsSpeech: 'yes'`) keeps the previous question on screen and shows its own media as the line under it (unless the previous step is a silent reading of this step's own question: then its own media, once). Any other step is not an answer step: do not show it as "answer now".
- Show only media bodies, never `delivery.instructions` or `delivery.prompt`.

`StepFacts` carries `expectsSpeech`, but not `category` nor `media_from` today: read them from the plan step in `session.plan.steps` (`PlanStep.category`, and `media_from` inside `PlanStep.delivery`, which `StepDelivery` does not type). Asked of the SDK. The [pitfall](/docs/resources/pitfalls/#pf-question-shown-as-answer) has the symptom.

When the catalogue gives no duration (`unknown` lists `'duration'`), how long the step lasts is your decision: close it with your button, and write the decision down per protocol.

## Marking in deferred: the manifest

When recording, the SDK writes a manifest with the video. Each `ManifestStep` says where the step starts and ends in the video and who ended it:

| Field | What it is |
| --- | --- |
| `ManifestStep.startMediaMs` | Where the step starts in the video, in milliseconds |
| `ManifestStep.endMediaMs` | Where the step ends in the video, in milliseconds |
| `ManifestStep.mediaPid` | The media you declared for the step |
| `ManifestStep.endedBy` | How it was closed: `'timer'` (`run()`), `'gesture'` (`end()`) or, since 0.28.0, `'source'` (the end of the source closed it: a file or a stream that finished, whether `run()` or `end()` was waiting). The type also allows `'media'` and `'cap'`, which the Guide does not document. The processor does not read it |

And the rest of the manifest:

| Field | What it is |
| --- | --- |
| `RecordingManifest.prerollMs` | The pre-roll recorded before the first step |
| `RecordingManifest.stopMediaMs` | Where the recording stops in the video |
| `RecordingManifest.hidden` | When the tab was hidden during the recording |
| `RecordingManifest.orientationChanges` | The orientation changes of the device |
| `RecordingManifest.audioGaps` | The gaps in the audio |
| `RecordingManifest.videoGaps` | The gaps in the video |
| `RecordingManifest.stimulus` | The timing of the video stimulus, if you tracked one |
| `RecordingManifest.interpreted` | Since 0.29.0: `false` when Engine said the protocol is not interpreted (Measurement), and absent otherwise. It only goes with `triggerAnalysis: false`, and the processing service's replay then sends no taxonomy and asks for no analysis either |

The pre-roll and post-roll are recording options (`prerollMs`, `postrollMs`), not manifest fields. If you show a video stimulus, `session.trackStimulus(order, el)` records its timing. The manifest is not frozen yet: read only these fields.

## Helpers

From `@neurologyca/capture/helpers`: `keepScreenOn` (the screen stays on), `guardUnload` (asks before closing the tab while a capture or upload runs), `classifyMediaError` (turns a camera error into a reason you can show), `waitForPortraitOrientation` and `startOrAsk` (on a `POSITION_RESERVED` it asks the person, and repeats with `discardPending: true` only on their yes; since 0.29.0, if the holder is already receiving frames, `holder.receivingPackets: true`, it does not ask: Engine would refuse the discard, so it throws the `POSITION_RESERVED` again, and you try later). `keepScreenOn` and `classifyMediaError` follow the browsers' public documentation. `guardUnload` depends on the browser: in Chrome for Android, closing the tab does not show the dialog, so do not count on it to keep a capture open on a phone. When recording, the local copy is what protects you there: on the next visit, `listPendingRecordings()` returns the recordings that were not uploaded (as `RecordingResult`); upload each one with `uploadRecording` or discard it with its `release()`. Live, a closed tab ends the capture: capture again.

## What the SDK does not do with a plan

`flowMode: 'free'` mechanics (microinterventions) are not supported (`E_FLOW_UNSUPPORTED`), phases are not driven, and there are no interruptions: your page handles them.

## The same loop, live and recorded

What each step shows comes from `stepPrompt`, the rule of [the section above](#steps-without-a-clock-or-without-text) and of the playground: the question (`text`), the line under it on an answer step (`cue`), `kind` (`'question'`, `'answer'` or `'silent'`) and the `mediaPid` to declare. Show «answer now» only when `kind` is `'answer'`. It reads `category` and `media_from` from the plan step, because `StepFacts` does not carry them:

```js
// What each step shows, from the plan: its own question; without one, the previous step's (media_from 'previous_step')
// or the next step's text (media_from 'next_step', a silent reading step: it lends the text, never the media).
// An answer step (category 'answer' or expectsSpeech 'yes') keeps the previous question on screen, its own media under it.
// Only media bodies, never delivery.instructions or delivery.prompt.
import { describeStep } from '@neurologyca/capture/record' // the light entry: it does not download kodec

/**
 * @typedef {{ kind: 'question' | 'answer' | 'silent', text: string | null, cue: string | null, mediaPid: string | null }} StepPrompt
 * @param {import('@neurologyca/capture/record').PlanStep[]} steps session.plan.steps
 * @param {number} order the step to show
 * @param {Record<string, string>} choice the mediaPid you chose per step order; absent, the step's first text
 * @param {string} lang the spoken language
 * @returns {StepPrompt | null} mediaPid: what to declare in start({ mediaPid }), null when the step has no media of its own
 */
export function stepPrompt(steps, order, choice, lang) {
  const sorted = [...steps].sort((a, b) => a.order - b.order)
  const at = sorted.findIndex((s) => s.order === order)
  if (at < 0) return null
  const step = sorted[at], prev = sorted[at - 1], next = sorted[at + 1]
  /** @param {import('@neurologyca/capture/record').PlanStep} s */
  const pidOf = (s) => choice[String(s.order)] ?? describeStep(s).texts[0]?.mediaPid ?? null
  /** @param {import('@neurologyca/capture/record').PlanStep} s */
  const own = (s) => {
    const body = describeStep(s).texts.find((t) => t.mediaPid === pidOf(s))?.body
    return body ? (body[lang] ?? body.en ?? Object.values(body)[0] ?? null) : null
  }
  /** @param {import('@neurologyca/capture/record').PlanStep} s */
  const isAnswer = (s) => s.category === 'answer' || s.expectsSpeech === 'yes'
  const from = step.delivery?.media_from // not in StepFacts today: read it from the plan step
  /** @type {StepPrompt['kind']} */
  const kind = isAnswer(step) ? 'answer' : from === 'next_step' && step.expectsSpeech === 'no' ? 'silent' : 'question'
  const mine = own(step)
  if (mine === null) {
    const lent = from === 'previous_step' && prev ? stepPrompt(steps, prev.order, choice, lang)?.text ?? null
      : from === 'next_step' && next ? own(next) : null
    return lent === null ? null : { kind, text: lent, cue: null, mediaPid: null }
  }
  // Not after a next_step reading step: that one already showed this step's own question.
  if (isAnswer(step) && prev && !isAnswer(prev) && prev.delivery?.media_from !== 'next_step') {
    const question = stepPrompt(steps, prev.order, choice, lang)?.text ?? null
    if (question !== null) return { kind, text: question, cue: mine, mediaPid: pidOf(step) }
  }
  return { kind, text: mine, cue: null, mediaPid: pidOf(step) }
}
```

Live, a button closes each step once its minimum is reached, and its maximum closes it on its own:

```js
// The step loop, live: a button closes each step once its minimum is reached, and its maximum closes it on its own.
// stepPrompt (step-prompt.js) says what each step shows: the question, and on an answer step the question kept on screen.
import { stepPrompt } from './step-prompt.js'

/**
 * @param {import('@neurologyca/capture').ProtocolClientHandle} client
 * @param {{ userProtocolId: string, protocolMechanicId: string }} pair from your backend
 * @param {HTMLButtonElement} next your "Next" button
 * @param {(p: import('./step-prompt.js').StepPrompt | null, remainingMs: number | null) => void} show your screen
 * @param {{ choice?: Record<string, string>, lang?: string }} [o] the question chosen per step, the spoken language
 */
export async function captureSteps(client, pair, next, show, { choice = {}, lang = 'en' } = {}) {
  const session = await client.captureProtocol({ ...pair, source: { type: 'webcam' } })
  const steps = session.plan.steps
  for (const step of steps) {
    const prompt = stepPrompt(steps, step.order, choice, lang)
    await session.step(step.order).start({ mediaPid: prompt?.mediaPid ?? null }) // settles at once when live
    await new Promise((/** @type {(v?: unknown) => void} */ done) => {
      const tick = setInterval(() => {
        const t = session.stepTiming()
        next.disabled = t?.minReached === false // enabled once the minimum is reached
        show(prompt, t?.remainingMs ?? null) // "answer now" only when prompt.kind is 'answer'
        if (t?.maxReached) finish()
      }, 250)
      const finish = () => { clearInterval(tick); next.onclick = null; done() }
      next.onclick = finish
    })
    await session.step(step.order).end()
  }
  return session.stop() // CaptureOutcome
}
```

Recorded, the steps are marked in the video, and the manifest says where each one starts and ends:

```js
// The same loop, recorded: the steps are marked in the video, and the manifest says where each one starts and ends.
import { createRecordingClient, uploadRecording } from '@neurologyca/capture/record'
import { stepPrompt } from './step-prompt.js'

/**
 * @param {{ userProtocolId: string, protocolMechanicId: string }} pair from your backend
 * @param {HTMLButtonElement} next your "Next" button
 * @param {(p: import('./step-prompt.js').StepPrompt | null) => void} show your screen
 */
export async function recordSteps(pair, next, show) {
  const recorder = createRecordingClient({ baseUrl: '/engine', taxonomy: { contextId: 'measurement' }, stepDurations: 'warn' })
  const session = await recorder.captureProtocol({ ...pair, source: { type: 'webcam' }, triggerAnalysis: false })
  const steps = session.plan.steps
  for (const step of steps) {
    const prompt = stepPrompt(steps, step.order, {}, 'en')
    show(prompt)
    await session.step(step.order).start({ mediaPid: prompt?.mediaPid ?? null }) // can reject when recording
    await new Promise((done) => { next.onclick = () => { next.onclick = null; done(undefined) } })
    await session.step(step.order).end()
  }
  const recording = await session.stop() // the video and its manifest, kept in this browser until uploaded
  console.log(recording.manifest.steps.map((s) => [s.order, s.startMediaMs, s.endMediaMs, s.endedBy]))
  return uploadRecording(recording, { url: '/uploads' }) // your receiver
}
```
