Operator runbook for issue 01

1. Open a run

Assign one run ID per system, edition, task and attempt. Record UTC date, operator, device and firmware, service build if exposed, plan, retention, camera ID, fixed view, resolution, frame rate, time zone, schedule, rule revision, display threshold, notification channel and clock offsets. Where the service does not expose a build, write “unknown”. Obtain access to the proposed hardware or service, plus permission for the footage, site and recipients. If access is still pending, preserve the empty result.

2. Check the input path

Download the manifest and the replay files. From a checkout, run the commands below from the lab folder: the first checks the packaged files, the second prints the actual container timestamps for the marked crossing clip.

python3 validate.py
ffprobe -v error -select_streams v:0 -show_entries frame=best_effort_timestamp_time -of csv=p=0 corpus/media/F01.mp4

Start with C03, whose rectangle lasts from 1.0 s up to 3.0 s. Compare preview and export against that interval. Then use F01 and check that the approaching sedan is visible at 2.7 s. Preserve the original MP4, case ID, digest and returned export. Replay all 15 once, logging accepted, decoded, refused or outside scope for each. Reset the test channel between files, or use isolated sessions if grouping spans clips.

For a documented upload path, submit one file once and log the job identifier. For a compatible stream path, use a test relay or encoder approved for that system. The example below sends one file at its recorded rate to an operator-provisioned RTSP server that accepts publishers. Enter the test relay address in REPLAY_RTSP_URL first, point the receiving client at that relay, verify timestamps and stop after the file ends.

ffmpeg -re -i corpus/media/F01.mp4 -map 0:v:0 -c:v copy -an -f rtsp -rtsp_transport tcp "$REPLAY_RTSP_URL"

A product that requires native cameras, a recorder or a partner station may not accept this relay. Use its supported path and, where possible, log the actual input bytes it receives. If field equipment is required, skip the file-ingest task and record why. A vendor upload's accepted extensions must be verified at intake. Icarian documents uploads and several stream paths. A03 · Connection guide · A12 · Upload workflow

3. Run the assigned task

The eight rows below correspond to the report. Longer intervals in these procedures call for a separate authorized site session; the tiny fixture pack stays reserved for system checks.

System	Setup and action	What to compute and retain
Sightline	Schedule a one-hour gate observation. Record sender stop and restart at minute 20 and 25. Save the PDF, camera status history and rule settings. Use PDF intervals if available; log displayed time precision. A06 · Reports	Divide observed monitored seconds by 3,600. Subtract the actual logged sender gap from the PDF gap, in seconds. Carry rounding into the result or mark unavailable if the report cannot resolve it.
Spot AI	Connect the intended channels to the recorder. Prepare a sealed key for two distinct target clips. Give the operator the same two written site/time/vehicle prompts used on Connector. Start each timer on prompt display and stop at correct playback. V01 · System	Count correct target clips opened within 180 s over two assigned searches. Preserve each time, wrong opens and timeouts. Confirm the key against the exported interval.
Coram	Keep Point and camera powered. Record one moving time marker on the LAN. Drop only the authorized test WAN for five minutes. After recovery export that interval. V03 · Point	Divide seconds with decodable advancing marker by actual interruption seconds. Measure the longest contiguous missing or frozen segment. Keep capture, WAN log and export.
Lumana	Fix the camera list, licensed feed count, protocol and retention. Connect each eligible feed to the proposed Core. Record UTC when the connection is made and when the first archived interval plays in VMS+. Allow ten minutes per feed. V06 · Core and VMS+	Count feeds with a playable archive within ten minutes over attempted eligible feeds. For successes give elapsed minutes per feed; list timeouts separately. Keep the configuration and playback records.
Verkada	Use an eligible third-party camera on a licensed Connector channel. Repeat the two sealed Spot prompts on the same scenes, one attempt per prompt. Counterbalance operator order across systems. V09 · Connector	Use the same 180-second stop, correctness key and two-search denominator. Record correct playback times and timeouts. Keep the channel type with each result.
Eagle Eye Networks	Seed two distinct UTC markers on a camera configured to retain at least nine days. At eight days of age try direct playback of each, then have a second operator try Smart Video Search. Record edition. V10 · Edition table	Directly playable/all seeded measures retention. For search, record retrieved/retained eligible markers. Mark out-of-window searches outside scope and keep the edition limit in the report.
Pano AI	Arrange a partner-approved historical or simulated smoke exercise with known positive references and a test destination. Use an authorized analyst confirmation log and actual receipt record, both synchronized. Record exercise mode and eligibility. V12 · Confirmation path	Count confirmed notifications over eligible exercises and measure receipt minus analyst-confirmation time per delivered message. A historical case without an instrumented receipt can contribute no live delay.
LVT Essentials	At an authorized temporary post log the scheduled shift, power conditions and intended view. Include a visible advancing clock in the permitted field of view. Request three supervised strobe activations and timestamp each. V14 · Package	Export the shift, count decodable advancing seconds over scheduled seconds and log each missing interval. Count witnessed activations over attempted activations. Preserve the observation sheet and package name.

4. Score the alert experiment

For products offering the same alert task, add a separate gate trial. Predeclare the rule, schedule, enrolled sites and observation period, and annotate field sessions blind to the outputs. Write CSVs with the fields below. Use the matching procedure for TP, FP and FN and the measurement protocol for negative windows and hours. Log overlapping coverage intervals only once per camera, and preserve raw event IDs and incident IDs. Icarian, for one, can group repeated activity in a single incident. A05 · Grouping

reference_id,camera,rule,class,start_utc,qualifies_utc,end_utc,reviewers,eligibility
output_id,incident_id,camera,rule,class,qualifies_utc,received_utc,raw_file
camera,scheduled_start_utc,scheduled_end_utc,monitored_start_utc,monitored_end_utc,evidence
run_id,case_or_task,config_id,attempt,status,numerator,denominator,unit,evidence,reviewer

Keep each header in its own sheet and link raw exports by filename. Exported person or vehicle tracks, where available, are a distinct unit from alert receipts. Use the timing procedure on matched live notifications. Retain FN and delivery failures even when a delay cannot be measured.

5. Close the record

Have a second reviewer replay the disputed intervals and recompute the report cell from the raw evidence. Enter the run ID, date, unit, numerator, denominator and scope beside the value. A refusal, missing log, unsupported task or zero denominator never becomes a zero score. Publish the exclusions and update the quarterly record. Redact credentials and identifiable field evidence before release; keep the originals under controlled access for the reviewers.

Record identifiers · L01 · L02 · A03 · A05 · A06 · A12 · V01 · V03 · V06 · V09 · V10 · V12 · V14