VWU.AI · Published

In-Cabin Action Annotation: Define Temporal Boundaries

In-Cabin Action Annotation: Define Temporal Boundaries — Visible action → Unknown interval → Boundary review
Original concept diagram, not measured data. Review the evidence at each step before expanding the dataset.

A useful in-cabin action label needs a reproducible start rule, an end rule and a policy for moments the camera cannot resolve. A clip labeled “phone use” is ambiguous if one annotator marks the first reach and another starts when fingers touch the screen. For driver monitoring systems (DMS) and occupant monitoring systems (OMS), settle that distinction before accepting a video batch.

This guide proposes a boundary-labeling protocol for dataset buyers and annotation leads. It is a project design aid, not an industry standard or a claim about any supplied dataset. For broader scope and coverage questions, start with the driver monitoring data overview.

Key Takeaways - Define an observable action separately from an inferred state such as distraction. - Record unknown or occluded time explicitly; do not silently convert it into positive or negative evidence. - Review start error, end error and temporal overlap together before applying a negotiated acceptance rule.

Define the action before its timestamps

Write the action as a visible criterion that different people can apply to the same frames. For example, a proposed “phone manipulation” rule could begin when visible hand contact and manipulation are established, then end when that visible manipulation ceases. Specify whether merely holding a phone qualifies. This example is a proposed label definition, not a validated measure of driver attention.

Do not use a visible phone action as an automatic substitute for cognitive distraction. Keep actor identity, action class and visibility in separate fields. The driver and front passenger can perform similar movements at the same time, while only one belongs to the target actor.

For each action, collect a positive example, a close negative and a disputed boundary example. Explain why each qualifies or fails. A hand moving toward a console, for instance, need not meet a phone-specific rule when the object is unresolved. Preserve that uncertainty instead of asking annotators to guess intent.

Fix the clock and interval convention

Agree on one time basis before labeling. Record whether timestamps refer to the source recording or a trimmed clip, how clip offsets map to the source, and whether the delivered video was resampled. Retain the mapping between decoded frames and timestamps. A frame number is not a complete time reference without its indexing convention and the relevant timing information.

For this proposed protocol, use half-open intervals: [start, end) includes the start and excludes the end. This makes adjacent intervals meet without assigning the shared instant twice. It is a local storage convention; a benchmark or tool may require a different export representation.

Store seconds at a precision supported by the media. Do not manufacture subframe certainty by adding decimal places. If a source has variable frame timing, use its timestamps rather than assuming that a nominal frame rate describes every decoded frame.

Check one exported example against the source viewer before scaling production. Confirm the actor, the first included evidence, the first excluded moment and the clip-to-source offset. Keep both the media version and the export version so a later conversion cannot silently move the labels.

Separate occlusion, interruption and clip truncation

An action disappearing from view does not establish that it ended. It also does not prove that it continued. The CVAT objects sidebar documentation describes an Occluded property for object annotations. That flag concerns object visibility; it does not, by itself, establish the duration of an action. The documentation was checked on October 31, 2026.

Choose the action policy explicitly. One conservative option is to annotate visible action intervals, retain a separate unknown span, and link the visible segments with an event identifier when the project permits that association. State whether this link expresses a continuity hypothesis or a confirmed event identity. It must not quietly fill the gap in the training target.

Use separate reasons for these cases:

Case Proposed record Review question
Hands or object become hidden Unknown span with visibility reason Is action evidence actually available in another approved view?
Visible action stops and later resumes Separate intervals, optionally linked Does the definition treat the pause as a new event?
Clip begins during an action Left-truncated flag Is the real onset outside the available media?
Clip ends before the action resolves Right-truncated flag Is the offset observed or only imposed by the clip edge?

A clipped event boundary should not be presented as an observed onset or offset. Report truncated events separately when reviewing boundary agreement, or define precisely how the agreed metric handles them.

In-Cabin Action Annotation: Define Temporal Boundaries — Visible action → Unknown interval → Boundary review
Original concept diagram, not measured data. Review the evidence at each step before expanding the dataset.

Use temporal IoU without hiding endpoint errors

Temporal intersection over union (tIoU) compares the shared duration of two intervals with their combined duration. The ActivityNet evaluation utility, segment_iou, computes overlap from interval endpoints and divides it by the sum of both durations minus that overlap. Its continuous-time calculation has no extra “plus one frame” term. This living source was checked on October 31, 2026.

Here is a fictional arithmetic example, not a dataset result. A reference interval is [10, 14) seconds and a candidate is [11, 15). Their intersection lasts 3 seconds, their union lasts 5 seconds, and tIoU is 3 / 5 = 0.6. Both candidate endpoints are one second late. That last fact matters even when the overlap looks acceptable.

The same absolute shift affects short and long events differently. For another fictional example, shifting [10, 12) to [11, 13) gives an intersection of 1 second and a union of 3 seconds, so tIoU is about 0.33. Neither number establishes an acceptable boundary tolerance. Choose tolerances from the task and review the consequences on representative clips.

Record signed start and end differences as well as absolute errors. A consistently late onset and an early offset can require different guideline changes. Validate positive interval duration before calculation and define how unmatched, duplicate, truncated and unknown events enter the report.

Pairwise tIoU is not average precision. The ActivityNet detection evaluator uses scored detections, matching and class-level average precision; its default overlap thresholds run from 0.50 to 0.95. A single annotation pair cannot reproduce that detection evaluation. These defaults, checked on October 31, 2026, are not recommended universal acceptance thresholds for DMS or OMS labels.

Run an independent pilot and resolve the cause

Have annotators label the same pilot clips independently before they discuss disagreements. Include ambiguous object visibility, interrupted actions and events touching clip edges. Select this set for diagnostic coverage, and document its selection; do not present it as a random estimate of the entire dataset unless it was sampled that way.

During adjudication, inspect the media around each disputed endpoint. Classify the cause: class definition, actor assignment, timing conversion, visibility uncertainty or a simple marking error. A new numeric tolerance will not fix a disagreement about what “phone use” means.

Save the original annotations, the final decision and its reason. If the rule changes, identify earlier clips affected by that change and review them before freezing the batch. An adjudicated reference is a documented decision under a guideline, not proof that every ambiguous frame has a unique objective answer.

Put boundary evidence in the delivery contract

Ask for a compact export that preserves the decision context. A proposed minimum is recording_id, clip_id, a pseudonymous actor_id, action_class, start_seconds, end_seconds, visibility or uncertainty state, truncation flags, event_id when applicable, and guideline_version. Define missing values and permitted states in a data dictionary.

The acceptance package should include the media timing map, the guideline and examples, pilot disagreements, adjudication notes, export checks and unresolved exceptions. The video annotation quality checklist covers the broader batch acceptance process.

Agree in advance which events are eligible for each metric, how unmatched events are counted, and how results are broken down by action and visibility. Review a sample of apparent passes as well as failures. A favorable aggregate can conceal one poorly defined action or a recurring timestamp conversion error.

Finally, keep this label review separate from evaluation-set independence. Accurate boundaries do not prevent related recordings from leaking across partitions; use the dataset split leakage guide for that distinct check.

Editorial scope

VWU.AI publishes this technical explainer using the primary documentation cited inline. The diagrams, field choices and arithmetic examples are illustrative proposals, not customer results, measured model performance or a statement of certified capability. Source files and documentation can change after the retrieval date. For publisher information, see about VWU.AI. To flag a technical correction, use the website contact section.