VWU.AI ·

DMS and OMS Training Data: Procurement and Acceptance Checklist

DMS and OMS Training Data: Procurement and Acceptance Checklist — Task definition → Sample validation → Delivery acceptance
Original concept diagram, not measured data. Review the evidence at each step before expanding the dataset.

Before buying training data for a driver monitoring system (DMS) or occupant monitoring system (OMS), define the model task, behavior labels and acceptance method. A request for “distraction videos” does not explain which actions count, how to label occlusion or how the buyer will decide that delivery is complete.

This checklist is for algorithm, data and procurement teams planning captured or synthetic video projects. It offers practical recommendations grounded in public research and documentation, rather than an automotive certification standard. VWU.AI's in-cabin behavior data service follows requirements review, sample validation and project-specific delivery. The examples below are illustrative, not customer deliverables or reported performance results.

Key Takeaways

On this page: Task · Examples · Timing · Metadata · Splits · Acceptance · Checklist · Sources

1. Specify the prediction the model must make

Turn broad concepts such as fatigue, distraction and passenger interaction into observable events that reviewers can label consistently. “The driver is interacting with a phone” is a different task from “a phone is visible”: the first needs a relationship among an actor, an object and an action.

The Drive&Act project uses a hierarchy of activities, actions and object interactions. Its structure illustrates why annotation levels belong in dataset design. Its labels and usage terms do not automatically transfer to a commercial project. Drive&Act official project

Requirement Question to resolve
Prediction unit Does the model classify a frame, clip, time interval or continuous event?
Target actor How are the driver, front passenger and rear passengers identified?
Observability When eyes, hands or objects are hidden, is the label unknown, ignored or another state?
Label relationships Can turning around and phone interaction overlap, or does one label take precedence?
Deployment conditions Which camera positions, imaging modalities, steering configurations and lighting conditions matter?

Check: ask the model owner and annotation lead to label the same sample independently. Record disagreements and revise definitions. Do not turn an unobservable mental state into a certain label without stating what additional evidence supports it.

2. Pair positive events with boundaries and negative examples

Choose negative examples that resemble the target action but fail its definition. A review that only asks whether the desired action appears can miss ambiguous labels.

These examples are discussion aids, not universal annotation rules:

Target event Example positive definition Negative or unknown cases to discuss
Phone interaction The target actor holds a phone and performs the agreed interaction A mounted phone; a passenger using a phone; the object is occluded
Turning around Head motion toward the rear reaches the agreed visible state Checking a mirror; eye movement alone; the face is not visible
Eye closure Visible eyes remain closed during a defined event Ordinary blinking; sunglasses; blur prevents a judgment
Occupant contact A specified actor makes the defined contact with an identified target Passing an object; adjusting clothing; occluded contact; uncertain target

For each label, record inclusion, exclusion, unknown and review-needed cases. Observable eye closure or yawning should not automatically become a diagnosis of physiological drowsiness without the additional evidence the task requires.

Agree the sample mix using task risk, gaps in existing data and validation results. A single positive-to-negative ratio is not suitable for every project. Published scenario videos help discuss appearance; the project still needs its own label definitions.

3. Define event boundaries and video specifications

For temporal annotations, specify how an event starts, how it ends and how boundary error is measured. Labels can use milliseconds or frame indices, but the unit must be explicit. Frame-based labels also need a defined relationship between frame rate and timestamps.

The following fictional record demonstrates a possible format. Its values are examples, not agreed delivery specifications:

{
  "clip_id": "example-001",
  "source_group_id": "source-example-a",
  "origin": "synthetic",
  "camera_view": "front",
  "duration_ms": 6000,
  "events": [
    {
      "actor_id": "driver-01",
      "label": "phone_interaction",
      "start_ms": 1800,
      "end_ms": 4200,
      "visibility": "visible"
    }
  ],
  "label_schema_version": "example-v1"
}

The reader should validate 0 ≤ start_ms < end_ms ≤ duration_ms and check actor IDs and labels against the agreed dictionaries. Mark events truncated by a clip boundary instead of assuming the action ends with the file.

Record the container, codec, resolution, frame rate, duration, imaging modality and audio requirements separately. A grayscale appearance does not establish equivalence to a real near-infrared sensor. If the deployment uses that modality, test the imaging assumptions during sample validation. Both parties must confirm feasible formats, annotation fields and production scope.

4. Use metadata to audit coverage and provenance

A delivery manifest should explain where a clip came from, why it belongs in the dataset and how it may be used. Datasheets for Datasets proposes documenting motivation, composition, collection and recommended uses. That provides a public framework for discussing delivery documentation. Gebru and colleagues, revised 2021

Agree fields according to the project and retain unknown values explicitly:

Do not infer unverified demographic attributes from appearance. For synthetic data, describe which generation-source records are available and what remains unknown. For captured data, agree the required permissions, privacy handling and retention arrangements. The project's compliance owner should resolve legal questions.

DMS and OMS Training Data: Procurement and Acceptance Checklist — Task definition → Sample validation → Delivery acceptance
Original concept diagram, not measured data. Review the evidence at each step before expanding the dataset.

5. Set group boundaries before clipping and augmentation

Adjacent clips from one recording and repeated clips of one person can carry related information. scikit-learn's grouped cross-validation documentation explains that repeated subject samples can allow a model to learn person-specific features, motivating group-aware evaluation. scikit-learn grouped cross-validation

For an in-cabin project, turn that principle into auditable source groups:

  1. Choose grouping dimensions based on the generalization question: subjects, original recordings, capture sessions or generation sources.
  2. Assign clips, duplicates and augmented variants from the same source to a group before allocating training, validation and test sets.
  3. If several relationships must remain isolated, merge connected samples across those relationships. Simply concatenating IDs does not necessarily keep a person out of multiple groups.
  4. Check group intersections and duplicates, then check behavior and condition coverage in each split. Isolation and coverage need to be assessed together.

These are applications of the grouping principle, not results automatically guaranteed by a tool. Synthetic variants also need source tracking. Multiple outputs from one seed image should not be treated as independent evaluation evidence without examination.

6. Validate samples before agreeing batch release

Run representative samples through the actual data reader, annotation review and model-evaluation process. Then freeze a versioned acceptance checklist. Selecting only visually attractive examples will not test difficult boundaries.

Acceptance layer What to check Evidence to retain
Files Decoding, agreed specifications and manifest-to-label correspondence Validation logs and missing-item list
Scenarios Actors, seats, actions and targets match the request Item-level decisions and rejection reasons
Labels Classes, relationships, visibility and event boundaries follow the rules Review records, disagreements and rule versions
Coverage Agreed target conditions, negatives and rare combinations are represented Counts broken down by condition
Independence Source groups, duplicates and splits follow the evaluation design Group-overlap and duplicate reports
Model Task metrics on independent, relevant evaluation data Condition-level results, limitations and reproducible settings

Before the batch, agree the sampling procedure, defect categories, rework scope and reinspection process. Thresholds depend on the task and its risks; this guide does not prescribe an unvalidated universal pass rate.

Sign off dataset quality separately from model performance. Evaluate the contribution of synthetic training data using an independent test set representative of target conditions. Visual quality alone cannot establish deployment benefit. Model Cards for Model Reporting emphasizes intended use, evaluation and limitations; it is a reporting framework, not a DMS approval certificate. Mitchell and colleagues, 2019

Final checklist before requesting a quote

Use the dataset requirements form to describe missing scenarios and delivery needs. VWU.AI discusses sample feasibility and production scope for the specific request. Pricing, quantity, schedules and usage terms are agreed per project.

Editorial ownership and sources

Published by VWU.AI for training-data requirements discussions. This guide synthesizes public research and official documentation; it does not report a new experiment. Tables, fields and JSON are illustrative recommendations, not customer data, model scores or delivery promises. Technical sources were checked on October 26, 2026.

The in-cabin illustration is an existing synthetic visual from the website, not a customer capture. Send corrections or project questions to chan@vwu.ai, or read the VWU.AI service overview.

Browse more procurement and acceptance guides