Originally published 2026-11-12 · VWU.AI
Isekai3D and Paired Visual Data: Aligning Views, Depth and Actions
Validate time, camera geometry, object identity and action labels before accepting paired visual data.

The Isekai3D SDG research page, published October 26, 2026, describes recording an Unreal Engine gameplay simulation and replaying it from different viewpoints to collect RGB, depth, normals and instance IDs with aligned camera, character and action metadata. It is a gameplay research platform. The checks below are proposed for dataset acceptance.
Define what pairing guarantees
Record the simulation run, event and clock, camera calibration, output frame IDs, depth convention, instance IDs and action-label version. Separate temporal, geometric, identity and action requirements.
Start with a short reviewable event
Pick a short motion with clear phases and collect paired frames before, during and after the event. Save the synchronized files and metadata in a sample card so another reviewer can repeat the check.
Check time before geometry
Confirm timestamps, frame indexing, sampling and gaps across modalities. Then check camera intrinsics and extrinsics, units, depth convention and projection using points visible in shared views. A time offset and a camera transform error can produce similar symptoms.
Separate identity, visibility and action
The same object can retain its instance ID when one camera cannot see it. Record occlusion independently. Distinguish an action command, visible motion and a confirmed result. Do not derive one directly from another.
Diagnose a disappearing cup
In a hypothetical example, camera A shows a cup moved to a tray, while it disappears briefly from camera B’s RGB image but appears in depth. Check frame pairing first, then projection, occlusion, instance masks and render passes. Do not discard the whole sample before isolating the mismatch.
Scale checks to the batch
Automate checks for missing fields, unmatched frames, invalid depth and conflicting IDs. Inspect difficult motion and occlusion manually. Group replay variants by underlying run in train and test splits.
Validate the downstream task
Well-paired output is a dataset quality result. Test whether it helps the intended model on an independent real target set and report boundary slices separately.