docs/DATASET_CARD.md

EventActivityNet Dataset Card

Dataset Summary

EventActivityNet v1.0 provides generated event voxel tensors derived from ActivityNet videos and ActivityNet Captions annotations. Two representations, 5-bin and 9-bin, cover the same canonical 3,263 videos, split assignments, annotations, and nested release scales. The release does not contain native event-camera recordings.

Dataset Details

Field	Value
Dataset name	EventActivityNet
Version	v1.0
Source lineage	ActivityNet and ActivityNet Captions
Source video lineage	non-interpolated/original-rate ActivityNet videos
Canonical videos	3,263
Train / validation	2,316 / 947
Verified action classes	200
Representations	5-bin and 9-bin HDF5 event voxel tensors
Public tar payload	approximately 8.57 TB

Representation Statistics

Representation	Path	Event shape	Train shards	Validation shards	HDF5 bytes	Tar bytes
5-bin	data_5bin/	(T5, 5, H, W)	157	62	4,355,745,895,245	4,355,753,021,440
9-bin	data_9bin/	(T9, 9, H, W)	157	62	4,214,122,096,103	4,214,129,203,200

The two variants are alternative groupings of adjacent-frame transition slices. No quality or superiority claim is attached to either grouping.

Scale Statistics

Scale	Videos	Hours	Train	Validation	Classes	Event-friendly
Large	3,263	106.94	2,316	947	200	65.31%
Medium	1,537	50.00	1,074	463	200	64.80%
Small	667	20.00	473	194	200	64.62%

Small is a strict subset of Medium, and Medium is a strict subset of Large. Scale membership is manifest-based and shared by both representations.

Source and Provenance

The canonical Large set matches the recovered original Large subset manifest. The recovered curation implementation merged ActivityNet Captions train and validation metadata before sampling. Verified principles include seed 2025, initial class-balanced sampling with max(5, int(class_ratio * class_count)), duration stratification at the 33% and 66% quantiles, and event-friendly enrichment using caption keywords or first-frame darkness.

The source videos follow the original-rate, non-interpolated ActivityNet lineage. FPS and resolution vary by video.

Generation Pipeline

The recovered implementation follows:

1. ActivityNet video loader;
2. mp4_to_h5.mp4_to_h5_stream();
3. EventEmulatorGPU.video_to_voxel();
4. event-slice grouping with B=5 or B=9;
5. HDF5 writer.

Both variants preserve source resolution, use one generated slice per adjacent decoded-frame transition, store events as int16, and store start/count arrays as int64/int32. No learned V2V checkpoint is required for HDF5 generation.

Event-Friendly Definition

A video is event-friendly when a caption contains run, fast, sprint, night, dark, or slow-motion, or when normalized first-frame mean brightness is below 0.4.

Scale	Event-friendly videos	Caption-keyword matches	Dark-first-frame matches	Both	Neither
Large	2,131	630	1,954	453	1,132
Medium	996	297	914	215	541
Small	431	118	396	83	236

Dataset Structure

Each canonical video has one HDF5 member per representation. Physical payload is split into 157 train and 62 validation shards under each representation directory. Shared annotations, source timing, and scale manifests are not duplicated. Representation-specific shard metadata is under metadata/5bin/ and metadata/9bin/.

Data Format

For bin count B in {5, 9}, events has shape (T_B, B, H, W) and:

T_B = ceil((N - 1) / B)

voxel_event_start[t] is the first adjacent-frame transition index in group t; voxel_event_count[t] is the number of valid transition slices. The last group may be partial and unused bins are zero-filled. See DATASET_FORMAT.md.

Intended Uses

• generated event voxel representation learning;
• activity recognition;
• caption-aligned video/event modeling;
• comparison across nested scales;
• comparison of 5-bin and 9-bin temporal groupings without treating either as native-event ground truth.

Out-of-Scope Uses

• identifying people or biometric recognition;
• surveillance deployment;
• consequential decisions about individuals;
• uses or redistribution that violate ActivityNet or ActivityNet Captions terms.

Limitations

• Tensors are generated from conventional videos, not captured by an event camera.
• The release is curated and is not an unbiased conversion of all ActivityNet videos.
• Source frame rate and resolution vary by video.
• Seconds-level timing is approximate for within-video variable-frame-rate streams because construction follows decoded frame order rather than per-frame presentation timestamps.
• Medium and Small are deterministic nested v1.0 scales, not historical original subsets.
• The verified release contains 200 classes. ActivityNet references to 203 classes do not describe the recovered release manifest.
• The verified Large duration is 106.94 hours; historical 107.3-hour wording is approximate.

Licensing and Citation

ActivityNet and ActivityNet Captions terms and citation obligations remain in force. EventActivityNet grants no additional rights beyond the source datasets. See LICENSE_NOTES.md and the repository CITATION.cff.

Integrity Verification

Both representations contain exactly 3,263 readable HDF5 members with the required three-dataset schema. Final remote verification confirmed 438 expected tar shards, exact recorded sizes and checksums, no missing or unexpected tar paths, and no stale payload under the former data/ namespace.