docs/DATASET_CARD.md

# EventActivityNet Dataset Card

## Dataset Summary

EventActivityNet v1.0 provides generated event voxel tensors derived from
ActivityNet videos and ActivityNet Captions annotations. Two representations,
5-bin and 9-bin, cover the same canonical 3,263 videos, split assignments,
annotations, and nested release scales. The release does not contain native
event-camera recordings.

## Dataset Details

| Field | Value |
|---|---|
| Dataset name | EventActivityNet |
| Version | v1.0 |
| Source lineage | ActivityNet and ActivityNet Captions |
| Source video lineage | non-interpolated/original-rate ActivityNet videos |
| Canonical videos | 3,263 |
| Train / validation | 2,316 / 947 |
| Verified action classes | 200 |
| Representations | 5-bin and 9-bin HDF5 event voxel tensors |
| Public tar payload | approximately 8.57 TB |

## Representation Statistics

| Representation | Path | Event shape | Train shards | Validation shards | HDF5 bytes | Tar bytes |
|---|---|---|---:|---:|---:|---:|
| 5-bin | `data_5bin/` | `(T5, 5, H, W)` | 157 | 62 | 4,355,745,895,245 | 4,355,753,021,440 |
| 9-bin | `data_9bin/` | `(T9, 9, H, W)` | 157 | 62 | 4,214,122,096,103 | 4,214,129,203,200 |

The two variants are alternative groupings of adjacent-frame transition
slices. No quality or superiority claim is attached to either grouping.

## Scale Statistics

| Scale | Videos | Hours | Train | Validation | Classes | Event-friendly |
|---|---:|---:|---:|---:|---:|---:|
| Large | 3,263 | 106.94 | 2,316 | 947 | 200 | 65.31% |
| Medium | 1,537 | 50.00 | 1,074 | 463 | 200 | 64.80% |
| Small | 667 | 20.00 | 473 | 194 | 200 | 64.62% |

Small is a strict subset of Medium, and Medium is a strict subset of Large.
Scale membership is manifest-based and shared by both representations.

## Source and Provenance

The canonical Large set matches the recovered original Large subset manifest.
The recovered curation implementation merged ActivityNet Captions train and
validation metadata before sampling. Verified principles include seed `2025`,
initial class-balanced sampling with
`max(5, int(class_ratio * class_count))`, duration stratification at the 33%
and 66% quantiles, and event-friendly enrichment using caption keywords or
first-frame darkness.

The source videos follow the original-rate, non-interpolated ActivityNet
lineage. FPS and resolution vary by video.

## Generation Pipeline

The recovered implementation follows:

1. ActivityNet video loader;
2. `mp4_to_h5.mp4_to_h5_stream()`;
3. `EventEmulatorGPU.video_to_voxel()`;
4. event-slice grouping with `B=5` or `B=9`;
5. HDF5 writer.

Both variants preserve source resolution, use one generated slice per adjacent
decoded-frame transition, store `events` as `int16`, and store start/count
arrays as `int64`/`int32`. No learned V2V checkpoint is required for HDF5
generation.

## Event-Friendly Definition

A video is event-friendly when a caption contains `run`, `fast`, `sprint`,
`night`, `dark`, or `slow-motion`, or when normalized first-frame mean
brightness is below `0.4`.

| Scale | Event-friendly videos | Caption-keyword matches | Dark-first-frame matches | Both | Neither |
|---|---:|---:|---:|---:|---:|
| Large | 2,131 | 630 | 1,954 | 453 | 1,132 |
| Medium | 996 | 297 | 914 | 215 | 541 |
| Small | 431 | 118 | 396 | 83 | 236 |

## Dataset Structure

Each canonical video has one HDF5 member per representation. Physical payload
is split into 157 train and 62 validation shards under each representation
directory. Shared annotations, source timing, and scale manifests are not
duplicated. Representation-specific shard metadata is under `metadata/5bin/`
and `metadata/9bin/`.

## Data Format

For bin count `B` in `{5, 9}`, `events` has shape `(T_B, B, H, W)` and:

```text
T_B = ceil((N - 1) / B)
```

`voxel_event_start[t]` is the first adjacent-frame transition index in group
`t`; `voxel_event_count[t]` is the number of valid transition slices. The last
group may be partial and unused bins are zero-filled. See
[DATASET_FORMAT.md](DATASET_FORMAT.md).

## Intended Uses

- generated event voxel representation learning;
- activity recognition;
- caption-aligned video/event modeling;
- comparison across nested scales;
- comparison of 5-bin and 9-bin temporal groupings without treating either as native-event ground truth.

## Out-of-Scope Uses

- identifying people or biometric recognition;
- surveillance deployment;
- consequential decisions about individuals;
- uses or redistribution that violate ActivityNet or ActivityNet Captions terms.

## Limitations

- Tensors are generated from conventional videos, not captured by an event camera.
- The release is curated and is not an unbiased conversion of all ActivityNet videos.
- Source frame rate and resolution vary by video.
- Seconds-level timing is approximate for within-video variable-frame-rate streams because construction follows decoded frame order rather than per-frame presentation timestamps.
- Medium and Small are deterministic nested v1.0 scales, not historical original subsets.
- The verified release contains 200 classes. ActivityNet references to 203 classes do not describe the recovered release manifest.
- The verified Large duration is 106.94 hours; historical 107.3-hour wording is approximate.

## Licensing and Citation

ActivityNet and ActivityNet Captions terms and citation obligations remain in
force. EventActivityNet grants no additional rights beyond the source datasets.
See [LICENSE_NOTES.md](LICENSE_NOTES.md) and the repository `CITATION.cff`.

## Integrity Verification

Both representations contain exactly 3,263 readable HDF5 members with the
required three-dataset schema. Final remote verification confirmed 438 expected
tar shards, exact recorded sizes and checksums, no missing or unexpected tar
paths, and no stale payload under the former `data/` namespace.