sebastientaylor commited on
Commit
73e0032
Β·
verified Β·
1 Parent(s): 5368caa

Correct the validation-pipeline description, in-progress wording, and throughput example

Browse files
Files changed (1) hide show
  1. README.md +9 -8
README.md CHANGED
@@ -101,7 +101,7 @@ EdgeFirst/yolov8-det/
101
  - **Linux aarch64** β€” ONNX Runtime / TFLite (ARM64 generic Linux)
102
  - **Apple macOS** β€” ONNX Runtime + CoreML ANE / GPU / CPU (FP16)
103
  - **NXP i.MX 8M Plus** β€” 2.3 TOPS, TFLite INT8
104
- - **NXP i.MX 95** β€” 2.0 TOPS, eIQ Neutron TFLite *(YOLOv5 / YOLOv8 only; YOLO11 / YOLO26 not yet supported on eIQ Neutron)*
105
  - **NXP Ara240** β€” 40 eTOPS, .DVM
106
  - **RPi5 + Hailo-8L** β€” 13 TOPS, HailoRT HEF
107
  - **NVIDIA Jetson Orin** β€” 67–157 TOPS, TensorRT
@@ -116,11 +116,13 @@ Each training session produces a single set of weights in [EdgeFirst Studio](htt
116
 
117
  ### EdgeFirst Profiler
118
 
119
- The on-target validation agent. Given a model and dataset, it runs full inference on the target device, captures per-image predictions in EdgeFirst Arrow/Parquet, and emits a Perfetto trace alongside. Loads each runtime through its native delegate β€” VX Delegate on NXP i.MX 8M Plus, eIQ Neutron on NXP i.MX 95, NXP Ara SDK on Ara240, HailoRT on RPi5 + Hailo, TensorRT on Jetson β€” so timing reflects deployed-application reality.
120
 
121
- ### EdgeFirst Validator
122
 
123
- The off-target post-processor. Consumes predictions + Perfetto trace, computes the 12-metric COCO accuracy tuple via `pycocotools` (or `lvis-api` for large-vocabulary datasets), and rebuilds per-stage timing summaries from the trace. Results attach to the Studio validation session as a structured YAML payload β€” the same payload this Model Zoo reads.
 
 
124
 
125
  ### EdgeFirst HAL
126
 
@@ -128,12 +130,11 @@ The [EdgeFirst Hardware Abstraction Layer](https://github.com/EdgeFirstAI/hal) p
128
 
129
  ### Latency and pipelined throughput
130
 
131
- Two timing surfaces per validation:
132
 
133
- - **`timing.inline`** β€” per-image `preprocess_ms` / `inference_ms` / `postprocess_ms` with min / mean / median / p95 / p99 / max. The universal contract every producer fills.
134
- - **`timing.trace`** β€” full per-stage breakdown from the Perfetto trace (typically 25–33 stages), plus end-to-end FPS distribution.
135
 
136
- Throughput exceeds the sum of stage latencies because the runtime pipelines I/O, preprocessing, NPU inference, and decode across frames. The Model Zoo headlines `trace.fps.median` as the throughput number, not the derived `1000 / (preprocess + inference + postprocess)`. Example: YOLOv5n on NXP i.MX 95 Neutron has per-stage means 21.7 + 12.2 + 15.8 ms (naive β‰ˆ 20 FPS) but pipelined throughput of 56 FPS median.
137
 
138
  ---
139
 
 
101
  - **Linux aarch64** β€” ONNX Runtime / TFLite (ARM64 generic Linux)
102
  - **Apple macOS** β€” ONNX Runtime + CoreML ANE / GPU / CPU (FP16)
103
  - **NXP i.MX 8M Plus** β€” 2.3 TOPS, TFLite INT8
104
+ - **NXP i.MX 95** β€” 2.0 TOPS, eIQ Neutron TFLite *(YOLOv5 and YOLOv8 published; YOLO11 and YOLO26 are in progress β€” they compile, load and run on the NPU while validation accuracy is resolved with NXP)*
105
  - **NXP Ara240** β€” 40 eTOPS, .DVM
106
  - **RPi5 + Hailo-8L** β€” 13 TOPS, HailoRT HEF
107
  - **NVIDIA Jetson Orin** β€” 67–157 TOPS, TensorRT
 
116
 
117
  ### EdgeFirst Profiler
118
 
119
+ The on-target measurement engine. Given a model and a dataset it runs the full pipeline on the target device β€” capture, preprocess, inference, postprocess β€” and **computes the accuracy there too**: COCO detection and segmentation metrics, the deployment confusion breakdown, and every timing block are calculated in-process, on the same machine that ran the model. There is no Python and no `pycocotools` on the device. It writes `metrics.yaml`, per-image predictions in EdgeFirst Arrow/Parquet, and a Perfetto trace.
120
 
121
+ Each runtime loads through its native delegate β€” VX Delegate on NXP i.MX 8M Plus, eIQ Neutron on NXP i.MX 95, NXP Ara SDK on Ara240, HailoRT on RPi5 + Hailo, TensorRT on Jetson, QNN on Qualcomm Hexagon, CoreML on Apple β€” so the timing reflects deployed-application reality rather than a benchmark harness.
122
 
123
+ ### EdgeFirst Studio
124
+
125
+ Where results are published, compared and browsed β€” not where they are computed. The profiler publishes its metrics, predictions, charts and trace to a `v-XXXX` validation session, and that session is what every figure in this Model Zoo cites. A run does not need Studio at all: point the profiler at a local model, a directory of images and a ground-truth file and it writes the same outputs to disk with nothing uploaded.
126
 
127
  ### EdgeFirst HAL
128
 
 
130
 
131
  ### Latency and pipelined throughput
132
 
133
+ Per-frame latency is **work-time** β€” the sum of that frame's own stage durations β€” not wall-clock from capture to result. In an offline batch the capture workers race ahead of the bottleneck, so a wall-clock sojourn balloons with queue backlog and overstates per-image cost.
134
 
135
+ Throughput is measured separately, from the rate at which finished results actually emerge. The Model Zoo headlines **realized FPS** (`realized_fps_scalar`), the measured steady-state rate, not the derived `1000 / (preprocess + inference + postprocess)`. The two disagree because the runtime overlaps stages across frames.
 
136
 
137
+ Example: YOLOv5 Nano on NXP i.MX 95 eIQ Neutron, INT8, smart decoder, on an NXP FRDM-IMX95 board β€” per-stage means of 12.8 + 12.9 + 18.0 ms sum to 43.7 ms, which divides out to roughly 23 FPS. Measured throughput is **55.2 FPS** ([`v-8ad`](https://edgefirst.studio/public/validation/v-8ad/details?mode=charts)), and the run is bound by postprocess rather than by the NPU.
138
 
139
  ---
140