HOWC : Hierarchical Open-World Classification for road perception

A training-free layer that turns a flat object detector into a hierarchical, open-set one. Every detection is resolved by taxonomic abstraction: the most specific level the evidence safely supports, or an explicit UNKNOWN OBSTACLE, never a confident wrong leaf.

_{The live tool. A deer, a class no closed detector is asked to output on the road, is not dropped and not mislabeled: it resolves to Living Being, with the full taxonomy descent path and an independent segmentation cross-check shown alongside.}

Watch the 3-minute explainer

---

The problem

A closed-set detector must assign every object one of a fixed set of labels. On an object outside that set (a horse-drawn carriage, road debris, livestock on a rural road) it can only do one of two unsafe things: force a confident but wrong specific label, or drop the object entirely. Real incidents trace back to exactly this failure.

How it works

HOWC puts a taxonomy over the detector and, per object, aggregates leaf probability mass up the tree. It commits only as deep as one branch holds enough mass; otherwise it stops at a coarser node. A per-branch safety floor (Vehicle, Living Being, Static Object) means it never collapses into a useless bare Object: below the floor, it emits an explicit UNKNOWN OBSTACLE with a fully inspectable decision path.

How it decides (worked examples)

Each detection produces a decision path: probability mass per node, the safety floor, and the resulting action.

ABSTRACTED. YOLO calls a distant truck "car" (0.33). The mass is ambiguous below the Vehicle floor, so HOWC does not guess a leaf: it reports Vehicle, still useful, never wrong.

UNKNOWN. A rural-road anomaly YOLO calls "person" (0.72). The mass splits above the floor across Living Being / Vehicle / Static, so HOWC refuses a category and flags UNKNOWN OBSTACLE.

Beyond 2D: a depth cue for flat detections

A 2D box cannot tell a real vehicle from a flat picture of one (a billboard, a truck livery). The open-world study adds a monocular-depth relief cue: detections that are geometrically flat get flagged, and the depth field feeds the open-world proposal front-end.

_{Left: 3D mode flags flat detections. Right: the monocular depth field (nearer = bright). This is a feasibility signal, not a solved module, and the paper is explicit that no single 2D cue suffices.}

Results (honest)

Out-of-vocabulary objects (v3 ground-truth leave-classes-out on COCO, n=235). Hold seven classes out of the taxonomy, then classify their real annotated crops:

On out-of-vocabulary objects	Flat / closed head	HOWC
Confident wrong specific label	100%	0%
... of which in the wrong super-category	37%	0%
Safely handled (right super-cat, or honest UNKNOWN)	0%	94%
Correct super-category recovered	—	26%
Conservatively flagged UNKNOWN	—	69%

On known, in-taxonomy objects: 0% off-branch (categorical) errors with ~24% calibrated abstention, versus a flat arg-max head's ~53% off-branch errors.

The safety/specificity trade-off is a dial, not a fixed point. Every hierarchical configuration stays 100% safe on novel objects while still usefully classifying up to 76% of known ones; the flat baseline sits alone at 0% novel-safe.

What this is, and is not. This is a safety result, not a specificity one. On closed-set COCO mAP a trained YOLO is more accurate, and we say so. The contribution is the abstraction layer over open-vocabulary (CLIP) features: it never makes a confident categorical mistake on an unknown object, at the honest cost of 69% abstention. The point is the tail, not the leaderboard.

An independent second path

The box classification is cross-checked against a class-agnostic segmentation path: does the pixel evidence back up the box verdict? Across sampled detections it confirms or stays neutral 63% of the time and flags 37% for review, with 0% hard conflicts, and it agrees most where it should (on abstracted and unknown objects).

Run it locally

Self-contained: code + taxonomy + a Gradio app. Training-free (pretrained YOLO + CLIP zero-shot); first run downloads weights (~360 MB).

pip install -r requirements.txt
python app.py            # Gradio UI: upload an image, see the taxonomy decision path
python app.py --share    # same, but also prints a temporary public URL (~72h)

Papers

• Open-world (v3): Open-World Hierarchical Perception: Taxonomic Abstraction over Class-Agnostic Proposals for the Safe Handling of Out-of-Vocabulary Road Objects, F. Schaller, arXiv:2608.07577.
• Foundational (v1): Hierarchical Taxonomic Abstraction for the Safe Handling of Novel Objects in Autonomous Driving Perception, doi:10.5281/zenodo.21593472.
• Source & full history: https://github.com/freshNfunky/IE2025-Research-Paper

@misc{schaller2026howc,
  title         = {Open-World Hierarchical Perception: Taxonomic Abstraction over
                   Class-Agnostic Proposals for the Safe Handling of
                   Out-of-Vocabulary Road Objects},
  author        = {Schaller, Felix},
  year          = {2026},
  eprint        = {2608.07577},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2608.07577}
}

License

CC BY-NC 4.0, matching the paper.