Hugging Face's logo

institutional
/
institutional-books-visual-elements-detection

Object Detection
ultralytics
YOLOv26
yolo
document-analysis
visual-elements
cultural-heritage
digitized-books
Eval Results (legacy)
Model card Files
xet
 Community

You need to agree to share your contact information to access this model

Terms of Use

This model is shared by the Institutional Data Initiative for research and public-interest use (the “Service”). These terms are intended to support experimentation while encouraging collaboration and feedback as we refine the model and work with contributing institutions to define shared, long-term norms for open data reuse. To share questions or feedback, contact us at contact@institutional.org.

By accessing or downloading the model or otherwise using the Service, you agree to the following:

  1. Noncommercial Use Only
    You may use the Service solely for noncommercial purposes. Open-source projects and other public-use efforts are welcome, even if they may indirectly support commercial use, so long as they are unaffiliated with commercial actors or intent.

    If you are affiliated with a commercial organization or plan to use the Service for commercial purposes (including AI model training), you will contact us first at contact@institutional.org.

  2. No Redistribution
    You may not share or redistribute the Service or any of the data provided through the Service, in whole or in part, including through public repositories or aggregators. If you want others to access it, please direct them to the attribution link.

  3. Derivative Works
    You may create derivative works for noncommercial use, but you may not make available any such derivative works that substantially reproduce the original model. Only outputs that are significantly transformed and cannot substitute for the original—such as evaluations, summary statistics, or visualizations—may be shared, with attribution.

  4. Attribution
    If you use the model in public-facing work, you must include attribution substantially similar to:

    Institutional Books provided by the Institutional Data Initiative with source material contributed by Harvard Library, available at https://institutional.org.

    Minor modifications to fit citation style or formatting are permitted, provided the essential elements remain intact.

  5. Provisional Terms
    These terms apply only to this release and may change. We are actively working with contributing institutions to develop a long-term framework for responsible, open data sharing.

  6. DISCLAIMER OF WARRANTY
    TO THE FULLEST EXTENT PERMITTED BY APPLICABLE LAW, ACCESS TO “SERVICE IS PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND (EXPRESS, IMPLIED, OR OTHERWISE), INCLUDING, WITHOUT LIMITATION, ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NONINFRINGEMENT. WE DO NOT WARRANT THAT THE SERVICE WILL OPERATE IN AN UNINTERRUPTED OR ERROR-FREE MANNER OR THAT THE SERVICE IS FREE OF VIRUSES OR OTHER HARMFUL COMPONENTS. WITHOUT LIMITING THE FOREGOING, WE DO NOT WARRANT THAT (A) THE SERVICE WILL MEET YOUR REQUIREMENTS OR EXPECTATIONS OR ACHIEVE THE INTENDED PURPOSES; (B) THE SERVICE WILL NOT EXPERIENCE OUTAGES OR OTHERWISE WILL BE UNINTERRUPTED, TIMELY, OR SECURE; (C) THE INFORMATION OR SERVICES OBTAINED THROUGH OR FROM THE SERVICE WILL BE ACCURATE, COMPLETE, CURRENT, ERROR-FREE, OR RELIABLE; (D) ANY DEFECTS IN OR ON THE SERVICE WILL BE CORRECTED; OR (E) THAT ANY POLLS OR OTHER SOLICITATIONS OF INFORMATION POSTED THROUGH THE SERVICE BY YOU OR OTHER USERS ARE SAFE OR APPROPRIATE FOR YOUR OR OTHER USERS’ PARTICIPATION. WE MAKE NO REPRESENTATION OR WARRANTY REGARDING YOUR ABILITY TO TRANSMIT AND RECEIVE INFORMATION FROM OR THROUGH THE SERVICE, AND YOU AGREE AND ACKNOWLEDGE THAT YOUR ABILITY TO ACCESS THE SERVICE MAY BE IMPAIRED.

  7. LIMITATION OF LIABILITY
    EXCEPT INSOFAR AS THE FOLLOWING LIMITATION MAY BE PROHIBITED BY APPLICABLE LAW, WE SHALL NOT BE LIABLE TO YOU OR TO ANY THIRD PARTY FOR ANY DIRECT, CONSEQUENTIAL, INDIRECT, PUNITIVE, SPECIAL, OR INCIDENTAL DAMAGES, WHETHER FORESEEABLE OR UNFORESEEABLE (INCLUDING, BUT NOT LIMITED TO, LOSS OF PROFITS OR EARNING POWER, LOSS OF DATA, LOSSES DUE TO ERRORS OR INTERRUPTION IN AVAILABILITY OF THE SERVICE, UNAVAILABILITY OF ANY SERVICE, SERVER, OR COMMUNICATIONS FACILITY, OR DAMAGES DUE TO ACTS OR OMISSIONS OF OTHERS USING THE SERVICE), ARISING OUT OF OR RELATING TO THE SERVICE, INCLUDING WITHOUT LIMITATION YOUR AND OTHERS’ USE OF OR INABILITY TO USE THE SERVICE, OR YOUR RELIANCE UPON INFORMATION OBTAINED FROM OR THROUGH THE SERVICE, WHETHER BASED IN CONTRACT, TORT, STATUTORY, OR OTHER LAW. OUR TOTAL CUMULATIVE LIABILITY TO YOU ARISING OUT OF OR RELATED TO THE SERVICE (INCLUDING, WITHOUT LIMITATION, IN THE WAYS DESCRIBED IN THE PRECEDING SENTENCE), WHETHER BASED IN CONTRACT, TORT, STATUTORY, OR OTHER LAW, WILL NOT EXCEED THE AMOUNT, IF ANY, THAT YOU PAID US TO USE THE SERVICE IN THE TWELVE MONTHS PRECEDING THE CLAIM. THE DISCLAIMERS AND LIMITATIONS SET FORTH IN THIS SECTION SHALL APPLY, TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, WHETHER OR NOT WE HAVE BEEN NEGLIGENT OR OTHERWISE AT FAULT.
    YOU ACKNOWLEDGE THAT, FOR PURPOSES OF THE FOREGOING DISCLAIMERS AND LIMITATIONS, AS WELL AS THE INDEMNITY PROVISION IN SECTION 8 BELOW, THE TERMS “WE,” “OUR,” “US,” “INSTITUTIONAL DATA INITIATIVE,” AND “IDI” INCLUDE THE CORPORATE BODY PRESIDENT AND FELLOWS OF HARVARD COLLEGE, ALSO KNOWN AS HARVARD UNIVERSITY, AND ITS VARIOUS SCHOOLS, THE MEMBERS OF ITS GOVERNING BOARDS, AND ITS OFFICERS, FACULTY MEMBERS, EMPLOYEES, FELLOWS, AND TO THE EXTENT WORKING ON IDI, ITS STUDENTS, CONTRACTORS, AND REPRESENTATIVES.

  8. Indemnification
    You agree to indemnify us and hold us harmless from any and all claims, liabilities, damages, losses and expenses, including reasonable attorneys’ fees and costs, relating to or arising out of (a) your use or attempted use of the Service in violation of these Terms of Service; or (b) your violation of any law or rights of any third party in connection with your use of the Service.

  9. Governing Law/ Jurisdiction
    You agree that the Terms of Service and any claim or dispute arising out of or relating to the Service or Terms of Service will be governed by the laws of the Commonwealth of Massachusetts, excluding its conflicts of laws principles. You agree that all such claims and disputes will be heard and resolved exclusively in the federal or state courts located in and serving Middlesex or Suffolk County, Massachusetts, U.S.A. You consent to the personal jurisdiction of those courts over you for this purpose, and you waive and agree not to assert any objection to such proceedings in those courts (including any defense or objection of lack of proper jurisdiction or inconvenience of forum).

  10. Whole Agreement/ Amendment
    These Terms of Service constitute the entire agreement between you and Harvard with respect to your use of the Service. We reserve the right to amend these Terms of Service at any time. The Service will post notice of changes to the terms on this webpage, and by accessing the Service after modifications to these Terms of Service have been posted, you agree to be bound by all the modified terms. Accordingly, you should periodically revisit this page to review the then-current Terms of Service.

Log in or Sign Up to review the conditions and access this model content.

📚 Institutional Books — Visual Elements Detection Model (YOLOv26n)

A single-class object detection model trained to localize visual elements (illustrations, photographs, charts, maps, decorative elements, etc.) in digitized book page scans. This model is the detection component of the Institutional Books Visual Elements Pipeline.

More information:

See also:

The Institutional Data Initiative at Harvard Law School Library works with knowledge institutions—from libraries and museums to cultural groups and government agencies—to refine and publish their collections as data. Reach out to collaborate on your collections.

Outline

Model Description

This model performs binary detection (visual element vs. background) on page scans from digitized book collections. It outputs bounding boxes for all visual elements on a page, regardless of their type. Classification into specific categories (Image/Illustration, Chart/Graph, Music, Ex Libris/Decorative, Artifact) is handled by a separate downstream model.

By decoupling localization from categorization, each task is optimized independently. The detection model focuses exclusively on identifying visual regions, while fine-grained classification is delegated to a separate classification model.

  • Architecture: YOLOv26n (nano variant)
  • Parameters: 2.4 million
  • Classes: 1 (visual element)
  • Framework: Ultralytics

Intended Use

This model is designed to detect visual elements in page scans from large-scale digitized book collections. It was developed as part of a pipeline to extract, classify, and caption visual content from the Institutional Books collection.

Primary use cases:

  • Locating illustrations, photographs, maps, charts, and decorative elements in digitized historical documents
  • Preprocessing step for downstream classification, captioning, and embedding generation
  • Large-scale visual element extraction from library and archive collections

Out of scope:

  • General-purpose object detection
  • Detection in born-digital documents (PDFs, web pages)
  • Scene or natural image detection

Training

Dataset

The training set comprises 4,528 page images sourced from 271 distinct volumes in the Institutional Books collection, containing 6,103 bounding box annotations. Annotations are type-agnostic (single class: visual element).

  • Splits: 80% train / 10% validation / 10% test
  • Annotation process: Iterative — initial manual annotation of 109 samples, followed by model-assisted pre-annotation of additional batches, all manually reviewed and corrected.

Training Configuration

Parameter Value
Image size (training) 800×800 px
Batch size 30 per GPU
Max epochs 400
Early stopping patience 50
Best checkpoint epoch 200
Box loss weight 8.3
DFL loss 1.5
Scale augmentation 0.7
Hardware 4× NVIDIA L40S
Training time 1.4 hours

The elevated box loss weight and DFL loss prioritize precise localization over classification confidence, reflecting the single-class detection approach. The scale augmentation factor introduces size variations to improve robustness across scans of varying resolutions.

Evaluation

Evaluated on the held-out test split (10% of the training dataset):

Metric Value
Test pages 453
Test annotations 658
Precision 0.95
Recall 0.96

Inference Configuration

The following settings are used in production inference:

Parameter Value Rationale
Image size 640×640 px Increased throughput with limited impact on accuracy
Confidence threshold 0.3 Conservative; maximizes recall, false positives pruned downstream. Alternative recommendation is 0.75.
NMS IoU threshold 0.2 Retains distinct but spatially close elements in densely illustrated pages

Usage

With Ultralytics 

from ultralytics import YOLO

model = YOLO("path/to/best.pt")

results = model.predict(
    "page_scan.jpg",
    imgsz=640,
    conf=0.3,
    iou=0.2,
)

for result in results:
    for box in result.boxes:
        print(f"bbox: {box.xyxy.tolist()[0]}, conf: {float(box.conf):.3f}")

With HuggingFace Hub 

from huggingface_hub import hf_hub_download
from ultralytics import YOLO

model_path = hf_hub_download(
    repo_id="institutional/institutional-books-visual-elements-detection",
    filename="weights/best.pt",
)

model = YOLO(model_path)
results = model.predict("page_scan.jpg", imgsz=640, conf=0.3, iou=0.2)

Model Files

File Description
weights/best.pt Best checkpoint (epoch 200) — use this for inference
weights/last.pt Final training checkpoint
weights/best.torchscript TorchScript export of best checkpoint

Limitations

  • Trained exclusively on public domain books. Performance on other digitized collections may vary.
  • Optimized for sparse documents (most pages contain 0–2 visual elements). Performance on extremely dense layouts has not been separately benchmarked.
  • Low NMS IoU threshold (0.2) may merge overlapping detections in rare cases where distinct elements share significant spatial overlap.

Terms of Use

This model is shared by the Institutional Data Initiative for research and public-interest use (the “Service”). These terms are intended to support experimentation while encouraging collaboration and feedback as we refine the model and work with contributing institutions to define shared, long-term norms for open data reuse. To share questions or feedback, contact us at contact@institutional.org.

By accessing or downloading the model or otherwise using the Service, you agree to the following:

  1. Noncommercial Use Only
    You may use the Service solely for noncommercial purposes. Open-source projects and other public-use efforts are welcome, even if they may indirectly support commercial use, so long as they are unaffiliated with commercial actors or intent.

    If you are affiliated with a commercial organization or plan to use the Service for commercial purposes (including AI model training), you will contact us first at contact@institutional.org.

  2. No Redistribution
    You may not share or redistribute the Service or any of the data provided through the Service, in whole or in part, including through public repositories or aggregators. If you want others to access it, please direct them to the attribution link.

  3. Derivative Works
    You may create derivative works for noncommercial use, but you may not make available any such derivative works that substantially reproduce the original model. Only outputs that are significantly transformed and cannot substitute for the original—such as evaluations, summary statistics, or visualizations—may be shared, with attribution.

  4. Attribution
    If you use the model in public-facing work, you must include attribution substantially similar to:

    Institutional Books provided by the Institutional Data Initiative with source material contributed by Harvard Library, available at https://institutional.org.

    Minor modifications to fit citation style or formatting are permitted, provided the essential elements remain intact.

  5. Provisional Terms
    These terms apply only to this release and may change. We are actively working with contributing institutions to develop a long-term framework for responsible, open data sharing.

  6. DISCLAIMER OF WARRANTY
    TO THE FULLEST EXTENT PERMITTED BY APPLICABLE LAW, ACCESS TO “SERVICE IS PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND (EXPRESS, IMPLIED, OR OTHERWISE), INCLUDING, WITHOUT LIMITATION, ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NONINFRINGEMENT. WE DO NOT WARRANT THAT THE SERVICE WILL OPERATE IN AN UNINTERRUPTED OR ERROR-FREE MANNER OR THAT THE SERVICE IS FREE OF VIRUSES OR OTHER HARMFUL COMPONENTS. WITHOUT LIMITING THE FOREGOING, WE DO NOT WARRANT THAT (A) THE SERVICE WILL MEET YOUR REQUIREMENTS OR EXPECTATIONS OR ACHIEVE THE INTENDED PURPOSES; (B) THE SERVICE WILL NOT EXPERIENCE OUTAGES OR OTHERWISE WILL BE UNINTERRUPTED, TIMELY, OR SECURE; (C) THE INFORMATION OR SERVICES OBTAINED THROUGH OR FROM THE SERVICE WILL BE ACCURATE, COMPLETE, CURRENT, ERROR-FREE, OR RELIABLE; (D) ANY DEFECTS IN OR ON THE SERVICE WILL BE CORRECTED; OR (E) THAT ANY POLLS OR OTHER SOLICITATIONS OF INFORMATION POSTED THROUGH THE SERVICE BY YOU OR OTHER USERS ARE SAFE OR APPROPRIATE FOR YOUR OR OTHER USERS’ PARTICIPATION. WE MAKE NO REPRESENTATION OR WARRANTY REGARDING YOUR ABILITY TO TRANSMIT AND RECEIVE INFORMATION FROM OR THROUGH THE SERVICE, AND YOU AGREE AND ACKNOWLEDGE THAT YOUR ABILITY TO ACCESS THE SERVICE MAY BE IMPAIRED.

  7. LIMITATION OF LIABILITY
    EXCEPT INSOFAR AS THE FOLLOWING LIMITATION MAY BE PROHIBITED BY APPLICABLE LAW, WE SHALL NOT BE LIABLE TO YOU OR TO ANY THIRD PARTY FOR ANY DIRECT, CONSEQUENTIAL, INDIRECT, PUNITIVE, SPECIAL, OR INCIDENTAL DAMAGES, WHETHER FORESEEABLE OR UNFORESEEABLE (INCLUDING, BUT NOT LIMITED TO, LOSS OF PROFITS OR EARNING POWER, LOSS OF DATA, LOSSES DUE TO ERRORS OR INTERRUPTION IN AVAILABILITY OF THE SERVICE, UNAVAILABILITY OF ANY SERVICE, SERVER, OR COMMUNICATIONS FACILITY, OR DAMAGES DUE TO ACTS OR OMISSIONS OF OTHERS USING THE SERVICE), ARISING OUT OF OR RELATING TO THE SERVICE, INCLUDING WITHOUT LIMITATION YOUR AND OTHERS’ USE OF OR INABILITY TO USE THE SERVICE, OR YOUR RELIANCE UPON INFORMATION OBTAINED FROM OR THROUGH THE SERVICE, WHETHER BASED IN CONTRACT, TORT, STATUTORY, OR OTHER LAW. OUR TOTAL CUMULATIVE LIABILITY TO YOU ARISING OUT OF OR RELATED TO THE SERVICE (INCLUDING, WITHOUT LIMITATION, IN THE WAYS DESCRIBED IN THE PRECEDING SENTENCE), WHETHER BASED IN CONTRACT, TORT, STATUTORY, OR OTHER LAW, WILL NOT EXCEED THE AMOUNT, IF ANY, THAT YOU PAID US TO USE THE SERVICE IN THE TWELVE MONTHS PRECEDING THE CLAIM. THE DISCLAIMERS AND LIMITATIONS SET FORTH IN THIS SECTION SHALL APPLY, TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, WHETHER OR NOT WE HAVE BEEN NEGLIGENT OR OTHERWISE AT FAULT.
    YOU ACKNOWLEDGE THAT, FOR PURPOSES OF THE FOREGOING DISCLAIMERS AND LIMITATIONS, AS WELL AS THE INDEMNITY PROVISION IN SECTION 8 BELOW, THE TERMS “WE,” “OUR,” “US,” “INSTITUTIONAL DATA INITIATIVE,” AND “IDI” INCLUDE THE CORPORATE BODY PRESIDENT AND FELLOWS OF HARVARD COLLEGE, ALSO KNOWN AS HARVARD UNIVERSITY, AND ITS VARIOUS SCHOOLS, THE MEMBERS OF ITS GOVERNING BOARDS, AND ITS OFFICERS, FACULTY MEMBERS, EMPLOYEES, FELLOWS, AND TO THE EXTENT WORKING ON IDI, ITS STUDENTS, CONTRACTORS, AND REPRESENTATIVES.

  8. Indemnification
    You agree to indemnify us and hold us harmless from any and all claims, liabilities, damages, losses and expenses, including reasonable attorneys’ fees and costs, relating to or arising out of (a) your use or attempted use of the Service in violation of these Terms of Service; or (b) your violation of any law or rights of any third party in connection with your use of the Service.

  9. Governing Law/ Jurisdiction
    You agree that the Terms of Service and any claim or dispute arising out of or relating to the Service or Terms of Service will be governed by the laws of the Commonwealth of Massachusetts, excluding its conflicts of laws principles. You agree that all such claims and disputes will be heard and resolved exclusively in the federal or state courts located in and serving Middlesex or Suffolk County, Massachusetts, U.S.A. You consent to the personal jurisdiction of those courts over you for this purpose, and you waive and agree not to assert any objection to such proceedings in those courts (including any defense or objection of lack of proper jurisdiction or inconvenience of forum).

  10. Whole Agreement/ Amendment
    These Terms of Service constitute the entire agreement between you and Harvard with respect to your use of the Service. We reserve the right to amend these Terms of Service at any time. The Service will post notice of changes to the terms on this webpage, and by accessing the Service after modifications to these Terms of Service have been posted, you agree to be bound by all the modified terms. Accordingly, you should periodically revisit this page to review the then-current Terms of Service.

Citation 

@misc{mendez2026institutionalbooksvisual,
      title={Institutional Books - Visual Elements: An open-source pipeline for extracting, classifying, deduplicating, and captioning visual elements from digital book collections}, 
      author={Jimmy Mendez and Matteo Cargnelutti and David Lowry-Duda and Catherine Brobston and Salwa Ismail and Greg Leppert and Amanda Watson and Jonathan Zittrain},
      year={2026},
      eprint={2608.18957},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2608.18957}, 
}
Downloads last month
25
Inference Providers NEW
Object Detection
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Dataset used to train institutional/institutional-books-visual-elements-detection

Collection including institutional/institutional-books-visual-elements-detection

Paper for institutional/institutional-books-visual-elements-detection

Evaluation results

  • Precision on Institutional Books Visual Elements (test split)
    self-reported
    0.950
  • Recall on Institutional Books Visual Elements (test split)
    self-reported
    0.960