| | --- |
| | license: mit |
| | language: |
| | - en |
| | tags: |
| | - statement-extraction |
| | - named-entity-recognition |
| | - t5 |
| | - gemma |
| | - seq2seq |
| | - nlp |
| | - information-extraction |
| | - corp-o-rate |
| | pipeline_tag: text2text-generation |
| | --- |
| | |
| | # Statement Extractor (T5-Gemma 2) |
| |
|
| | A fine-tuned T5-Gemma 2 model for extracting structured statements from text. Part of [corp-o-rate.com](https://corp-o-rate.com). |
| |
|
| | ## Model Description |
| |
|
| | This model extracts subject-predicate-object triples from unstructured text, with automatic entity type recognition and coreference resolution. |
| |
|
| | - **Architecture**: T5-Gemma 2 (270M-270M, 540M total parameters) |
| | - **Training Data**: 77,515 examples from corporate and news documents |
| | - **Final Eval Loss**: 0.209 |
| | - **Max Input Length**: 4,096 tokens |
| | - **Max Output Length**: 2,048 tokens |
| |
|
| | ## Usage |
| |
|
| | ### Python |
| |
|
| | ```python |
| | from transformers import AutoModelForSeq2SeqLM, AutoTokenizer |
| | import torch |
| | |
| | model = AutoModelForSeq2SeqLM.from_pretrained( |
| | "Corp-o-Rate-Community/statement-extractor", |
| | torch_dtype=torch.bfloat16, |
| | trust_remote_code=True, |
| | ) |
| | tokenizer = AutoTokenizer.from_pretrained( |
| | "Corp-o-Rate-Community/statement-extractor", |
| | trust_remote_code=True, |
| | ) |
| | |
| | text = "Apple Inc. announced a commitment to carbon neutrality by 2030." |
| | inputs = tokenizer(f"<page>{text}</page>", return_tensors="pt") |
| | outputs = model.generate(**inputs, max_new_tokens=2048, num_beams=4) |
| | result = tokenizer.decode(outputs[0], skip_special_tokens=True) |
| | print(result) |
| | ``` |
| |
|
| | ### Input Format |
| |
|
| | Wrap your text in `<page>` tags: |
| |
|
| | ``` |
| | <page>Your text here...</page> |
| | ``` |
| |
|
| | ### Output Format |
| |
|
| | The model outputs XML with extracted statements: |
| |
|
| | ```xml |
| | <statements> |
| | <stmt> |
| | <subject type="ORG">Apple Inc.</subject> |
| | <object type="EVENT">carbon neutrality by 2030</object> |
| | <predicate>committed to</predicate> |
| | <text>Apple Inc. committed to achieving carbon neutrality by 2030.</text> |
| | </stmt> |
| | </statements> |
| | ``` |
| |
|
| | ## Entity Types |
| |
|
| | | Type | Description | |
| | |------|-------------| |
| | | ORG | Organizations (companies, agencies) | |
| | | PERSON | People (names, titles) | |
| | | GPE | Geopolitical entities (countries, cities) | |
| | | LOC | Locations (mountains, rivers) | |
| | | PRODUCT | Products (devices, services) | |
| | | EVENT | Events (announcements, meetings) | |
| | | WORK_OF_ART | Creative works (reports, books) | |
| | | LAW | Legal documents | |
| | | DATE | Dates and time periods | |
| | | MONEY | Monetary values | |
| | | PERCENT | Percentages | |
| | | QUANTITY | Quantities and measurements | |
| |
|
| | ## Demo |
| |
|
| | Try the interactive demo at [statement-extractor.vercel.app](https://statement-extractor.vercel.app) |
| |
|
| | ## Training |
| |
|
| | - Base model: `google/t5gemma-2-270m-270m` |
| | - Training examples: 77,515 |
| | - Final eval loss: 0.209 |
| | - Training with refinement phase (LR=1e-6, epochs=0.2) |
| | - Beam search: num_beams=4 |
| | |
| | ## About corp-o-rate |
| | |
| | This model is part of [corp-o-rate.com](https://corp-o-rate.com) - an AI-powered platform for ESG analysis and corporate accountability. |
| | |
| | ## License |
| | |
| | MIT License |
| | |
