| | --- |
| | license: apache-2.0 |
| | datasets: |
| | - dedoc/paragraph_dataset |
| | language: |
| | - ru |
| | - en |
| | metrics: |
| | - f1 |
| | - accuracy |
| | --- |
| | |
| | # Paragraph classifier |
| |
|
| | The classifier is used for binary classification of text lines in PDF or scanned documents. |
| |
|
| | For each document line, it determines: |
| |
|
| | * line is a beginning of a new paragraph or |
| |
|
| | * line is a continuation of the previous paragraph |
| |
|
| | For each line, feature vector is formed based on line's text and formatting, please see |
| | `dedoc/structure_extractors/feature_extractors/paragraph_feature_extractor.py` in [dedoc](https://github.com/ispras/dedoc). |
| |
|
| |
|
| | * Training data are available at [the link](https://huggingface.co/datasets/dedoc/paragraph_dataset). |
| |
|
| | * Training script is [here](https://github.com/ispras/dedoc/blob/master/scripts/train/train_paragraph_classifier.py). |
