m4xi/texo-unimer-replication

Texo — UniMER replication (step 1750 checkpoint)

Full fine-tune of PP-FormulaNet-S on m4xi/unimer-merged for math formula recognition (image → LaTeX).

Base model

PP-FormulaNet-S (formulanet.pt from alephpi/FormulaNet): 58M parameter model with HGNetV2 encoder + MBart decoder. This is the distilled checkpoint from the original Texo paper, prior to any UniMER training.

Training

Dataset	m4xi/unimer-merged (~1.04M train samples, 98/2 train/val split)
Fine-tune strategy	Full (no LoRA)
Effective batch size	64 (16 per device × 4 grad accum)
Learning rate	1e-4, cosine decay
Precision	bf16
Steps	1,750 (~0.11 epochs, ~112K samples seen)
Hardware	NVIDIA RTX 4090, ~42h total runtime (crashed)

Checkpoint selection

The run crashed after ~70k steps. Validation BLEU peaked at step 1750 (BLEU=0.7247, edit_distance=0.0582) and degraded monotonically after that. save_top_k=5 retained only the five best-BLEU checkpoints, all of which fell within the first ~15% of epoch 1; consistent with rapid convergence from a strong pretrained base.

Usage

import torch
from task import FormulaNetLit  # Texo src/task.py

model = FormulaNetLit.load_from_checkpoint("texo-cp-1750.ckpt", map_location="cpu")
model.eval()

outputs = model.generate(pixel_values, num_beams=1, do_sample=False, max_new_tokens=512)
pred = model.tokenizer.batch_decode(outputs, skip_special_tokens=True)