File size: 2,990 Bytes
63519e7 46ccbfa 63519e7 46ccbfa 63519e7 46ccbfa 63519e7 46ccbfa 63519e7 46ccbfa 63519e7 46ccbfa 63519e7 eee2a88 63519e7 46ccbfa | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 | ---
language: en
license: mit
library_name: pytorch
base_model: FacebookAI/roberta-base
pipeline_tag: question-answering
tags:
- extractive-qa
- grounded-qa
- hallucination-free
- selective-prediction
- abstention
- rag
datasets:
- rajpurkar/squad_v2
metrics:
- exact_match
model-index:
- name: grounded-pointer-qa
results:
- task:
type: question-answering
dataset:
name: SQuAD v2 (held-out test half, retrieval setting)
type: rajpurkar/squad_v2
metrics:
- type: exact_match
value: 74.6
name: EM (EM-optimal gate)
- type: precision
value: 91.7
name: Answered precision at 90%-precision gate (9% coverage)
---
# Grounded Pointer QA
An extractive question-answering model that **cannot hallucinate by
construction**: its output layer can only point at spans inside retrieved
passages of your documents. It has no vocabulary to generate from. A trained
abstention head refuses when the loaded knowledge does not contain the answer,
decoding is deterministic (argmax, so the same question over the same documents
gives the same answer every time), and knowledge is **hot-swappable**: point it
at a new folder of text files and it answers from those, with no retraining.
Built on `roberta-base` (125M params) with pointer and abstention heads,
finetuned on SQuAD v2 against real TF-IDF retrieval. The model never saw gold
passages during training, only what the retriever actually returned, so it
learned to abstain on retrieval misses too.
## Operating modes
The checkpoint ships with a calibrated confidence gate
(`P(answerable) x P(span)`), selected on a calibration split and verified on a
disjoint held-out test split:
| Mode | Gate | Coverage | Answered precision | EM |
|---|---|---|---|---|
| "Right or silent" (shipped default) | 0.965 | 9% | 91.7% | 57.1 |
| EM-optimal | 0.295 | 45% | 72.3% | 74.6 |
Pass a lower `gate` to `ask()` for more coverage at lower precision.
## Usage
```python
# files needed: proqa.pt, modeling_proqa.py (both in this repo)
# pip install torch transformers scikit-learn numpy (plus pypdf for PDFs)
from modeling_proqa import GroundedQA
qa = GroundedQA("proqa.pt")
qa.load_folder("path/to/your/notes") # .txt / .md / .pdf
qa.ask("when does the vendor contract expire?")
# {'answer': '30 November 2026', 'source': '...expires on 30 November 2026...',
# 'confidence': 0.98}
qa.ask("what is the capital of France?") # not in your docs
# {'answer': None, 'source': None, 'confidence': 0.0}
```
## Roadmap
Today the model answers one self-contained question at a time. A conversational
grounding layer (multi-turn context and follow-up questions, with every answer
still a verified quote) is coming.
## Training
One NVIDIA RTX 5060 Ti (16 GB): about 2.5 h finetune (batch 8 x 4 passages x
384 tokens, bf16, lr 2e-5, 2 epochs) plus a calibration pass.
|