Card: link the idealens package and GitHub repo; full card
Browse files
README.md
CHANGED
|
@@ -1,32 +1,71 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
extra_gated_prompt: "Access is granted individually. Please say who you are and what you intend to use the weights for."
|
| 5 |
---
|
| 6 |
|
| 7 |
-
#
|
| 8 |
|
| 9 |
-
|
| 10 |
-
`text-embedding-3-large` vectors (3072 dimensions). It answers **whose ideas a
|
| 11 |
-
document contains**, not who typed the sentences.
|
| 12 |
|
| 13 |
-
|
| 14 |
-
-
|
| 15 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
|
| 17 |
-
##
|
| 18 |
|
| 19 |
-
|
| 20 |
-
AI is the positive class, so the reported score is `predict_proba(x)[:, 0]` = P(human), and
|
| 21 |
-
the detector fires when that falls below a calibrated threshold. Thresholds are quantiles
|
| 22 |
-
over held-out human documents and are **not** included here: a cut from one model or input
|
| 23 |
-
form is meaningless against another's scores.
|
| 24 |
|
| 25 |
-
|
|
|
|
|
|
|
|
|
|
| 26 |
|
| 27 |
-
|
| 28 |
-
predictions. The refit was checked against those published test predictions and reproduces
|
| 29 |
-
them to **max |difference| = 2.06e-13**, so it is the same
|
| 30 |
-
model that produced the reported numbers rather than an approximation.
|
| 31 |
|
| 32 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
tags:
|
| 6 |
+
- ai-text-detection
|
| 7 |
+
- idea-provenance
|
| 8 |
+
datasets:
|
| 9 |
+
- rishanthrajendhran/WildOutlines
|
| 10 |
extra_gated_prompt: "Access is granted individually. Please say who you are and what you intend to use the weights for."
|
| 11 |
---
|
| 12 |
|
| 13 |
+
# IdeaLens-LogisticClassifier
|
| 14 |
|
| 15 |
+
IdeaLens-LogisticClassifier is an idea-level detector: it judges **whose ideas a document contains**, not who wrote its words, so a document whose ideas are a person's counts as human however much of its prose an AI wrote. It is one of the detectors released with [IdeaLens](https://huggingface.co/rishanthrajendhran/IdeaLens) and trained on the same data.
|
|
|
|
|
|
|
| 16 |
|
| 17 |
+
| | |
|
| 18 |
+
|---|---|
|
| 19 |
+
| Model | logistic regression over OpenAI `text-embedding-3-large` embeddings |
|
| 20 |
+
| Reads | the whole role-labelled outline, embedded as one text |
|
| 21 |
+
| Training data | [WildOutlines](https://huggingface.co/datasets/rishanthrajendhran/WildOutlines), train split |
|
| 22 |
+
| Output | P(human); a document is flagged as AI when P(human) is below a cut |
|
| 23 |
+
| Default cut | 0.05954 (global, 1% false-positive rate) |
|
| 24 |
+
| Hardware | CPU only; embedding calls need `OPENAI_API_KEY` |
|
| 25 |
|
| 26 |
+
## Usage
|
| 27 |
|
| 28 |
+
The [idealens](https://github.com/RishanthRajendhran/IdeaLens) package ([PyPI](https://pypi.org/project/idealens/)) runs the whole pipeline: it assigns each document one of the eight formats, extracts the outline with the prompt, role vocabulary and worked examples the detectors were trained with, and scores it with this model and the thresholds in this repo.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
|
| 30 |
+
```bash
|
| 31 |
+
pip install "idealens[openai]"
|
| 32 |
+
idealens run docs.jsonl -o scores.jsonl --model IdeaLens-LogisticClassifier
|
| 33 |
+
```
|
| 34 |
|
| 35 |
+
Input is JSONL with a `text` field per document. To score outlines you already have, use `idealens score outlines.jsonl -o scores.jsonl --model IdeaLens-LogisticClassifier`. Score outlines as extracted; the paraphrasing step is only for training data.
|
|
|
|
|
|
|
|
|
|
| 36 |
|
| 37 |
+
In Python:
|
| 38 |
+
|
| 39 |
+
```python
|
| 40 |
+
import idealens as il
|
| 41 |
+
|
| 42 |
+
with il.Detector("IdeaLens-LogisticClassifier") as det:
|
| 43 |
+
records = il.run(texts, det) # classify, extract and score
|
| 44 |
+
print(records[0]["p_human"], records[0]["verdict"]["ai"])
|
| 45 |
+
```
|
| 46 |
+
|
| 47 |
+
## Thresholds
|
| 48 |
+
|
| 49 |
+
`thresholds.json` holds this model's cuts at 0.1%, 0.5%, 1%, 2% and 5% false-positive rates, fitted on the 80,000 human
|
| 50 |
+
documents of WildOutlines' `calibration` split: one global cut per rate, plus per-format and per-topic cuts. The package applies them. A cut
|
| 51 |
+
fitted for one model does not transfer to another model's scores. For documents unlike English web text, fit cuts on
|
| 52 |
+
human documents from your own domain with `idealens calibrate`.
|
| 53 |
+
|
| 54 |
+
## Related
|
| 55 |
+
|
| 56 |
+
- [IdeaLens](https://huggingface.co/rishanthrajendhran/IdeaLens): the main idea-level detector, with full documentation
|
| 57 |
+
- [ProseLens](https://huggingface.co/rishanthrajendhran/ProseLens): its prose-level counterpart
|
| 58 |
+
- [idealens](https://github.com/RishanthRajendhran/IdeaLens): the Python package that runs these models
|
| 59 |
+
- [WildOutlines](https://huggingface.co/datasets/rishanthrajendhran/WildOutlines): the training corpus
|
| 60 |
+
|
| 61 |
+
## Citation
|
| 62 |
+
|
| 63 |
+
```bibtex
|
| 64 |
+
@article{idealens2026,
|
| 65 |
+
title = {IdeaLens: Detecting AI Ideas in Long-form Writing},
|
| 66 |
+
author = {Anonymous},
|
| 67 |
+
journal = {arXiv preprint arXiv:TBD},
|
| 68 |
+
year = {2026},
|
| 69 |
+
url = {https://arxiv.org/abs/TBD}
|
| 70 |
+
}
|
| 71 |
+
```
|