duttaprat commited on
Commit
7cd4d3a
·
verified ·
1 Parent(s): 167656c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +200 -200
README.md CHANGED
@@ -1,307 +1,307 @@
1
  ---
2
  language:
3
- - en
4
  license: apache-2.0
5
  library_name: transformers
 
6
  tags:
7
- - genomics
8
- - virology
9
- - dnabert
10
- - foundation-model
11
- - hvilm
12
- - pathogenicity
13
- - transmissibility
14
- - host-tropism
15
- - viral-genomics
 
16
  datasets:
17
- - VIRION
18
- - BV-BRC
19
- - VHDB
20
- - duttaprat/HVUE
21
  pipeline_tag: feature-extraction
22
  widget:
23
- - text: "ATGCGTACGTTAGCCGATCG"
24
- example_title: "Viral Sequence Example"
25
  ---
26
 
27
  # HViLM-base: A Foundation Model for Viral Genomics
28
 
29
  <div align="center">
30
 
31
- [![Paper](https://img.shields.io/badge/Paper-RECOMB%202026-blue)](https://github.com/duttaprat/HViLM)
32
- [![GitHub](https://img.shields.io/badge/Code-GitHub-black)](https://github.com/duttaprat/HViLM)
 
33
  [![License](https://img.shields.io/badge/License-Apache%202.0-green.svg)](LICENSE)
34
- [![Hugging Face](https://img.shields.io/badge/🤗%20Hugging%20Face-HViLM--base-yellow)](https://huggingface.co/duttaprat/HViLM-base)
35
 
36
  </div>
37
 
 
 
 
 
 
38
  ## Model Description
39
 
40
- **HViLM (Human Virome Language Model)** is the first foundation model specifically designed for comprehensive viral risk assessment through multi-task prediction of pathogenicity, host tropism, and transmissibility. Built through continued pre-training of [DNABERT-2](https://github.com/MAGICS-LAB/DNABERT_2) on 5 million viral genome sequences from the [VIRION database](https://virion.verena.org), HViLM captures universal viral genomic patterns relevant for human disease risk assessment.
 
 
41
 
42
- **Paper**: *HViLM: A Foundation Model for Viral Genomics Enables Multi-Task Prediction of Pathogenicity, Transmissibility, and Host Tropism* (RECOMB 2026)
 
43
 
44
- **Authors**: Pratik Dutta, Jack Vaska, Pallavi Surana, Rekha Sathian, Max Chao, Zhihan Zhou, Han Liu, and Ramana V. Davuluri
45
 
46
- **Code & Benchmarks**: [GitHub Repository](https://github.com/duttaprat/HViLM)
47
 
48
  ---
49
 
50
- ## Key Features
51
 
52
- - 🦠 **Viral-specialized pre-training** on 5M sequences from 10.8M genomes spanning 45+ viral families
53
- - 🎯 **Multi-task predictions** across 3 epidemiologically critical tasks:
54
- - **Pathogenicity classification**: 95.32% average accuracy
55
- - **Host tropism prediction**: 96.25% accuracy
56
- - **Transmissibility assessment**: 97.36% average accuracy
57
- - 📊 **[HVUE Benchmark](https://huggingface.co/datasets/duttaprat/HVUE)**: 7 curated datasets totaling 60K+ viral sequences
58
- - 🔍 **Mechanistic interpretability**: Identifies transcription factor binding site mimicry (42 conserved motifs)
59
- - ⚡ **Parameter-efficient fine-tuning**: LoRA adaptation (~0.3M trainable parameters per task)
60
- - 🚀 **State-of-the-art performance**: Outperforms Nucleotide Transformer, GENA-LM, and DNABERT-MB
61
 
62
- ---
 
 
 
 
 
 
63
 
64
- ## Model Architecture
65
 
66
- HViLM is built upon **DNABERT-2** (117M parameters), which uses the MosaicBERT architecture with:
67
- - **Tokenization**: Byte Pair Encoding (BPE) with vocabulary size 4,096
68
- - **Max sequence length**: 1,000 base pairs
69
- - **Hidden size**: 768
70
- - **Attention heads**: 12
71
- - **Layers**: 12
72
- - **Positional encoding**: Attention with Linear Biases (ALiBi)
73
 
74
- **Continued pre-training**:
75
- - **Objective**: Masked Language Modeling (MLM)
76
- - **Training data**: 5M viral sequence chunks (non-overlapping, 1000 bp)
77
- - **Data source**: VIRION database (clustered at 80% identity with MMseqs2)
78
- - **Training**: 10 epochs, AdamW optimizer, learning rate 5e-5
79
- - **Hardware**: 4x NVIDIA A100 GPUs (72 hours)
80
- - **Performance**: 94.2% MLM accuracy on validation set
81
 
82
- ---
 
 
 
 
 
83
 
84
- ## Installation
85
 
86
- ```bash
87
- pip install transformers torch
88
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
89
 
90
  ---
91
 
92
  ## Quick Start
93
 
94
- ### Basic Usage: Extract Sequence Embeddings
95
 
96
  ```python
97
- from transformers import AutoTokenizer, AutoModel
98
  import torch
 
 
 
99
 
100
- # Load model and tokenizer
101
  tokenizer = AutoTokenizer.from_pretrained(
102
- "duttaprat/HViLM-base",
103
- trust_remote_code=True # Required for custom architecture
104
  )
 
105
  model = AutoModel.from_pretrained(
106
- "duttaprat/HViLM-base",
107
- trust_remote_code=True
108
  )
109
 
110
- # Example: Get embeddings for a viral sequence
111
- viral_sequence = "ATGCGTACGTTAGCCGATCGATTACGCGTACGTAGCTAGCTAGCT"
112
 
113
- # Tokenize
114
  inputs = tokenizer(
115
- viral_sequence,
116
  return_tensors="pt",
117
  truncation=True,
118
- max_length=512,
119
- padding=True
120
  )
121
 
122
- # Generate embeddings
123
  with torch.no_grad():
124
  outputs = model(**inputs)
125
- embeddings = outputs.last_hidden_state # [batch_size, seq_len, 768]
126
 
127
- print(f"Sequence embeddings shape: {embeddings.shape}")
 
 
 
 
128
 
129
- # Mean pooling for sequence-level representation
130
- attention_mask = inputs['attention_mask']
131
- mask_expanded = attention_mask.unsqueeze(-1).expand(embeddings.size()).float()
132
- sum_embeddings = torch.sum(embeddings * mask_expanded, dim=1)
133
- sum_mask = torch.clamp(mask_expanded.sum(dim=1), min=1e-9)
134
- mean_embeddings = sum_embeddings / sum_mask
135
 
136
- print(f"Mean sequence embedding shape: {mean_embeddings.shape}") # [batch_size, 768]
137
- ```
138
 
139
- ### Fine-tuning on Your Own Task
140
 
141
- For fine-tuning HViLM on custom viral classification tasks, please refer to the [GitHub repository](https://github.com/duttaprat/HViLM) for complete training scripts and examples.
142
 
143
  ```python
144
- # Example fine-tuning setup (see GitHub for complete code)
145
- from transformers import AutoModel, TrainingArguments, Trainer
146
- from peft import LoraConfig, get_peft_model
147
-
148
- # Load base model
149
- model = AutoModel.from_pretrained("duttaprat/HViLM-base", trust_remote_code=True)
150
-
151
- # Configure LoRA for parameter-efficient fine-tuning
152
- lora_config = LoraConfig(
153
- r=8, # rank
154
- lora_alpha=16, # scaling factor
155
- target_modules=["query", "value"], # attention layers
156
- lora_dropout=0.1,
157
- bias="none"
158
- )
159
 
160
- # Apply LoRA
161
- model = get_peft_model(model, lora_config)
162
 
163
- # Add classification head and train (see GitHub for details)
 
 
 
 
164
  ```
165
 
166
- ---
167
 
168
- ## Performance on HVUE Benchmark
 
169
 
170
- ### Pathogenicity Classification
171
 
172
- | Dataset | Sequences | Accuracy | F1-Score | MCC |
173
- |---------|-----------|----------|----------|-----|
174
- | CINI | 159 | **87.74%** | 86.98 | 74.48 |
175
- | BVBRC-CoV | 18,066 | **98.26%** | 98.26 | 96.52 |
176
- | BVBRC-Calici | 31,089 | **99.95%** | 99.93 | 99.90 |
177
- | **Average** | **49,314** | **95.32%** | **95.06** | **90.30** |
178
 
179
- ### Host Tropism Prediction
 
 
180
 
181
- | Dataset | Sequences | Accuracy | F1-Score | MCC |
182
- |---------|-----------|----------|----------|-----|
183
- | VHDB | 9,428 | **96.25%** | 91.34 | 91.24 |
184
 
185
- ### Transmissibility Assessment (R₀-based Classification)
 
186
 
187
- | Viral Family | Sequences | Accuracy | F1-Score | MCC |
188
- |--------------|-----------|----------|----------|-----|
189
- | Coronaviridae | ~3,000 | **97.45%** | 97.37 | 93.43 |
190
- | Orthomyxoviridae | ~2,500 | **95.62%** | 95.44 | 91.07 |
191
- | Caliciviridae | ~1,800 | **99.95%** | 99.95 | 99.90 |
192
- | **Average** | **~7,300** | **97.36%** | **97.59** | **94.80** |
193
 
194
- **Comparison with baselines**: HViLM consistently outperforms Nucleotide Transformer 500M-1000g, GENA-LM, and DNABERT-MB across all tasks.
195
 
196
- ---
 
 
197
 
198
- ## Interpretability: Transcription Factor Mimicry
199
 
200
- HViLM's attention mechanisms reveal biologically meaningful pathogenicity determinants through **molecular mimicry of host regulatory elements**:
 
201
 
202
- - **42 conserved motifs** identified in high-attention regions of pathogenic coronaviruses
203
- - **10 vertebrate transcription factors** targeted, including:
204
- - **Irf1** (Interferon Regulatory Factor 1): 8 convergent motifs for immune evasion
205
- - **Foxq1**: Multiple motifs for epithelial cell tropism
206
- - **ZNF354A**: 6 motifs for chromatin regulation
207
-
208
- This demonstrates that HViLM captures genuine biological mechanisms rather than spurious correlations.
209
 
210
  ---
211
 
212
- ## Training Data
213
 
214
- ### Pre-training Corpus
215
 
216
- - **Source**: [VIRION database](https://virion.verena.org) (476,242 virus-host associations)
217
- - **Genomes**: 10,817,265 unique NCBI accession numbers
218
- - **Processing**:
219
- - Segmented into non-overlapping 1000 bp chunks
220
- - Clustered with MMseqs2 at 80% identity threshold
221
- - **Final dataset**: 5 million unique sequences
222
- - **Coverage**: 45+ viral families across all Baltimore classification groups
223
 
 
 
 
 
 
 
224
 
225
- ---
226
 
227
- ## HVUE Benchmark Datasets
 
 
228
 
229
- The **Human Virome Understanding Evaluation (HVUE)** benchmark consists of 7 curated datasets:
230
 
231
- ### Pathogenicity Prediction (3 datasets)
232
- - **CINI**: 159 sequences, 4 viral families, manual literature curation
233
- - **BVBRC-CoV**: 18,066 coronaviruses
234
- - **BVBRC-Calici**: 31,089 caliciviruses
235
 
236
- ### Host Tropism Prediction (1 dataset)
237
- - **VHDB**: 9,428 sequences, 30 viral families
238
- - Binary classification: human-tropic (13.1%) vs non-human-tropic (86.9%)
239
 
240
- ### Transmissibility Prediction (3 datasets)
241
- - **Coronaviridae**: R₀-based classification (R₀<1 vs R₀≥1)
242
- - **Orthomyxoviridae**: R₀-based classification
243
- - **Caliciviridae**: R₀-based classification
 
244
 
245
- All datasets available at: **[🤗 duttaprat/HVUE](https://huggingface.co/datasets/duttaprat/HVUE)**
246
 
247
- ### Download and Use
248
- ```python
249
- from datasets import load_dataset
250
 
251
- # Load specific task
252
- host_tropism = load_dataset("duttaprat/HVUE", data_dir="Host_Tropism")
253
- pathogenicity = load_dataset("duttaprat/HVUE", data_dir="Pathogenecity")
254
- transmissibility = load_dataset("duttaprat/HVUE", data_dir="Transmissibility")
255
 
256
- # Load specific split
257
- train_data = load_dataset("duttaprat/HVUE", data_files="Host_Tropism/train.csv")
258
- ```
 
 
 
 
259
 
260
  ---
261
 
262
- ## Reproducing Paper Results
263
 
264
- ### Step 1: Download HVUE Benchmark
265
- ```python
266
- from datasets import load_dataset
267
 
268
- # Download all datasets
269
- host_tropism = load_dataset("duttaprat/HVUE", data_dir="Host_Tropism")
270
- pathogenicity = load_dataset("duttaprat/HVUE", data_dir="Pathogenecity")
271
- transmissibility = load_dataset("duttaprat/HVUE", data_dir="Transmissibility")
272
- ```
273
 
274
- ### Step 2: Fine-tune and Evaluate
 
 
 
 
275
 
276
- To reproduce the results reported in the paper, clone the repository and follow the fine-tuning instructions:
277
 
 
278
 
 
279
 
280
- ```bash
281
- # Clone repository
282
- git clone https://github.com/duttaprat/HViLM.git
283
- cd HViLM
284
 
285
- # Install dependencies
286
- pip install -r requirements.txt
287
 
288
- # Reproduce pathogenicity results on CINI dataset
289
- cd finetune
290
- bash scripts/run_patho_cini.sh
291
 
292
- # Reproduce host tropism results
293
- bash scripts/run_tropism_vhdb.sh
294
 
295
- # Reproduce transmissibility results
296
- bash scripts/run_r0_coronaviridae.sh
297
- ```
 
 
 
 
 
 
 
 
298
 
299
- For detailed instructions, see the [GitHub repository](https://github.com/duttaprat/HViLM).
 
 
 
 
 
 
300
 
301
  ---
302
 
303
  ## Citation
304
- If you use HViLM in your research, please cite our paper:
 
305
 
306
  ```bibtex
307
  @article{dutta2026hvilm,
@@ -313,37 +313,37 @@ If you use HViLM in your research, please cite our paper:
313
  publisher={Cold Spring Harbor Laboratory}
314
  }
315
  ```
 
 
 
316
  ---
317
 
318
  ## Model Card Authors
319
 
320
- - **Pratik Dutta** (Senior Research Scientist, Stony Brook University)
321
- - **Ramana V. Davuluri** (Professor, Stony Brook University)
322
 
323
  ---
324
 
325
  ## Contact
326
 
327
- - **Email**: pratik.dutta@stonybrook.edu
328
- - **Lab**: [Davuluri Lab, Stony Brook University](https://davulurilab.github.io/)
329
- - **GitHub Issues**: [Report bugs or request features](https://github.com/duttaprat/HViLM/issues)
330
 
331
  ---
332
 
333
  ## Acknowledgments
334
 
335
- This work builds upon [DNABERT-2](https://github.com/MAGICS-LAB/DNABERT_2) by Zhou et al. Pre-training data from the [VIRION database](https://virion.verena.org) maintained by the Viral Emergence Research Initiative (Verena).
336
-
337
-
338
 
339
  ---
340
 
341
  ## License
342
 
343
- This model is released under the **Apache License 2.0**.
344
 
345
  ---
346
 
347
  ## Disclaimer
348
 
349
- HViLM is a research tool for computational biology and should not be used as the sole basis for clinical or public health decisions. Predictions should be validated through experimental methods and expert analysis.
 
1
  ---
2
  language:
3
+ - en
4
  license: apache-2.0
5
  library_name: transformers
6
+ base_model: zhihan1996/DNABERT-2-117M
7
  tags:
8
+ - genomics
9
+ - virology
10
+ - dnabert
11
+ - foundation-model
12
+ - hvilm
13
+ - viral-genomics
14
+ - pathogenicity
15
+ - transmissibility
16
+ - host-tropism
17
+ - hvue-v2
18
  datasets:
19
+ - duttaprat/HVUE-v2
 
 
 
20
  pipeline_tag: feature-extraction
21
  widget:
22
+ - text: "ATGCGTACGTTAGCCGATCG"
23
+ example_title: "Virus sequence example"
24
  ---
25
 
26
  # HViLM-base: A Foundation Model for Viral Genomics
27
 
28
  <div align="center">
29
 
30
+ [![Preprint](https://img.shields.io/badge/bioRxiv-2026-B31B1B)](https://www.biorxiv.org/content/10.64898/2026.03.18.712700v1)
31
+ [![Code](https://img.shields.io/badge/Code-GitHub-black)](https://github.com/duttaprat/HViLM)
32
+ [![Dataset](https://img.shields.io/badge/Dataset-HVUE--v2-yellow)](https://huggingface.co/datasets/duttaprat/HVUE-v2)
33
  [![License](https://img.shields.io/badge/License-Apache%202.0-green.svg)](LICENSE)
 
34
 
35
  </div>
36
 
37
+ > [!IMPORTANT]
38
+ > **HVUE v2 supersedes the original HVUE benchmark.**
39
+ > The original HVUE v1 benchmark contained substantial cross-split sequence similarity that could inflate held-out performance estimates. HVUE v2 was rebuilt using source-sequence clustering **before** train/validation/test assignment and chunking, followed by exact- and near-match leakage auditing.
40
+ > **Use [duttaprat/HVUE-v2](https://huggingface.co/datasets/duttaprat/HVUE-v2) for current HViLM evaluation.**
41
+
42
  ## Model Description
43
 
44
+ **HViLM (Human Virome Language Model)** is a genomic foundation model adapted to virus sequences through continued pre-training of [DNABERT-2](https://huggingface.co/zhihan1996/DNABERT-2-117M). HViLM-base was trained on approximately **5 million non-redundant virus-derived sequence fragments** from the [VIRION](https://virion.verena.org/) resource, representing approximately 9,000 virus species across 45+ families.
45
+
46
+ The architecture and tokenizer remain those of DNABERT-2; continued pre-training updates the model weights using a masked-language-modeling objective on the virus-focused corpus.
47
 
48
+ **Preprint:** *HViLM: A Foundation Model for Viral Genomics Enables Multi-Task Prediction of Pathogenicity, Transmissibility, and Host Tropism*
49
+ [bioRxiv 2026.03.18.712700](https://www.biorxiv.org/content/10.64898/2026.03.18.712700v1)
50
 
51
+ **Authors:** Pratik Dutta, Jack Vaska, Pallavi Surana, Rekha Sathian, Max Chao, Zhihan Zhou, Han Liu, and Ramana V. Davuluri
52
 
53
+ **Code:** [github.com/duttaprat/HViLM](https://github.com/duttaprat/HViLM)
54
 
55
  ---
56
 
57
+ ## HViLM Model Family
58
 
59
+ HViLM-base is the continued-pretrained foundation model. Official task-specific models fine-tuned on HVUE v2 are released as standalone checkpoints:
 
 
 
 
 
 
 
 
60
 
61
+ | Resource | Purpose |
62
+ |---|---|
63
+ | [HViLM-base](https://huggingface.co/duttaprat/HViLM-base) | Continued-pretrained foundation model / sequence representations |
64
+ | [HViLM-Patho](https://huggingface.co/duttaprat/HViLM-Patho) | Pathogenicity classification |
65
+ | [HViLM-R0](https://huggingface.co/duttaprat/HViLM-R0) | Transmissibility classification |
66
+ | [HViLM-Tropism](https://huggingface.co/duttaprat/HViLM-Tropism) | Human host-tropism classification |
67
+ | [HVUE-v2](https://huggingface.co/datasets/duttaprat/HVUE-v2) | Leakage-controlled benchmark |
68
 
69
+ The complete project is also grouped in the **HViLM: Human Virome Language Model** collection on the [duttaprat Collections page](https://huggingface.co/duttaprat/collections).
70
 
71
+ ---
 
 
 
 
 
 
72
 
73
+ ## Key Features
 
 
 
 
 
 
74
 
75
+ - **Virus-focused continued pre-training:** approximately 5M non-redundant fragments derived from VIRION-linked virus sequences.
76
+ - **DNABERT-2 initialization:** preserves the DNABERT-2 architecture and BPE tokenizer while adapting model weights to virus sequence data.
77
+ - **Three official downstream models:** pathogenicity, transmissibility, and host tropism.
78
+ - **HVUE v2 evaluation:** cluster-aware splitting before chunking, with multiple similarity stringencies and sequence lengths.
79
+ - **Parameter-efficient downstream adaptation:** official task models were trained with LoRA.
80
+ - **Public reproducibility resources:** base model, three task-specific checkpoints, HVUE v2 benchmark, and project code are released publicly.
81
 
82
+ ---
83
 
84
+ ## Model Architecture and Continued Pre-training
85
+
86
+ HViLM-base is derived from **DNABERT-2 (117M parameters)**.
87
+
88
+ | Property | Value |
89
+ |---|---|
90
+ | Architecture | MosaicBERT / DNABERT-2 |
91
+ | Parameters | ~117M |
92
+ | Hidden size | 768 |
93
+ | Transformer layers | 12 |
94
+ | Attention heads | 12 |
95
+ | Tokenization | Byte Pair Encoding (BPE) |
96
+ | Positional method | ALiBi |
97
+ | Continued-pretraining objective | Masked Language Modeling |
98
+ | Pretraining fragment length | 1000 nt |
99
+ | Final virus-focused corpus | ~5M non-redundant fragments |
100
+ | Redundancy reduction | MMseqs2 clustering at 80% identity / 80% coverage |
101
+ | Optimizer | AdamW |
102
+ | Learning rate | 5e-5 |
103
+ | Training | 10 epochs |
104
+ | Hardware | 4 × NVIDIA A100 GPUs |
105
+ | Approximate training time | 72 hours |
106
+ | Held-out MLM accuracy | 94.2% |
107
+
108
+ **Sequence-length note:** HViLM uses BPE tokenization, so nucleotide length and model-token length are not equivalent. The continued-pretraining corpus used 1000-nt sequence fragments; downstream configurations are described by nucleotide length in HVUE v2.
109
 
110
  ---
111
 
112
  ## Quick Start
113
 
114
+ ### Extract sequence representations from HViLM-base
115
 
116
  ```python
 
117
  import torch
118
+ from transformers import AutoTokenizer, AutoModel
119
+
120
+ model_id = "duttaprat/HViLM-base"
121
 
 
122
  tokenizer = AutoTokenizer.from_pretrained(
123
+ model_id,
124
+ trust_remote_code=True,
125
  )
126
+
127
  model = AutoModel.from_pretrained(
128
+ model_id,
129
+ trust_remote_code=True,
130
  )
131
 
132
+ sequence = "ATGCGTACGTTAGCCGATCGATTACGCGTACGTAGCTAGCTAGCT"
 
133
 
 
134
  inputs = tokenizer(
135
+ sequence,
136
  return_tensors="pt",
137
  truncation=True,
138
+ padding=True,
 
139
  )
140
 
 
141
  with torch.no_grad():
142
  outputs = model(**inputs)
 
143
 
144
+ token_embeddings = outputs.last_hidden_state
145
+ print(token_embeddings.shape)
146
+ ```
147
+
148
+ For sequence-level representations, pooling strategy should be chosen according to the downstream task rather than treated as a fixed property of HViLM-base.
149
 
150
+ ---
 
 
 
 
 
151
 
152
+ ## Use the Official Fine-tuned Models
 
153
 
154
+ If the goal is one of the three HVUE v2 tasks, users can load the corresponding task model directly; `HViLM-base` does not need to be loaded separately.
155
 
156
+ ### Pathogenicity
157
 
158
  ```python
159
+ from transformers import AutoTokenizer, AutoModelForSequenceClassification
 
 
 
 
 
 
 
 
 
 
 
 
 
 
160
 
161
+ model_id = "duttaprat/HViLM-Patho"
 
162
 
163
+ tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
164
+ model = AutoModelForSequenceClassification.from_pretrained(
165
+ model_id,
166
+ trust_remote_code=True,
167
+ )
168
  ```
169
 
170
+ Labels:
171
 
172
+ - `0` `NON_PATHOGENIC`
173
+ - `1` → `PATHOGENIC`
174
 
175
+ ### Transmissibility
176
 
177
+ Use:
 
 
 
 
 
178
 
179
+ ```text
180
+ duttaprat/HViLM-R0
181
+ ```
182
 
183
+ Labels:
 
 
184
 
185
+ - `0` `R0_LT_1`
186
+ - `1` → `R0_GE_1`
187
 
188
+ ### Host Tropism
 
 
 
 
 
189
 
190
+ Use:
191
 
192
+ ```text
193
+ duttaprat/HViLM-Tropism
194
+ ```
195
 
196
+ Labels:
197
 
198
+ - `0` `NON_HUMAN_TROPIC`
199
+ - `1` → `HUMAN_TROPIC`
200
 
201
+ See the individual model cards for full usage examples and task-specific limitations.
 
 
 
 
 
 
202
 
203
  ---
204
 
205
+ ## HVUE v2 Benchmark
206
 
207
+ [HVUE v2](https://huggingface.co/datasets/duttaprat/HVUE-v2) is the current evaluation benchmark for HViLM. It replaces HVUE v1.
208
 
209
+ The benchmark was reconstructed to reduce supervised train-test leakage using the following ordering:
 
 
 
 
 
 
210
 
211
+ 1. consolidate and deduplicate source sequences;
212
+ 2. cluster source sequences with MMseqs2;
213
+ 3. assign complete clusters to train/validation/test splits;
214
+ 4. chunk sequences only **after** split assignment;
215
+ 5. remove exact duplicate chunks;
216
+ 6. audit cross-split exact and near matches.
217
 
218
+ HVUE v2 includes:
219
 
220
+ - **Pathogenicity**
221
+ - **Transmissibility**
222
+ - **Host Tropism**
223
 
224
+ Across the benchmark, configurations evaluate different sequence lengths (500, 1000, and 2000 nt where applicable), sequence-similarity stringencies, and temporal generalization where reliable collection-date metadata are available.
225
 
226
+ ### Primary HViLM Results
 
 
 
227
 
228
+ The primary results below use the **standard 1000-nt configuration** for each task.
 
 
229
 
230
+ | Task | HVUE v2 configuration | Accuracy | F1 | MCC | Official model |
231
+ |---|---|---:|---:|---:|---|
232
+ | Pathogenicity | `standard_capped_1000bp` | **92.39** | **91.32** | **83.10** | [HViLM-Patho](https://huggingface.co/duttaprat/HViLM-Patho) |
233
+ | Transmissibility | `standard_capped_1000bp` | **87.50** | **86.16** | **72.66** | [HViLM-R0](https://huggingface.co/duttaprat/HViLM-R0) |
234
+ | Host Tropism | `standard_95_1000bp` | **96.49** | **74.49** | **48.99** | [HViLM-Tropism](https://huggingface.co/duttaprat/HViLM-Tropism) |
235
 
236
+ The directory/configuration identifiers retain `bp` for release stability; manuscript and descriptive text use **nt** for nucleotide sequence length.
237
 
238
+ ### Interpretation of the HVUE v2 Results
 
 
239
 
240
+ Under leakage-controlled evaluation, the effect of virus-focused continued pre-training is **task dependent**:
 
 
 
241
 
242
+ - **Pathogenicity:** HViLM improves F1 by 1.28 points over vanilla DNABERT-2 (91.32 vs. 90.04).
243
+ - **Transmissibility:** HViLM and DNABERT-2 are close (86.16 vs. 85.81 F1), and HViLM is essentially tied with DNABERT-MB (86.16 vs. 86.15 F1).
244
+ - **Host Tropism:** HViLM shows the largest F1 improvement, reaching 74.49 compared with 64.82 for class-balanced DNABERT-2.
245
+
246
+ These results support a more specific conclusion than the original HVUE v1 evaluation: virus-focused continued pre-training provides its clearest benefit on the more challenging Host Tropism task, while gains on Pathogenicity and Transmissibility are smaller.
247
+
248
+ For complete baseline comparisons, hard-split evaluations, temporal evaluations, and sequence-length analyses, see the [HViLM GitHub repository](https://github.com/duttaprat/HViLM) and [HVUE-v2](https://huggingface.co/datasets/duttaprat/HVUE-v2).
249
 
250
  ---
251
 
252
+ ## Training Data
253
 
254
+ ### Continued-pretraining corpus
 
 
255
 
256
+ HViLM-base was trained using virus sequences associated with the **VIRION** resource.
257
+
258
+ Processing included:
 
 
259
 
260
+ - retrieval and quality control of VIRION-linked nucleotide sequences;
261
+ - removal of short sequences and exact duplicates;
262
+ - segmentation into non-overlapping 1000-nt fragments;
263
+ - MMseqs2 clustering at 80% sequence identity and 80% coverage;
264
+ - selection of approximately 5M representative fragments for continued pre-training.
265
 
266
+ The corpus spans approximately 9,000 virus species and 45+ virus families across the Baltimore classification groups.
267
 
268
+ ---
269
 
270
+ ## Interpretability
271
 
272
+ Attention-guided analyses associated with the HViLM study identified **candidate sequence motifs** in pathogenic coronavirus sequences, including motifs with similarity to vertebrate transcription-factor binding motifs such as IRF1, FOXQ1, and ZNF354A.
 
 
 
273
 
274
+ These observations are **hypothesis-generating**. Sequence similarity between virus motifs and host transcription-factor binding motifs does not by itself establish molecular mimicry, causal regulation, immune evasion, or another biological mechanism. Experimental validation and additional controls are required for mechanistic interpretation.
 
275
 
276
+ ---
 
 
277
 
278
+ ## Limitations
 
279
 
280
+ - HVUE v2 controls supervised split leakage through source-level clustering and auditing, but sequence-similarity thresholds cannot eliminate every form of biological relatedness.
281
+ - The complete historical training exposure of the original DNABERT-2 model cannot be reconstructed; therefore, absence of all possible ancestral pretraining exposure to benchmark-related sequences cannot be guaranteed.
282
+ - Host association is biologically context-dependent and may include multi-host, zoonotic, and reverse-zoonotic relationships; the benchmark uses a simplified binary formulation.
283
+ - R₀-based transmissibility labels simplify a continuous, context-dependent epidemiological quantity into a binary benchmark task.
284
+ - Performance differences between closely matched models should not be interpreted as statistically meaningful without uncertainty estimates or repeated evaluations.
285
+ - Attention-based motif analyses should be considered exploratory rather than direct evidence of mechanism.
286
+ - HViLM predictions are research outputs and are not intended to replace experimental, clinical, epidemiological, or public-health assessment.
287
+
288
+ ---
289
+
290
+ ## Reproducibility and Resources
291
 
292
+ - **Base model:** [duttaprat/HViLM-base](https://huggingface.co/duttaprat/HViLM-base)
293
+ - **Pathogenicity model:** [duttaprat/HViLM-Patho](https://huggingface.co/duttaprat/HViLM-Patho)
294
+ - **Transmissibility model:** [duttaprat/HViLM-R0](https://huggingface.co/duttaprat/HViLM-R0)
295
+ - **Host Tropism model:** [duttaprat/HViLM-Tropism](https://huggingface.co/duttaprat/HViLM-Tropism)
296
+ - **Benchmark:** [duttaprat/HVUE-v2](https://huggingface.co/datasets/duttaprat/HVUE-v2)
297
+ - **Code:** [github.com/duttaprat/HViLM](https://github.com/duttaprat/HViLM)
298
+ - **Collections:** [duttaprat's Hugging Face Collections](https://huggingface.co/duttaprat/collections)
299
 
300
  ---
301
 
302
  ## Citation
303
+
304
+ If you use HViLM in your research, please cite:
305
 
306
  ```bibtex
307
  @article{dutta2026hvilm,
 
313
  publisher={Cold Spring Harbor Laboratory}
314
  }
315
  ```
316
+
317
+ If you use DNABERT-2 directly or build on its architecture, please also cite the DNABERT-2 publication.
318
+
319
  ---
320
 
321
  ## Model Card Authors
322
 
323
+ - **Pratik Dutta** Stony Brook University
324
+ - **Ramana V. Davuluri** Stony Brook University
325
 
326
  ---
327
 
328
  ## Contact
329
 
330
+ - **GitHub Issues:** [github.com/duttaprat/HViLM/issues](https://github.com/duttaprat/HViLM/issues)
331
+ - **Lab:** [Davuluri Lab, Stony Brook University](https://davulurilab.github.io/)
 
332
 
333
  ---
334
 
335
  ## Acknowledgments
336
 
337
+ HViLM builds on DNABERT-2 by Zhou et al. Continued-pretraining data were derived from the VIRION resource maintained by the Viral Emergence Research Initiative (Verena).
 
 
338
 
339
  ---
340
 
341
  ## License
342
 
343
+ HViLM-base is released under the **Apache License 2.0**.
344
 
345
  ---
346
 
347
  ## Disclaimer
348
 
349
+ HViLM is a research model for computational biology. It should not be used as the sole basis for clinical, diagnostic, epidemiological, biosurveillance, or public-health decisions. Model outputs should be interpreted alongside appropriate biological evidence and expert assessment.