chumengl commited on
Commit
d48a15b
·
verified ·
1 Parent(s): 8b5917a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +46 -44
README.md CHANGED
@@ -1,44 +1,46 @@
1
- ---
2
- datasets:
3
- - Skylion007/openwebtext
4
- language:
5
- - en
6
- library_name: transformers
7
- license: apache-2.0
8
- metrics:
9
- - perplexity
10
- pipeline_tag: text-generation
11
- ---
12
-
13
- # LangFlow
14
-
15
- LangFlow is a continuous diffusion language model that operates in embedding space. Unlike discrete diffusion models (MDLM, SEDD, DUO), LangFlow performs diffusion directly on continuous token embeddings, enabling smoother denoising dynamics.
16
-
17
- ## Using LangFlow
18
-
19
- To use the pre-trained model for text generation, use the following snippet:
20
-
21
- ```python
22
- from transformers import AutoModelForMaskedLM, AutoTokenizer
23
-
24
- tokenizer = AutoTokenizer.from_pretrained('gpt2')
25
- model = AutoModelForMaskedLM.from_pretrained('chumengl/langflow-owt', trust_remote_code=True)
26
-
27
- # Generate samples
28
- samples = model.generate_samples(num_samples=5, num_steps=128)
29
- texts = tokenizer.batch_decode(samples, skip_special_tokens=True)
30
- for text in texts:
31
- print(text)
32
- ```
33
-
34
- ## Model Details
35
-
36
- - **Architecture**: DiT (Diffusion Transformer) backbone with adaptive layer normalization
37
- - **Context Length**: 1024 tokens
38
- - **Parameters**: ~130M non-embedding parameters (similar to GPT-2 medium)
39
- - **Training**: 1M steps on OpenWebText corpus
40
- - **Tokenizer**: GPT-2 tokenizer (50,257 vocab size)
41
-
42
- ## Model Card Contact
43
-
44
- Chumeng Liang (chumengl@illinois.edu)
 
 
 
1
+ ---
2
+ datasets:
3
+ - Skylion007/openwebtext
4
+ papers:
5
+ - arxiv: 2604.11748
6
+ language:
7
+ - en
8
+ library_name: transformers
9
+ license: apache-2.0
10
+ metrics:
11
+ - perplexity
12
+ pipeline_tag: text-generation
13
+ ---
14
+
15
+ # LangFlow
16
+
17
+ LangFlow is a continuous diffusion language model that operates in embedding space. Unlike discrete diffusion models (MDLM, SEDD, DUO), LangFlow performs diffusion directly on continuous token embeddings, enabling smoother denoising dynamics.
18
+
19
+ ## Using LangFlow
20
+
21
+ To use the pre-trained model for text generation, use the following snippet:
22
+
23
+ ```python
24
+ from transformers import AutoModelForMaskedLM, AutoTokenizer
25
+
26
+ tokenizer = AutoTokenizer.from_pretrained('gpt2')
27
+ model = AutoModelForMaskedLM.from_pretrained('chumengl/langflow-owt', trust_remote_code=True)
28
+
29
+ # Generate samples
30
+ samples = model.generate_samples(num_samples=5, num_steps=128)
31
+ texts = tokenizer.batch_decode(samples, skip_special_tokens=True)
32
+ for text in texts:
33
+ print(text)
34
+ ```
35
+
36
+ ## Model Details
37
+
38
+ - **Architecture**: DiT (Diffusion Transformer) backbone with adaptive layer normalization
39
+ - **Context Length**: 1024 tokens
40
+ - **Parameters**: ~130M non-embedding parameters (similar to GPT-2 medium)
41
+ - **Training**: 1M steps on OpenWebText corpus
42
+ - **Tokenizer**: GPT-2 tokenizer (50,257 vocab size)
43
+
44
+ ## Model Card Contact
45
+
46
+ Chumeng Liang (chumengl@illinois.edu)