Initial commit

README.md

@@ -0,0 +1,56 @@
+---
+language: vi
+datasets:
+- cc100
+tags:
+- summarization
+
+license: mit
+
+widget:
+- text: "vietnews: VietAI là tổ chức phi lợi nhuận với sứ mệnh ươm mầm tài năng về trí tuệ nhân tạo và xây dựng một cộng đồng các chuyên gia trong lĩnh vực trí tuệ nhân tạo đẳng cấp quốc tế tại Việt Nam."
+---
+
+# ViT5-large Finetuned on `vietnews` Abstractive Summarization
+
+
+State-of-the-art pretrained Transformer-based encoder-decoder model for Vietnamese.
+[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/vit5-pretrained-text-to-text-transformer-for/abstractive-text-summarization-on-vietnews)](https://paperswithcode.com/sota/abstractive-text-summarization-on-vietnews?p=vit5-pretrained-text-to-text-transformer-for)
+
+
+## How to use
+For more details, do check out [our Github repo](https://github.com/vietai/ViT5) and [eval script](https://github.com/vietai/ViT5/blob/main/eval/Eval_vietnews_sum.ipynb). 
+
+```python
+from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
+​
+tokenizer = AutoTokenizer.from_pretrained("VietAI/vit5-large-vietnews-summarization")  
+model = AutoModelForSeq2SeqLM.from_pretrained("VietAI/vit5-large-vietnews-summarization")
+model.cuda()
+​
+sentence = "VietAI là tổ chức phi lợi nhuận với sứ mệnh ươm mầm tài năng về trí tuệ nhân tạo và xây dựng một cộng đồng các chuyên gia trong lĩnh vực trí tuệ nhân tạo đẳng cấp quốc tế tại Việt Nam."
+text =  "vietnews: " + sentence + " </s>"
+encoding = tokenizer(text, return_tensors="pt")
+input_ids, attention_masks = encoding["input_ids"].to("cuda"), encoding["attention_mask"].to("cuda")
+outputs = model.generate(
+    input_ids=input_ids, attention_mask=attention_masks,
+    max_length=256,
+    early_stopping=True
+)
+for output in outputs:
+    line = tokenizer.decode(output, skip_special_tokens=True, clean_up_tokenization_spaces=True)
+    print(line)
+```
+
+## Citation
+```
+@inproceedings{phan-etal-2022-vit5,
+    title = "{V}i{T}5: Pretrained Text-to-Text Transformer for {V}ietnamese Language Generation",
+    author = "Phan, Long and Tran, Hieu and Nguyen, Hieu and Trinh, Trieu H.",
+    booktitle = "Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop",
+    year = "2022",
+    publisher = "Association for Computational Linguistics",
+    url = "https://aclanthology.org/2022.naacl-srw.18",
+    pages = "136--142",
+}
+```

config.json

@@ -0,0 +1,38 @@
+{
+  "architectures": [
+    "T5ForConditionalGeneration"
+  ],
+  "d_ff": 2816,
+  "d_kv": 64,
+  "d_model": 1024,
+  "decoder_start_token_id": 0,
+  "dropout_rate": 0.1,
+  "eos_token_id": 1,
+  "feed_forward_proj": "gated-gelu",
+  "initializer_factor": 1.0,
+  "is_encoder_decoder": true,
+  "layer_norm_epsilon": 1e-06,
+  "model_type": "t5",
+  "num_decoder_layers": 24,
+  "num_heads": 16,
+  "num_layers": 24,
+  "output_past": true,
+  "pad_token_id": 0,
+  "relative_attention_max_distance": 128,
+  "relative_attention_num_buckets": 32,
+  "task_specific_params": {
+    "summarization": {
+      "early_stopping": true,
+      "length_penalty": 2.0,
+      "max_length": 256,
+      "no_repeat_ngram_size": 3,
+      "num_beams": 4,
+      "prefix": "summarization: "
+    }
+  },
+  "tie_word_embeddings": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.18.0",
+  "use_cache": true,
+  "vocab_size": 36096
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7c524804e16b992b38ed32b14cf495399e0c70dd0b26aa414f4d93d5acca4254
+size 3165287717

spiece.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:59986b62f9f0b90edafb9b073ea7b93d21114a5841219a1ea2399ade73f729c6
+size 820370

tf_model.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2195c610ae03a6b541d53d3dc35b328860ba4148a85a02021696c0fa27d01500
+size 3166229632