Ephraimmm
/

pidgin14-decoder

@@ -1,199 +1,96 @@
 ---
 library_name: transformers
-tags: []
 ---
-# Model Card for Model ID
-<!-- Provide a quick summary of what the model is/does. -->
-## Model Details
-### Model Description
-<!-- Provide a longer summary of what this model is. -->
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-### Model Sources [optional]
-<!-- Provide the basic links for the model. -->
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-## Uses
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-### Direct Use
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-[More Information Needed]
-### Downstream Use [optional]
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-[More Information Needed]
-### Out-of-Scope Use
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-[More Information Needed]
-## Bias, Risks, and Limitations
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-[More Information Needed]
-### Recommendations
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-## How to Get Started with the Model
-Use the code below to get started with the model.
-[More Information Needed]
 ## Training Details
-### Training Data
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-[More Information Needed]
-### Training Procedure
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-#### Preprocessing [optional]
-[More Information Needed]
-#### Training Hyperparameters
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-#### Speeds, Sizes, Times [optional]
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-[More Information Needed]
-## Evaluation
-<!-- This section describes the evaluation protocols and provides the results. -->
-### Testing Data, Factors & Metrics
-#### Testing Data
-<!-- This should link to a Dataset Card if possible. -->
-[More Information Needed]
-#### Factors
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-[More Information Needed]
-#### Metrics
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-[More Information Needed]
-### Results
-[More Information Needed]
-#### Summary
-## Model Examination [optional]
-<!-- Relevant interpretability work for the model goes here -->
-[More Information Needed]
-## Environmental Impact
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-## Technical Specifications [optional]
-### Model Architecture and Objective
-[More Information Needed]
-### Compute Infrastructure
-[More Information Needed]
-#### Hardware
-[More Information Needed]
-#### Software
-[More Information Needed]
-## Citation [optional]
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-**BibTeX:**
-[More Information Needed]
-**APA:**
-[More Information Needed]
-## Glossary [optional]
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-[More Information Needed]
-## More Information [optional]
-[More Information Needed]
-## Model Card Authors [optional]
-[More Information Needed]
-## Model Card Contact
-[More Information Needed]

 ---
+language:
+- pcm
+- en
+tags:
+- transformers
+- encoder-decoder
+- gpt2
+- pidgin
+- nigerian-pidgin
+- nlp
+- text-generation
 library_name: transformers
 ---
+# Pidgin14 Decoder (GPT-2-medium-based)
+## Overview
+This repository hosts the **decoder-side tokenizer** for `pidgin14`, an encoder-decoder sequence-to-sequence system for Nigerian Pidgin English ("Naija") built by [Ephraim](https://huggingface.co/Ephraimmm) at Analytics Intelligence.
+`pidgin14` is composed of two halves published as separate repositories:
+- **Encoder** — [`Ephraimmm/pidgin14-encoder`](https://huggingface.co/Ephraimmm/pidgin14-encoder), based on AfriBERTa, reads source text and produces contextual representations.
+- **Decoder** (this repo) — based on GPT-2-medium, consumes the encoder's representations via cross-attention and generates the output text.
+The two halves are combined and trained together as a single `EncoderDecoderModel`, whose full weights are published at [`Ephraimmm/pidgin14`](https://huggingface.co/Ephraimmm/pidgin14). The architecture facts below are taken directly from that combined model's `config.json` (`decoder` sub-config), since this component repository itself contains only tokenizer files (`tokenizer.json`, `tokenizer_config.json`, `special_tokens_map.json`, `vocab.json`, `merges.txt`) and not a standalone `config.json` or weight file.
+## Architecture Details
+From the `decoder` sub-configuration of the combined `Ephraimmm/pidgin14` model:
+| Field | Value |
+|---|---|
+| Base model | `gpt2-medium` |
+| Model type | `gpt2` (architecture class `GPT2LMHeadModel`), configured with `add_cross_attention: true` so it can act as the decoder half of an `EncoderDecoderModel` |
+| Layers (`n_layer`) | 24 |
+| Hidden size (`n_embd`) | 1024 |
+| Attention heads (`n_head`) | 16 |
+| Context length (`n_positions` / `n_ctx`) | 1024 |
+| Vocabulary size | 50,257 |
+| Activation function | `gelu_new` |
+Tokenizer shipped in **this** repository:
+- Tokenizer class: `GPT2Tokenizer` (byte-level BPE)
+- Vocabulary size: 50,257 tokens (`vocab.json` with 50,000 merge rules in `merges.txt`) — this matches the standard, unmodified GPT-2 tokenizer vocabulary rather than a Pidgin-specific retrained vocabulary.
+- Special token: `<|endoftext|>` used as bos/eos/pad/unk (token id 50256).
+- `decoder_start_token_id`: 50256 (per the combined model's config).
 ## Training Details
+- Fine-tuned from: `gpt2-medium`, used as the decoder half of the `pidgin14` `EncoderDecoderModel` (with cross-attention layers added to attend to the encoder's outputs).
+- Framework: Hugging Face `transformers` (the combined model's config records `transformers_version: 4.44.2`).
+- Stored precision: `float32` (per the combined model's config).
+- No `trainer_state.json`, training-step/epoch counts, optimizer settings, or training-dataset identifiers are published in this repository or in the combined `Ephraimmm/pidgin14` repository. These details are therefore omitted rather than estimated.
+## Intended Use
+- Generating Nigerian Pidgin English and/or English text as the second stage of the `pidgin14` sequence-to-sequence pipeline (e.g. translation, paraphrasing, conversational response generation).
+- Research and experimentation on low-resource West African language NLP.
+- Must be paired with the [`pidgin14-encoder`](https://huggingface.co/Ephraimmm/pidgin14-encoder) tokenizer and the trained weights in [`Ephraimmm/pidgin14`](https://huggingface.co/Ephraimmm/pidgin14) to produce output.
+## How to Use
+```python
+from transformers import AutoTokenizer, EncoderDecoderModel
+# Tokenizers for each half of the system
+encoder_tokenizer = AutoTokenizer.from_pretrained("Ephraimmm/pidgin14-encoder")
+decoder_tokenizer = AutoTokenizer.from_pretrained("Ephraimmm/pidgin14-decoder")
+# The trained combined encoder-decoder weights
+model = EncoderDecoderModel.from_pretrained("Ephraimmm/pidgin14")
+text = "How you dey?"
+inputs = encoder_tokenizer(text, return_tensors="pt")
+output_ids = model.generate(
+    **inputs,
+    decoder_start_token_id=decoder_tokenizer.bos_token_id,
+    max_length=50,
+)
+print(decoder_tokenizer.decode(output_ids[0], skip_special_tokens=True))
+```
+## Limitations
+- This repository provides the **tokenizer only** for the decoder half of `pidgin14`; it is not a usable standalone model and contains no weight file or `config.json` of its own.
+- Must be paired with [`Ephraimmm/pidgin14-encoder`](https://huggingface.co/Ephraimmm/pidgin14-encoder) and the weights in [`Ephraimmm/pidgin14`](https://huggingface.co/Ephraimmm/pidgin14) to perform any task.
+- The tokenizer vocabulary is the stock GPT-2 (English-oriented) byte-level BPE vocabulary and was not retrained on Pidgin-specific text, which may reduce tokenization efficiency for Pidgin-specific spellings and slang.
+- Nigerian Pidgin English is a low-resource language with substantial dialectal and orthographic variation; outputs should be reviewed for fluency and correctness before use.
+- No evaluation metrics, benchmark results, or training-dataset documentation are published for this model. Outputs should be independently validated before any production use.
+- License terms are not specified in the repository; users should contact the author before commercial reuse.
+## Author
+Developed by [Ephraimmm](https://huggingface.co/Ephraimmm)