flanT5-MoE-7X0.1B

flanT5-MoE-7X0.1B is a compact Mixture of Experts (MoE) text-to-text generation model published by WithIn Us AI. It is built from google/flan-t5-small and presented as a 7-expert MoE model intended for lightweight instruction-following, coding-oriented prompting, reasoning-style tasks, summarization, and dialogue-style generation.

This model is designed for users who want a small and efficient T5-family checkpoint for experimentation, local inference, and structured prompt-to-output workflows.

Model Summary

This model is intended for:

• instruction-following tasks
• lightweight coding and programming prompts
• reasoning-style text generation
• summarization and dialogue tasks
• compact text-to-text workflows
• experimentation with small Mixture of Experts architectures

Because this model follows the T5 / Flan-T5 text-to-text format, it generally works best when prompted with clear task instructions rather than open-ended casual chat.

Base Model

This model is based on:

• google/flan-t5-small

Model Architecture

According to the current repository metadata and README, this model is a 7-expert Mixture of Experts merge built from repeated google/flan-t5-small experts with different prompt affinities. The README describes the expert routing themes as:

• instruction / task
• reasoning / logic
• creative / writing
• code / programming
• science / facts
• math / calculation
• summary / dialogue

The current model page also lists the model as a T5, Mixture of Experts, Merge, mergekit, and lazymergekit project.

Training Data

The current repository metadata lists the following datasets in the model lineage:

• deepmind/code_contests
• svakulenk0/qrecc
• djaym7/wiki_dialog

These suggest a blend of coding, conversational retrieval, and dialogue-oriented supervision.

Intended Use

This model is intended for:

• compact instruction-based generation
• small coding assistant experiments
• educational prompt-response systems
• summarization and reformulation
• dialogue-style text transformation
• lightweight research into MoE behavior

Recommended Use Cases

This model can be useful for:

• generating short code-oriented responses
• answering structured prompts
• summarizing small passages
• rewriting text into cleaner or more concise form
• handling lightweight reasoning prompts
• experimenting with expert-routed small models

Out-of-Scope Use

This model should not be relied on for:

• legal advice
• medical advice
• financial advice
• safety-critical automation
• unsupervised production deployment
• high-stakes factual systems without human verification

All generated outputs should be reviewed before real-world use.

Repository Contents

The repository currently includes standard Transformers artifacts such as:

• config.json
• generation_config.json
• model.safetensors
• tokenizer.json
• tokenizer_config.json

Prompting Guidance

This model works best with direct task instructions.

Example prompt styles

Instruction following

Explain recursion in simple terms for a beginner.

Coding

Write a short Python function that removes duplicate values from a list while preserving order.

Summarization

Summarize the following paragraph in three bullet points.

Dialogue / QA

Rewrite this answer so it sounds clearer and more helpful.

Strengths

This model may be especially useful for:

• compact inference
• structured text-to-text tasks
• small-scale coding prompts
• efficient summarization
• lightweight experimentation with MoE routing
• educational and prototype workflows

Limitations

Like other compact language models, this model may:

• hallucinate facts or APIs
• produce incomplete code
• struggle with long-context tasks
• simplify complex reasoning too aggressively
• require prompt iteration for best results
• underperform larger models on advanced technical work

Human review is recommended.

Attribution

WithIn Us AI is the publisher of this model release.

Credit for upstream assets remains with their original creators, including:

• Google for google/flan-t5-small
• DeepMind for deepmind/code_contests
• the creators of svakulenk0/qrecc
• the creators of djaym7/wiki_dialog

License

This repository is currently marked:

• Apache-2.0

Use of this model should remain consistent with the repository license and any applicable upstream terms.

Acknowledgments

Thanks to:

• WithIn Us AI
• Google
• DeepMind
• the dataset creators listed above
• the Hugging Face ecosystem
• the open-source merge and MoE tooling community

Disclaimer

This model may produce inaccurate, incomplete, or biased outputs. Review important outputs carefully before real-world use.