The Mind's Eye: A Multi-Faceted Reward Framework for Guiding Visual Metaphor Generation
Paper
•
2508.18569
•
Published
AI & ML interests
None defined yet.
Recent Activity
View all activity
mindhunter23
authored
a
paper
3 months ago
b1l4lx1
authored
a
paper
5 months ago
romainhuet
updated
a
Space
6 months ago
romainhuet
published
a
Space
6 months ago
xxiaoyali
authored
7
papers
6 months ago
ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information
Paper
•
2106.16038
•
Published
OpenViDial 2.0: A Larger-Scale, Open-Domain Dialogue Generation Dataset with Visual Contexts
Paper
•
2109.12761
•
Published
Instruction Tuning for Large Language Models: A Survey
Paper
•
2308.10792
•
Published
•
1
Reinforcement Learning Enhanced LLMs: A Survey
Paper
•
2412.10400
•
Published
FaceID-6M: A Large-Scale, Open-Source FaceID Customization Dataset
Paper
•
2503.07091
•
Published
•
3
Dice Loss for Data-imbalanced NLP Tasks
Paper
•
1911.02855
•
Published
CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learning
Paper
•
2507.14111
•
Published
•
24
mindhunter23
authored
a
paper
9 months ago
mindhunter23
authored
2
papers
11 months ago
mindhunter23
authored
a
paper
about 1 year ago
Post
4836
What a brilliant week for Open Source AI!
Qwen 2.5 Coder by Alibaba - 0.5B / 1.5B / 3B / 7B / 14B/ 32B (Base + Instruct) Code generation LLMs, with 32B tackling giants like Gemnini 1.5 Pro, Claude Sonnet
Qwen/qwen25-coder-66eaa22e6f99801bf65b0c2f
LLM2CLIP from Microsoft - Leverage LLMs to train ultra-powerful CLIP models! Boosts performance over the previous SOTA by ~17%
microsoft/llm2clip-672323a266173cfa40b32d4c
Athene v2 Chat & Agent by NexusFlow - SoTA general LLM fine-tuned from Qwen 2.5 72B excels at Chat + Function Calling/ JSON/ Agents
Nexusflow/athene-v2-6735b85e505981a794fb02cc
Orca Agent Instruct by Microsoft - 1 million instruct pairs covering text editing, creative writing, coding, reading comprehension, etc - permissively licensed
microsoft/orca-agentinstruct-1M-v1
Ultravox by FixieAI - 70B/ 8B model approaching GPT4o level, pick any LLM, train an adapter with Whisper as Audio Encoder
reach-vb/ultravox-audio-language-model-release-67373b602af0a52b2a88ae71
JanusFlow 1.3 by DeepSeek - Next iteration of their Unified MultiModal LLM Janus with RectifiedFlow
deepseek-ai/JanusFlow-1.3B
Common Corpus by Pleais - 2,003,039,184,047 multilingual, commercially permissive and high quality tokens!
PleIAs/common_corpus
I'm sure I missed a lot, can't wait for the next week!
Put down in comments what I missed! 🤗
Qwen 2.5 Coder by Alibaba - 0.5B / 1.5B / 3B / 7B / 14B/ 32B (Base + Instruct) Code generation LLMs, with 32B tackling giants like Gemnini 1.5 Pro, Claude Sonnet
Qwen/qwen25-coder-66eaa22e6f99801bf65b0c2f
LLM2CLIP from Microsoft - Leverage LLMs to train ultra-powerful CLIP models! Boosts performance over the previous SOTA by ~17%
microsoft/llm2clip-672323a266173cfa40b32d4c
Athene v2 Chat & Agent by NexusFlow - SoTA general LLM fine-tuned from Qwen 2.5 72B excels at Chat + Function Calling/ JSON/ Agents
Nexusflow/athene-v2-6735b85e505981a794fb02cc
Orca Agent Instruct by Microsoft - 1 million instruct pairs covering text editing, creative writing, coding, reading comprehension, etc - permissively licensed
microsoft/orca-agentinstruct-1M-v1
Ultravox by FixieAI - 70B/ 8B model approaching GPT4o level, pick any LLM, train an adapter with Whisper as Audio Encoder
reach-vb/ultravox-audio-language-model-release-67373b602af0a52b2a88ae71
JanusFlow 1.3 by DeepSeek - Next iteration of their Unified MultiModal LLM Janus with RectifiedFlow
deepseek-ai/JanusFlow-1.3B
Common Corpus by Pleais - 2,003,039,184,047 multilingual, commercially permissive and high quality tokens!
PleIAs/common_corpus
I'm sure I missed a lot, can't wait for the next week!
Put down in comments what I missed! 🤗
Post
2126
Smol TTS models are here! OuteTTS-0.1-350M - Zero shot voice cloning, built on LLaMa architecture, CC-BY license! 🔥
> Pure language modeling approach to TTS
> Zero-shot voice cloning
> LLaMa architecture w/ Audio tokens (WavTokenizer)
> BONUS: Works on-device w/ llama.cpp ⚡
Three-step approach to TTS:
> Audio tokenization using WavTokenizer (75 tok per second)
> CTC forced alignment for word-to-audio token mapping
> Structured prompt creation w/ transcription, duration, audio tokens
The model is extremely impressive for 350M parameters! Kudos to the
OuteAI team on such a brilliant feat - I'd love to see this be applied on larger data and smarter backbones like SmolLM 🤗
Check out the models here: OuteAI/outetts-6728aa71a53a076e4ba4817c
> Pure language modeling approach to TTS
> Zero-shot voice cloning
> LLaMa architecture w/ Audio tokens (WavTokenizer)
> BONUS: Works on-device w/ llama.cpp ⚡
Three-step approach to TTS:
> Audio tokenization using WavTokenizer (75 tok per second)
> CTC forced alignment for word-to-audio token mapping
> Structured prompt creation w/ transcription, duration, audio tokens
The model is extremely impressive for 350M parameters! Kudos to the
OuteAI team on such a brilliant feat - I'd love to see this be applied on larger data and smarter backbones like SmolLM 🤗
Check out the models here: OuteAI/outetts-6728aa71a53a076e4ba4817c
reach-vb
authored
a
paper
about 1 year ago
Post
3252
Smol models ftw! AMD released AMD OLMo 1B - beats OpenELM, tiny llama on MT Bench, Alpaca Eval - Apache 2.0 licensed 🔥
> Trained with 1.3 trillion (dolma 1.7) tokens on 16 nodes, each with 4 MI250 GPUs
> Three checkpoints:
- AMD OLMo 1B: Pre-trained model
- AMD OLMo 1B SFT: Supervised fine-tuned on Tulu V2, OpenHermes-2.5, WebInstructSub, and Code-Feedback datasets
- AMD OLMo 1B SFT DPO: Aligned with human preferences using Direct Preference Optimization (DPO) on UltraFeedback dataset
Key Insights:
> Pre-trained with less than half the tokens of OLMo-1B
> Post-training steps include two-phase SFT and DPO alignment
> Data for SFT:
- Phase 1: Tulu V2
- Phase 2: OpenHermes-2.5, WebInstructSub, and Code-Feedback
> Model checkpoints on the Hub & Integrated with Transformers ⚡️
Congratulations & kudos to AMD on a brilliant smol model release! 🤗
amd/amd-olmo-6723e7d04a49116d8ec95070
> Trained with 1.3 trillion (dolma 1.7) tokens on 16 nodes, each with 4 MI250 GPUs
> Three checkpoints:
- AMD OLMo 1B: Pre-trained model
- AMD OLMo 1B SFT: Supervised fine-tuned on Tulu V2, OpenHermes-2.5, WebInstructSub, and Code-Feedback datasets
- AMD OLMo 1B SFT DPO: Aligned with human preferences using Direct Preference Optimization (DPO) on UltraFeedback dataset
Key Insights:
> Pre-trained with less than half the tokens of OLMo-1B
> Post-training steps include two-phase SFT and DPO alignment
> Data for SFT:
- Phase 1: Tulu V2
- Phase 2: OpenHermes-2.5, WebInstructSub, and Code-Feedback
> Model checkpoints on the Hub & Integrated with Transformers ⚡️
Congratulations & kudos to AMD on a brilliant smol model release! 🤗
amd/amd-olmo-6723e7d04a49116d8ec95070
Post
2700
What a great day for Open Science!
@AIatMeta
released models, datasets, and code for many of its research artefacts! 🔥
1. Meta Segment Anything Model 2.1: An updated checkpoint with improved results on visually similar objects, small objects and occlusion handling. A new developer suite will be added to make it easier for developers to build with SAM 2.
Model checkpoints: reach-vb/sam-21-6702d40defe7611a8bafa881
2. Layer Skip: Inference code and fine-tuned checkpoints demonstrating a new method for enhancing LLM performance.
Model checkpoints: facebook/layerskip-666b25c50c8ae90e1965727a
3. SALSA: New code enables researchers to benchmark AI-based attacks to validate security for post-quantum cryptography.
Repo: https://github.com/facebookresearch/LWE-benchmarking
4. Meta Lingua: A lightweight and self-contained codebase designed to train language models at scale.
Repo: https://github.com/facebookresearch/lingua
5. Meta Open Materials: New open source models and the largest dataset to accelerate AI-driven discovery of new inorganic materials.
Model checkpoints: https://huggingface.co/fairchem/OMAT24
6. MEXMA: A new research paper and code for our novel pre-trained cross-lingual sentence encoder covering 80 languages.
Model checkpoint: facebook/MEXMA
7. Self-Taught Evaluator: a new method for generating synthetic preference data to train reward models without relying on human annotations.
Model checkpoint: facebook/Self-taught-evaluator-llama3.1-70B
8. Meta Spirit LM: An open-source language model for seamless speech and text integration.
Repo: https://github.com/facebookresearch/spiritlm
1. Meta Segment Anything Model 2.1: An updated checkpoint with improved results on visually similar objects, small objects and occlusion handling. A new developer suite will be added to make it easier for developers to build with SAM 2.
Model checkpoints: reach-vb/sam-21-6702d40defe7611a8bafa881
2. Layer Skip: Inference code and fine-tuned checkpoints demonstrating a new method for enhancing LLM performance.
Model checkpoints: facebook/layerskip-666b25c50c8ae90e1965727a
3. SALSA: New code enables researchers to benchmark AI-based attacks to validate security for post-quantum cryptography.
Repo: https://github.com/facebookresearch/LWE-benchmarking
4. Meta Lingua: A lightweight and self-contained codebase designed to train language models at scale.
Repo: https://github.com/facebookresearch/lingua
5. Meta Open Materials: New open source models and the largest dataset to accelerate AI-driven discovery of new inorganic materials.
Model checkpoints: https://huggingface.co/fairchem/OMAT24
6. MEXMA: A new research paper and code for our novel pre-trained cross-lingual sentence encoder covering 80 languages.
Model checkpoint: facebook/MEXMA
7. Self-Taught Evaluator: a new method for generating synthetic preference data to train reward models without relying on human annotations.
Model checkpoint: facebook/Self-taught-evaluator-llama3.1-70B
8. Meta Spirit LM: An open-source language model for seamless speech and text integration.
Repo: https://github.com/facebookresearch/spiritlm
- 3 replies