DELLA-Merging: Reducing Interference in Model Merging through Magnitude-Based Sampling
Paper
•
2406.11617
•
Published
•
8
Image-to-Text
Transformers
Safetensors
mistral3
mergekit
Merge
esper
shining-valiant
valiant
mistral
mistral-common
ministral-3-14b
ministral
reasoning
code
code-reasoning
code-instruct
python
javascript
dev-ops
jenkins
terraform
scripting
powershell
azure
aws
gcp
cloud
science
science-reasoning
physics
biology
chemistry
earth-science
astronomy
machine-learning
artificial-intelligence
compsci
computer-science
information-theory
ML-Ops
math
cuda
deep-learning
agentic
LLM
neuromorphic
self-improvement
complex-systems
cognition
linguistics
philosophy
logic
epistemology
simulation
game-theory
knowledge-management
creativity
problem-solving
architect
engineer
developer
creative
analytical
expert
rationality
conversational
chat
instruct
PlumEsper
This is a merge of pre-trained language models created using mergekit, combining the specialty and general reasoning skills of Esper 3.1 14b and Shining Valiant 3 14b.
Merge Details
Merge Method
This model was merged using the DELLA merge method using mistralai/Ministral-3-14B-Reasoning-2512 as a base.
Models Merged
The following models were included in the merge:
- ValiantLabs/Ministral-3-14B-Reasoning-2512-ShiningValiant3
- ValiantLabs/Ministral-3-14B-Reasoning-2512-Esper3.1
Configuration
The following YAML configuration was used to produce this model:
merge_method: della
dtype: bfloat16
parameters:
normalize: true
tokenizer:
source: "base"
models:
- model: ValiantLabs/Ministral-3-14B-Reasoning-2512-Esper3.1
parameters:
density: 0.5
weight: 0.8
- model: ValiantLabs/Ministral-3-14B-Reasoning-2512-ShiningValiant3
parameters:
density: 0.5
weight: 0.8
base_model: mistralai/Ministral-3-14B-Reasoning-2512
- Downloads last month
- 7
Model tree for sequelbox/Ministral-3-14B-Reasoning-2512-PlumEsper1.1
Merge model
this model