s1ghhh/Llama-3-70b-Drop
Text Generation • 71B • Updated • 8 • 4
LLM-Drop
Resources for studies on redundancy in LLMs, including layer dropping and representation-hierarchy-based pruning.
-
-
s1ghhh/Llama-2-70b-Drop
Text Generation • Updated • 5 • 2 -
s1ghhh/Llama-2-13b-Drop8Attn
13B • Updated • 5 • 2 -
s1ghhh/Llama-2-13b-Drop4Attn
13B • Updated • 6 • 2 -
s1ghhh/Mistral-7B-v0.1-Drop8Attn
7B • Updated • 6 • 2 -
s1ghhh/Llama-2-13b-Drop8MLP
13B • Updated • 3 • 2 -
s1ghhh/Llama-2-13b-Drop8Block
13B • Updated • 5 • 2 -
s1ghhh/Llama-2-13b-Drop4MLP
13B • Updated • 5 • 2 -
s1ghhh/Llama-2-13b-Drop4Block
13B • Updated • 6 • 2 -
s1ghhh/Mistral-7B-v0.1-Drop4Attn
7B • Updated • 4 • 2 -
s1ghhh/Mistral-7B-v0.1-Drop4Block
7B • Updated • 4 • 2 -
s1ghhh/Mistral-7B-v0.1-Drop8MLP
7B • Updated • 8 • 2 -
s1ghhh/Mistral-7B-v0.1-Drop8Block
7B • Updated • 8 • 2 -
s1ghhh/Mistral-7B-v0.1-Drop4MLP
7B • Updated • 3 • 2 -
Demystifying When Pruning Works via Representation Hierarchies
Paper • 2603.24652 • Published • 20 -
What Matters in Transformers? Not All Attention is Needed
Paper • 2406.15786 • Published • 33