KVpop -- Key-Value Cache Compression with Predictive Online Pruning
Paper • 2607.05061 • Published • 18
Lukas
sirluk
AI & ML interests
None yet
Recent Activity
submitted a paper 2 days ago
KVpop -- Key-Value Cache Compression with Predictive Online Pruning upvoted a paper 28 days ago
On Subquadratic Architectures: From Applications to Principles commentedon a paper 4 months ago
Effective Distillation to Hybrid xLSTM ArchitecturesOrganizations
submitted a
paper to Daily Papers 2 days ago
upvoted a paper 28 days ago
commented a
paper 4 months ago
authored a
paper 4 months ago
submitted a
paper to Daily Papers 4 months ago
upvoted an article 4 months ago
Article
TiRex on the Edge
BobWue
• • 14 updated a
Space 8 months ago
published a
Space 8 months ago
commented on Efficient LLM Pretraining: Packed Sequences and Masked Attention 10 months ago
Can you maybe share a full script to reproduce this error?
upvoted an article about 1 year ago
Article
xLSTM-based time series model TiRex significantly outperforms competing models in forecasting accuracy
BobWue
• • 12 commented on Efficient LLM Pretraining: Packed Sequences and Masked Attention about 1 year ago
Yes, this sounds like the model just learns to not attend to earlier tokens when an eos token comes after. With a packed sequence mask you can enforce this explicitly by just masking out previous tokens.
commented on Efficient LLM Pretraining: Packed Sequences and Masked Attention about 1 year ago
Hey @shantanuagarwal , glad you enjoyed the article! Even though I havent tried it out myself you should be able to leverage pytorch flexattention api for this. Have a look at the tutorial here https://pytorch.org/blog/flexattention/. Section "Document Masking/Jagged Sequences" talks about these packed sequence masks.
upvoted a paper over 1 year ago
authored a
paper over 1 year ago
upvoted 2
papers over 1 year ago
published an article almost 2 years ago
Article
Efficient LLM Pretraining: Packed Sequences and Masked Attention
sirluk
• • 71 apply_chat_template method not working correctly for llama 3 tokenizer
3
#35 opened almost 2 years ago
by sirluk
liked a
dataset about 2 years ago
published an article over 2 years ago
Article
Multilabel Classification using Mistral-7B on a single GPU with quantization and LoRA
sirluk
• • 26