Efficient Knowledge Distillation for LLMs: Offline Top-K Logits and a Fused Chunked KL Loss
Paper • 2608.03796 • Published • 13
Iñigo Pikabea
inigopm
AI & ML interests
None yet
Recent Activity
upvoted an article 2 days ago
Making Knowledge Distillation Cheap Enough to Run at Scale updated a dataset 3 months ago
inigopm/RealWorldQA-IberOrganizations
None yet
upvoted a paper 2 days ago
upvoted an article 2 days ago
Article
Making Knowledge Distillation Cheap Enough to Run at Scale
MultiverseComputingCAI
• • 25 updated a
dataset 3 months ago
published a
dataset 3 months ago
updated a
dataset 4 months ago
published a
dataset 4 months ago
liked a
model 8 months ago
updated a
model 8 months ago
updated a
collection 8 months ago
liked a
dataset 9 months ago
published a
model 9 months ago
updated a
model 10 months ago
updated a
dataset 11 months ago
authored 2
papers 12 months ago
upvoted a paper 12 months ago
upvoted an article about 1 year ago
Article
SmolLM3: smol, multilingual, long-context reasoner
- +21
eliebak, cmpatino, anton-l, edbeeching, m-ric, nouamanetazi, akseljoonas, guipenedo, hynky, clefourrier, SaylorTwift, kashif, qgallouedec, hlarcher, glutamatt, Xenova, reach-vb, ngxson, craffel, lewtun, loubnabnl, lvwerra, thomwolf
• • 788 liked a
model about 1 year ago
published a
model about 1 year ago
updated a
collection about 1 year ago