Published collection
SparDA uses Forecast projections to select next-layer KV blocks
This method targets the cost of KV cache selection in long-context reasoning and may improve sparse attention efficiency.
- Published entries
- 1
- Sources
- 1
- Date range
- 2026-08-12
- Primary labels
- Research · 论文 · Training Methods · SparDA · NVIDIA · DeepSeek
Published evidence
Every entry keeps its summary and a path back to the source context.
SparDA uses Forecast projections to select next-layer KV blocks
The post says NVIDIA researchers built SparDA, adding a Forecast projection beyond Q, K, and V to use the previous layer to predict the KV blocks needed by the next layer; this variant makes decoding 1.7x faster and improves long-reasoning accuracy by 6.5 points.
Original source