Article

BitDecoding: Unlocking Tensor Cores for Long-Context LLMs Decoding with Low-Bit KV Cache
[HPCA 2026] BitDecoding is a GPU-optimized framework that enhances decoding efficiency for long-context LLMs by unlocking Tensor Cores with low-bit KV cache, achieving significant speedups and reducing latency in long-context generation scenarios.
Dayou Du, Shijie Cao, Jianyi Cheng, Luo Mai, Ting Cao, Mao Yang
PDF Code
BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillation
[ACL 2024] BitDistiller synergizes Quantization-Aware Training and Knowledge Distillation, significantly enhancing sub-4-bit LLMs’ performance on complex tasks with reduced data and training resource demands.
Dayou Du, Yijia Zhang, Shijie Cao, Jiaqi Guo, Ting Cao, Xiaowen Chu, Ningyi Xu
PDF Code