inference-optimization/Llama-3.1-8B-Instruct-QKV-Cache-FP8-Per-Head
8B
•
Updated
•
37
KV Cache Quantization
Collection on FP8 Quantization of Weights, Activations and KV Cache
-
-
inference-optimization/Llama-3.1-8B-Instruct-QKV-Cache-FP8-Per-Tensor
8B • Updated • 19 -
inference-optimization/Llama-3.1-8B-Instruct-FP8-dynamic-QKV-Cache-FP8-Per-Head
8B • Updated • 24 -
inference-optimization/Llama-3.1-8B-Instruct-FP8-dynamic-QKV-Cache-FP8-Per-Tensor
8B • Updated • 14 -
inference-optimization/Llama-3.3-70B-Instruct-QKV-Cache-FP8-Per-Head
71B • Updated • 2 -
inference-optimization/Llama-3.3-70B-Instruct-QKV-Cache-FP8-Per-Tensor
71B • Updated • 2 -
inference-optimization/Llama-3.3-70B-Instruct-FP8-dynamic-QKV-Cache-FP8-Per-Head
71B • Updated • 1 -
inference-optimization/Llama-3.3-70B-Instruct-FP8-dynamic-QKV-Cache-FP8-Per-Tensor
71B • Updated • 4 -
inference-optimization/Qwen3-32B-QKV-Cache-FP8-Per-Head
33B • Updated • 1 -
inference-optimization/Qwen3-32B-QKV-Cache-FP8-Per-Tensor
33B • Updated • 2 -
inference-optimization/Qwen3-32B-FP8-dynamic-QKV-Cache-FP8-Per-Head
33B • Updated • 1 -
inference-optimization/Qwen3-32B-FP8-dynamic-QKV-Cache-FP8-Per-Tensor
33B • Updated • 2