The Era of 1-bit LLMs: ternary parameters for cost-effective computing

1 crossposts

Technology @lemmy.ml ☆ Yσɠƚԋσʂ ☆ @lemmy.ml 4 mo. ago

1-bit LLM performs similarly to full-precision Transformer LLMs with the same model size and training tokens but is much more efficient in terms of latency, memory, throughput, and energy consumption.

arxiv.org /abs/2402.17764