Skip Navigation
[Paper] The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
  • They said their's is "comparable with the 8-bit models". Its all tradeoffs. It isn't clear to me where you allocate your compute/memory budget. I've noticed that full 7b 16 bit models often produce better results for me than some much larger quantied models. It will be interesting to find the sweet spot.

  • Afroman - Lemon Pound Cake
    1
    Spaghetti Again
    0
    Why privacy matters.
    odysee.com This will change the fabric of society...

    Privacy is assaulted in broad daylight and everything we do is recorded with an unforgiving permanence. We trade away our private data for services and conveniences -- not realizing that the governmen...

    This will change the fabric of society...
    0
    InitialsDiceBearhttps://github.com/dicebear/dicebearhttps://creativecommons.org/publicdomain/zero/1.0/„Initials” (https://github.com/dicebear/dicebear) by „DiceBear”, licensed under „CC0 1.0” (https://creativecommons.org/publicdomain/zero/1.0/)TH
    Throwaway4669332255 @lemmy.world
    Posts 23
    Comments 51