The author explores efficient SIMD-friendly unpacking of ternary numbers, aiming for the theoretical ideal bits per ternary digit. They define a “trit” as a ternary digit, and highlight the need for a block size that balances information density and hardware convenience. Surprisingly, 5 trits fit tightly into 8 bits at 1.6 bits per trit, achieving 99.06% efficiency. They introduce a method involving fixed-point numbers to easily extract digits when unpacking with SIMD, bypassing the need for modulo operations. The author’s technique has been tested and proven to be lossless, and will be implemented in the upcoming 1.625 bpw quant in llama.cpp for BitNet b1.58.
https://compilade.net/blog/ternary-packing