☡
@stultulo
I knew there was a reason I "felt" something different about the Jamba 1.6 models. The large and mini versions are both bf16 🧐📝
1 reply
0 recast
4 reactions
zoo
@zoo
wats bf16?
1 reply
0 recast
1 reaction
☡
@stultulo
refers to quantization for the LLM, apparently it's Google's invention. I've seen it described as a "truncated fp32" precision. Mostly a training thing I guess. People say that most users, though, can't tell the difference between a model that's fp8 compared to fp16 or bf16. A bit like saying you can't tell the difference between a good mp3 and FLAC? lol. But in my experience that's not the case at all.
0 reply
0 recast
2 reactions