I knew there was a reason I "felt" something different about the Jamba 1.6 models. The large and mini versions are both bf16 🧐📝

refers to quantization for the LLM, apparently it's Google's invention. I've seen it described as a "truncated fp32" precision. Mostly a training thing I guess.

People say that most users, though, can't tell the difference between a model that's fp8 compared to fp16 or bf16. A bit like saying you can't tell the difference between a good mp3 and FLAC? lol. But in my experience that's not the case at all.