Skip to main content

Questions tagged [quantization]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers

Is a quantized Mistral LLM slower than non-quantized Mistral LLM?

For reference, I've been playing around with Mistral 7B v0.1 and v0.3 but did not like that I was limited by A100 availability on google colab. so I wanted to try 4 bit and 8 bit quantized models. but ...
user23971971's user avatar
0 votes
2 answers

Does 1-bit quantization (layers with boolean tensors) machine learning exist?

Does 1-bit quantization machine learning exist? Pytorch's docs on "Quantization" define it as: techniques for performing computations and storing tensors at lower bitwidths than floating ...
Geremia's user avatar
  • 205