You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is there an easy way to add Q6_K quantization (or eventually Q5_K) to the converter ? I find this particular quantization to be a good compromise between Q8_0 and Q4_K on my system.
The weights actually supported are : f32, f16, q4_0, q4_1, q5_0, q5_1, q8_0, q2_k, q3_k, q4_k