Skip to content

Request : Q6 GGUF quantization #431

Open
@Danamir

Description

@Danamir

Is there an easy way to add Q6_K quantization (or eventually Q5_K) to the converter ? I find this particular quantization to be a good compromise between Q8_0 and Q4_K on my system.

The weights actually supported are : f32, f16, q4_0, q4_1, q5_0, q5_1, q8_0, q2_k, q3_k, q4_k

Cheers,

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions