Skip to content

cli: Remove type restrictions #489

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Nov 30, 2024
Merged

cli: Remove type restrictions #489

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
fdc466e
cli: allow any wtype
stduhpf Nov 28, 2024
d1392d6
cli: ignore deprecated types
stduhpf Nov 28, 2024
f52e9c5
cli: capitalize "K" in K quants names
stduhpf Nov 28, 2024
d24116a
cli: ignore internal (invalid) types
stduhpf Nov 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 19 additions & 24 deletions examples/cli/main.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -346,30 +346,25 @@ void parse_args(int argc, const char** argv, SDParams& params) {
invalid_arg = true;
break;
}
std::string type = argv[i];
if (type == "f32") {
params.wtype = SD_TYPE_F32;
} else if (type == "f16") {
params.wtype = SD_TYPE_F16;
} else if (type == "q4_0") {
params.wtype = SD_TYPE_Q4_0;
} else if (type == "q4_1") {
params.wtype = SD_TYPE_Q4_1;
} else if (type == "q5_0") {
params.wtype = SD_TYPE_Q5_0;
} else if (type == "q5_1") {
params.wtype = SD_TYPE_Q5_1;
} else if (type == "q8_0") {
params.wtype = SD_TYPE_Q8_0;
} else if (type == "q2_k") {
params.wtype = SD_TYPE_Q2_K;
} else if (type == "q3_k") {
params.wtype = SD_TYPE_Q3_K;
} else if (type == "q4_k") {
params.wtype = SD_TYPE_Q4_K;
} else {
fprintf(stderr, "error: invalid weight format %s, must be one of [f32, f16, q4_0, q4_1, q5_0, q5_1, q8_0, q2_k, q3_k, q4_k]\n",
type.c_str());
std::string type = argv[i];
bool found = false;
std::string valid_types = "";
for (size_t i = 0; i < SD_TYPE_COUNT; i++) {
auto trait = ggml_get_type_traits((ggml_type)i);
std::string name(trait->type_name);
if (i)
valid_types += ", ";
valid_types += name;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the type enum is non contiguous, so we check for something here, maybe empty name?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch

if (type == name) {
params.wtype = (enum sd_type_t)i;
found = true;
break;
}
}
if (!found) {
fprintf(stderr, "error: invalid weight format %s, must be one of [%s]\n",
type.c_str(),
valid_types.c_str());
exit(1);
}
} else if (arg == "--lora-model-dir") {
Expand Down
Loading