Skip to content

Commit ceff347

Browse files
committed
updating LMFormatEnforcer metrics
1 parent d1044cc commit ceff347

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ Benchmark various LLM Structured Output frameworks: Instructor, Mirascope, Langc
2424
| [Mirascope](https://github.com/mirascope/mirascope) | gpt-3.5-turbo-0125 | 0.995 | 1.280 |
2525
| [Instructor](https://github.com/jxnl/instructor) | gpt-3.5-turbo-0125 | 0.992 | 1.299 |
2626
| [Marvin](https://github.com/PrefectHQ/marvin) | gpt-3.5-turbo-0125 | 0.971 | 1.151 |
27+
| [LMFormatEnforcer](https://github.com/noamgat/lm-format-enforcer) | unsloth/llama-3-8b-Instruct-bnb-4bit | 0.950 | 7.248 |
2728

2829
## 🧪 Benchmark methodology
2930

@@ -99,7 +100,7 @@ The easiest way to create a new framework is to reference the `./frameworks/inst
99100
| [Outlines](https://github.com/outlines-dev/outlines) | ✅ HF Transformers | 💭 Planning | 💭 Planning |
100101
| [LM format enforcer](https://github.com/noamgat/lm-format-enforcer) | ✅ HF Transformers | 💭 Planning | 💭 Planning |
101102
| [Jsonformer](https://github.com/1rgs/jsonformer) | ❌ No Enum Support | 💭 Planning | 💭 Planning |
102-
| [Strictjson](https://github.com/tanchongmin/strictjson) |No Pydantic Support | 💭 Planning | 💭 Planning |
103+
| [Strictjson](https://github.com/tanchongmin/strictjson) |Non-standard schema | 💭 Planning | 💭 Planning |
103104
| [Guidance](https://github.com/guidance-ai/guidance) | 🚧 In Progress | 💭 Planning | 💭 Planning |
104105
| [DsPy](https://dspy-docs.vercel.app/docs/building-blocks/typed_predictors) | 🚧 In Progress | 💭 Planning | 💭 Planning |
105106
| [Langchain](https://python.langchain.com/v0.2/docs/tutorials/extraction/) | 🚧 In Progress | 💭 Planning | 💭 Planning |

0 commit comments

Comments
 (0)