| - dataset: |
| id: LocalLLaMA/typed-decisions |
| task_id: accuracy |
| value: 0.743 |
| date: '2026-10-08' |
| source: |
| url: https://huggingface.co/datasets/LocalLLaMA/typed-decisions |
| name: Typed Decisions |
| user: SeaWolf-AI |
| notes: 'General, zero-shot: never trained on the Typed Decisions train split, its workflows or its question schemas. Test split, 400 cases, 2,000 decisions, one request per case with the state and all five questions (README request shape), all answered, zero errors. Accuracy = agreement with the gold label; KL = KL(gold || prediction); Brier summed over options (scorer reproduces the README Uniform reference). One forward pass per question, no generated tokens.' |
| - dataset: |
| id: LocalLLaMA/typed-decisions |
| task_id: kl_from_gold |
| value: 0.204 |
| date: '2026-10-08' |
| source: |
| url: https://huggingface.co/datasets/LocalLLaMA/typed-decisions |
| name: Typed Decisions |
| user: SeaWolf-AI |
| notes: 'General, zero-shot: never trained on the Typed Decisions train split, its workflows or its question schemas. Test split, 400 cases, 2,000 decisions, one request per case with the state and all five questions (README request shape), all answered, zero errors. Accuracy = agreement with the gold label; KL = KL(gold || prediction); Brier summed over options (scorer reproduces the README Uniform reference). One forward pass per question, no generated tokens.' |
| - dataset: |
| id: LocalLLaMA/typed-decisions |
| task_id: brier |
| value: 0.097 |
| date: '2026-10-08' |
| source: |
| url: https://huggingface.co/datasets/LocalLLaMA/typed-decisions |
| name: Typed Decisions |
| user: SeaWolf-AI |
| notes: 'General, zero-shot: never trained on the Typed Decisions train split, its workflows or its question schemas. Test split, 400 cases, 2,000 decisions, one request per case with the state and all five questions (README request shape), all answered, zero errors. Accuracy = agreement with the gold label; KL = KL(gold || prediction); Brier summed over options (scorer reproduces the README Uniform reference). One forward pass per question, no generated tokens.' |
|
|
