Claude Sonnet
All 1 published result of this model on the leaderboard — every version, method and dataset — with the publications they come from. Results on different datasets are not directly comparable.
- Version
- 4.0
- Approach
- Prompting
- Dataset
- Size
- 4.0: ~175B · Estimate of a related model (see “About model sizes” on the leaderboard)
- Best Top-1 (diagnosis) 74.30 % Chatbot default · HeadAI-pt-315
Publications
- https://doi.org/10.1186/s12883-025-04473-1 — 1 result on HeadAI-pt-315
Results
| Version · badges | Approach | Dataset | Top-1 (diagnosis) | Top-3 (diagnosis) | Top-1 (group) | Top-3 (group) | Cohen's Kappa | Links |
|---|---|---|---|---|---|---|---|---|
| 4.0 Peer-reviewed publication ↗ |
Chatbot default
Approach · Prompting Chatbot defaultThe public chatbot with its default settings, given the vignette as written. Prompting approachesA general-purpose language model diagnoses from the case text, steered only by the prompt: zero- or few-shot, chain-of-thought, or with diagnosis lists or ICHD-3 criteria pasted into the prompt.
|
External dataset · HeadAI-pt-315 Head.AI validation vignettes (original, Brazilian Portuguese)315 expert-written clinical headache vignettes from the Head.AI validation study. Results as reported in the paper: correct diagnosis as the first hypothesis. Top-3 is omitted because the rank counts in the paper do not sum to n. Results on this dataset are not directly comparable with results on a platform release or on another dataset.
|
74.30 % | — | — | — | — | Paper |