Gemma 3 12B IT
All 5 published results of this model on the leaderboard — every version, method and dataset — with the publications they come from. Results on different datasets are not directly comparable.
- Version
- google/gemma-3-12b-it
- Approaches
- Prompting, Retrieval-augmented
- Dataset
- Size
- google/gemma-3-12b-it: ~12B · Disclosed by the developer (see “About model sizes” on the leaderboard)
Publications
- https://doi.org/10.3233/SHTI260922 — 5 results on Release 1.0
Results
Results marked Platform team were produced by the people who run this platform — disclosed because the same team curates the data and verifies results.
| Version · badges | Approach | Dataset | Top-1 (diagnosis) | Top-3 (diagnosis) | Top-1 (group) | Top-3 (group) | Cohen's Kappa | Links |
|---|---|---|---|---|---|---|---|---|
| google/gemma-3-12b-it Verified Peer-reviewed publication ↗ Disclosure Produced by the platform teamThis result was produced by people who also run this platform and published in their own paper. The same team curates the case data, moderates submissions and verifies results, so its own results carry a potential conflict of interest. They are flagged so you can weigh them accordingly and filter them out.
|
RAG (ICHD-3 retrieval)
Approach · Retrieval-augmented RAG (ICHD-3 retrieval)For each case, the most similar ICHD-3 diagnosis entries are retrieved by embedding similarity and added to the prompt; the model decides. Retrieval-augmented approachesFor each case, relevant ICHD-3 knowledge is retrieved from a text collection and added to the prompt; the language model decides.
|
Platform release Release 1.0A published, checksummed release of this platform's community cases, validated by two independent reviewers. About this resultEnglish variants (n=305) of release 1.0; vLLM, temperature 0 (Qwen3: 0.6), runs March 2026. Top-3 from a separate top-3 run; group = ICHD-3 top-level group of the first diagnosis (computed from the paper's run data).
|
55.41 % | 59.30 % | 89.51 % | — | — | Paper |
| google/gemma-3-12b-it Verified Peer-reviewed publication ↗ Disclosure Produced by the platform teamThis result was produced by people who also run this platform and published in their own paper. The same team curates the case data, moderates submissions and verifies results, so its own results carry a potential conflict of interest. They are flagged so you can weigh them accordingly and filter them out.
|
Few-shot
Approach · Prompting Few-shotCase text, candidate list and worked example cases in the prompt. Prompting approachesA general-purpose language model diagnoses from the case text, steered only by the prompt: zero- or few-shot, chain-of-thought, or with diagnosis lists or ICHD-3 criteria pasted into the prompt.
|
Platform release Release 1.0A published, checksummed release of this platform's community cases, validated by two independent reviewers. About this resultEnglish variants (n=305) of release 1.0; vLLM, temperature 0 (Qwen3: 0.6), runs March 2026. Top-3 from a separate top-3 run; group = ICHD-3 top-level group of the first diagnosis (computed from the paper's run data).
|
47.54 % | 52.80 % | 86.23 % | — | — | Paper |
| google/gemma-3-12b-it Verified Peer-reviewed publication ↗ Disclosure Produced by the platform teamThis result was produced by people who also run this platform and published in their own paper. The same team curates the case data, moderates submissions and verifies results, so its own results carry a potential conflict of interest. They are flagged so you can weigh them accordingly and filter them out.
|
ICHD-3 criteria in context
Approach · Prompting ICHD-3 criteria in contextThe full ICHD-3 ontology (simplified criteria, descriptions, clinical features) is placed in the system prompt. Prompting approachesA general-purpose language model diagnoses from the case text, steered only by the prompt: zero- or few-shot, chain-of-thought, or with diagnosis lists or ICHD-3 criteria pasted into the prompt.
|
Platform release Release 1.0A published, checksummed release of this platform's community cases, validated by two independent reviewers. About this resultEnglish variants (n=305) of release 1.0; vLLM, temperature 0 (Qwen3: 0.6), runs March 2026. Top-3 from a separate top-3 run; group = ICHD-3 top-level group of the first diagnosis (computed from the paper's run data).
|
45.90 % | 43.60 % | 81.31 % | — | — | Paper |
| google/gemma-3-12b-it Verified Peer-reviewed publication ↗ Disclosure Produced by the platform teamThis result was produced by people who also run this platform and published in their own paper. The same team curates the case data, moderates submissions and verifies results, so its own results carry a potential conflict of interest. They are flagged so you can weigh them accordingly and filter them out.
|
Chain-of-thought
Approach · Prompting Chain-of-thoughtCase text and candidate list; the model reasons step by step before naming the diagnosis. Prompting approachesA general-purpose language model diagnoses from the case text, steered only by the prompt: zero- or few-shot, chain-of-thought, or with diagnosis lists or ICHD-3 criteria pasted into the prompt.
|
Platform release Release 1.0A published, checksummed release of this platform's community cases, validated by two independent reviewers. About this resultEnglish variants (n=305) of release 1.0; vLLM, temperature 0 (Qwen3: 0.6), runs March 2026. Top-3 from a separate top-3 run; group = ICHD-3 top-level group of the first diagnosis (computed from the paper's run data).
|
45.57 % | 53.80 % | 86.23 % | — | — | Paper |
| google/gemma-3-12b-it Verified Peer-reviewed publication ↗ Disclosure Produced by the platform teamThis result was produced by people who also run this platform and published in their own paper. The same team curates the case data, moderates submissions and verifies results, so its own results carry a potential conflict of interest. They are flagged so you can weigh them accordingly and filter them out.
|
Zero-shot
Approach · Prompting Zero-shotCase text plus the list of candidate ICHD-3 codes; single best diagnosis, no examples. Prompting approachesA general-purpose language model diagnoses from the case text, steered only by the prompt: zero- or few-shot, chain-of-thought, or with diagnosis lists or ICHD-3 criteria pasted into the prompt.
|
Platform release Release 1.0A published, checksummed release of this platform's community cases, validated by two independent reviewers. About this resultEnglish variants (n=305) of release 1.0; vLLM, temperature 0 (Qwen3: 0.6), runs March 2026. Top-3 from a separate top-3 run; group = ICHD-3 top-level group of the first diagnosis (computed from the paper's run data).
|
42.30 % | — | 83.61 % | — | — | Paper |