MMLU-ITA

MMLU Italian is the localized version of the Massive Multitask Language Understanding benchmark, designed to evaluate language models across a wide range of academic and professional subjects — but in Italian. It includes over 50 topics, from STEM to humanities and social sciences, and offers a straightforward way to assess a model’s general knowledge and reasoning ability in Italian.

Vitruvian_Scientist-14B

74.50%

Vitruvian_Explainer-14B

74.40%

Qwen_2.5_14B

74.00%

Vitruvian_Smart-12B

67.10%

Mistral_small_22B

65.16%

Qwen2_7B

62.39%

Velvet_14B

58.78%

LLaMA_3.1_8B

58.43%

Fastweb-MIIA-7B

57.26%

Maestrale_Chat_v0.4_7B

56.27%

iGenius_Italia_9B

42.16%

Minerva_7B

39.30%

Question Example

1/3

Since the benchmark tasks are in Italian, both questions and answers are shown in their original language to preserve fidelity and meaning