HMU-Bench

A BENCHMARK FOR OMNI MODELS

HMU-Bench

Music understanding.
From the whole to the detail.

A bilingual benchmark for hierarchical music understanding.
Full-track audio. Symbolic notation. Questions that go deeper.

Zheqi Dai*, Boyang Wang*, Zhiyue Wu, Yiyang Xu, Xinming Shi,
Ruibin Yuan, Xuerui Yang, Chao Yan, Qiuqiang Kong

AUDIO / SYMBOLIC●
TRACK01/ 92
0:00FULL TRACK3:28
ABC ANNOTATIONK:E  ·  M:4/4  ·  Q:143

Emaj7 → C♯m7 → F♯m7

Global structure → Local detail↗
92 tracksComplete musical pieces
460 questionsAudio · per language
1,029 questionsABC notation · per language
EN / 中文Aligned bilingual evaluation

01 / SELECTED EXAMPLES

Inside the benchmark.

Three audio pieces and twelve notation tasks.
The complete benchmark is available in the release.

0:003:28
COMPLETE TRACK
Music in context.

Every question uses the complete track. Local questions identify a specific passage.

Full-track ABC annotation Reference material

THE EVALUATION SETUP

One question.
The whole piece.

Audio questions range from global musical style and structure to local melody, harmony, and rhythm. The model receives the complete track for each question; timestamps and bar ranges locate the passage when needed.

ABC questions pair notation excerpts with natural-language questions. Reference answers, audio annotations, model responses, and compact judge scores support inspection and evaluation.

Read the evaluation protocol ↗

02 / REPORTED RESULTS

Results from the paper.

Tables 1 & 3 · arXiv:2610.15773
Mean judge score on a 0–4 scale.

Chinese and English evaluated separately

OPEN DATA

Take a closer look.

Questions, reference answers, prompts, model outputs,
and the full audio release.

Get the release ↗