Closed-source ASR models on the Open ASR leaderboards

๐Ÿ”’ = closed-source API evaluated in this run on a seeded random sample of 500 utterances per test set (SADA: 494 with available audio; Earnings22: full set, 341 chunks / 6 sessions), scored with each leaderboard's own normalizer and metric. Other rows: published leaderboard scores (full test sets). English average = mean of the 9 public test sets; models missing one of them have no average, as on the leaderboard. Click a column to sort. API code & parameters
* Extended normalization: applied identically to reference and prediction, on top of the official normalizer. It only removes writing-convention differences, never recognition errors. Arabic: annotation tags such as [ู…ูˆุณูŠู‚ู‰] removed; all Unicode punctuation/symbols removed (e.g. em dash); tatweel removed; digits converted to Arabic number words with case/dual forms unified to the colloquial form (e.g. ุณุจุนูˆู† โ†’ ุณุจุนูŠู†, ุงู„ูุงู† โ†’ ุงู„ููŠู†, ุชุณุนู…ุงุฆุฉ โ†’ ุชุณุน ู…ูŠู‡); ta marbuta ุฉ โ†’ ู‡; alif maqsura ู‰ โ†’ ูŠ; Maghrebi letters ฺจ โ†’ ู‚, ฺข โ†’ ู (like the official ฺค โ†’ ู); elongations collapsed (ู„ุงุงุง โ†’ ู„ุง); hesitation/laughter tokens (ุขุข, ุงู…ู…, ู‡ู‡ู‡ู‡, ู‡ุงู‡ุงู‡ุง), false-start fragments (a bare ุงู„ or ุง) and stutters (repeated words) removed; split/merged compounds (ุนุจุฏ ุงู„ู„ู‡ / ุนุจุฏุงู„ู„ู‡) not counted as errors (kaldialign merge_compounds, as the English leaderboard does). The MASC references always write final ุฉ as ู‡. English: colloquial forms unified (cause โ†’ because, kinda โ†’ kind of, gonna โ†’ going to, towards โ†’ toward, okay โ†’ ok, ...), Q1โ€“Q4 โ†’ 1stโ€“4th quarter, stutters ("I I I") removed. Other rows use the official normalizers only, so * rows are not a like-for-like comparison.