Records · kv:2610.00043v1

Reproduction of huggingface/open_asr_leaderboard: Whisper large-v3-turbo on the seven English test sets

K-Veritas Team

Submitted by K-Veritas Team

Published 2026-10-21

Independent reproduction · Artifact evaluation · Natural language processing

PaperReproduction of: original code

Abstract

Ran transformers/run_whisper.sh at the pinned commit, unchanged: openai/whisper-large-v3-turbo with batch size 64 on the full test splits of AMI, GigaSpeech, VoxPopuli, Earnings-22, LibriSpeech test-clean and test-other and SPGISpeech from hf-audio/open-asr-leaderboard, reporting WER and RTFx per set, then normalizer/eval_utils.score_results. The dataset and model weights were downloaded before the sealed run, which ran offline. Folders for the other toolkits (nemo_asr, espnet, ctranslate2 and so on) are withheld as hash-only entries since this run does not use them. PyTorch 2.11 (CUDA 12.8), transformers 5.19, datasets 5.1 with torchcodec for audio decoding. One NVIDIA RTX 5060 Ti.

Sealed report

Cite this record

@misc{kveritas261000043v1,
  title={Reproduction of huggingface/open_asr_leaderboard: Whisper large-v3-turbo on the seven English test sets},
  author={K-Veritas Team},
  year={2026},
  howpublished={K-Veritas Records},
  note={kv:2610.00043v1},
  url={https://kveritas.org/records/2610.00043v1},
}

References

1. [1] Mamadou K. Keita and Christopher Homan. Computer Science Conferences Should Require Nonrepudiable Experimental Results. NeurIPS 2026 Position Paper Track. arXiv:2605.08586.

Sealed with K-Veritas [1]. Listed, not endorsed. A record shows these results came from the sealed code, unchanged. Whether the work is sound is for the reader to judge.