Records · kv:2610.00033v1

Reproduction of state-spaces/mamba: Mamba-370M zero-shot evaluation

K-Veritas Team

Submitted by K-Veritas Team

Published 2026-10-10

Independent reproduction · Artifact evaluation · Natural language processing

PaperReproduction of: original code

Abstract

Ran the README zero-shot evaluation for the released state-spaces/mamba-370m checkpoint with lm-evaluation-harness and the mamba_ssm backend, on the full LAMBADA, HellaSwag, PIQA, ARC-Easy, ARC-Challenge and WinoGrande sets, batch size 64. mamba_ssm and causal-conv1d were compiled at the pinned commits for sm_120 with CUDA 12.8 (MAMBA_KEEP_CUDA_BUILD=TRUE for the Mamba-1 kernels); apache-tvm-ffi was pinned to 0.1.9 so the bundled tilelang imports. One NVIDIA RTX 5060 Ti.

Sealed report

Cite this record

@misc{kveritas261000033v1,
  title={Reproduction of state-spaces/mamba: Mamba-370M zero-shot evaluation},
  author={K-Veritas Team},
  year={2026},
  howpublished={K-Veritas Records},
  note={kv:2610.00033v1},
  url={https://kveritas.org/records/2610.00033v1},
}

References

1. [1] Mamadou K. Keita and Christopher Homan. Computer Science Conferences Should Require Nonrepudiable Experimental Results. NeurIPS 2026 Position Paper Track. arXiv:2605.08586.

Sealed with K-Veritas [1]. Listed, not endorsed. A record shows these results came from the sealed code, unchanged. Whether the work is sound is for the reader to judge.