Records · kv:2610.00034v1

Reproduction of jiaweizzhao/GaLore: LLaMA-60M pre-training on C4 with GaLore

K-Veritas Team

Submitted by K-Veritas Team

Published 2026-10-11

Independent reproduction · Artifact evaluation · Machine learning

PaperReproduction of: original code

Abstract

Pre-trained LLaMA-60M on streamed C4 with the GaLore AdamW optimizer using the README command at the pinned commit (rank 128, scale 0.25, projection update every 200 steps, lr 0.01, bf16), for 2,000 steps instead of 10,000 to fit the time budget, with evaluation every 500 steps. Changes needed on this machine: micro-batch 64 with the same total batch of 512 (256 does not fit 16 GB), transformers 4.40.2 instead of the pinned 4.31 (no Python 3.12 build), and the validation set loaded as allenai/c4 instead of the retired c4 name. One NVIDIA RTX 5060 Ti.

Sealed report

Cite this record

@misc{kveritas261000034v1,
  title={Reproduction of jiaweizzhao/GaLore: LLaMA-60M pre-training on C4 with GaLore},
  author={K-Veritas Team},
  year={2026},
  howpublished={K-Veritas Records},
  note={kv:2610.00034v1},
  url={https://kveritas.org/records/2610.00034v1},
}

References

1. [1] Mamadou K. Keita and Christopher Homan. Computer Science Conferences Should Require Nonrepudiable Experimental Results. NeurIPS 2026 Position Paper Track. arXiv:2605.08586.

Sealed with K-Veritas [1]. Listed, not endorsed. A record shows these results came from the sealed code, unchanged. Whether the work is sound is for the reader to judge.