Records · kv:2610.00029v1

Reproduction of pytorch/torchtitan: Llama 3 debug-model pretraining on one GPU

K-Veritas Team

Submitted by K-Veritas Team

Published 2026-10-08

Independent reproduction · Artifact evaluation · Natural language processing

PaperReproduction of: original code

Abstract

Ran the README quick start at the pinned commit on a single GPU: run_train.sh with NGPU=1 and the shipped llama3_debugmodel recipe (small Llama 3 architecture, the c4_test sample data bundled in the repository, AdamW lr 8e-4 with linear decay), extended from 10 to 2,000 steps through a two-line recipe wrapper (kv_recipes.py) because recipe settings are no longer command-line options. PyTorch nightly 2.16.0.dev20261007 for CUDA 13.0, as the README requires a nightly build. One NVIDIA RTX 5060 Ti.

Sealed report

Cite this record

@misc{kveritas261000029v1,
  title={Reproduction of pytorch/torchtitan: Llama 3 debug-model pretraining on one GPU},
  author={K-Veritas Team},
  year={2026},
  howpublished={K-Veritas Records},
  note={kv:2610.00029v1},
  url={https://kveritas.org/records/2610.00029v1},
}

References

1. [1] Mamadou K. Keita and Christopher Homan. Computer Science Conferences Should Require Nonrepudiable Experimental Results. NeurIPS 2026 Position Paper Track. arXiv:2605.08586.

Sealed with K-Veritas [1]. Listed, not endorsed. A record shows these results came from the sealed code, unchanged. Whether the work is sound is for the reader to judge.