Records · kv:2610.00028v1

Reproduction of karpathy/llama2.c: 15M-parameter Llama 2 trained on TinyStories

K-Veritas Team

Submitted by K-Veritas Team

Published 2026-10-08

Independent reproduction · Benchmark or leaderboard entry · Natural language processing

Reproduction of: original code

Abstract

Trained the README's smallest Llama 2 configuration (dim 288, 6 layers, 6 heads, about 15M parameters, Llama 2 tokenizer) on the pretokenized TinyStories dataset at the pinned commit with train.py defaults (batch 128, 4 gradient accumulation steps, bf16, torch.compile, seed 1337) for 15,000 iterations instead of the README's 20,000 to fit the time budget; the learning-rate schedule follows the shorter run. The exported model was then sampled with the C inference program run.c (256 tokens, temperature 0.8, seed 42). One NVIDIA RTX 5060 Ti.

Sealed report

Cite this record

@misc{kveritas261000028v1,
  title={Reproduction of karpathy/llama2.c: 15M-parameter Llama 2 trained on TinyStories},
  author={K-Veritas Team},
  year={2026},
  howpublished={K-Veritas Records},
  note={kv:2610.00028v1},
  url={https://kveritas.org/records/2610.00028v1},
}

References

1. [1] Mamadou K. Keita and Christopher Homan. Computer Science Conferences Should Require Nonrepudiable Experimental Results. NeurIPS 2026 Position Paper Track. arXiv:2605.08586.

Sealed with K-Veritas [1]. Listed, not endorsed. A record shows these results came from the sealed code, unchanged. Whether the work is sound is for the reader to judge.