Records · kv:2610.00006v1

Reproduction of karpathy/llm.c: GPT-2 124M fine-tuning on tinyshakespeare in CUDA

K-Veritas Team

Submitted by K-Veritas Team

Published 2026-10-07

Independent reproduction · Benchmark or leaderboard entry · Natural language processing

Reproduction of: original code

Abstract

Ran the llm.c quick start from the pinned commit: downloaded the starter pack (GPT-2 124M bf16 weights, GPT-2 tokenizer, tokenized tinyshakespeare) with dev/download_starter_pack.sh, built train_gpt2cu with make (bf16, no cuDNN, single GPU, compute capability 12.0, CUDA 12.8 toolkit from conda-forge), and ran ./train_gpt2cu with its default settings: one epoch (74 steps) of fine-tuning at B=4, T=1024, AdamW lr 3e-4, with validation loss every 20 steps and sample generations. Training loss per step, validation loss, average iteration time and throughput are parsed from the program output. Single NVIDIA RTX 5060 Ti 16 GB.

Sealed report

Cite this record

@misc{kveritas261000006v1,
  title={Reproduction of karpathy/llm.c: GPT-2 124M fine-tuning on tinyshakespeare in CUDA},
  author={K-Veritas Team},
  year={2026},
  howpublished={K-Veritas Records},
  note={kv:2610.00006v1},
  url={https://kveritas.org/records/2610.00006v1},
}

References

1. [1] Mamadou K. Keita and Christopher Homan. Computer Science Conferences Should Require Nonrepudiable Experimental Results. NeurIPS 2026 Position Paper Track. arXiv:2605.08586.

Sealed with K-Veritas [1]. Listed, not endorsed. A record shows these results came from the sealed code, unchanged. Whether the work is sound is for the reader to judge.