Records · kv:2610.00017v1

Reproduction of meta-pytorch/torchtune: Qwen2.5-0.5B LoRA fine-tuning recipe

K-Veritas Team

Submitted by K-Veritas Team

Published 2026-10-07

Independent reproduction · Other · Natural language processing

Reproduction of: original code

Abstract

Ran the lora_finetune_single_device recipe with the shipped config qwen2_5/0.5B_lora_single_device at the pinned commit: LoRA (rank 32) fine-tuning of Qwen2.5-0.5B-Instruct on alpaca-cleaned for one epoch, bf16, batch 2 with 8 accumulation steps, seed 42; only the model and output paths were overridden. With the config's default learning rate (2e-3, no gradient clipping) the loss falls until the end of warmup at step 100, then diverges and does not recover. One NVIDIA RTX 5060 Ti.

Sealed report

Cite this record

@misc{kveritas261000017v1,
  title={Reproduction of meta-pytorch/torchtune: Qwen2.5-0.5B LoRA fine-tuning recipe},
  author={K-Veritas Team},
  year={2026},
  howpublished={K-Veritas Records},
  note={kv:2610.00017v1},
  url={https://kveritas.org/records/2610.00017v1},
}

References

1. [1] Mamadou K. Keita and Christopher Homan. Computer Science Conferences Should Require Nonrepudiable Experimental Results. NeurIPS 2026 Position Paper Track. arXiv:2605.08586.

Sealed with K-Veritas [1]. Listed, not endorsed. A record shows these results came from the sealed code, unchanged. Whether the work is sound is for the reader to judge.