Records · kv:2610.00014v1

Reproduction of LAION-AI/CLIP_benchmark: zero-shot classification of OpenCLIP ViT-B-32 (LAION-2B) on CIFAR-10, CIFAR-100 and STL-10

K-Veritas Team

Submitted by K-Veritas Team

Published 2026-10-07

Independent reproduction · Artifact evaluation · Computer vision

PaperReproduction of: original code

Abstract

Ran the clip_benchmark CLI from the pinned commit (installed editable) to evaluate the OpenCLIP ViT-B-32 model with the laion2b_s34b_b79k weights in zero-shot classification on the CIFAR-10, CIFAR-100 and STL-10 test sets, with the default English class names and prompt templates, batch size and AMP settings. Top-1, top-5 and mean per-class recall are read from the result JSON files the tool wrote during the sealed run. Datasets downloaded by torchvision and weights fetched from Hugging Face before the sealed run. Single NVIDIA RTX 5060 Ti 16 GB, PyTorch 2.11 cu128, Python 3.12.

Sealed report

Cite this record

@misc{kveritas261000014v1,
  title={Reproduction of LAION-AI/CLIP_benchmark: zero-shot classification of OpenCLIP ViT-B-32 (LAION-2B) on CIFAR-10, CIFAR-100 and STL-10},
  author={K-Veritas Team},
  year={2026},
  howpublished={K-Veritas Records},
  note={kv:2610.00014v1},
  url={https://kveritas.org/records/2610.00014v1},
}

References

1. [1] Mamadou K. Keita and Christopher Homan. Computer Science Conferences Should Require Nonrepudiable Experimental Results. NeurIPS 2026 Position Paper Track. arXiv:2605.08586.

Sealed with K-Veritas [1]. Listed, not endorsed. A record shows these results came from the sealed code, unchanged. Whether the work is sound is for the reader to judge.