Log your results¶

Print one line per result. Any language.

print("KVERITAS_METRIC name=val_accuracy value=0.9471 step=100", flush=True)

Every other line prints as usual.

Tip

Use flush=True in Python so lines arrive in order.

Lines¶

Line	Use
`KVERITAS_METRIC`	A number, optionally per step. The main one.
`KVERITAS_PHASE`	A stage boundary. Adds a snapshot and a hardware reading.
`KVERITAS_CLAIM`	A headline number, committed when computed.
`KVERITAS_INPUT`	The random seed.
`KVERITAS_MODEL`	Model size, architecture, precision. Feeds the compute certificate.
`KVERITAS_WORKLOAD`	Dataset size, epochs, batch. Feeds the compute certificate.
`KVERITAS_ARTIFACT`	A model or dataset file, by hash or commitment.

Full list: Output lines.

Metric¶

KVERITAS_METRIC name=<id> value=<number> [step=<label>]

Field	Rule
`name`	Letter first, then letters, digits, underscores. `val_accuracy`, `f1`.
`value`	Integer, decimal, or scientific (`1e-4`).
`step`	Optional. Epoch, global step, any label. Same name over steps makes a curve.

print(f"KVERITAS_METRIC name=train_loss value={loss:.4f} step={epoch}", flush=True)

The last value of a name is its final result.

Phase¶

print("KVERITAS_PHASE name=train", flush=True)
print("KVERITAS_PHASE name=evaluate", flush=True)

Marks a stage. The report shows when each one happened.

Claim¶

The number you would put in a paper.

print(f"KVERITAS_CLAIM metric=test_accuracy value={acc:.4f}", flush=True)

Field	Rule
`metric`	The metric the claim is about.
`value`	The committed number.
`phase`	Optional phase.

Seed¶

torch.manual_seed(42)
print("KVERITAS_INPUT src=seed:42", flush=True)

Model card¶

print(f"KVERITAS_MODEL params={n_params} arch=resnet50 precision=fp16", flush=True)
print(f"KVERITAS_WORKLOAD dataset_size={n_train} epochs={epochs} batch_size={batch}", flush=True)

Line	Fields
`KVERITAS_MODEL`	`params=<int> arch=<name> precision=<fp16\\|bf16\\|fp32>`
`KVERITAS_WORKLOAD`	`dataset_size=<int> epochs=<float> batch_size=<int> [seq_len=<int>]`

Used by the compute check.

Artifact¶

print("KVERITAS_ARTIFACT role=model name=resnet50 path=model.pt visibility=private", flush=True)
print("KVERITAS_ARTIFACT role=dataset name=mmlu path=data/mmlu.bin visibility=public", flush=True)

See Models and data.

Picked up automatically¶

Source	Captured
Keras	Every metric in `model.history.history` after `fit()`.
scikit-learn	`cv_results_` and `best_score_` from grid search and cross-validation.
Variables	Numbers left in metric-like names (`accuracy`, `loss`, `f1`) at the end.
Environment	`pip freeze` of the interpreter that ran the code. Its digest is always signed.

Examples¶

PyTorchHugging FaceKerasscikit-learnShell, R, Julia

torch.manual_seed(42)
print("KVERITAS_INPUT src=seed:42", flush=True)
n_params = sum(p.numel() for p in model.parameters())
print(f"KVERITAS_MODEL params={n_params} arch=resnet18 precision=fp32", flush=True)

print("KVERITAS_PHASE name=train", flush=True)
for epoch in range(epochs):
    loss = train_one_epoch(model, train_loader)
    acc = evaluate(model, val_loader)
    print(f"KVERITAS_METRIC name=train_loss value={loss:.4f} step={epoch}", flush=True)
    print(f"KVERITAS_METRIC name=val_accuracy value={acc:.4f} step={epoch}", flush=True)

print("KVERITAS_PHASE name=test", flush=True)
print(f"KVERITAS_CLAIM metric=test_accuracy value={evaluate(model, test_loader):.4f}", flush=True)

from transformers import TrainerCallback

class KVeritas(TrainerCallback):
    def on_evaluate(self, args, state, control, metrics=None, **kwargs):
        for key, value in (metrics or {}).items():
            if isinstance(value, (int, float)):
                name = key.replace("eval_", "").replace("/", "_")
                print(f"KVERITAS_METRIC name={name} value={value:.6f} step={state.global_step}", flush=True)

trainer = Trainer(model=model, args=args, train_dataset=train, eval_dataset=dev, callbacks=[KVeritas()])
trainer.train()
final = trainer.evaluate()
print(f"KVERITAS_CLAIM metric=accuracy value={final['eval_accuracy']:.6f}", flush=True)

class KVeritas(tf.keras.callbacks.Callback):
    def on_epoch_end(self, epoch, logs=None):
        for name, value in (logs or {}).items():
            print(f"KVERITAS_METRIC name={name.replace('/', '_')} value={value:.6f} step={epoch}", flush=True)

model.fit(x_train, y_train, validation_data=(x_val, y_val), epochs=20, callbacks=[KVeritas()])

model.fit(X_train, y_train)
acc = accuracy_score(y_test, model.predict(X_test))
print(f"KVERITAS_METRIC name=test_accuracy value={acc:.4f}", flush=True)
print(f"KVERITAS_CLAIM metric=test_accuracy value={acc:.4f}", flush=True)

# Shell
echo "KVERITAS_METRIC name=val_accuracy value=0.9471 step=100"
# R
cat(sprintf("KVERITAS_METRIC name=val_accuracy value=%.4f step=%d\n", acc, epoch))
# Julia
println("KVERITAS_METRIC name=val_accuracy value=$(acc) step=$(epoch)")

Results saved to a file¶

Read the file in the same run and print the lines. Never type values by hand.

#!/usr/bin/env bash
python evaluate.py --out results.json
python - <<'PY'
import json
r = json.load(open("results.json"))
print(f"KVERITAS_CLAIM metric=map value={r['map']}")
PY

kveritas run -- bash run.sh