Getting Started

Instrument your script

K-Veritas records what your script prints. To capture a number, a stage, a seed, or a headline claim, print one line in a fixed format. You do not change how your model trains, and it works in any language.

How capture works

When you run kveritas run -- python train.py, K-Veritas watches your script's standard output. Any line that starts with a KVERITAS_ directive is parsed and bound into the signed report. Every other line prints exactly as before, so adding these directives never breaks your existing logs.

Print immediately. Use flush=True (Python) so each line is captured in the order it happened. Buffered output can arrive out of order or be lost if the run is interrupted.

You may not need any of these. Common frameworks are auto-detected, so metrics can be captured with no markers at all. See Auto-detected below. Use the directives when you want full control over exactly what is recorded.

KVERITAS_METRIC — record a number

The one you will use most. Print it every time you have a metric worth keeping.

KVERITAS_METRIC name=<id> value=<number> [step=<label>]

name	An identifier: starts with a letter, then letters, digits, or underscores. Example: val_accuracy, train_loss, f1.
value	A number: integer, decimal, or scientific (1e-4). Format floats yourself, e.g. {acc:.4f}.
step	Optional label for the point in time: an epoch index, a global step, 100, step_5. Same name across steps forms a trajectory.

print(f"KVERITAS_METRIC name=train_loss value={loss:.4f} step={epoch}", flush=True)
print(f"KVERITAS_METRIC name=val_accuracy value={acc:.4f} step={epoch}", flush=True)

Repeat the same name with different step values to record a curve; the final value is treated as the result for that metric.

KVERITAS_PHASE — mark a stage

Marks a boundary between stages of the run (for example data loading, training, evaluation). Each phase records a hardware snapshot and a provenance checkpoint, so the report shows a timeline of what happened when.

print("KVERITAS_PHASE name=train", flush=True)
print("KVERITAS_PHASE name=evaluate", flush=True)

KVERITAS_CLAIM — commit a headline number

A claim is the number you would put in a paper or a table, committed at the moment it was computed. A reviewer can later check the paper's claim against the sealed one.

print(f"KVERITAS_CLAIM metric=test_accuracy value={test_acc:.4f}", flush=True)

metric	The metric name the claim is about.
value	The committed number.
phase	Optional phase this claim belongs to.

KVERITAS_INPUT — commit a seed

Commits the random seed your run used, so the report states exactly which seed produced the result. Print it right after you set the seed.

SEED = 42
torch.manual_seed(SEED)
print(f"KVERITAS_INPUT src=seed:{SEED}", flush=True)

KVERITAS_MODEL and KVERITAS_WORKLOAD — declare the model card

These feed compute-cost attestation: given the model size and the workload, K-Veritas checks the declared work against physical limits. Print them once, near the start.

n_params = sum(p.numel() for p in model.parameters())
print(f"KVERITAS_MODEL params={n_params} arch=resnet50 precision=fp16", flush=True)
print(f"KVERITAS_WORKLOAD dataset_size={n_train} epochs={epochs} batch_size={batch}", flush=True)

KVERITAS_MODEL	params=<int> arch=<name> precision=<fp16\|bf16\|fp32>
KVERITAS_WORKLOAD	dataset_size=<int> epochs=<float> batch_size=<int> [seq_len=<int>]

KVERITAS_ARTIFACT — attest a model or dataset

Binds a file (a checkpoint, a dataset) to the report. visibility=public records a plain hash of its identity; visibility=private records a salted commitment that reveals nothing about the contents.

print("KVERITAS_ARTIFACT role=model name=resnet50 path=model.pt visibility=private", flush=True)
print("KVERITAS_ARTIFACT role=dataset name=imagenet path=data/ visibility=public", flush=True)

A small helper

To keep your training loop clean, wrap the metric line once and reuse it.

def kv_metric(name, value, step=None):
    line = f"KVERITAS_METRIC name={name} value={value:.6f}"
    if step is not None:
        line += f" step={step}"
    print(line, flush=True)

kv_metric("val_accuracy", 0.9142, step=epoch)

Framework snippets

PyTorch

import torch

SEED = 42
torch.manual_seed(SEED)
print(f"KVERITAS_INPUT src=seed:{SEED}", flush=True)

model = build_model()
n_params = sum(p.numel() for p in model.parameters())
print(f"KVERITAS_MODEL params={n_params} arch=resnet18 precision=fp32", flush=True)

print("KVERITAS_PHASE name=train", flush=True)
for epoch in range(epochs):
    train_loss = train_one_epoch(model, train_loader)
    val_acc = evaluate(model, val_loader)
    print(f"KVERITAS_METRIC name=train_loss value={train_loss:.4f} step={epoch}", flush=True)
    print(f"KVERITAS_METRIC name=val_accuracy value={val_acc:.4f} step={epoch}", flush=True)

print("KVERITAS_PHASE name=test", flush=True)
test_acc = evaluate(model, test_loader)
print(f"KVERITAS_CLAIM metric=test_accuracy value={test_acc:.4f}", flush=True)

Hugging Face Trainer

Add a callback that forwards the Trainer's eval metrics. Your datasets load as usual; nothing about them leaves your machine.

from transformers import Trainer, TrainerCallback

class KVeritas(TrainerCallback):
    def on_evaluate(self, args, state, control, metrics=None, **kwargs):
        for key, value in (metrics or {}).items():
            if isinstance(value, (int, float)):
                name = key.replace("eval_", "").replace("/", "_")
                print(f"KVERITAS_METRIC name={name} value={value:.6f} step={state.global_step}", flush=True)

trainer = Trainer(model=model, args=training_args, train_dataset=ds_train,
                  eval_dataset=ds_eval, callbacks=[KVeritas()])
trainer.train()

final = trainer.evaluate()
print(f"KVERITAS_CLAIM metric=accuracy value={final['eval_accuracy']:.6f}", flush=True)

TensorFlow / Keras

A one-method callback covers every logged metric. Keras model.history is also auto-detected, so this is optional if you only need the final values.

import tensorflow as tf

class KVeritas(tf.keras.callbacks.Callback):
    def on_epoch_end(self, epoch, logs=None):
        for name, value in (logs or {}).items():
            print(f"KVERITAS_METRIC name={name.replace('/', '_')} value={value:.6f} step={epoch}", flush=True)

model.fit(x_train, y_train, validation_data=(x_val, y_val),
          epochs=20, callbacks=[KVeritas()])

scikit-learn

Print the score after fitting. Cross-validation results from GridSearchCV and cross_validate are auto-detected.

from sklearn.metrics import accuracy_score

model.fit(X_train, y_train)
acc = accuracy_score(y_test, model.predict(X_test))
print(f"KVERITAS_METRIC name=test_accuracy value={acc:.4f}", flush=True)
print(f"KVERITAS_CLAIM metric=test_accuracy value={acc:.4f}", flush=True)

Other languages

The directives are plain text, so print them from anything.

# Shell
echo "KVERITAS_METRIC name=val_accuracy value=0.9471 step=100"

# R
cat(sprintf("KVERITAS_METRIC name=val_accuracy value=%.4f step=%d\n", acc, epoch))

# Julia
println("KVERITAS_METRIC name=val_accuracy value=$(acc) step=$(epoch)")

Auto-detected (no markers needed)

K-Veritas reads these automatically after your run, so simple experiments need no directives.

Keras	Every metric in model.history.history after fit().
scikit-learn	cv_results_ and best_score_ from grid search and cross-validation.
Local variables	Numbers left in metric-like variables (accuracy, loss, f1) at the end of the run.

Auto-detection covers the common path. For a specific step-by-step curve, an explicit claim, a seed, or a model card, use the directives above.

Run and seal

Once your script prints what you want, wrap it and seal a report.

kveritas init
kveritas run -- python train.py --epochs 50
kveritas seal --output report.pdf
kveritas verify report.pdf

See the Reference for the full command surface and the exact directive grammar.