pycsamt.format.provenance#

Machine-learning-model provenance for a PCSF/PCSM AI-inversion result.

This is deliberately distinct from pycsamt.metadata.ProvenanceMeta, which describes survey/data provenance (who collected/submitted an EM dataset). ModelProvenance describes the model that produced a resistivity result – its architecture, training framework, checkpoint, and hyperparameters – so a shared .pcsf/.pcsm file is independently checkable by anyone re-running the model, not just re-readable.

Every field is optional free text: this is meant to be usable by literally any AI/DL inversion tool, including third-party ones with no dependency on pycsamt at all, so nothing here is enforced beyond basic typing.

Functions

compute_checkpoint_hash(path, *[, chunk_size])

SHA-256 hex digest of a model-checkpoint file, streamed in chunks.

Classes

ModelProvenance([architecture, framework, ...])

Describe the AI/DL model that produced a resistivity result.
class pycsamt.format.provenance.ModelProvenance(architecture='', framework='', framework_version='', checkpoint='', checkpoint_sha256='', training_data='', hyperparameters=<factory>, random_seed=None, git_commit='', authors=<factory>, contact='', notes='', extra=<factory>)[source]

Bases: PyCSAMTObject

Describe the AI/DL model that produced a resistivity result.

Parameters:

  • architecture (str, optional) – Free-text model family/architecture, e.g. "UNet", "GCN", "ResNet18", or any third-party name.

  • framework (str, optional) – e.g. "pytorch"/"2.3.0", "tensorflow"/"2.16.1".

  • framework_version (str, optional) – e.g. "pytorch"/"2.3.0", "tensorflow"/"2.16.1".

  • checkpoint (str, optional) – Path or identifier of the trained weights used to produce the result (e.g. a filename, a model-hub id, a DOI).

  • checkpoint_sha256 (str, optional) – SHA-256 hex digest of the checkpoint file, so a reader can verify they are re-running the exact weights this result claims – see compute_checkpoint_hash().

  • training_data (str, optional) – Free-text reference to the training dataset (name, DOI, path).

  • hyperparameters (dict, default {}) – Free-form training/model hyperparameters.

  • random_seed (int, optional) – Seed used for training and/or inference, when reproducibility depends on it.

  • git_commit (str, optional) – Commit hash of the code that produced this result.

  • authors (list of str, default []) – Model authors/maintainers.

  • contact (str, optional) – Contact e-mail or URL for questions about this result.

  • notes (str, optional) – Free-text notes not covered by the fields above.

  • extra (dict, default {}) – Unmodelled provenance fields, retained losslessly.

Examples

>>> from pycsamt.format.provenance import ModelProvenance
>>> prov = ModelProvenance(
...     architecture="UNet",
...     framework="pytorch",
...     framework_version="2.3.0",
...     checkpoint="unet_v3.pt",
...     random_seed=42,
... )
>>> prov.to_dict()["architecture"]
'UNet'
architecture: str = ''
framework: str = ''
framework_version: str = ''
checkpoint: str = ''
checkpoint_sha256: str = ''
training_data: str = ''
hyperparameters: dict[str, Any]
random_seed: int | None = None
git_commit: str = ''
authors: list[str]
contact: str = ''
notes: str = ''
extra: dict[str, Any]
validate()[source]

Validate object state.

Subclasses can override this hook. The base implementation intentionally accepts all states.

Return type:

None

to_dict()[source]

Plain-dict form, suitable for PCSFModel.metadata['model_provenance'].

Return type:

dict[str, Any]

pycsamt.format.provenance.compute_checkpoint_hash(path, *, chunk_size=1048576)[source]

SHA-256 hex digest of a model-checkpoint file, streamed in chunks.

Parameters:

  • path (path-like) – The checkpoint file to hash (e.g. a .pt/.h5/.onnx file).

  • chunk_size (int, default 1 MiB) – Read block size; large checkpoints are hashed without loading the whole file into memory.

Returns:

Lowercase hex digest, directly comparable to ModelProvenance.checkpoint_sha256.

Return type:

str

Examples

>>> from pycsamt.format.provenance import compute_checkpoint_hash
>>> compute_checkpoint_hash("unet_v3.pt")
'3b1c...'