2.25.3.6. pycsamt.ai.experiments#

Reproducible experiment configuration, seed plans, dataset references, and acceptance-gate records.

Reproducible experiment configuration and artifact provenance.

Experiment records will capture configuration hashes, seeds, realization splits, frequency and depth grids, normalisation state, software versions, metrics, and artifact checksums. Runtime outputs and large artifacts are not package resources and must remain outside source control.

class pycsamt.ai.experiments.SeedPlan(root_seed, namespace='pycsamt.ai')#

Bases: object

Derive stable, labeled child seeds from one experiment root seed.

Parameters:

  • root_seed (int) – Non-negative root seed smaller than 2**64.

  • namespace (str, default="pycsamt.ai") – Domain separator preventing identical labels in unrelated projects from producing the same child sequence.

Examples

Child seeds are label-stable and request-order independent:

>>> plan = SeedPlan(42, namespace="willy")
>>> plan.derive("dataset") == plan.derive("dataset")
True
>>> plan.derive("dataset") != plan.derive("network")
True
root_seed: int#
namespace: str = 'pycsamt.ai'#
derive(label)#

Derive one unsigned 32-bit seed for a named subsystem.

Parameters:

label (str) – Stable descriptive label such as "geology", "noise", or "network/seed-0".

Returns:

Deterministic value in [0, 2**32).

Return type:

int

Raises:

ValueError – If label is empty.

Examples

>>> child = SeedPlan(1).derive("training")
>>> 0 <= child < 2**32
True
derive_many(labels)#

Derive seeds for multiple unique labels.

Parameters:

labels (sequence of str) – Unique non-empty subsystem labels.

Returns:

Labels in caller order mapped to stable child seeds.

Return type:

dict

Examples

>>> sorted(SeedPlan(2).derive_many(["data", "model"]))
['data', 'model']
to_dict()#

Return a JSON-serializable seed plan.

Returns:

Schema version, root seed, and namespace.

Return type:

dict

Examples

>>> SeedPlan(3).to_dict()["root_seed"]
3
classmethod from_dict(data)#

Restore a validated seed plan.

Parameters:

data (mapping) – State returned by to_dict().

Returns:

Immutable seed plan.

Return type:

SeedPlan

Examples

>>> SeedPlan.from_dict(SeedPlan(4).to_dict()) == SeedPlan(4)
True
class pycsamt.ai.experiments.DatasetReference(dataset_id, manifest_hash, split_hash, normalization_hash=None, manifest_uri=None)#

Bases: object

Pin an experiment to exact dataset preparation artifacts.

Parameters:

  • dataset_id (str) – Portable identifier from the dataset manifest.

  • manifest_hash (str) – SHA-256 digests of the complete dataset manifest and realization split.

  • split_hash (str) – SHA-256 digests of the complete dataset manifest and realization split.

  • normalization_hash (str or None, optional) – Digest of fitted normalization state. It may be absent for an audit or forward-only experiment that has not normalized data.

  • manifest_uri (str or None, optional) – Informational local path or remote URI. Integrity relies on the hash, not on this location.

Examples

>>> reference = DatasetReference("willy-v1", "a" * 64, "b" * 64)
>>> reference.dataset_id
'willy-v1'
dataset_id: str#
manifest_hash: str#
split_hash: str#
normalization_hash: str | None = None#
manifest_uri: str | None = None#
to_dict()#

Return a JSON-serializable artifact reference.

Returns:

Schema version and pinned artifact identifiers.

Return type:

dict

Examples

>>> DatasetReference("d", "a" * 64, "b" * 64).to_dict()[
...     "schema_version"
... ]
1
classmethod from_dict(data)#

Restore a validated dataset reference.

Parameters:

data (mapping) – State returned by to_dict().

Returns:

Immutable pinned reference.

Return type:

DatasetReference

Examples

>>> ref = DatasetReference("d", "a" * 64, "b" * 64)
>>> DatasetReference.from_dict(ref.to_dict()) == ref
True
class pycsamt.ai.experiments.AcceptanceCriterion(metric, operator, threshold, description='')#

Bases: object

Predeclare one numerical condition required for an experiment gate.

Parameters:

  • metric (str) – Exact metric key, preferably namespaced, for example "test.impedance_nrms".

  • operator ({"<", "<=", ">", ">=", "=="}) – Comparison applied as observed operator threshold.

  • threshold (float) – Finite value fixed before results are inspected.

  • description (str, optional) – Human-readable scientific justification.

Examples

>>> criterion = AcceptanceCriterion("test.impedance_nrms", "<=", 2.0)
>>> criterion.evaluate(1.7)
True
>>> criterion.evaluate(2.5)
False
metric: str#
operator: str#
threshold: float#
description: str = ''#
evaluate(observed)#

Evaluate an observed metric against the frozen threshold.

Parameters:

observed (float) – Finite measured value.

Returns:

Result of observed operator threshold.

Return type:

bool

Raises:

ValueError – If observed is NaN or infinite.

Examples

>>> AcceptanceCriterion("coverage", ">=", 0.9).evaluate(0.95)
True
to_dict()#

Return a JSON-serializable criterion.

Returns:

Metric, comparison, threshold, and description.

Return type:

dict

Examples

>>> AcceptanceCriterion("x", "<", 1).to_dict()["operator"]
'<'
classmethod from_dict(data)#

Restore a validated acceptance criterion.

Parameters:

data (mapping) – State returned by to_dict().

Returns:

Immutable numerical gate condition.

Return type:

AcceptanceCriterion

Examples

>>> c = AcceptanceCriterion("x", ">", 0)
>>> AcceptanceCriterion.from_dict(c.to_dict()) == c
True
class pycsamt.ai.experiments.GateEvaluation(passed, criteria, observed, missing=(), complete=True)#

Bases: object

Immutable result of evaluating configured acceptance criteria.

Parameters:

  • passed (bool) – Whether every criterion passed and no required metric was missing.

  • criteria (mapping of str to bool) – Per-metric pass/fail results.

  • observed (mapping of str to float) – Finite observed values for metrics that were available.

  • missing (sequence of str) – Required metric keys absent from the supplied result mapping.

  • complete (bool, default=True) – Whether every configured criterion was evaluated. A partial status report can never pass the final gate.

Examples

>>> result = GateEvaluation(True, {"nrms": True}, {"nrms": 1.2}, ())
>>> result.failed_metrics
()
passed: bool#
criteria: Mapping[str, bool]#
observed: Mapping[str, float]#
missing: tuple[str, ...] = ()#
complete: bool = True#
property failed_metrics: tuple[str, ...]#

Return metrics that failed their configured comparisons.

Returns:

Failed available metrics; missing metrics are reported separately.

Return type:

tuple of str

Examples

>>> GateEvaluation(False, {"x": False}, {"x": 2.0}).failed_metrics
('x',)
to_dict()#

Return a JSON-serializable gate result.

Returns:

Overall and per-metric outcomes.

Return type:

dict

Examples

>>> GateEvaluation(True, {"x": True}, {"x": 0.5}).to_dict()["passed"]
True
class pycsamt.ai.experiments.ExperimentConfig(experiment_id, stage, dataset, seeds, model, training, physics=<factory>, acceptance=(), description='', tags=(), created_utc=None, schema_version=1)#

Bases: object

Immutable source of truth for one reproducible inversion experiment.

Parameters:

  • experiment_id (str) – Portable unique experiment identifier.

  • stage (str) – Roadmap stage such as "baseline", "forward_2d", or "field_evaluation".

  • dataset (DatasetReference) – Exact dataset, split, and optional normalization artifacts.

  • seeds (SeedPlan) – Root and namespace used for labeled child seeds.

  • model (mapping) – Finite JSON-compatible configuration sections. Their schemas are owned by later model/trainer/solver adapters; this class freezes and hashes them without importing optional dependencies.

  • training (mapping) – Finite JSON-compatible configuration sections. Their schemas are owned by later model/trainer/solver adapters; this class freezes and hashes them without importing optional dependencies.

  • physics (mapping) – Finite JSON-compatible configuration sections. Their schemas are owned by later model/trainer/solver adapters; this class freezes and hashes them without importing optional dependencies.

  • acceptance (sequence of AcceptanceCriterion, optional) – Criteria fixed before results are viewed. Metric names must be unique.

  • description (str, optional) – Human-readable experiment objective.

  • tags (sequence of str, optional) – Unique searchable labels.

  • created_utc (str or None, optional) – Timezone-aware ISO-8601 timestamp normalized to UTC.

  • schema_version (int, default=1) – Configuration schema version.

Examples

>>> dataset = DatasetReference("willy-v1", "a" * 64, "b" * 64)
>>> config = ExperimentConfig(
...     "m0-baseline",
...     "baseline",
...     dataset,
...     SeedPlan(42, "willy"),
...     model={"architecture": "unet"},
...     training={"epochs": 100},
...     acceptance=[AcceptanceCriterion("test.nrms", "<=", 2.0)],
... )
>>> len(config.config_hash)
64
>>> config.child_seed("network") == config.seeds.derive("network")
True
experiment_id: str#
stage: str#
dataset: DatasetReference#
seeds: SeedPlan#
model: Mapping[str, Any]#
training: Mapping[str, Any]#
physics: Mapping[str, Any]#
acceptance: tuple[AcceptanceCriterion, ...] = ()#
description: str = ''#
tags: tuple[str, ...] = ()#
created_utc: str | None = None#
schema_version: int = 1#
property config_hash: str#

Return the canonical digest of the complete configuration.

Returns:

SHA-256 digest covering every serialized field.

Return type:

str

Examples

>>> d = DatasetReference("d", "a" * 64, "b" * 64)
>>> len(
...     ExperimentConfig(
...         "e", "baseline", d, SeedPlan(0), {}, {}
...     ).config_hash
... )
64
child_seed(label)#

Derive a stable subsystem seed from this experiment.

Parameters:

label (str) – Subsystem label passed to SeedPlan.derive().

Returns:

Deterministic unsigned 32-bit seed.

Return type:

int

Examples

>>> d = DatasetReference("d", "a" * 64, "b" * 64)
>>> c = ExperimentConfig("e", "baseline", d, SeedPlan(0), {}, {})
>>> c.child_seed("data") == c.child_seed("data")
True
evaluate_gate(metrics, *, require_all=True)#

Evaluate observed metrics against predeclared criteria.

Parameters:

  • metrics (mapping of str to float) – Observed metric values.

  • require_all (bool, default=True) – Treat absent configured metrics as missing failures. When false, absent metrics are omitted; this is useful only for partial status reports and cannot prove the final gate passed unless all criteria were supplied.

Returns:

Immutable overall, per-metric, observed, and missing results.

Return type:

GateEvaluation

Examples

>>> d = DatasetReference("d", "a" * 64, "b" * 64)
>>> c = ExperimentConfig(
...     "e",
...     "baseline",
...     d,
...     SeedPlan(0),
...     {},
...     {},
...     acceptance=[AcceptanceCriterion("nrms", "<=", 2)],
... )
>>> c.evaluate_gate({"nrms": 1.5}).passed
True
to_dict()#

Return the complete JSON-compatible configuration.

Returns:

Mutable schema-1 representation.

Return type:

dict

Examples

>>> d = DatasetReference("d", "a" * 64, "b" * 64)
>>> ExperimentConfig(
...     "e", "baseline", d, SeedPlan(0), {}, {}
... ).to_dict()["stage"]
'baseline'
write_json(path, *, overwrite=True)#

Write a deterministic UTF-8 JSON configuration file.

Parameters:

  • path (str or pathlib.Path) – Destination file.

  • overwrite (bool, default=True) – Permit replacement of an existing file.

Returns:

Destination path.

Return type:

pathlib.Path

Examples

>>> from tempfile import TemporaryDirectory
>>> d = DatasetReference("d", "a" * 64, "b" * 64)
>>> c = ExperimentConfig("e", "baseline", d, SeedPlan(0), {}, {})
>>> with TemporaryDirectory() as directory:
...     path = c.write_json(Path(directory) / "experiment.json")
...     loaded = ExperimentConfig.read_json(path)
>>> loaded.config_hash == c.config_hash
True
classmethod from_dict(data)#

Restore and validate a serialized configuration.

Parameters:

data (mapping) – State returned by to_dict().

Returns:

Immutable source-of-truth configuration.

Return type:

ExperimentConfig

Examples

>>> d = DatasetReference("d", "a" * 64, "b" * 64)
>>> c = ExperimentConfig("e", "baseline", d, SeedPlan(0), {}, {})
>>> ExperimentConfig.from_dict(
...     c.to_dict()
... ).config_hash == c.config_hash
True
classmethod read_json(path)#

Read and validate a JSON experiment configuration.

Parameters:

path (str or pathlib.Path) – Existing UTF-8 JSON file.

Returns:

Validated immutable configuration.

Return type:

ExperimentConfig

Examples

>>> from tempfile import TemporaryDirectory
>>> d = DatasetReference("d", "a" * 64, "b" * 64)
>>> c = ExperimentConfig("e", "baseline", d, SeedPlan(0), {}, {})
>>> with TemporaryDirectory() as directory:
...     path = c.write_json(Path(directory) / "config.json")
...     loaded = ExperimentConfig.read_json(path)
>>> loaded.experiment_id
'e'

pycsamt.ai.experiments.config

Immutable source-of-truth configuration for AI inversion experiments.