pycsamt.ai.experiments.config#

Immutable source-of-truth configuration for AI inversion experiments.

This module describes what an experiment is and which evidence must pass. It intentionally does not start training jobs, import machine-learning frameworks, or construct Maxwell solvers. Runtime orchestration and result records belong to later modules in pycsamt.ai.experiments.

Classes

`AcceptanceCriterion`(metric, operator, threshold)	Predeclare one numerical condition required for an experiment gate.
`DatasetReference`(dataset_id, manifest_hash, ...)	Pin an experiment to exact dataset preparation artifacts.
`ExperimentConfig`(experiment_id, stage, ...)	Immutable source of truth for one reproducible inversion experiment.
`GateEvaluation`(passed, criteria, observed[, ...])	Immutable result of evaluating configured acceptance criteria.
`SeedPlan`(root_seed[, namespace])	Derive stable, labeled child seeds from one experiment root seed.

class pycsamt.ai.experiments.config.SeedPlan(root_seed, namespace='pycsamt.ai')[source]

Bases: object

Derive stable, labeled child seeds from one experiment root seed.

Parameters:

root_seed (int) – Non-negative root seed smaller than 2**64.
namespace (str, default="pycsamt.ai") – Domain separator preventing identical labels in unrelated projects from producing the same child sequence.

Examples

Child seeds are label-stable and request-order independent:

>>> plan = SeedPlan(42, namespace="willy")
>>> plan.derive("dataset") == plan.derive("dataset")
True
>>> plan.derive("dataset") != plan.derive("network")
True

root_seed: int

namespace: str = 'pycsamt.ai'

derive(label)[source]

Derive one unsigned 32-bit seed for a named subsystem.

Parameters:: label (str) – Stable descriptive label such as "geology", "noise", or "network/seed-0".
Returns:: Deterministic value in [0, 2**32).
Return type:: int
Raises:: ValueError – If label is empty.

Examples

>>> child = SeedPlan(1).derive("training")
>>> 0 <= child < 2**32
True

derive_many(labels)[source]

Derive seeds for multiple unique labels.

Parameters:: labels (sequence of str) – Unique non-empty subsystem labels.
Returns:: Labels in caller order mapped to stable child seeds.
Return type:: dict

Examples

>>> sorted(SeedPlan(2).derive_many(["data", "model"]))
['data', 'model']

to_dict()[source]

Return a JSON-serializable seed plan.

Returns:: Schema version, root seed, and namespace.
Return type:: dict

Examples

>>> SeedPlan(3).to_dict()["root_seed"]
3

classmethod from_dict(data)[source]

Restore a validated seed plan.

Parameters:: data (mapping) – State returned by to_dict().
Returns:: Immutable seed plan.
Return type:: SeedPlan

Examples

>>> SeedPlan.from_dict(SeedPlan(4).to_dict()) == SeedPlan(4)
True

class pycsamt.ai.experiments.config.DatasetReference(dataset_id, manifest_hash, split_hash, normalization_hash=None, manifest_uri=None)[source]

Bases: object

Pin an experiment to exact dataset preparation artifacts.

Parameters:

dataset_id (str) – Portable identifier from the dataset manifest.
manifest_hash (str) – SHA-256 digests of the complete dataset manifest and realization split.
split_hash (str) – SHA-256 digests of the complete dataset manifest and realization split.
normalization_hash (str or None, optional) – Digest of fitted normalization state. It may be absent for an audit or forward-only experiment that has not normalized data.
manifest_uri (str or None, optional) – Informational local path or remote URI. Integrity relies on the hash, not on this location.

Examples

>>> reference = DatasetReference("willy-v1", "a" * 64, "b" * 64)
>>> reference.dataset_id
'willy-v1'

dataset_id: str

manifest_hash: str

split_hash: str

normalization_hash: str | None = None

manifest_uri: str | None = None

to_dict()[source]

Return a JSON-serializable artifact reference.

Returns:: Schema version and pinned artifact identifiers.
Return type:: dict

Examples

>>> DatasetReference("d", "a" * 64, "b" * 64).to_dict()[
...     "schema_version"
... ]
1

classmethod from_dict(data)[source]

Restore a validated dataset reference.

Parameters:: data (mapping) – State returned by to_dict().
Returns:: Immutable pinned reference.
Return type:: DatasetReference

Examples

>>> ref = DatasetReference("d", "a" * 64, "b" * 64)
>>> DatasetReference.from_dict(ref.to_dict()) == ref
True

class pycsamt.ai.experiments.config.AcceptanceCriterion(metric, operator, threshold, description='')[source]

Bases: object

Predeclare one numerical condition required for an experiment gate.

Parameters:

metric (str) – Exact metric key, preferably namespaced, for example "test.impedance_nrms".
operator ({"<", "<=", ">", ">=", "=="}) – Comparison applied as observed operator threshold.
threshold (float) – Finite value fixed before results are inspected.
description (str, optional) – Human-readable scientific justification.

Examples

>>> criterion = AcceptanceCriterion("test.impedance_nrms", "<=", 2.0)
>>> criterion.evaluate(1.7)
True
>>> criterion.evaluate(2.5)
False

metric: str

operator: str

threshold: float

description: str = ''

evaluate(observed)[source]

Evaluate an observed metric against the frozen threshold.

Parameters:: observed (float) – Finite measured value.
Returns:: Result of observed operator threshold.
Return type:: bool
Raises:: ValueError – If observed is NaN or infinite.

Examples

>>> AcceptanceCriterion("coverage", ">=", 0.9).evaluate(0.95)
True

to_dict()[source]

Return a JSON-serializable criterion.

Returns:: Metric, comparison, threshold, and description.
Return type:: dict

Examples

>>> AcceptanceCriterion("x", "<", 1).to_dict()["operator"]
'<'

classmethod from_dict(data)[source]

Restore a validated acceptance criterion.

Parameters:: data (mapping) – State returned by to_dict().
Returns:: Immutable numerical gate condition.
Return type:: AcceptanceCriterion

Examples

>>> c = AcceptanceCriterion("x", ">", 0)
>>> AcceptanceCriterion.from_dict(c.to_dict()) == c
True

class pycsamt.ai.experiments.config.GateEvaluation(passed, criteria, observed, missing=(), complete=True)[source]

Bases: object

Immutable result of evaluating configured acceptance criteria.

Parameters:

passed (bool) – Whether every criterion passed and no required metric was missing.
criteria (mapping of str to bool) – Per-metric pass/fail results.
observed (mapping of str to float) – Finite observed values for metrics that were available.
missing (sequence of str) – Required metric keys absent from the supplied result mapping.
complete (bool, default=True) – Whether every configured criterion was evaluated. A partial status report can never pass the final gate.

Examples

>>> result = GateEvaluation(True, {"nrms": True}, {"nrms": 1.2}, ())
>>> result.failed_metrics
()

passed: bool

criteria: Mapping[str, bool]

observed: Mapping[str, float]

missing: tuple[str, ...] = ()

complete: bool = True

property failed_metrics: tuple[str, ...][source]

Return metrics that failed their configured comparisons.

Returns:: Failed available metrics; missing metrics are reported separately.
Return type:: tuple of str

Examples

>>> GateEvaluation(False, {"x": False}, {"x": 2.0}).failed_metrics
('x',)

to_dict()[source]

Return a JSON-serializable gate result.

Returns:: Overall and per-metric outcomes.
Return type:: dict

Examples

>>> GateEvaluation(True, {"x": True}, {"x": 0.5}).to_dict()["passed"]
True

class pycsamt.ai.experiments.config.ExperimentConfig(experiment_id, stage, dataset, seeds, model, training, physics=<factory>, acceptance=(), description='', tags=(), created_utc=None, schema_version=1)[source]

Bases: object

Immutable source of truth for one reproducible inversion experiment.

Parameters:

experiment_id (str) – Portable unique experiment identifier.
stage (str) – Roadmap stage such as "baseline", "forward_2d", or "field_evaluation".
dataset (DatasetReference) – Exact dataset, split, and optional normalization artifacts.
seeds (SeedPlan) – Root and namespace used for labeled child seeds.
model (mapping) – Finite JSON-compatible configuration sections. Their schemas are owned by later model/trainer/solver adapters; this class freezes and hashes them without importing optional dependencies.
training (mapping) – Finite JSON-compatible configuration sections. Their schemas are owned by later model/trainer/solver adapters; this class freezes and hashes them without importing optional dependencies.
physics (mapping) – Finite JSON-compatible configuration sections. Their schemas are owned by later model/trainer/solver adapters; this class freezes and hashes them without importing optional dependencies.
acceptance (sequence of AcceptanceCriterion, optional) – Criteria fixed before results are viewed. Metric names must be unique.
description (str, optional) – Human-readable experiment objective.
tags (sequence of str, optional) – Unique searchable labels.
created_utc (str or None, optional) – Timezone-aware ISO-8601 timestamp normalized to UTC.
schema_version (int, default=1) – Configuration schema version.

Examples

>>> dataset = DatasetReference("willy-v1", "a" * 64, "b" * 64)
>>> config = ExperimentConfig(
...     "m0-baseline",
...     "baseline",
...     dataset,
...     SeedPlan(42, "willy"),
...     model={"architecture": "unet"},
...     training={"epochs": 100},
...     acceptance=[AcceptanceCriterion("test.nrms", "<=", 2.0)],
... )
>>> len(config.config_hash)
64
>>> config.child_seed("network") == config.seeds.derive("network")
True

experiment_id: str

stage: str

dataset: DatasetReference

seeds: SeedPlan

model: Mapping[str, Any]

training: Mapping[str, Any]

physics: Mapping[str, Any]

acceptance: tuple[AcceptanceCriterion, ...] = ()

description: str = ''

tags: tuple[str, ...] = ()

created_utc: str | None = None

schema_version: int = 1

property config_hash: str[source]

Return the canonical digest of the complete configuration.

Returns:: SHA-256 digest covering every serialized field.
Return type:: str

Examples

>>> d = DatasetReference("d", "a" * 64, "b" * 64)
>>> len(
...     ExperimentConfig(
...         "e", "baseline", d, SeedPlan(0), {}, {}
...     ).config_hash
... )
64

child_seed(label)[source]

Derive a stable subsystem seed from this experiment.

Parameters:: label (str) – Subsystem label passed to SeedPlan.derive().
Returns:: Deterministic unsigned 32-bit seed.
Return type:: int

Examples

>>> d = DatasetReference("d", "a" * 64, "b" * 64)
>>> c = ExperimentConfig("e", "baseline", d, SeedPlan(0), {}, {})
>>> c.child_seed("data") == c.child_seed("data")
True

evaluate_gate(metrics, *, require_all=True)[source]

Evaluate observed metrics against predeclared criteria.

Parameters:

metrics (mapping of str to float) – Observed metric values.
require_all (bool, default=True) – Treat absent configured metrics as missing failures. When false, absent metrics are omitted; this is useful only for partial status reports and cannot prove the final gate passed unless all criteria were supplied.

Returns:

Immutable overall, per-metric, observed, and missing results.

Return type:

GateEvaluation

Examples

>>> d = DatasetReference("d", "a" * 64, "b" * 64)
>>> c = ExperimentConfig(
...     "e",
...     "baseline",
...     d,
...     SeedPlan(0),
...     {},
...     {},
...     acceptance=[AcceptanceCriterion("nrms", "<=", 2)],
... )
>>> c.evaluate_gate({"nrms": 1.5}).passed
True

to_dict()[source]

Return the complete JSON-compatible configuration.

Returns:: Mutable schema-1 representation.
Return type:: dict

Examples

>>> d = DatasetReference("d", "a" * 64, "b" * 64)
>>> ExperimentConfig(
...     "e", "baseline", d, SeedPlan(0), {}, {}
... ).to_dict()["stage"]
'baseline'

write_json(path, *, overwrite=True)[source]

Write a deterministic UTF-8 JSON configuration file.

Parameters:

path (str or pathlib.Path) – Destination file.
overwrite (bool, default=True) – Permit replacement of an existing file.

Returns:

Destination path.

Return type:

pathlib.Path

Examples

>>> from tempfile import TemporaryDirectory
>>> d = DatasetReference("d", "a" * 64, "b" * 64)
>>> c = ExperimentConfig("e", "baseline", d, SeedPlan(0), {}, {})
>>> with TemporaryDirectory() as directory:
...     path = c.write_json(Path(directory) / "experiment.json")
...     loaded = ExperimentConfig.read_json(path)
>>> loaded.config_hash == c.config_hash
True

classmethod from_dict(data)[source]

Restore and validate a serialized configuration.

Parameters:: data (mapping) – State returned by to_dict().
Returns:: Immutable source-of-truth configuration.
Return type:: ExperimentConfig

Examples

>>> d = DatasetReference("d", "a" * 64, "b" * 64)
>>> c = ExperimentConfig("e", "baseline", d, SeedPlan(0), {}, {})
>>> ExperimentConfig.from_dict(
...     c.to_dict()
... ).config_hash == c.config_hash
True

classmethod read_json(path)[source]

Read and validate a JSON experiment configuration.

Parameters:: path (str or pathlib.Path) – Existing UTF-8 JSON file.
Returns:: Validated immutable configuration.
Return type:: ExperimentConfig

Examples

>>> from tempfile import TemporaryDirectory
>>> d = DatasetReference("d", "a" * 64, "b" * 64)
>>> c = ExperimentConfig("e", "baseline", d, SeedPlan(0), {}, {})
>>> with TemporaryDirectory() as directory:
...     path = c.write_json(Path(directory) / "config.json")
...     loaded = ExperimentConfig.read_json(path)
>>> loaded.experiment_id
'e'