pycsamt.ai.domain_gap.report#

Quantitative comparison between simulated and field feature distributions.

The M3 acceptance gate requires that “simulated and field feature distributions are compared quantitatively” rather than judged by eye. This module computes per-feature summary statistics and a two-sample Kolmogorov-Smirnov test between a simulated (corrupted synthetic) survey and a real field survey, on features derived from the shared SurveyData contract so the comparison never depends on how either survey was produced.

Functions

`compare_feature_distributions`(simulated, ...)	Compare two 1-D samples of the same feature quantitatively.
`compare_survey_distributions`(simulated, field, *)	Compare simulated and field surveys across several canonical features.

Classes

`DistributionComparisonReport`(comparisons)	Collection of `FeatureComparison` results across features.
`FeatureComparison`(feature, simulated_stats, ...)	Quantitative comparison of one feature between two distributions.

class pycsamt.ai.domain_gap.report.FeatureComparison(feature, simulated_stats, field_stats, ks_statistic, ks_pvalue, mean_difference, std_ratio)[source]

Bases: object

Quantitative comparison of one feature between two distributions.

Parameters:

feature (str) – Name of the compared feature.
simulated_stats (mapping) – count, mean, std, median of each sample.
field_stats (mapping) – count, mean, std, median of each sample.
ks_statistic (float) – Two-sample Kolmogorov-Smirnov statistic and p-value; NaN when either sample is empty.
ks_pvalue (float) – Two-sample Kolmogorov-Smirnov statistic and p-value; NaN when either sample is empty.
mean_difference (float) – simulated mean - field mean.
std_ratio (float) – simulated std / field std; NaN when the field std is zero.

feature: str

simulated_stats: Mapping[str, float]

field_stats: Mapping[str, float]

ks_statistic: float

ks_pvalue: float

mean_difference: float

std_ratio: float

to_dict()[source]

Return a JSON-serializable representation.

Returns:: All fields, with statistics mappings converted to plain dicts.
Return type:: dict

Examples

>>> import numpy as np
>>> comparison = compare_feature_distributions(
...     np.array([1.0, 2.0, 3.0]),
...     np.array([1.0, 2.0, 3.0]),
...     feature="custom",
... )
>>> comparison.to_dict()["feature"]
'custom'

class pycsamt.ai.domain_gap.report.DistributionComparisonReport(comparisons)[source]

Bases: object

Collection of FeatureComparison results across features.

Parameters:: comparisons (mapping) – Feature name to FeatureComparison.

comparisons: Mapping[str, FeatureComparison]

worst_feature()[source]

Return the feature name with the largest KS statistic.

Returns:: Feature name whose simulated/field distributions differ most, ignoring features with a NaN statistic (empty samples).
Return type:: str
Raises:: ValueError – If every feature has a NaN KS statistic.

Examples

>>> import numpy as np
>>> from pycsamt.ai.data.contracts import SurveyData
>>> z = np.full((2, 4, 1), 100 + 50j)
>>> survey = SurveyData(
...     z, np.linspace(100, 1, 4), ["A", "B"], ["xy"], np.zeros((2, 2))
... )
>>> report = compare_survey_distributions(survey, survey)
>>> report.worst_feature() in report.comparisons
True

to_dict()[source]

Return a JSON-serializable representation.

Returns:: Mapping of feature name to its comparison dict.
Return type:: dict

Examples

>>> import numpy as np
>>> from pycsamt.ai.data.contracts import SurveyData
>>> z = np.full((2, 4, 1), 100 + 50j)
>>> survey = SurveyData(
...     z, np.linspace(100, 1, 4), ["A", "B"], ["xy"], np.zeros((2, 2))
... )
>>> report = compare_survey_distributions(survey, survey)
>>> sorted(report.to_dict())
['error_to_magnitude_ratio', 'log_impedance_magnitude', 'phase_deg']

pycsamt.ai.domain_gap.report.compare_feature_distributions(simulated, field, *, feature)[source]

Compare two 1-D samples of the same feature quantitatively.

Parameters:

simulated (array-like) – Feature values already extracted from each survey (see compare_survey_distributions() to extract them from SurveyData directly).
field (array-like) – Feature values already extracted from each survey (see compare_survey_distributions() to extract them from SurveyData directly).
feature (str) – Label recorded on the returned FeatureComparison.

Returns:

Summary statistics, mean difference, std ratio, and two-sample KS test between the samples.

Return type:

FeatureComparison

Examples

>>> import numpy as np
>>> rng = np.random.default_rng(0)
>>> comparison = compare_feature_distributions(
...     rng.normal(0, 1, 200), rng.normal(0, 1, 200), feature="demo"
... )
>>> comparison.ks_pvalue > 0.01
True

pycsamt.ai.domain_gap.report.compare_survey_distributions(simulated, field, *, features=('log_impedance_magnitude', 'phase_deg', 'error_to_magnitude_ratio'))[source]

Compare simulated and field surveys across several canonical features.

Parameters:

simulated (SurveyData) – Simulated (e.g. corrupted synthetic) survey.
field (SurveyData) – Real field survey, ideally sharing the simulated survey’s frequency band and component set for a meaningful comparison.
features (sequence of str, default features) – Any of "log_impedance_magnitude", "phase_deg", or "error_to_magnitude_ratio". The error-ratio feature is skipped (empty sample) for a survey without declared errors.

Returns:

One FeatureComparison per requested feature.

Return type:

DistributionComparisonReport

Examples

>>> import numpy as np
>>> from pycsamt.ai.data.contracts import SurveyData
>>> z = np.full((2, 4, 1), 100 + 50j)
>>> survey = SurveyData(
...     z, np.linspace(100, 1, 4), ["A", "B"], ["xy"], np.zeros((2, 2))
... )
>>> report = compare_survey_distributions(survey, survey)
>>> report.comparisons["phase_deg"].mean_difference
0.0