pycsamt.ai.domain_gap.survey_fit#

Fit plausible CorruptionConfig ranges from real data.

This module is the bridge between a real field survey (AMT, CSAMT, MT, or otherwise) and the numpy-only pycsamt.ai.domain_gap.simulator. It is deliberately split in two:

survey_data_from_sites()

Converts EDI/Sites/APISurvey input into a canonical SurveyData, using only the quantities the M1 data contract defines. This path stays numpy-only once the bridge itself has run.

fit_corruption_config()

Derives plausible noise/error-floor/dropout ranges purely from a SurveyData’s own declared errors and coverage. No EDI or pandas dependency is required at this stage.

fit_distortion_priors_from_sites()

Calls the heavier, pandas-based pycsamt.emtools.gb/pycsamt.emtools.ss diagnostics directly on real sites to estimate empirical static-shift and galvanic-distortion spreads. This is the one part of M3 that genuinely depends on a real survey’s own QC diagnostics rather than literature defaults.

The frequency grid required by SurveyData is not silently interpolated across stations with different sampling: mismatched frequency grids raise a clear error, per the plan’s non-negotiable principles.

Functions

fit_corruption_config(survey, *[, ...])

Derive plausible noise/dropout ranges from a real survey's QC.

fit_distortion_priors_from_sites(sites, *[, ...])

Estimate empirical static-shift and distortion spreads from real EDI.

survey_data_from_sites(sites, *[, crs, ...])

Bridge EDI/Sites/APISurvey input to canonical SurveyData.
pycsamt.ai.domain_gap.survey_fit.survey_data_from_sites(sites, *, crs=None, freq_rtol=1e-06, station_spacing=500.0, recursive=True, on_dup='replace', verbose=0, metadata=None)[source]

Bridge EDI/Sites/APISurvey input to canonical SurveyData.

Parameters:

  • sites (Any) – Anything accepted by pycsamt.emtools._core.ensure_sites(): a filesystem path/glob/directory, EDIFile/EDICollection, Site/Sites, APISurvey, or an iterable of these.

  • crs (str, optional) – Coordinate reference system identifier to record. Station positions are always projected with a local equirectangular approximation (see pycsamt.ai.inversion._sites_bridge.sites_to_coords_3d()); pass a CRS string only if it genuinely describes that projection.

  • freq_rtol (float, default=1e-6) – Relative tolerance used when checking that every station shares the same frequency grid.

  • station_spacing (float, default=500.0) – Forwarded to the coordinate bridge as a uniform-grid fallback spacing, used only when no station reports finite coordinates.

  • recursive (bool) – Forwarded to ensure_sites.

  • on_dup (str) – Forwarded to ensure_sites.

  • verbose (int) – Forwarded to ensure_sites.

  • metadata (dict, optional) – Extra provenance recorded on the returned survey.

Returns:

Canonical survey with full xx, xy, yx, yy components, with impedance and its declared error converted from Site’s EDI-native [mV/km]/[nT] convention to SI (V/A), matching SurveyData’s default ImpedanceConvention.

Return type:

SurveyData

Raises:

ValueError – If no station has usable impedance data, or stations do not share a common frequency grid within freq_rtol.

Notes

This function performs no frequency interpolation: a survey whose stations were sampled on different frequency grids must be resolved by an explicit, survey-matched frequency selector (an M1 concern) before reaching this bridge.

Examples

>>> survey = survey_data_from_sites(
...     "data/AMT/WILLY_DATA/L18PLT", recursive=False, verbose=0
... )
>>> survey.components
('xx', 'xy', 'yx', 'yy')
pycsamt.ai.domain_gap.survey_fit.fit_corruption_config(survey, *, severity_scale=1.0)[source]

Derive plausible noise/dropout ranges from a real survey’s QC.

Only quantities already present in the canonical SurveyData contract are used: the impedance_error-to-|Z| ratio for heteroscedastic noise and error floor, and coverage() for dropout rates.

Parameters:

  • survey (SurveyData) – Real (or realistically corrupted) survey to profile.

  • severity_scale (float, default=1.0) – Multiplier applied to every fitted range/rate, letting a caller derive a milder or harsher preset from the same empirical fit.

Returns:

Configuration whose noise range spans the interquartile range of the observed relative error, whose error floor is the fifth percentile of that ratio, and whose dropout rates equal the observed missing fractions. Distortion and outlier parameters are left at zero; see fit_distortion_priors_from_sites() for those.

Return type:

CorruptionConfig

Raises:

ValueError – If survey has no declared impedance_error to profile.

Examples

>>> import numpy as np
>>> from pycsamt.ai.data.contracts import SurveyData
>>> z = np.full((4, 6, 2), 100 + 50j)
>>> err = np.full((4, 6, 2), 3.0)
>>> survey = SurveyData(
...     z,
...     np.linspace(1000, 1, 6),
...     ["A", "B", "C", "D"],
...     ["xy", "yx"],
...     np.zeros((4, 2)),
...     impedance_error=err,
... )
>>> config = fit_corruption_config(survey)
>>> config.noise_level_range[0] >= 0.0
True
pycsamt.ai.domain_gap.survey_fit.fit_distortion_priors_from_sites(sites, *, recursive=True, on_dup='replace', verbose=0, **kwargs)[source]

Estimate empirical static-shift and distortion spreads from real EDI.

This is the one M3 entry point that genuinely depends on the heavier, pandas-based EM diagnostics in pycsamt.emtools.gb and pycsamt.emtools.ss, run directly on real sites (e.g. a WILLY line) rather than on the numpy-only SurveyData contract.

Parameters:

  • sites (Any) – Anything accepted by pycsamt.emtools._core.ensure_sites().

  • recursive (bool) – Forwarded to the underlying diagnostics.

  • on_dup (str) – Forwarded to the underlying diagnostics.

  • verbose (int) – Forwarded to the underlying diagnostics.

  • **kwargs (Any) – Forwarded to pycsamt.emtools.gb.groom_bailey_table().

Returns:

static_shift_log10_sigma, distortion_gain_log10_sigma, distortion_twist_deg_sigma, distortion_shear_sigma, and distortion_anisotropy_sigma, each the population standard deviation of the corresponding per-station fitted parameter across stations with a successful fit. A parameter is 0.0 when fewer than two stations produced a usable fit.

Return type:

dict

Examples

>>> priors = fit_distortion_priors_from_sites(
...     "data/AMT/WILLY_DATA/L18PLT", recursive=False, verbose=0
... )
>>> sorted(priors)
['distortion_anisotropy_sigma', 'distortion_gain_log10_sigma', 'distortion_shear_sigma', 'distortion_twist_deg_sigma', 'static_shift_log10_sigma']