2.25.3.17. pycsamt.ai.processing.distortion#

DistortionTypeClassifier — learned galvanic-distortion triage.

Why this does not re-solve the physics#

pycsamt.emtools.ss already estimates static shift with four different spatial-statistics methods (AMA, LOESS, bilateral filter, reference-median), and pycsamt.emtools.gb already fits a full Groom-Bailey galvanic-distortion decomposition (groom_bailey_table(), apply_groom_bailey()) by nonlinear least squares. DistortionTypeClassifier is not meant to replace either – it is meant to triage: given a station’s phase-tensor invariants and its already-fitted Groom-Bailey parameters, label it as clean, static-shift-only, or in need of the full decomposition, the same way DimensionalityClassifier refines classify_dimensionality()’s threshold rule into a smoother multi-feature classifier rather than inventing a new definition of dimensionality. The value is routing a survey’s many stations toward the right existing tool quickly, not a new distortion model.

Classes#

  1. clean – resistivity close to the along-line spatial trend, twist and shear close to 0; no correction needed.

  2. static-shift-only – resistivity departs from the spatial trend but twist and shear stay small: a pure multiplicative offset, the case pycsamt.emtools.ss targets directly.

  3. distorted – twist and/or shear are non-negligible: rotation- and anisotropy-like distortion that a static-shift correction alone cannot fix, the case apply_groom_bailey() targets.

Feature vector (per station)#

[beta_abs, ellipt_abs, delta_log10_rho, twist_deg, shear, anisotropy] – beta_abs/ellipt_abs from phase_features_table() (median over frequency, already reused by DimensionalityClassifier); twist_deg/shear/anisotropy from groom_bailey_table(), frequency-independent by construction (Groom-Bailey assumes one real distortion matrix per station); delta_log10_rho, the station’s log10-resistivity deviation from the spatial trend along the line, from estimate_ss_ama(). build_distortion_features_table() builds this table directly from a site collection.

Note

Groom-Bailey’s own fitted distortion matrix is normalised to unit determinant at every iteration (_normalise_distortion()) – the textbook convention, since the absolute gain is degenerate with the unknown regional resistivity and genuinely unrecoverable from one station’s data alone. Its gain column is therefore always exactly 1.0 and carries no information about static shift; delta_log10_rho, a real cross-station spatial comparison, is used here instead.

Self-training labels#

Mirroring DimensionalityClassifier’s _rule_labels / from_features_table pattern: default training labels come from simple thresholds on |twist_deg|, |shear|, and |delta_log10_rho| when no label_col is supplied, so the network starts out approximating a transparent rule and can be refined with real labels later exactly the way DimensionalityClassifier is.

Functions

build_distortion_features_table(sites, *[, ...])

Build the per-station feature table DistortionTypeClassifier expects.

Classes

DistortionTypeClassifier([hidden, dropout, ...])

Learned triage classifier for galvanic-distortion regime.
class pycsamt.ai.processing.distortion.DistortionTypeClassifier(hidden=(32, 16), dropout=0.2, n_classes=3, lr=0.001, device=None)[source]

Bases: BaseEMProcessor

Learned triage classifier for galvanic-distortion regime.

Parameters:

  • hidden (tuple of int, default (32, 16)) – Hidden-layer widths of the shared MLP backbone – deliberately smaller than DimensionalityClassifier’s default, since this problem is station-level (tens to low hundreds of samples per survey) rather than station-frequency-level.

  • dropout (float, default 0.2) – Dropout probability in each hidden layer.

  • n_classes (int, default 3) – Number of distortion-regime classes; see the module docstring.

  • lr (float, default 1e-3) – Default learning rate; can be overridden in fit().

  • device (str or None)

Notes

Falls back to a random-forest classifier (scikit-learn) when neither PyTorch nor TensorFlow is available; transform() then returns the forest’s own class probabilities.

Examples

>>> from pycsamt.ai.processing.distortion import (
...     DistortionTypeClassifier, build_distortion_features_table,
... )
>>> feats = build_distortion_features_table(sites)
>>> clf = DistortionTypeClassifier.from_features_table(
...     feats, epochs=80,
... )
>>> table = clf.predict_table(sites)
classmethod from_features_table(df, *, label_col=None, shift_th=0.1, twist_th=10.0, shear_th=0.1, **fit_kwargs)[source]

Construct and train a classifier from a build_distortion_features_table() DataFrame.

Parameters:

  • df (DataFrame)

  • label_col (str or None) – Column for pre-computed labels; None – the default – falls back to rule-based self-training labels (see the module docstring).

  • shift_th (float) – Rule-based thresholds (used when label_col is absent).

  • twist_th (float) – Rule-based thresholds (used when label_col is absent).

  • shear_th (float) – Rule-based thresholds (used when label_col is absent).

  • **fit_kwargs – Passed to fit() (e.g. epochs=80).

Return type:

DistortionTypeClassifier

fit(X, y=None, *, epochs=80, batch_size=64, lr=None, val_frac=0.15, seed=None, verbose=True)[source]

Train the distortion-triage classifier.

Parameters:

  • X (ndarray (n_samples, 6) or DataFrame)

  • y (int ndarray (n_samples,) or None) – Class labels (0=clean, 1=static-shift-only, 2=distorted). None falls back to the rule-based self-training labels described in the module docstring.

  • epochs (int) – Training hyper-parameters. epochs defaults low (80) for a quick look; on a real, small survey (tens of stations, as most are) this is often too few for the self-trained network to converge reliably – observed on 28-station data, repeated fits at the default can disagree on which regime is the majority class, not just on a minority label appearing or not. Prefer more epochs (e.g. 150-200) and treat a single run’s predict_table() output as one plausible smoothing of the rule boundary rather than a converged answer – see the user guide for a concrete before/after comparison.

  • batch_size (int) – Training hyper-parameters. epochs defaults low (80) for a quick look; on a real, small survey (tens of stations, as most are) this is often too few for the self-trained network to converge reliably – observed on 28-station data, repeated fits at the default can disagree on which regime is the majority class, not just on a minority label appearing or not. Prefer more epochs (e.g. 150-200) and treat a single run’s predict_table() output as one plausible smoothing of the rule boundary rather than a converged answer – see the user guide for a concrete before/after comparison.

  • lr (float | None) – Training hyper-parameters. epochs defaults low (80) for a quick look; on a real, small survey (tens of stations, as most are) this is often too few for the self-trained network to converge reliably – observed on 28-station data, repeated fits at the default can disagree on which regime is the majority class, not just on a minority label appearing or not. Prefer more epochs (e.g. 150-200) and treat a single run’s predict_table() output as one plausible smoothing of the rule boundary rather than a converged answer – see the user guide for a concrete before/after comparison.

  • val_frac (float) – Training hyper-parameters. epochs defaults low (80) for a quick look; on a real, small survey (tens of stations, as most are) this is often too few for the self-trained network to converge reliably – observed on 28-station data, repeated fits at the default can disagree on which regime is the majority class, not just on a minority label appearing or not. Prefer more epochs (e.g. 150-200) and treat a single run’s predict_table() output as one plausible smoothing of the rule boundary rather than a converged answer – see the user guide for a concrete before/after comparison.

  • seed (int | None) – Training hyper-parameters. epochs defaults low (80) for a quick look; on a real, small survey (tens of stations, as most are) this is often too few for the self-trained network to converge reliably – observed on 28-station data, repeated fits at the default can disagree on which regime is the majority class, not just on a minority label appearing or not. Prefer more epochs (e.g. 150-200) and treat a single run’s predict_table() output as one plausible smoothing of the rule boundary rather than a converged answer – see the user guide for a concrete before/after comparison.

  • verbose (bool) – Training hyper-parameters. epochs defaults low (80) for a quick look; on a real, small survey (tens of stations, as most are) this is often too few for the self-trained network to converge reliably – observed on 28-station data, repeated fits at the default can disagree on which regime is the majority class, not just on a minority label appearing or not. Prefer more epochs (e.g. 150-200) and treat a single run’s predict_table() output as one plausible smoothing of the rule boundary rather than a converged answer – see the user guide for a concrete before/after comparison.

Return type:

self

transform(X)[source]

Compute class probabilities.

Returns:

proba

Return type:

ndarray, shape (n_samples, n_classes)

Parameters:

X (ndarray | DataFrame)

predict(X)[source]

Predict the distortion regime (0=clean, 1=static-shift-only, 2=distorted).

Parameters:

X (ndarray | DataFrame)

Return type:

ndarray

predict_table(sites, *, band=None, recursive=True, on_dup='replace', strict=False, verbose=0)[source]

Classify an entire site collection and return a result DataFrame.

Parameters:

  • sites (SiteCollection or compatible)

  • band ((f_lo, f_hi) or None) – Passed to build_distortion_features_table().

  • recursive (bool) – Passed to ensure_sites().

  • on_dup (str) – Passed to ensure_sites().

  • strict (bool) – Passed to ensure_sites().

  • verbose (int) – Passed to ensure_sites().

Returns:

df – Columns: station, the six input features, regime (0/1/2), regime_label (str), confidence.

Return type:

DataFrame

property history_: dict[str, list][source]

Training history recorded by the last fit() call.

Returns:

history – {"train_loss": [...], "val_loss": [...]}, one value per epoch. Empty when the random-forest fallback was used (no epoch loop) or before fit() has been called. Pass directly to plot_training_history().

Return type:

dict

pycsamt.ai.processing.distortion.build_distortion_features_table(sites, *, band=None, recursive=True, on_dup='replace', strict=False, verbose=0)[source]

Build the per-station feature table DistortionTypeClassifier expects.

Aggregates phase_features_table()’s per-(station, frequency) beta_abs / ellipt_abs to a per-station median, then merges with groom_bailey_table()’s per-station twist_deg / shear / anisotropy and estimate_ss_ama()’s per-station delta_log10_rho – one row per station present in all three. Groom-Bailey’s own gain column is not used; see the module docstring for why.

Parameters:

  • sites (SiteCollection or compatible)

  • band ((f_lo, f_hi) or None) – Frequency band passed to groom_bailey_table().

  • recursive (bool) – Passed to ensure_sites().

  • on_dup (str) – Passed to ensure_sites().

  • strict (bool) – Passed to ensure_sites().

  • verbose (int) – Passed to ensure_sites().

Returns:

df – Columns: station, beta_abs, ellipt_abs, delta_log10_rho, twist_deg, shear, anisotropy. Empty (but correctly columned) when no station has estimates from all three sources.

Return type:

DataFrame