2.25.3.16. pycsamt.ai.processing.denoise#

EMDenoiser — convolutional autoencoder for MT impedance tensor denoising.

Trains a 1-D convolutional autoencoder (CAE) on clean (or simulated) MT data. At inference time the trained network suppresses broadband Gaussian noise and narrowband artefacts (powerline harmonics, dead-band contamination) while preserving the smooth spectral shape of the true signal.

Architecture#

The network is a symmetric encoder-decoder:

\[\mathbf{z} = f_{\text{enc}}(\mathbf{x} + \boldsymbol{\varepsilon}), \quad \hat{\mathbf{x}} = f_{\text{dec}}(\mathbf{z})\]

where \(\boldsymbol{\varepsilon}\) is synthetic training noise and the network minimises \(\|\mathbf{x} - \hat{\mathbf{x}}\|_2^2\).

Input tensor shape#

(n_samples, n_components, n_freqs)

  • n_components = 4 (default) → [log|Zxy|, \phi_{xy}, log|Zyx|, \phi_{yx}]

  • n_components = 8 → all four impedance tensor components

Working directly with site collections#

EMDenoiser.apply() accepts and returns a site collection directly – extracting features, denoising, and writing the reconstructed impedance back onto each station’s own frequency grid – so the network can be dropped into the same sites-in / sites-out chain as the rule-based functions in pycsamt.emtools.remove_noise.

Functions

prepare_z_features(sites, *[, n_components, ...])

Extract an impedance feature array from a site collection.

Classes

EMDenoiser([n_freqs, n_components, ...])

Convolutional autoencoder for MT impedance tensor denoising.
class pycsamt.ai.processing.denoise.EMDenoiser(n_freqs=None, n_components=4, *, channels=(64, 128, 64), dropout=0.1, device=None)[source]

Bases: BaseEMProcessor

Convolutional autoencoder for MT impedance tensor denoising.

The network is trained on clean (or synthetic) impedance data by adding controlled noise and learning to reconstruct the original signal. Supports both PyTorch and TensorFlow backends via pycsamt.backends.

Parameters:

  • n_freqs (int or None, default None) – Number of frequency channels in the input. When None, inferred from X.shape[2] on the first call to fit().

  • n_components ({4, 8}, default 4) – Feature channels per frequency: 4 = off-diagonal \(Z_{xy}\), \(Z_{yx}\) amplitudes and phases; 8 = all four tensor components.

  • channels (tuple of int, default (64, 128, 64)) – Channel widths for the encoder/decoder stages.

  • dropout (float, default 0.1) – Dropout probability in the encoder bottleneck.

  • device (str or None) – "cpu", "cuda", "mps", "/GPU:0", or None for auto-detect via the active backend.

Notes

When neither PyTorch nor TensorFlow is available the denoiser falls back to a per-channel Gaussian smoothing filter (requires scipy).

Call prepare_z_features() to convert a site collection to the (n_samples, n_components, n_freqs) array expected here, or use apply() to go straight from a site collection to a corrected one without handling the array form yourself.

Examples

>>> import numpy as np
>>> X_clean = np.random.randn(200, 4, 32).astype("float32")
>>> den = EMDenoiser()
>>> den.fit(X_clean, epochs=5, verbose=False)
EMDenoiser(n_freqs=32, n_components=4)
>>> X_denoised = den.transform(X_clean)

Sites in, corrected sites out — the same convention used by rule-based functions such as notch_powerline():

>>> clean_sites = den.apply(sites)
fit(X, *, noise_level=0.05, epochs=50, batch_size=64, lr=0.001, val_frac=0.1, seed=None, verbose=True)[source]

Train the denoiser on clean impedance data.

Parameters:

  • X (ndarray, shape (n_samples, n_components, n_freqs)) – Clean training data.

  • noise_level (float) – Standard deviation of additive Gaussian noise relative to the per-channel standard deviation of X.

  • epochs (int) – Maximum number of training epochs.

  • batch_size (int)

  • lr (float) – Initial Adam learning rate.

  • val_frac (float) – Fraction of training samples held out for validation.

  • seed (int or None)

  • verbose (bool)

Return type:

self

transform(X)[source]

Apply the trained denoiser.

Parameters:

X (ndarray, shape (n_samples, n_components, n_freqs)) – Noisy impedance data.

Returns:

X_denoised

Return type:

ndarray, same shape as input

apply(sites, *, freq_ref=None, log_amp=True, inplace=False, recursive=True, on_dup='replace', strict=False, verbose=0)[source]

Denoise a site collection’s impedance tensor in place.

Extracts features with prepare_z_features(), denoises them with transform(), then writes the reconstructed impedance back onto each site’s own frequency grid — the same sites-in / sites-out convention used by rule-based correction functions such as notch_powerline() (mutate Z.z, hand the mutation to a shared copy-or-mutate-in-place helper).

Parameters:

  • sites (SiteCollection or compatible)

  • freq_ref (ndarray or None) – Common frequency grid (Hz) the network was trained on. Must match the grid used to prepare the training data — when None (the default), the first site’s own grid is used, matching prepare_z_features()’s own default. Pass the same explicit array at both training-data preparation time and here whenever sites is not the exact collection the model was trained on.

  • log_amp (bool, default True) – Must match the log_amp used to build the training features.

  • inplace (bool, default False) – When False (the default), sites is left untouched and a corrected copy is returned. When True, sites is modified in place and returned.

  • recursive (bool) – Passed to ensure_sites().

  • on_dup (str) – Passed to ensure_sites().

  • strict (bool) – Passed to ensure_sites().

  • verbose (int) – Passed to ensure_sites().

Returns:

corrected – A site collection with the same stations, with each station’s Z.z replaced by the network’s denoised reconstruction. Tensor components not modelled by n_components — the diagonal \(Z_{xx}, Z_{yy}\) when n_components=4 — are left untouched.

Return type:

Sites

Examples

>>> den = EMDenoiser().fit(X_clean, epochs=60)
>>> clean_sites = den.apply(sites)
property history_: dict[str, list][source]

Training history recorded by the last fit() call.

Returns:

history – {"train_loss": [...], "val_loss": [...]}, one value per epoch. Empty when the scipy Gaussian fallback was used (no epoch loop) or before fit() has been called. Pass directly to plot_training_history().

Return type:

dict

pycsamt.ai.processing.denoise.prepare_z_features(sites, *, n_components=4, log_amp=True, freq_ref=None)[source]

Extract an impedance feature array from a site collection.

Parameters:

  • sites (SiteCollection, dict, or list of Z objects) – MT impedance data in any format accepted by ensure_sites().

  • n_components ({4, 8}) – 4 uses off-diagonal components only; 8 includes diagonal.

  • log_amp (bool) – If True, amplitudes are transformed as \(\log_{10}(|Z| + \varepsilon)\) before packing.

  • freq_ref (ndarray or None) – Common frequency grid to interpolate onto. When None, the grid of the first site is used.

Returns:

X – Feature array ready for EMDenoiser.

Return type:

ndarray, shape (n_sites, n_components, n_freqs)