2.25.3.14. pycsamt.ai.processing.anomaly#

AnomalyDetector — unsupervised profile-level anomaly detection.

A fully-connected autoencoder is trained on per-site feature vectors derived from a survey profile. Sites whose reconstruction error significantly exceeds the training distribution are flagged as anomalous — they may represent bad data, equipment problems, or genuinely anomalous geology that warrants closer inspection.

Architecture#

\[\hat{\mathbf{x}} = g(f(\mathbf{x}))\]

where \(f: \mathbb{R}^d \to \mathbb{R}^k\) is the encoder (\(k \ll d\)) and \(g: \mathbb{R}^k \to \mathbb{R}^d\) is the decoder. The anomaly score for site \(i\) is

\[s_i = \|\mathbf{x}_i - \hat{\mathbf{x}}_i\|_2^2 / d.\]

A site is flagged when \(s_i\) exceeds the threshold_percentile-th percentile of training scores.

When neither PyTorch nor TensorFlow is available the detector falls back to PCA-based reconstruction using sklearn.decomposition.PCA.

Working directly with site collections#

AnomalyDetector.apply() accepts a site collection directly and returns one with every flagged station removed, via Sites.select() – the sites-in / sites-out counterpart of transform() / flag_anomalies(), shaped as a station filter rather than a per-station mutation.

Classes

AnomalyDetector([n_features, latent_dim, ...])

Profile-level unsupervised anomaly detector.

class pycsamt.ai.processing.anomaly.AnomalyDetector(n_features=None, latent_dim=32, *, channels=(128, 64), threshold_percentile=95.0, device=None)[source]

Bases: BaseEMProcessor

Profile-level unsupervised anomaly detector.

Parameters:

n_features (int or None, default None) – Feature vector length per site (typically n_freqs × n_components). When None, inferred from X.shape[1] on the first call to fit().
latent_dim (int, default 32) – Bottleneck dimension of the autoencoder.
channels (tuple of int, default (128, 64)) – Hidden layer widths in the encoder (decoder is mirrored).
threshold_percentile (float, default 95.0) – Sites with reconstruction error above this percentile of the training distribution are flagged as anomalous.
device (str or None) – Compute device. Ignored when using the PCA fallback.

Notes

When neither PyTorch nor TensorFlow is installed the detector silently uses a PCA reconstruction model via scikit-learn. The interface is identical; only the accuracy differs.

Examples

>>> import numpy as np
>>> X = np.random.randn(100, 120).astype("float32")
>>> det = AnomalyDetector(latent_dim=16)
>>> det.fit(X, epochs=10, verbose=False)
AnomalyDetector(n_features=120, latent_dim=16)
>>> scores = det.transform(X)
>>> flags = det.flag_anomalies(X)

fit(X, *, epochs=80, batch_size=32, lr=0.001, val_frac=0.1, seed=None, verbose=True)[source]

Train the anomaly detector on profile data.

Parameters:

X (ndarray, shape (n_sites, n_features)) – Per-site feature vectors (normal / clean data).
epochs (int) – Training epochs (ignored when using PCA fallback).
batch_size (int)
lr (float)
val_frac (float)
seed (int or None)
verbose (bool)

Return type:

self

transform(X)[source]

Compute per-site reconstruction error (anomaly score).

Parameters:: X (ndarray, shape (n_samples, n_features))
Returns:: scores – Per-site mean squared reconstruction error.
Return type:: ndarray, shape (n_samples,)

flag_anomalies(X)[source]

Return a boolean mask of anomalous sites.

Parameters:: X (ndarray, shape (n_samples, n_features))
Returns:: flags
Return type:: bool ndarray, shape (n_samples,)

apply(sites, *, n_components=4, freq_ref=None, log_amp=True, threshold=None, recursive=True, on_dup='replace', strict=False, verbose=0)[source]

Drop anomalous stations, sites-in / sites-out.

Builds one feature vector per station with prepare_z_features(), flattened to (n_sites, n_components * n_freqs) — the same convention used to fit a detector on site data in the first place — scores each station, and returns a new site collection with every flagged station removed via Sites.select().

Unlike EMDenoiser.apply, there is no inplace option: dropping stations is a selection, not a per-station mutation, and Sites has no in-place removal — select() always returns a new container, and so does this method.

Parameters:

sites (SiteCollection or compatible)
n_components (int) – Passed to prepare_z_features(). Must reproduce the exact feature layout used to build the training data — in particular, n_components * len(freq_ref) must equal the fitted n_features, or a ValueError is raised rather than silently scoring a mismatched feature vector.
freq_ref (ndarray | None) – Passed to prepare_z_features(). Must reproduce the exact feature layout used to build the training data — in particular, n_components * len(freq_ref) must equal the fitted n_features, or a ValueError is raised rather than silently scoring a mismatched feature vector.
log_amp (bool) – Passed to prepare_z_features(). Must reproduce the exact feature layout used to build the training data — in particular, n_components * len(freq_ref) must equal the fitted n_features, or a ValueError is raised rather than silently scoring a mismatched feature vector.
threshold (float or None) – Overrides the fitted threshold_ for this call.
recursive (bool) – Passed to ensure_sites().
on_dup (str) – Passed to ensure_sites().
strict (bool) – Passed to ensure_sites().
verbose (int) – Passed to ensure_sites().

Returns:

corrected – A new site collection with flagged stations removed.

Return type:

Sites

Examples

>>> det = AnomalyDetector(latent_dim=8)
>>> det.fit(X, epochs=80)
>>> clean_sites = det.apply(sites)

property history_: dict[str, list][source]

Training history recorded by the last fit() call.

Returns:: history – {"train_loss": [...], "val_loss": [...]}, one value per epoch. Empty when the PCA fallback was used (no epoch loop) or before fit() has been called. Pass directly to plot_training_history().
Return type:: dict

property threshold_: float | None[source]

Fitted anomaly-score threshold.

The percentile (threshold_percentile) of the training reconstruction-error distribution used by flag_anomalies(). None before fit().