pycsamt.forward.batch#

Parallelised synthetic EM dataset generator.

generate_dataset() is the single entry point. It creates a large collection of (data, model) pairs suitable for training 1-D neural network inverters.

Quick start#

>>> import numpy as np
>>> from pycsamt.forward.batch import generate_dataset
>>> ds = generate_dataset(
...     solver="mt1d",
...     n_samples=5_000,
...     freqs=np.logspace(-3, 4, 30),
...     n_layers=(3, 7),
...     noise_level=0.05,
...     seed=42,
...     n_jobs=4,
...     output="mt1d_train.npz",
... )
>>> ds.X.shape          # (5000, 60) — log10(rho_a) + phase at 30 freqs
>>> ds.y.shape          # (5000, 9) — log10(rho) and thickness for up to 7 layers

Dataset layout#

X

Feature matrix, shape (n_samples, n_features). For MT: [log10(rho_a_0), …, log10(rho_a_{nf-1}), phi_0, …, phi_{nf-1}]. For TEM: [log10(|dBz/dt|_0), …, log10(|dBz/dt|_{nt-1})].

y

Target matrix, shape (n_samples, n_params). [log10(rho_0), …, log10(rho_{nl-1}), thick_0, …, thick_{nl-2}]. Padded with NaN for samples with fewer than max_layers layers.

meta

Structured array with per-sample metadata: n_layers, depth_max, noise_level, seed.

Functions

generate_dataset([solver, n_samples, freqs, ...])

Generate a batch of synthetic (data, model) pairs for ML training.

generate_dataset_3d([solver, n_surveys, ...])

Generate a pseudo-3D synthetic dataset for GCNInverter3D training.

Classes

ForwardDataset(X, y[, freqs, times, meta, ...])

Numpy container for a batch of (features, targets) EM samples.

SurveyDataset3D(X, y, coords[, freqs, meta, ...])

Numpy container for a pseudo-3D multi-station survey dataset.
pycsamt.forward.batch.generate_dataset(solver='mt1d', n_samples=10000, *, freqs=None, times=None, n_layers=(3, 7), rho_range=(1.0, 10000.0), depth_max=2000.0, loop_radius=50.0, noise_level=0.05, noise_type='gaussian', geology=None, include_phase=True, seed=None, n_jobs=1, output=None, verbose=True)[source]#

Generate a batch of synthetic (data, model) pairs for ML training.

Parameters:

  • solver ({'mt1d', 'tem1d', 'csamt1d'}) – Forward solver to use.

  • n_samples (int) – Total number of samples.

  • freqs (ndarray or None) – Frequencies [Hz] for MT/CSAMT. If None, uses np.logspace(-3, 4, 30) for MT.

  • times (ndarray or None) – Times [s] for TEM. If None, uses np.logspace(-6, -2, 25).

  • n_layers (int or (lo, hi)) – Fixed number of layers, or a range from which the count is drawn uniformly at random per sample.

  • rho_range ((low, high)) – Resistivity bounds in Ω·m.

  • depth_max (float) – Maximum depth of the model [m].

  • loop_radius (float) – TEM transmitter loop radius [m].

  • noise_level (float) – Relative noise standard deviation. 0 = noise-free.

  • noise_type (str) – Noise model: 'gaussian', 'multiplicative', 'field'.

  • geology (str or None) – If given, models are drawn from LayeredModel.from_geology() using this geological scenario name.

  • include_phase (bool) – Include impedance phase in the MT feature vector.

  • seed (int or None) – Base random seed. Worker seeds are derived deterministically.

  • n_jobs (int) – Number of parallel worker processes. -1 uses all CPU cores.

  • output (str or None) – If given, save the dataset to a .npz file at this path.

  • verbose (bool) – Print progress.

Return type:

ForwardDataset

Examples

>>> import numpy as np
>>> from pycsamt.forward.batch import generate_dataset
>>> ds = generate_dataset(n_samples=100, seed=0)
>>> ds.X.shape[0]
100
class pycsamt.forward.batch.ForwardDataset(X, y, freqs=None, times=None, meta=None, solver='mt1d')[source]#

Bases: object

Numpy container for a batch of (features, targets) EM samples.

Parameters:

  • X (ndarray, shape (n_samples, n_features)) – Model responses (log-scaled data).

  • y (ndarray, shape (n_samples, n_params)) – Model parameters (log10 resistivity + thickness).

  • freqs (ndarray or None) – Frequencies used (MT/CSAMT).

  • times (ndarray or None) – Times used (TEM).

  • meta (structured ndarray) – Per-sample metadata (n_layers, noise_level, …).

  • solver (str) – Solver used to generate this dataset.

X: ndarray#
y: ndarray#
freqs: ndarray | None = None#
times: ndarray | None = None#
meta: ndarray | None = None#
solver: str = 'mt1d'#
save(path)[source]#

Save to a compressed .npz file.

Parameters:

path (str)

Return type:

None

classmethod load(path)[source]#

Load from a .npz file produced by save().

Parameters:

path (str)

Return type:

ForwardDataset

split(val_frac=0.1, test_frac=0.1, seed=None)[source]#

Split into train / validation / test sets.

Returns:

(train, val, test)

Return type:

tuple of ForwardDataset

Parameters:
pycsamt.forward.batch.generate_dataset_3d(solver='mt1d', n_surveys=1000, *, n_stations=25, n_layers=4, freqs=None, extent=10000.0, corr_length=2000.0, log_rho_mean=2.0, log_rho_std=0.5, thickness_range=(100.0, 2000.0), station_layout='grid', noise_level=0.03, noise_type='gaussian', include_phase=True, seed=None, n_jobs=1, output=None, verbose=True)[source]#

Generate a pseudo-3D synthetic dataset for GCNInverter3D training.

Each survey realisation consists of n_stations MT soundings whose per-layer log-resistivities are drawn from a spatially correlated Gaussian random field (GRF) with a squared-exponential covariance:

\[C(\mathbf{x}_i,\mathbf{x}_j) = \exp\!\left(-\frac{\|\mathbf{x}_i-\mathbf{x}_j\|^2} {2\,\ell^2}\right)\]

where \(\ell\) is corr_length. Layer thicknesses are drawn log-uniformly per survey but kept spatially constant (flat-layer assumption within one survey). All surveys share the same fixed station grid so that a single adjacency matrix covers the whole training set.

Parameters:

  • solver (str) – Forward solver. Currently only 'mt1d' is supported.

  • n_surveys (int) – Number of independent survey realisations.

  • n_stations (int) – Number of MT stations per survey.

  • n_layers (int) – Fixed number of earth layers (including halfspace).

  • freqs (ndarray or None) – Frequencies [Hz]. Defaults to np.logspace(-3, 4, 30).

  • extent (float) – Side length of the square survey area in metres.

  • corr_length (float) – Spatial correlation length of the GRF in metres. Set smaller than station spacing for uncorrelated models; larger for smooth lateral variation.

  • log_rho_mean (float) – Mean of log₁₀(ρ) for all layers (default 2 → 100 Ω·m).

  • log_rho_std (float) – Marginal standard deviation of the GRF in log₁₀(ρ) units.

  • thickness_range ((lo, hi)) – Bounds [m] for log-uniform layer thickness draws.

  • station_layout ('grid' or 'random') – 'grid' places stations on a regular grid of ⌈√n_stations⌉² points (default). 'random' draws uniform positions within [0, extent]².

  • noise_level (float) – Relative noise standard deviation applied to each response.

  • noise_type (str) – Noise model: 'gaussian', 'multiplicative', 'field'.

  • include_phase (bool) – Include impedance phase in the feature vector.

  • seed (int or None) – Base random seed. Worker seeds are derived deterministically.

  • n_jobs (int) – Number of parallel worker processes. -1 uses all CPU cores.

  • output (str or None) – If given, save the dataset to a compressed .npz file.

  • verbose (bool) – Print progress.

Returns:

Container with arrays X (n_surveys, n_stations, n_features), y (n_surveys, n_stations, n_params), and coords (n_stations, 2).

Return type:

SurveyDataset3D

Examples

>>> import numpy as np
>>> from pycsamt.forward.batch import generate_dataset_3d
>>> from pycsamt.ai.nets.gcn import build_adjacency
>>> ds = generate_dataset_3d(n_surveys=500, n_stations=16,
...                          n_layers=4, corr_length=2000., seed=0)
>>> ds.X.shape
(500, 16, 60)
>>> A = build_adjacency(ds.coords, radius=3000.)
>>> from pycsamt.ai.inversion.inv3d import GCNInverter3D
>>> inv = GCNInverter3D(n_features=ds.n_features, n_layers=4)
>>> inv.fit(ds.X, ds.y, adjacency=A, epochs=5, verbose=False)
class pycsamt.forward.batch.SurveyDataset3D(X, y, coords, freqs=None, meta=None, solver='mt1d')[source]#

Bases: object

Numpy container for a pseudo-3D multi-station survey dataset.

Designed for training GCNInverter3D. All surveys share the same fixed station grid so that a single pre-computed adjacency matrix covers the whole dataset.

Parameters:

  • X (ndarray, shape (n_surveys, n_stations, n_features)) – Per-station MT/CSAMT feature vectors (log-scaled data).

  • y (ndarray, shape (n_surveys, n_stations, n_params)) – Per-station model parameters [log10(ρ), thickness].

  • coords (ndarray, shape (n_stations, 2)) – Station (easting, northing) coordinates in metres.

  • freqs (ndarray or None) – Frequencies [Hz] used by the forward solver.

  • meta (structured ndarray or None) – Per-survey metadata (corr_length, noise_level).

  • solver (str) – Solver used to generate this dataset.

X: ndarray#
y: ndarray#
coords: ndarray#
freqs: ndarray | None = None#
meta: ndarray | None = None#
solver: str = 'mt1d'#
property n_surveys: int[source]#

Number of synthetic survey realisations.

property n_stations: int[source]#

Number of stations per survey.

property n_features: int[source]#

Feature dimensionality per station.

property n_params: int[source]#

Parameter dimensionality per station (2 × n_layers − 1).

save(path)[source]#

Save to a compressed .npz file.

Parameters:

path (str)

Return type:

None

classmethod load(path)[source]#

Load from a .npz file produced by save().

Parameters:

path (str)

Return type:

SurveyDataset3D

split(val_frac=0.1, test_frac=0.1, seed=None)[source]#

Split into train / validation / test sets along the survey axis.

Returns:

(train, val, test)

Return type:

tuple of SurveyDataset3D

Parameters: