Noise Removal And Spatial Filtering#

pycsamt.emtools.remove_noise is the main post-estimation cleaning module for CSAMT/AMT/MT transfer functions. It operates on impedance and tipper arrays already stored in EDI or Sites objects. It does not estimate transfer functions from time series, and it does not perform true remote-reference processing internally. Use it after data loading, and before inversion or final interpretation, when the survey contains power-line harmonics, isolated frequency spikes, station-local outliers, station-to-station jumps, or rows that should be filtered more strongly because their confidence is low.

The module has many functions because noise is not one problem. A single CSAMT line may need one or more of these treatments:

Step

Use when

Typical functions

Diagnose noise

You need to see which stations and frequencies are weak before editing the survey.

snr_table, emi_mitigation_report

Remove mains harmonics

Frequencies land near 50/60 Hz or harmonics.

notch_powerline

Smooth along frequency

Curves are locally jagged, but neighboring frequencies should be continuous.

smooth_logfreq, smooth_rho_phase

Remove isolated outliers

A few rows or stations are inconsistent with their local neighbors.

hampel_filter_freq, spatial_median_filter, rpca_offdiag_denoise

Enforce impedance consistency

Off-diagonal components should be closer to an anti-symmetric 1-D/2-D response.

enforce_offdiag_consistency

Mask or drop bad frequencies

Frequencies are globally weak or known to be contaminated.

mask_incoherent_freqs, drop_freqs_manual

Stabilize station profiles

One station is shifted relative to its neighbors, or an EMAP style spatial average is appropriate.

correct_static_shift, apply_emap_filter, confidence_gated_emap_filter

Verify the correction

You need before/after figures for the report or paper.

emap_filter_report, plot_emap_filter_profile, plot_emap_filter_psection, nr_qc_* plots

Full function signatures and parameter defaults are maintained in the API reference. This page focuses on practical usage, decision making, and reproducible code.

Loading A Survey Safely#

Noise removal functions accept the same inputs as most emtools workflows: a directory of EDI files, a single EDI path, an existing Sites object, or an iterable of station-like objects. For repeatable processing, load once with ensure_sites and keep the raw object unchanged while you test settings.

 1from pathlib import Path
 2
 3from pycsamt.emtools import ensure_sites
 4
 5edi_dir = Path("data/AMT/WILLY_DATA/L18PLT")
 6raw = ensure_sites(edi_dir, recursive=True, verbose=0)
 7
 8# Most remove_noise functions default to inplace=False. The returned
 9# object is therefore the processed survey, while raw remains the
10# baseline for before/after QC.
11processed = raw

Use inplace=False during exploration. Switch to inplace=True only inside a controlled pipeline where you no longer need the raw object in memory.

SNR Diagnostics#

The simplest diagnostic is snr_table. It returns one row per station and frequency. For each row, pyCSAMT computes an impedance-amplitude signal-to-noise ratio from Z and Z_err:

\[\mathrm{SNR} = \frac{\sqrt{\overline{|Z|^2}}} {\sqrt{\overline{|Z_{\mathrm{err}}|^2}}}\]

If the input EDI files do not contain impedance errors, the SNR values are NaN. That is useful information: it means later SNR-gated steps cannot make a data-driven decision from error bars.

 1from pycsamt.emtools import ensure_sites
 2from pycsamt.emtools.remove_noise import snr_table
 3
 4sites = ensure_sites("data/AMT/WILLY_DATA/L18PLT", recursive=True)
 5snr = snr_table(sites)
 6
 7print(snr.head())
 8print(snr["snr"].describe())
 9
10weak_rows = snr.loc[snr["snr"] < 3.0, ["station", "freq", "snr"]]
11weak_by_station = (
12    weak_rows.groupby("station", as_index=False)
13    .agg(n_weak=("freq", "size"), min_snr=("snr", "min"))
14    .sort_values(["n_weak", "min_snr"], ascending=[False, True])
15)
16print(weak_by_station.head(10))
station     freq        snr
0  18-001A  10400.0  24.379116
1  18-001A   8707.0  21.511108
2  18-001A   7289.0  21.241377
3  18-001A   6102.0  15.230757
4  18-001A   5108.0  13.140256
count    1484.000000
mean       14.064419
std         5.878163
min         2.187781
25%         9.679665
50%        13.271554
75%        17.294278
max        56.055512
Name: snr, dtype: float64
   station  n_weak   min_snr
2  18-024U       3  2.187781
1  18-022U       1  2.388301
0  18-018A       1  2.808353

Import snr_table from pycsamt.emtools.remove_noise when you are documenting this module specifically. The top-level pycsamt.emtools namespace also exposes an snr_table name used by the spectra tools, so the explicit module import avoids ambiguity.

Remote Reference And EMI Reporting#

The remove-noise layer works on estimated transfer functions. It cannot create a remote-referenced impedance tensor unless remote-reference processing was already performed by field software or an external processor. What it can do is make the mitigation record explicit: whether remote reference was attempted, whether it was available, which power-line notch settings were used, and how many harmonic samples were affected.

 1from pycsamt.emtools import ensure_sites
 2from pycsamt.emtools.remove_noise import emi_mitigation_report
 3
 4sites = ensure_sites("data/AMT/WILLY_DATA/L18PLT", recursive=True)
 5
 6emi = emi_mitigation_report(
 7    sites,
 8    remote_reference_attempted=False,
 9    remote_reference_reason=(
10        "No independent remote-reference time series were acquired; "
11        "post-estimation EMI mitigation was applied to the EDI transfer functions."
12    ),
13    mains_hz=50.0,
14    n_harm=30,
15    tol_hz=0.08,
16    notch_mode="interp",
17)
18
19print(
20    emi[
21        [
22            "station",
23            "remote_reference_attempted",
24            "remote_reference_available",
25            "n_frequency",
26            "harmonic_z_samples",
27            "harmonic_tipper_samples",
28            "applied_measures",
29        ]
30    ].head()
31)
   station  ...                                   applied_measures
0  18-001A  ...  notch_powerline(mode=interp, mains_hz=50, n_ha...
1  18-002U  ...  notch_powerline(mode=interp, mains_hz=50, n_ha...
2  18-003A  ...  notch_powerline(mode=interp, mains_hz=50, n_ha...
3  18-004A  ...  notch_powerline(mode=interp, mains_hz=50, n_ha...
4  18-005U  ...  notch_powerline(mode=interp, mains_hz=50, n_ha...

[5 rows x 7 columns]

This report is especially useful in manuscripts and reproducibility bundles because it states what was not done as clearly as what was done. If remote-reference EDIs are supplied, load those files and set the report fields accordingly.

Power-Line Notching#

notch_powerline finds frequencies within tol_hz of mains_hz * k for harmonics k = 1 .. n_harm. It can either set those rows to NaN with mode="mask" or replace them by interpolation from neighboring frequencies with mode="interp".

 1from pycsamt.emtools import ensure_sites, notch_powerline
 2
 3sites = ensure_sites("data/AMT/WILLY_DATA/L18PLT", recursive=True)
 4
 5notched = notch_powerline(
 6    sites,
 7    mains_hz=50.0,
 8    n_harm=30,
 9    tol_hz=0.08,
10    mode="interp",
11    also="both",
12    inplace=False,
13)

Use mode="mask" when you want contaminated rows to remain visibly missing for later QC. Use mode="interp" when the downstream workflow requires a complete frequency axis. Use also="z" for impedance only, also="tipper" for tipper only, or also="both" when both tensors share the same contaminated frequencies.

Sparse logarithmic CSAMT grids often do not land exactly on 50/60 Hz harmonics. In that case the function may make no changes at conservative tol_hz values. That is not a failure. It means the sampled frequencies are not close enough to the harmonic rows you asked it to treat.

Log-Frequency Smoothing#

smooth_logfreq applies a moving average directly to complex tensor rows along the frequency axis. It is a local smoother. It does not assume a global resistivity model; it only says that neighboring log-frequency samples should be less jagged.

 1from pycsamt.emtools import ensure_sites, smooth_logfreq
 2
 3sites = ensure_sites("data/AMT/WILLY_DATA/L18PLT", recursive=True)
 4
 5smoothed = smooth_logfreq(
 6    sites,
 7    win=5,
 8    kind="tri",
 9    also="both",
10    gate_snr=2.5,
11    inplace=False,
12)

win is the number of frequency rows used by the moving window. kind="tri" gives the center row more weight than the edges; kind="box" gives every row equal weight. gate_snr limits smoothing to rows that pass the chosen SNR logic inside the function. Set gate_snr=None when you deliberately want every row smoothed.

Rho/Phase Trend Smoothing#

smooth_rho_phase is more interpretive. It converts impedance components to apparent resistivity and phase, fits a polynomial trend in log-frequency space, unwraps phase, then rebuilds complex impedance from the smoothed curves.

 1from pycsamt.emtools import ensure_sites, smooth_rho_phase
 2
 3sites = ensure_sites("data/AMT/WILLY_DATA/L18PLT", recursive=True)
 4
 5trend = smooth_rho_phase(
 6    sites,
 7    components="offdiag",
 8    degree=3,
 9    min_points=None,
10    smooth_rho=True,
11    smooth_phase=True,
12    robust=True,
13    robust_iters=3,
14    blend=1.0,
15    inplace=False,
16)

Choose smooth_logfreq for local jitter. Choose smooth_rho_phase when you expect a station curve to follow a smooth global trend. components can be "offdiag", "diagonal", "all", or an explicit list such as ("xy", "yx"). blend lets you mix the filtered value with the original value; for example blend=0.5 moves only halfway toward the fitted trend.

Outlier And Spatial Denoising#

Outlier removal can operate along one station’s frequency curve, across neighboring stations, or across the whole survey matrix.

 1from pycsamt.emtools import (
 2    ensure_sites,
 3    hampel_filter_freq,
 4    rpca_offdiag_denoise,
 5    spatial_median_filter,
 6)
 7
 8sites = ensure_sites("data/AMT/WILLY_DATA/L18PLT", recursive=True)
 9
10# Per-station, along frequency. Best for isolated spikes in one curve.
11hampel = hampel_filter_freq(
12    sites,
13    win=3,
14    nsig=3.0,
15    on="z",
16    inplace=False,
17)
18
19# Across station order. Best for a station that jumps relative to its
20# neighbors at the same frequency.
21spatial = spatial_median_filter(
22    sites,
23    half_window=2,
24    lam=0.25,
25    on="z",
26    inplace=False,
27)
28
29# Survey-wide low-rank denoising of off-diagonal log magnitudes.
30# Best when the line has a common coherent trend plus sparse outliers.
31low_rank = rpca_offdiag_denoise(
32    sites,
33    rank=2,
34    inplace=False,
35)

hampel_filter_freq is conservative at typical nsig values. It does not remove a station merely because the station is high amplitude; it removes rows that are local outliers relative to the station’s own nearby frequencies. spatial_median_filter uses neighboring stations, so it should be applied only when station order has spatial meaning. rpca_offdiag_denoise can be powerful, but a low-rank model may damp a real station anomaly if that anomaly is not shared by the rest of the line. Always inspect before/after curves when using it.

Off-Diagonal Consistency#

For a simple 1-D or approximately 2-D response, the off-diagonal impedance terms often satisfy an anti-symmetric relationship: Zxy ~= -Zyx. enforce_offdiag_consistency blends the observed off-diagonal components toward that target.

 1from pycsamt.emtools import ensure_sites, enforce_offdiag_consistency
 2
 3sites = ensure_sites("data/AMT/WILLY_DATA/L18PLT", recursive=True)
 4
 5consistent = enforce_offdiag_consistency(
 6    sites,
 7    mode="anti",
 8    lam=0.5,
 9    inplace=False,
10)

lam=0 leaves the data unchanged. lam=1 fully replaces the selected components by the consistency target. Intermediate values are usually safer because real 3-D structure, galvanic distortion, and measurement geometry can all create departures from the ideal anti-symmetric relation.

Masking And Manual Frequency Drops#

mask_incoherent_freqs uses the survey SNR table to identify frequencies where too few stations pass an SNR threshold. It masks those frequencies rather than pretending they are reliable.

 1from pycsamt.emtools import (
 2    drop_freqs_manual,
 3    ensure_sites,
 4    mask_incoherent_freqs,
 5)
 6
 7sites = ensure_sites("data/AMT/WILLY_DATA/L18PLT", recursive=True)
 8
 9masked = mask_incoherent_freqs(
10    sites,
11    snr_thresh=2.5,
12    min_frac=0.4,
13    inplace=False,
14)
15
16trimmed = drop_freqs_manual(
17    sites,
18    drop_freqs=[50.0, 60.0],
19    tol_rel=0.005,
20    inplace=False,
21)

Use mask_incoherent_freqs for a rule based on the data. Use drop_freqs_manual when the field log, instrument notes, or a diagnostic figure identifies specific frequencies. Manual drops use a relative tolerance, so give the sampled frequency value when possible.

Group-Trend Shrinkage#

shrink_to_group_trend pulls station curves toward a group median trend. By default it is gated to harmonic rows, which makes it a targeted EMI treatment rather than a blanket smoothing operation.

 1from pycsamt.emtools import ensure_sites, shrink_to_group_trend
 2
 3sites = ensure_sites("data/AMT/WILLY_DATA/L18PLT", recursive=True)
 4
 5harmonic_shrink = shrink_to_group_trend(
 6    sites,
 7    lam=0.25,
 8    gate_harm=True,
 9    mains_hz=50.0,
10    n_harm=30,
11    tol_hz=0.08,
12    inplace=False,
13)
14
15all_rows_shrink = shrink_to_group_trend(
16    sites,
17    lam=0.25,
18    gate_harm=False,
19    inplace=False,
20)

Keep gate_harm=True when the problem is power-line contamination. Use gate_harm=False only when you have decided that the entire line should be pulled toward a common spatial trend.

Static Shift And EMAP Filters#

The module contains two related families of station-profile filters. correct_static_shift implements a Torres-Verdin and Bostick style Hanning adaptive moving-average correction. apply_emap_filter is a dispatcher for adaptive moving average ("ama"), fixed-length moving average ("flma"), and trimmed moving average ("tma").

 1from pycsamt.emtools import (
 2    apply_emap_filter,
 3    correct_static_shift,
 4    ensure_sites,
 5)
 6
 7sites = ensure_sites("data/AMT/WILLY_DATA/L18PLT", recursive=True)
 8
 9ama_static = correct_static_shift(
10    sites,
11    window_m=1500.0,
12    comp="xy",
13    inplace=False,
14)
15
16flma = apply_emap_filter(
17    sites,
18    method="flma",
19    window=5,
20    component="xy",
21    inplace=False,
22)
23
24tma = apply_emap_filter(
25    sites,
26    method="tma",
27    window=5,
28    component="xy",
29    inplace=False,
30)

These filters assume that station order and station spacing are meaningful. They are most useful on survey lines where neighboring stations should share a broad geoelectric trend and sudden station-to-station jumps are likely to be static shift or local noise. They are less appropriate when a sharp station-local feature is a known target.

EMAP Reports And Plots#

Before using an EMAP-filtered survey downstream, summarize where the filter changed the data.

 1import matplotlib.pyplot as plt
 2
 3from pycsamt.emtools import (
 4    apply_emap_filter,
 5    emap_filter_report,
 6    ensure_sites,
 7    plot_emap_filter_profile,
 8    plot_emap_filter_psection,
 9)
10
11sites = ensure_sites("data/AMT/WILLY_DATA/L18PLT", recursive=True)
12flma = apply_emap_filter(
13    sites,
14    method="flma",
15    window=5,
16    component="xy",
17    inplace=False,
18)
19
20report = emap_filter_report(
21    sites,
22    flma,
23    component="xy",
24)
25print(report.sort_values("rms_delta_log10_abs_z", ascending=False).head())
26
27fig, ax = plt.subplots(figsize=(8, 4))
28plot_emap_filter_profile(
29    sites,
30    flma,
31    method="flma",
32    component="xy",
33    ax=ax,
34)
35fig.tight_layout()
36fig.savefig("emap_filter_profile_xy.png", dpi=200)
37plt.close(fig)
38
39fig = plot_emap_filter_psection(
40    sites,
41    flma,
42    method="flma",
43    component="xy",
44)
45fig.savefig("emap_filter_psection_xy.png", dpi=200)
46plt.close(fig)
    station component  ...  median_delta_log10_abs_z  rms_delta_log10_abs_z
18  18-019U        xy  ...                  0.415765               0.505929
26  18-024U        xy  ...                  0.346431               0.401617
22  18-022U        xy  ...                  0.360391               0.391346
17  18-018A        xy  ...                  0.361853               0.377488
23  18-022V        xy  ...                  0.300769               0.310826

[5 rows x 5 columns]

plot_emap_filter_profile is the quickest way to check one frequency profile along the line. plot_emap_filter_psection shows the before/after/delta pseudo-section across stations and periods. Use both: one exposes station-to-station behavior clearly, and the other shows whether corrections concentrate in a narrow period band.

Confidence-Gated EMAP Filtering#

confidence_gated_emap_filter connects the noise-removal module to the QC module. It builds a frequency confidence table, computes an EMAP filtered candidate, then decides row by row how much filtered data to use:

  • confidence above ci_hi: preserve the original row;

  • confidence below ci_lo: use the filtered row;

  • confidence between the thresholds: blend original and filtered rows.

 1from pycsamt.emtools import confidence_gated_emap_filter, ensure_sites
 2
 3sites = ensure_sites("data/AMT/WILLY_DATA/L18PLT", recursive=True)
 4
 5result = confidence_gated_emap_filter(
 6    sites,
 7    method="flma",
 8    window=5,
 9    ci_hi=0.90,
10    ci_lo=0.50,
11    component="xy",
12)
13
14print(result.summary())
15print(result.report.head())
16print(result.decisions.head())
17
18gated_sites = result.sites
EMAPFilterResult(method='flma', confidence='composite', preserved=0, blended=1411, filtered=73)
   station  n_freq  ...  median_confidence  median_delta_log10_abs_z
0  18-001A      53  ...           0.711753                 -0.020918
1  18-002U      53  ...           0.749480                  0.015749
2  18-003A      53  ...           0.666613                  0.046988
3  18-004A      53  ...           0.735994                  0.004156
4  18-005U      53  ...           0.728841                 -0.025544

[5 rows x 8 columns]
   station  frequency_hz  period_s  ...  blend_weight   action  delta_log10_abs_z
0  18-001A       10400.0  0.000096  ...      0.109609  blended          -0.002075
1  18-001A        8707.0  0.000115  ...      0.134217  blended          -0.001228
2  18-001A        7289.0  0.000137  ...      0.142135  blended          -0.001559
3  18-001A        6102.0  0.000164  ...      0.236784  blended          -0.006566
4  18-001A        5108.0  0.000196  ...      0.437715  blended          -0.007875

[5 rows x 8 columns]

The return value is an EMAPFilterResult. It keeps the processed sites object, a station-level report, a row-level decisions table, the EMAP method, and the confidence thresholds. The convenience properties n_preserved, n_blended, and n_filtered are useful for logging.

 1from pycsamt.emtools import confidence_gated_emap_filter, ensure_sites
 2
 3sites = ensure_sites("data/AMT/WILLY_DATA/L18PLT", recursive=True)
 4result = confidence_gated_emap_filter(sites, method="flma")
 5
 6print(result.n_preserved)
 7print(result.n_blended)
 8print(result.n_filtered)
 9
10most_filtered = (
11    result.report.sort_values(
12        ["n_filtered", "median_confidence"],
13        ascending=[False, True],
14    )
15    .loc[:, ["station", "n_preserved", "n_blended", "n_filtered", "median_confidence"]]
16)
17print(most_filtered.head(10))
0
1411
73
    station  n_preserved  n_blended  n_filtered  median_confidence
22  18-022U            0         38          15           0.569810
24  18-023A            0         41          12           0.612583
5   18-006A            0         47           6           0.764254
12  18-013U            0         48           5           0.669948
20  18-021B            0         49           4           0.594338
17  18-018A            0         50           3           0.585752
26  18-024U            0         50           3           0.631934
11  18-012A            0         50           3           0.651210
2   18-003A            0         50           3           0.666613
13  18-014A            0         50           3           0.699489

This is usually safer than applying the same spatial filter everywhere. High-confidence rows remain close to the measurement, while low confidence rows are allowed to borrow more from station-neighbor structure.

Full Pipeline#

remove_noise_pipeline provides a compact chain for common cleaning: power-line notching, frequency smoothing, and optional group-trend shrinkage. It is convenient for batch processing, but you should still run the individual diagnostics first so you know which part of the pipeline is doing the work.

 1from pathlib import Path
 2
 3from pycsamt.emtools import ensure_sites, remove_noise_pipeline
 4
 5sites = ensure_sites("data/AMT/WILLY_DATA/L18PLT", recursive=True)
 6
 7cleaned = remove_noise_pipeline(
 8    sites,
 9    mains_hz=50.0,
10    n_harm=30,
11    tol_hz=0.08,
12    notch_mode="interp",
13    smooth_win=5,
14    smooth_kind="tri",
15    gate_snr=2.5,
16    group_shrink=False,
17    inplace=False,
18)
19
20output_dir = Path("outputs/remove_noise")
21output_dir.mkdir(parents=True, exist_ok=True)

The exact keyword names are intentionally close to the lower-level functions. Keep your pipeline call in a script or notebook with all parameters written out. That makes the processing reproducible and prevents hidden defaults from changing the interpretation later.

QC Figures For Noise Removal#

The dedicated nr_qc_* figures compare a raw survey with a named noise-removal method. They are designed to be used after a method is chosen, not as a substitute for choosing the method carefully.

 1import matplotlib.pyplot as plt
 2
 3from pycsamt.emtools import (
 4    ensure_sites,
 5    nr_qc_delta_offdiag_psection,
 6    nr_qc_harmonic_waterfall,
 7    nr_qc_snr_gain_profile,
 8    nr_qc_station_offdiag_curves,
 9)
10
11sites = ensure_sites("data/AMT/WILLY_DATA/L18PLT", recursive=True)
12
13fig, ax = plt.subplots(figsize=(9, 5))
14nr_qc_delta_offdiag_psection(
15    sites,
16    method="pipeline",
17    ax=ax,
18)
19fig.savefig("nr_qc_delta_offdiag_psection.png", dpi=200)
20plt.close(fig)
21
22fig, ax = plt.subplots(figsize=(8, 4))
23nr_qc_snr_gain_profile(
24    sites,
25    method="pipeline",
26    ax=ax,
27)
28fig.tight_layout()
29fig.savefig("nr_qc_snr_gain_profile.png", dpi=200)
30plt.close(fig)
31
32fig, ax = plt.subplots(figsize=(9, 5))
33nr_qc_harmonic_waterfall(
34    sites,
35    method="notch",
36    mains_hz=50.0,
37    n_harm=5,
38    tol_hz=25.0,
39    ax=ax,
40)
41fig.savefig("nr_qc_harmonic_waterfall.png", dpi=200)
42plt.close(fig)
43
44fig, ax = plt.subplots(figsize=(8, 4))
45nr_qc_station_offdiag_curves(
46    sites,
47    method="pipeline",
48    station="18-016A",
49    ax=ax,
50)
51fig.tight_layout()
52fig.savefig("nr_qc_station_offdiag_curves_18-016A.png", dpi=200)
53plt.close(fig)

nr_qc_delta_offdiag_psection shows where off-diagonal magnitude changed in station-period space. nr_qc_snr_gain_profile summarizes SNR gain by station. nr_qc_harmonic_waterfall is specific to power-line harmonic reduction. nr_qc_station_offdiag_curves is the plainest check: one station, before and after, on the same axes.

Choosing A Treatment#

Start with the least interpretive operation that addresses the observed problem.

  • If only known harmonic rows are contaminated, start with notch_powerline.

  • If curves are locally jagged but geologically plausible, try smooth_logfreq with a small window.

  • If the whole curve should be smooth in apparent resistivity and phase, try smooth_rho_phase and inspect phase behavior.

  • If one row is a spike, use hampel_filter_freq.

  • If one station differs from its neighbors at the same frequencies, use spatial_median_filter or an EMAP filter.

  • If many stations share a coherent trend but a few observations depart from it, test rpca_offdiag_denoise carefully.

  • If QC confidence is already available, prefer confidence_gated_emap_filter over applying one fixed-strength spatial filter everywhere.

Do not stack every function by default. Each step changes the transfer function. A defensible workflow has a diagnostic reason for every correction and a before/after figure showing the effect.

Reproducible Bundle#

A practical processing bundle usually contains four outputs:

 1from pathlib import Path
 2
 3import matplotlib.pyplot as plt
 4
 5from pycsamt.emtools import (
 6    ensure_sites,
 7    nr_qc_station_offdiag_curves,
 8    remove_noise_pipeline,
 9)
10from pycsamt.emtools.remove_noise import emi_mitigation_report, snr_table
11
12out = Path("outputs/remove_noise_l18plt")
13out.mkdir(parents=True, exist_ok=True)
14
15sites = ensure_sites("data/AMT/WILLY_DATA/L18PLT", recursive=True)
16
17snr_table(sites).to_csv(out / "snr_table_raw.csv", index=False)
18emi_mitigation_report(
19    sites,
20    remote_reference_attempted=False,
21    mains_hz=50.0,
22    n_harm=30,
23    tol_hz=0.08,
24    notch_mode="interp",
25).to_csv(out / "emi_mitigation_report.csv", index=False)
26
27cleaned = remove_noise_pipeline(
28    sites,
29    mains_hz=50.0,
30    notch_mode="interp",
31    smooth_win=5,
32    smooth_kind="tri",
33    gate_snr=2.5,
34    group_shrink=False,
35    inplace=False,
36)
37
38fig, ax = plt.subplots(figsize=(8, 4))
39
40nr_qc_station_offdiag_curves(
41    sites,
42    method="pipeline",
43    station="18-016A",
44    ax=ax,
45)
46fig.savefig(out / "station_18-016A_pipeline_offdiag.png", dpi=200)
47plt.close(fig)
../../_images/user-guide-emtools-remove-noise-17.png

Keep the raw SNR table, EMI report, processing script, and representative QC figures together. That gives reviewers enough information to understand both the data quality and the editing decisions.

Worked Example#

The example uses the real L18PLT survey where possible and small synthetic dense-frequency objects only where the real frequency grid does not contain power-line harmonic rows. It demonstrates SNR diagnostics, notching, smoothing, Hampel/spatial/RPCA denoising, consistency enforcement, masking, group-trend shrinkage, EMAP filtering, confidence-gated EMAP filtering, the full pipeline, and the dedicated QC plots.

Open the rendered gallery page here: Noise removal and spatial filtering (pycsamt.emtools.remove_noise).