EDI Recompute Workflow#

The pycsamt.site.recompute module provides a higher-level workflow for normalizing EDI files and rewriting them with pyCSAMT. It is intended for surveys where EDI files may come from another package or commercial software, and you want to rebuild the transfer-function blocks, derived resistivity/phase values, station names, and output folder layout in a controlled way.

Use this workflow when you need to:

  • read one EDI file, one line folder, a survey folder, an EDICollection, or an in-memory list of EDI objects;

  • rotate impedance and/or tipper components before writing;

  • keep only a frequency band before recomputation;

  • fill missing values before computing derived quantities;

  • recompute apparent resistivity and phase from impedance;

  • write new EDI files using the pyCSAMT writer;

  • preserve line folders or flatten all recomputed stations into one folder;

  • write a manifest that records source paths, output paths, line names, station names, and processing status.

The default output folder name is recomputed_edis. This is intentionally plain: users opening the project directory can immediately see that the folder contains EDI files regenerated by pyCSAMT.

Tool Map#

Tool

Scope

Main purpose

EDIRecomputer

Workflow class

Configure and run a reproducible recomputation/export workflow.

recompute_edis()

Convenience function

Run EDIRecomputer with keyword arguments.

recompute_edi()

One EDI object

Apply the same recomputation operations to one in-memory EDI object.

EDIRecomputeResult

Result object

Hold recomputed EDI objects, wrapped sites, output paths, and records.

EDIRecomputeRecord

Manifest row

Describe one station outcome.

Quick Start#

Recompute a single line folder and keep the line name in the output tree:

 1from pycsamt.site.recompute import recompute_edis
 2
 3result = recompute_edis(
 4    "WILLY_DATA/L18PLT",
 5    rotate_angle=30.0,
 6    template="{source_stem}.edi",
 7    overwrite=True,
 8    progress=True,
 9    verbose=1,
10)
11
12print(result.output_root)
13print(result.paths)

For the input:

1WILLY_DATA/
2  L18PLT/
3    S001.edi
4    S002.edi

the default output is:

1WILLY_DATA/
2  recomputed_edis/
3    L18PLT/
4      S001.edi
5      S002.edi
6    manifest.csv

Whole Survey Folders#

When the source directory contains subdirectories with EDI files, each subdirectory is treated as a survey line.

1WILLY_DATA/
2  L18PLT/
3    *.edi
4  L32PLT/
5    *.edi
6  L22PLT/
7    *.edi

Run:

 1from pycsamt.site.recompute import EDIRecomputer
 2
 3runner = EDIRecomputer(
 4    rotate_angle=30.0,
 5    rotate_components=("Z", "Tip"),
 6    recompute_resphase=True,
 7    template="{source_stem}.edi",
 8    overwrite=True,
 9)
10
11result = runner.run("WILLY_DATA")

The output preserves the line structure:

1WILLY_DATA/
2  recomputed_edis/
3    L18PLT/
4      *.edi
5    L32PLT/
6      *.edi
7    L22PLT/
8      *.edi
9    manifest.csv

Flattened Output#

Set preserve_line_dirs=False when all recomputed files should be written directly inside the output root. In that case, include {line} in the filename template to avoid collisions between stations from different lines.

1result = recompute_edis(
2    "WILLY_DATA",
3    preserve_line_dirs=False,
4    template="{line}_{source_stem}.edi",
5    overwrite=True,
6)

This writes:

1WILLY_DATA/
2  recomputed_edis/
3    L18PLT_S001.edi
4    L32PLT_S001.edi
5    L22PLT_S001.edi
6    manifest.csv

Operation Order#

For each station, EDIRecomputer applies operations in this order:

  1. load the EDI object, or use the provided in-memory EDI object;

  2. copy it by default, so the original object is not mutated;

  3. rotate selected components when rotate_angle is provided;

  4. subset the frequency range when fmin/fmax or keep_freq is provided;

  5. fill missing values when fill_missing_values is provided;

  6. recompute apparent resistivity and phase when recompute_resphase=True;

  7. apply the optional station rename_policy;

  8. write the EDI with pyCSAMT when write=True;

  9. append a record to the result manifest.

This order is important. For example, when a frequency band is selected, resistivity and phase are recomputed only for the retained frequency rows.

Rotation Control#

By default, both impedance and tipper are rotated when present.

1result = recompute_edis(
2    "WILLY_DATA/L18PLT",
3    rotate_angle=30.0,
4    rotate_components=("Z", "Tip"),
5)

Rotate only impedance:

1result = recompute_edis(
2    "WILLY_DATA/L18PLT",
3    rotate_angle=30.0,
4    rotate_components=("Z",),
5)

Rotate only tipper:

1result = recompute_edis(
2    "WILLY_DATA/L18PLT",
3    rotate_angle=30.0,
4    rotate_components=("Tip",),
5)

Frequency And Missing Values#

Keep a frequency band before recomputing derived quantities:

1result = recompute_edis(
2    "WILLY_DATA",
3    fmin=1.0,
4    fmax=1000.0,
5    recompute_resphase=True,
6)

Fill missing values before recomputation:

1result = recompute_edis(
2    "WILLY_DATA",
3    fill_missing_values="nan",
4)

Use "zero" only when zeros are meaningful for your downstream workflow. For quality-control workflows, "nan" is often safer because missing values remain visible in later checks.

Filename Templates#

The recompute workflow supports these filename template keys:

Key

Meaning

{station}

Station name after optional renaming.

{index}

Zero-based order across all recomputed inputs.

{line}

Inferred line folder name, such as L18PLT.

{source_stem}

Stem of the source EDI filename.

Examples:

 1recompute_edis(
 2    "WILLY_DATA",
 3    template="{source_stem}.edi",
 4)
 5
 6recompute_edis(
 7    "WILLY_DATA",
 8    preserve_line_dirs=False,
 9    template="{line}_{source_stem}.edi",
10)
11
12recompute_edis(
13    "WILLY_DATA",
14    template="{index:04d}_{station}.edi",
15)

If a template does not end with .edi, the extension is appended automatically.

Result Object#

The returned EDIRecomputeResult gives access to both in-memory objects and written paths.

1result = recompute_edis("WILLY_DATA/L18PLT")
2
3edis = result.edis
4sites = result.sites
5paths = result.paths
6failed = result.failed
7
8for record in result.records:
9    print(record.station, record.status, record.output)

The sites attribute is a pycsamt.site.base.Sites wrapper for convenient station-centric inspection. The edis property preserves every recomputed EDI object, including cases where imported files originally had duplicate station identifiers.

Manifest CSV#

By default, written workflows create:

recomputed_edis/manifest.csv

The manifest contains:

Column

Meaning

source

Original EDI path when known.

output

Recomputed EDI output path.

line

Inferred line name.

station

Station name after recomputation and optional renaming.

status

ok or failed.

message

Error text for failed rows.

You can also write a manifest manually:

1result = recompute_edis("WILLY_DATA", manifest_csv=False)
2result.to_manifest("reports/recompute_manifest.csv")

In-Memory Recompute#

Use write=False when you want recomputed objects but do not want files on disk.

1result = recompute_edis(
2    edis,
3    rotate_angle=15.0,
4    write=False,
5)
6
7recomputed_edis = result.edis

This is useful in notebooks, tests, or workflows that pass EDI objects directly to another pyCSAMT layer.

Command-Line Use#

The same workflow is exposed through:

1pycsamt site recompute WILLY_DATA/L18PLT --rotate 30 --progress

Recompute a whole survey folder and preserve line folders:

1pycsamt site recompute WILLY_DATA --rotate 30 --overwrite

Flatten all output files into one directory:

1pycsamt site recompute WILLY_DATA \
2    --flatten \
3    --template "{line}_{source_stem}.edi" \
4    --overwrite

Use an explicit output directory:

1pycsamt site recompute WILLY_DATA \
2    --output-dir cleaned_edis \
3    --template "{source_stem}.edi"

Dry-run without writing files:

1pycsamt site recompute WILLY_DATA/L18PLT --rotate 30 --dry-run

Available CLI options include --rotate, --components, --freq, --fill-missing, --skip-rho-phase, --datatype, --synthesize-spectra, --flatten, --output-dir, --output-name, --template, --no-manifest, --dry-run, --progress, and --overwrite.

Relationship To Editing And Export#

The recompute workflow builds on the lower-level editing and export tools:

Use pycsamt.site.edit when your code already owns a small number of EDI objects and needs one operation. Use pycsamt.site.recompute when the task is a complete survey delivery or a cleanup pass over EDI files imported from another software package.