Export And Reporting#

The export and reporting tools are the last mile of a site workflow. Export functions write cleaned EDI-like objects to disk or package them into an archive. Report classes summarize one station or a whole survey for terminal inspection, notebooks, QA logs, and downstream tables.

Use this page when you need to:

  • write one corrected station to a known file path;

  • batch-write a selected survey with deterministic file names;

  • create a manifest CSV for a delivery directory or zip archive;

  • package cleaned sites into a zip file;

  • generate single-station and survey-level report tables;

  • understand how terminal reports, dictionaries, and DataFrames differ.

If the goal is not only to write existing sites, but also to rotate, frequency-filter, recompute resistivity/phase, and regenerate EDI files from another software source, use EDI Recompute Workflow. The export helpers documented here are simpler writer utilities for objects that have already been prepared.

Tool Map#

Tool

Module

Main purpose

pycsamt.site.export.write_site()

pycsamt.site.export

Write one EDI-like object to a target path.

pycsamt.site.export.write_sites()

pycsamt.site.export

Batch-write many sites to an output directory with templated file names and an optional manifest CSV.

pycsamt.site.export.pack_zip()

pycsamt.site.export

Stage sites in a temporary directory, compress them, and optionally write a manifest.

pycsamt.site.report.SiteReport

pycsamt.site.report

Compute and display statistics for one pycsamt.site.base.Site.

pycsamt.site.report.SitesReport

pycsamt.site.report

Compute and display survey-level and per-station statistics for a collection.

Export Workflow#

Export normally happens after loading, selecting, editing, and quality checks.

 1from pycsamt.seg.collection import EDICollection
 2from pycsamt.site.export import write_sites
 3from pycsamt.site.selection import drop_empty, keep_finite_z
 4
 5sites = EDICollection.from_sources("data/edi")
 6
 7sites = drop_empty(sites)
 8sites = keep_finite_z(sites)
 9
10paths = write_sites(
11    sites,
12    outdir="deliveries/clean_edi",
13    template="{index:03d}_{station}.edi",
14    manifest_csv="deliveries/clean_edi_manifest.csv",
15)
16
17print(paths)

The export functions accept:

  • a pycsamt.site.base.Sites object;

  • an EDICollection;

  • a list or other iterable of EDI-like objects;

  • a single EDI-like object.

An object is EDI-like for export when it supports at least one common writer method: write(new_edifn=...), write(path), to_file(path), or save(path).

Writing One Site#

Use pycsamt.site.export.write_site() when the output path is already known and you are only writing one object.

 1from pycsamt.seg.edi import EDIFile
 2from pycsamt.site.export import write_site
 3
 4edi = EDIFile("data/edi/S01.edi")
 5
 6out = write_site(
 7    edi,
 8    "deliveries/single/S01_corrected.edi",
 9)
10
11print(out)

The parent directory is created automatically. If the target file already exists, overwrite behavior depends on the underlying EDI writer. For explicit collision control in batch exports, use pycsamt.site.export.write_sites() with exist_ok.

Batch Writing Sites#

Use pycsamt.site.export.write_sites() for a selected survey or station group.

 1from pycsamt.site.export import write_sites
 2
 3written = write_sites(
 4    sites,
 5    outdir="deliveries/line_01",
 6    template="{index:03d}_{station}",
 7    exist_ok=False,
 8    manifest_csv="deliveries/line_01_manifest.csv",
 9)
10
11for path in written:
12    print(path.name)

If the rendered file name does not end with .edi, the extension is added automatically. In the example above, a station named S01 at index 0 is written as 000_S01.edi.

Filename Templates#

Filename templates are rendered from a context collected from each EDI header and its input order.

Template key

Source

Notes

{station}

Station name helper.

Uses the same station-name resolution as the site utilities.

{index}

Input iteration order.

Zero-based integer. Supports Python format specifiers such as {index:03d}.

{lat}

EDI HEAD coordinate.

Missing values become NaN.

{lon}

EDI HEAD coordinate.

Also accepts long and longitude internally.

{elev}

EDI HEAD coordinate.

Missing values become NaN.

{chainage}

EDI object attribute.

Reads edi.chainage when present.

Examples:

 1write_sites(
 2    sites,
 3    "out/by_station",
 4    template="{station}.edi",
 5)
 6
 7write_sites(
 8    sites,
 9    "out/by_order",
10    template="{index:03d}_{station}",
11)
12
13write_sites(
14    sites,
15    "out/by_line_position",
16    template="{index:03d}_{station}_{chainage:.0f}m",
17)

Unknown template keys are rendered as empty strings. Keep templates simple and avoid path separators inside station names when building files for external delivery.

Collision Policy#

By default, pycsamt.site.export.write_sites() refuses to overwrite an existing file.

1from pycsamt.site.export import write_sites
2
3# Raises FileExistsError if any rendered file already exists.
4write_sites(
5    sites,
6    "deliveries/line_01",
7    template="{station}.edi",
8    exist_ok=False,
9)

Pass exist_ok=True when an export directory is intentionally being refreshed.

1write_sites(
2    sites,
3    "deliveries/line_01",
4    template="{station}.edi",
5    exist_ok=True,
6)

Manifest CSV#

Both pycsamt.site.export.write_sites() and pycsamt.site.export.pack_zip() can write a manifest CSV. The manifest is a compact audit table for exported stations.

Column

Meaning

index

Zero-based input order used during export.

station

Resolved station name.

lat

Latitude from the EDI header, or NaN.

lon

Longitude from the EDI header, or NaN.

elev

Elevation from the EDI header, or NaN.

chainage

edi.chainage when present, otherwise NaN.

filename

File name written in the output directory or archive.

path

Full exported file path for directory exports, or the zip archive path for archive exports.

 
 1import pandas as pd
 2
 3from pycsamt.site.export import write_sites
 4
 5write_sites(
 6    sites,
 7    "deliveries/clean_edi",
 8    template="{index:03d}_{station}",
 9    manifest_csv="deliveries/clean_edi_manifest.csv",
10)
11
12manifest = pd.read_csv("deliveries/clean_edi_manifest.csv")
13print(manifest[["index", "station", "filename"]])

The manifest is only written when at least one row exists.

Zip Packaging#

Use pycsamt.site.export.pack_zip() when you need a single archive for a delivery, web upload, or reproducible artifact.

 1from pycsamt.site.export import pack_zip
 2
 3archive = pack_zip(
 4    sites,
 5    out_zip="deliveries/line_01_clean.zip",
 6    template="{station}.edi",
 7    manifest_csv="deliveries/line_01_clean_manifest.csv",
 8)
 9
10print(archive)

The function writes each site to a temporary directory, then stores those files inside the archive with zipfile.ZIP_DEFLATED compression. The original EDI sources are not deleted or modified by packaging.

You can inspect the archive names with the standard library:

1from zipfile import ZipFile
2
3with ZipFile("deliveries/line_01_clean.zip", "r") as zf:
4    print(sorted(zf.namelist()))

Reporting Workflow#

Reports are read-only summaries. They do not modify site data.

 1from pycsamt.site.base import Sites
 2from pycsamt.site.report import SitesReport
 3
 4sites = Sites.from_any("data/edi")
 5
 6report = SitesReport(sites)
 7report.report(top=10)
 8
 9table = report.to_dataframe(api=False)
10print(table.head())

If the optional rich package is installed, terminal reports use rich panels and tables. Without rich, pyCSAMT falls back to plain-text output.

Single-Site Reports#

pycsamt.site.report.SiteReport computes statistics for one pycsamt.site.base.Site-like object.

 1from pycsamt.site.base import Site
 2from pycsamt.site.report import SiteReport
 3
 4site = Site(edi)
 5report = SiteReport(site)
 6
 7print(report.summary())
 8report.report()
 9
10info = report.to_dict()
11z_frame = report.to_dataframe("z", api=False)

The dictionary returned by pycsamt.site.report.SiteReport.to_dict() contains:

  • name, lat, lon, and elev;

  • nfreq, freq_min, freq_max, per_min, and per_max;

  • component availability for Zxx, Zxy, Zyx, and Zyy;

  • has_tipper;

  • mean and standard deviation for apparent resistivity and phase summaries of the xy and yx components;

  • per-component finite-data quality fractions.

The DataFrame returned by pycsamt.site.report.SiteReport.to_dataframe() delegates to the underlying site to_dataframe method. Use the kind argument to choose the station table representation supported by pycsamt.site.base.Site.

Survey Reports#

pycsamt.site.report.SitesReport computes one station record per site and a survey-level summary.

 1from pycsamt.site.report import SitesReport
 2
 3report = SitesReport(sites)
 4
 5print(report.summary())
 6report.report(top=20)
 7
 8records = report.to_dict()
 9frame = report.to_dataframe(api=False)
10
11print(len(records))
12print(frame.columns)

The survey DataFrame contains one row per station with columns:

 1station
 2lat
 3lon
 4elev
 5nfreq
 6freq_min
 7freq_max
 8has_Zxx
 9has_Zxy
10has_Zyx
11has_Zyy
12has_tipper
13rho_xy
14rho_xy_std
15rho_yx
16rho_yx_std
17phi_xy
18phi_xy_std
19phi_yx
20phi_yx_std

Use top to limit only the terminal display. It does not limit to_dict or to_dataframe.

API View Wrapping#

Report DataFrame methods accept api:

api=False

Return a plain pandas DataFrame.

api=True

Return a pyCSAMT API view wrapper around the DataFrame.

api=None

Defer to the global API-view configuration.

1from pycsamt.site.report import SiteReport, SitesReport
2
3one_station = SiteReport(site).to_dataframe("resphase", api=False)
4survey_view = SitesReport(sites).to_dataframe(api=True)
5
6print(type(one_station))
7print(survey_view.kind)

Use plain DataFrames for local analysis and API views when returning data from public-facing pyCSAMT APIs.

End-To-End Delivery Example#

The following pattern creates a cleaned station export, a manifest, an archive, and a survey report table.

 1from pathlib import Path
 2
 3from pycsamt.seg.collection import EDICollection
 4from pycsamt.site.edit import select_freq_all
 5from pycsamt.site.export import pack_zip, write_sites
 6from pycsamt.site.report import SitesReport
 7from pycsamt.site.selection import by_freq, drop_empty, keep_finite_z
 8
 9root = Path("deliveries/line_01")
10root.mkdir(parents=True, exist_ok=True)
11
12sites = EDICollection.from_sources("data/edi")
13sites = drop_empty(sites)
14sites = keep_finite_z(sites)
15sites = by_freq(sites, fmin=0.1, fmax=100.0)
16sites = select_freq_all(
17    sites,
18    fmin=0.1,
19    fmax=100.0,
20    inplace=False,
21)
22
23write_sites(
24    sites,
25    root / "edi",
26    template="{index:03d}_{station}",
27    manifest_csv=root / "manifest.csv",
28)
29
30pack_zip(
31    sites,
32    root / "edi.zip",
33    template="{index:03d}_{station}.edi",
34    manifest_csv=root / "zip_manifest.csv",
35)
36
37report = SitesReport(sites)
38report.report(top=15)
39summary = report.to_dataframe(api=False)
40summary.to_csv(root / "site_report.csv", index=False)

Common Mistakes#

Assuming pycsamt.site.export.write_site() enforces overwrite policy

The single-site writer delegates to the backend writer. Use pycsamt.site.export.write_sites() with exist_ok=False when you need explicit collision protection.

Forgetting that {index} is zero-based

The index in filenames and manifests is the input iteration position, starting from 0.

Using pycsamt.site.export.pack_zip() when you also need individual files

The zip function stages files in a temporary directory. Use pycsamt.site.export.write_sites() first when the directory tree itself is part of the delivery.

Expecting reports to clean data

Reports summarize the current objects. Run selection and editing before generating reports.

Confusing top with filtering

SitesReport.report(top=10) limits terminal display only. It does not remove stations from the report object.

Next Pages#