pycsamt.zonge.utils#

General‑purpose helpers for Zonge AVG / AMTAVG files and accompanying station profiles.

The file now supports

Kind‑1 legacy whitespace tables.
Kind‑2 comma‑separated tables with leading metadata.

A tidy pandas.DataFrame plus a metadata dict is returned regardless of flavour. Column names are normalised to a concise lower‑ case schema (station, freq, emag, rho, phase, …).

Functions

`chunk_by_frequency`(data, n_per_chunk, *[, ...])	Split data into equally sized chunks (one per frequency).
`classify_avg_format`(lines)	Return 1 or 2 depending on AVG flavour.
`detect_stn_header`(profile[, splitter])	Heuristically detect a `.stn` header and map token positions.
`extract_core_columns`(df, *[, keep])	Split a kind‑2 frame into core and extra columns.
`find_and_rename_column`(df, canonical_name)	Find a column by any of its aliases and rename it to the canonical name.
`load_avg`(path, *[, ll_columns, utm_zone, ...])	Read a Zonge AVG file and return a tidy DataFrame and metadata.
`number_stations`(n_stations, n_freq, *[, prefix])	Generate station IDs and a frequency-expanded copy.
`read_stn`(path)	Parse Zonge `.stn` files (legacy and extended forms).
`round_dipole_length`(length)	Round length to the nearest 5‑m increment.
`split_by_station`(df)	Split a tidy AVG DataFrame into per-station sub-frames.
`to_numeric_if_possible`(values)	Return numeric values when coercion succeeds, else original values.
`to_xarray`(df, *[, coords, data_vars, attrs])	Convert a tidy Zonge table to an `xarray.Dataset`.
`validate_stn_profile`(profile[, splitter])	Heuristically detect a `.stn` header and map token positions.
`write_avg`(core, extra, meta[, path, stamp, ...])	Serialize a DataFrame to a Zonge kind-2 AVG file.

pycsamt.zonge.utils.load_avg(path, *, ll_columns=('latitude', 'longitude'), utm_zone=None, inplace=False)[source]#

Read a Zonge AVG file and return a tidy DataFrame and metadata.

This function serves as the primary parser for both legacy (kind-1) and modern (kind-2) Zonge AVG files. It automatically detects the file format, parses the data accordingly, and standardizes all column names to a consistent, canonical schema.

Parameters:

path (str or pathlib.Path) – The filesystem path to the .avg file.
ll_columns (tuple[str, str], default ('latitude', 'longitude')) – Column names in the source data that contain latitude and longitude values in decimal degrees. This is not a standard Zonge field but is supported for custom data formats.
utm_zone (int, optional) – The UTM zone number to use for coordinate conversion. If None, the zone is auto-detected from the longitude.
inplace (bool, default False) – This parameter is deprecated and no longer has an effect, as the function always returns a new DataFrame.

Returns:

df (pandas.DataFrame) – A tidy DataFrame where all column names have been standardized to the internal canonical schema (e.g., ‘Resistivity’ becomes ‘rho’).
meta (dict) – A dictionary containing all header metadata from the file. For modern files, this includes a ‘blocks’ key with a list of per-station metadata blocks.

Raises:

FileNotFoundError – If the file specified by path does not exist.
AvgFileError – If the file format cannot be reliably classified as either legacy or modern.
AvgDataError – If parsing fails due to malformed or missing data within the file.

Return type:

tuple[DataFrame, dict[str, str]]

Notes

The standardization of column names is a key feature of this function. It ensures that all subsequent processing steps can rely on a consistent and predictable data structure, regardless of the input file’s original format. This is achieved by using the _CANONICAL_MAP from the schema module.

Examples

>>> from pycsamt.zonge.utils import load_avg
>>> # Load a modern AVG file
>>> df, meta = load_avg('data/avg/K2.avg')
>>> print(df.columns)
Index(['z_mwgt', 'freq', ..., 'station', 'comp', 'use'], dtype='object')
>>> print(meta['Survey.Type'])
CSAMT

See also

pycsamt.zonge.avg.AVG.from_file: The recommended high-level entry point for loading AVG data.
pycsamt.zonge.utils.write_avg: The corresponding function for writing AVG files.

pycsamt.zonge.utils.round_dipole_length(length)[source]#

Round length to the nearest 5‑m increment.

Parameters:: length (float | int)
Return type:: float

pycsamt.zonge.utils.validate_stn_profile(profile, splitter=None)#

Heuristically detect a .stn header and map token positions.

Scans non-comment lines, tolerates CSV and whitespace, quoted labels (e.g., 3x"dot3x"), and embedded label=value segments. Returns a score (matched labels) and a list of (canonical_label, index) for the best header line found. If no header is detected, returns (0, []).

Parameters:

profile (sequence of str) – Raw lines of the STN file.
splitter (str or None, default None) – Token delimiter. If None, auto-detect per line (comma if present, else whitespace).

Returns:

score (int) – Number of recognized header labels on the best line.
matches (list of (str, int)) – Pairs of canonical label name and column index.

Return type:

tuple[int, list[tuple[str, int]]]

pycsamt.zonge.utils.classify_avg_format(lines)[source]#

Return 1 or 2 depending on AVG flavour.

This function uses a multi-pass approach for robustness. It first checks for explicit headers and then falls back to analyzing structural clues like keyword format and data delimiters.

Parameters:: lines (Sequence[str]) – Raw text lines read from the AVG file.
Raises:: AvgFileError – When the function cannot detect a valid header.
Return type:: int

pycsamt.zonge.utils.extract_core_columns(df, *, keep=None)[source]#

Split a kind‑2 frame into core and extra columns.

Parameters:

df (pandas.DataFrame) – Parsed DataFrame from load_avg().
keep (Iterable[str] | None, optional) – Canonical column names to retain. Defaults to a built‑in minimal set.

Returns:

core (DataFrame) – Only the requested keep columns (plus station if absent).
extra (DataFrame) – All remaining columns.

Return type:

tuple[DataFrame, DataFrame]

pycsamt.zonge.utils.number_stations(n_stations, n_freq, *, prefix='S')[source]#

Generate station IDs and a frequency-expanded copy.

Parameters:

n_stations (int | Integral) – Positive integers. n_freq is the number of frequencies per station.
n_freq (int | Integral) – Positive integers. n_freq is the number of frequencies per station.
prefix (str) – String prepended to every station label.

Returns:

ids – ['S00', 'S01', …] up to n_stations – 1.
expanded – Each ID repeated n_freq times (ordered like the original table: all rows for S00, then S01, …).

Return type:

tuple[list[str], list[str]]

pycsamt.zonge.utils.chunk_by_frequency(data, n_per_chunk, *, drop_remainder=False)[source]#

Split data into equally sized chunks (one per frequency).

Parameters:

data (Sequence[Any] | ndarray) – Any 1-D array-like object. A copy is not made unless conversion is required.
n_per_chunk (int | Integral) – Items per chunk (e.g. number of stations for that frequency).
drop_remainder (bool) – If True, discard a final partial chunk; otherwise it is returned as-is.

Returns:

List of numpy.ndarray slices.

Return type:

chunks

Examples

>>> chunk_by_frequency([0, 1, 2, 3, 4], 2)
[array([0, 1]), array([2, 3]), array([4])]

pycsamt.zonge.utils.write_avg(core, extra, meta, path=None, *, stamp=True, float_fmt='%.6g', na_rep='*', header_spaces=False, banner_lines=None)[source]#

Serialize a DataFrame to a Zonge kind-2 AVG file.

This function serves as the core writer for creating modern, CSAVGW/ASTATIC-style .avg files. It takes a DataFrame with canonical column names, aggregates metadata, and formats the output into a structured text file with data blocks grouped by station.

Parameters:

core (pandas.DataFrame) – The main DataFrame containing the core measurement data. It is expected to have canonical column names (e.g., ‘rho’, ‘phase’, ‘pc_emag’).
extra (pandas.DataFrame or None) – An optional DataFrame containing additional columns to be merged with the core data before writing.
meta (dict, optional) – A dictionary of global metadata to be written as $keyword=value pairs in the file header.
path (str or pathlib.Path, optional) – The output file path. If None, a default filename like exported_kind2.avg is created in the current working directory.
stamp (bool, default True) – If True, a $Written=<timestamp> line is added to the header for provenance.
float_fmt (str, default "%.6g") – The format specifier for writing floating-point numbers in the data blocks.
na_rep (str, default "*") – The string representation for missing (NaN) values in the data blocks, conforming to the Zonge convention.
header_spaces (bool, default False) – If True, adds spaces around the equals sign in header keywords (e.g., $Key = Value).
banner_lines (sequence of str, optional) – A sequence of comment lines (starting with ‘\’) to be prepended to the file header, typically for hardware and processing information.

Returns:

The absolute path to the newly created .avg file.

Return type:

pathlib.Path

Notes

This function is designed to produce clean, compliant, and human-readable kind-2 AVG files. Its key behaviors include:

Column Renaming: It uses the CANON_TO_MODERN_MAP from the schema module to automatically convert the internal canonical column names back to the standard modern format (e.g., ‘rho’ becomes ‘ARes.mag’).
Block Grouping: If a ‘station’ column is present, the function intelligently groups the data by station and writes a separate data block for each, preceded by its specific $Rx.* metadata.
Smart Column Filtering: It automatically detects and omits placeholder columns (like ‘Choer’, ‘Gdp.Blk’) if they contain no valid data, preventing empty columns from cluttering the output file. It also excludes internal helper columns from the final output.
Aligned Formatting: The function uses a custom CSV formatter to produce neatly aligned, fixed-width-like columns within the data blocks, matching the appearance of files generated by Zonge’s proprietary software.