AVG - Main user-facing data container for Zonge datasets.
This module provides the AVG class, which serves as the
primary entry point and high-level facade for interacting with a
complete Zonge AVG dataset. It composes all other components
(Header, Z, Resistivity, Phase, and QC metrics) into a single,
convenient container.
Base class for AVG data handling and file writing.
This class serves as the core engine for reading, parsing,
and writing Zonge AVG data files. It handles the logic for
identifying file types (legacy vs. modern), transforming
legacy data, and populating a structured data model.
Parameters:
verbose (bool, default False) – If True, log messages will be printed to the console
during file reading and writing operations, providing
insight into the process.
Variables:
info (DataInfo) – An aggregator object that holds all the individual data
components (e.g., Header, Station, Z, Resistivity).
verbose (bool) – Controls the verbosity of logging output.
_kind ({1, 2} or None) – Indicates the detected file format: 1 for legacy, 2 for
modern. None if no file has been read.
_source_path (pathlib.Path or None) – The path to the source file that was read.
Notes
This class is typically not instantiated directly by users.
Instead, the AVG class, which
inherits from BaseAVG, is the primary user-facing entry
point.
The read method is the main entry point for data ingestion
and is designed to be flexible, accepting file paths or
in-memory data structures.
Examples
While direct use is uncommon, one could use BaseAVG like so:
Read, parse, and populate components from a data source.
This is the primary data ingestion method for the class. It
is designed to be flexible, accepting various data sources
and orchestrating the entire process of parsing, transforming,
and populating the internal data components.
meta (mapping, optional) – An optional dictionary of metadata. This is primarily
used when source is a pd.DataFrame to provide
header-like information.
Returns:
self – The method returns the instance of the class, allowing
for convenient method chaining.
AvgFileError – If a file source cannot be reliably classified as either
legacy (kind-1) or modern (kind-2).
AvgDataError – If parsing fails due to malformed or missing data.
Notes
The read method encapsulates a complete data processing
pipeline:
File Handling: If a path is provided, it reads the
file and uses classify_avg_format() to
determine the file kind.
Transformation: If a legacy (kind-1) file is
detected, it automatically uses the
LegacyAVGBase transformer to
convert the data to a modern, structured format.
Standardization: If a raw pd.DataFrame is
provided, it is wrapped in an AVGFrame, which
triggers the automatic standardization of column names
to the internal canonical schema.
Component Population: The final, clean DataFrame and
metadata are passed to the DataInfo component, which
in turn populates all the individual data components
(e.g., Station, Z, Resistivity).
Examples
>>> frompycsamt.zongeimportAVG>>> # Read from a modern file>>> avg=AVG().read('data/avg/K2.avg')>>>>>> # Read from a legacy file (will be auto-transformed)>>> avg_legacy=AVG().read('data/avg/K1.avg')>>>>>> # Read from an in-memory DataFrame>>> importpandasaspd>>> df=pd.DataFrame({'Freq':[1024],'Resistivity':[100]})>>> avg_from_df=AVG().read(df)
This method serializes the object’s data into a modern,
CSAVGW-style .avg file. It delegates the core writing
and formatting logic to the write_avg()
function.
Parameters:
path (str or pathlib.Path, optional) – The output file path. If None, a default filename
(e.g., K2_modern.avg) is created in the current
working directory.
stamp (bool, default True) – If True, a $Written=<timestamp> line is added to
the header for provenance.
float_fmt (str, default "%.6g") – The format specifier for floating-point numbers.
na_rep (str, default "*") – The string representation for missing (NaN) values.
header_spaces (bool, default False) – If True, adds spaces around the equals sign in header
keywords (e.g., $Key=Value).
Notes
This method intelligently prepares the data for writing by:
1. Assembling a complete header by calling the
This method reconstructs the classic, fixed-width format of
older Zonge AVG files. It uses the populated data components
to generate the header and formatted data rows.
Parameters:
path (str or pathlib.Path, optional) – The output file path. If None, a default filename
(e.g., K2_legacy.avg) is created in the current
working directory.
precision (int, default 4) – The number of decimal places for scientific notation
fields (e.g., ‘Emag’, ‘Resistivity’).
na_rep (str, default "*") – The string representation for missing (NaN) values.
Notes
The conversion from a modern, comprehensive data structure to
the legacy format involves several key steps:
Header Synthesis: It creates a minimal legacy header
using $ASPACE and $XMTR keywords derived from the
header component.
Column Translation: It translates the internal canonical
column names (e.g., ‘pc_emag’) back to their legacy
equivalents (e.g., ‘%Emag’) for correct formatting.
Data Filtering: It first filters out any artificial
data rows (e.g., those with NaN in the ‘rho’ column) to
ensure the output is clean.
Be aware that some information from a modern dataset may be
lost in this conversion, as the legacy format does not
support all the fields of the modern format.
This method acts as a high-level dispatcher, calling either
the to_legacy() or to_modern() method based on the
fmt parameter. It provides a single, convenient entry point
for file serialization.
Parameters:
path (str or pathlib.Path, optional) – The output file path. If None, a default filename will
be generated based on the source file’s name.
fmt ({'legacy', 'kind1', 'modern', 'kind2', 'auto'}, default 'auto') – The desired output format.
- ‘legacy’ or ‘kind1’: Writes a fixed-width legacy file.
- ‘modern’ or ‘kind2’: Writes a CSV-based modern file.
- ‘auto’: Defaults to the modern format.
**kwargs – Additional keyword arguments to be passed to the specific
writing method (to_legacy or to_modern). For example,
precision for to_legacy or float_fmt for
to_modern.
Provide a Bunch object with a summary of key attributes.
This property aggregates the most important descriptive
metadata and statistics from the loaded dataset into a single,
easy-to-read object. It is useful for quickly inspecting the
contents and overall scope of the survey data.
Returns:
A Bunch object (which behaves like a dictionary with
attribute-style access) containing key information such
as the source filename, data format kind, project name,
number of stations and frequencies, and their respective
ranges.
The summary property is read-only and is generated on-demand
by introspecting the various data components (e.g., header,
station, frequency). If no data has been loaded, it will
return a Bunch with a ‘status’ key indicating this.
The Bunch object has a user-friendly string representation,
making it ideal for printing in interactive sessions.
Return a shallow, JSON-serializable representation.
This method aggregates the keyword dictionaries from all
header components into a single dictionary, providing a
complete, serializable view of the survey’s metadata.
An AVG instance loads a raw text file, normalizes
its content, and hydrates a full suite of data components,
including stations, frequencies, impedance, quality metrics,
and header blocks. This makes the entire survey accessible
through strongly-typed attributes [1]_.
Parameters:
verbose (bool, default False) – If True, log messages will be printed to the console
during file reading and writing operations.
Returns a JSON-serializable dictionary of all header
metadata.
Notes
Zonge Engineering produced two AVG file formats:
Kind-1 (Legacy): Fixed-width text tables with minimal
metadata.
Kind-2 (Modern): CSV-based format with rich
$key=value headers, used by modern AMTAVG and ASTATIC.
The loader automatically detects the format and transforms it
into a canonical, tidy DataFrame. All subsequent processing is
format-agnostic. Zonge placeholders (*) are converted to
NaN.
Examples
Load a file, access a component, and write back:
>>> frompycsamt.zongeimportAVG>>> avg=AVG.from_file('LCS01.avg',verbose=True)>>> # Access resistivity for a specific component>>> rho_xy=avg.resistivity.frame[... avg.resistivity.frame.comp=='ExHy'... ]>>> avg.write('LCS01_clean.avg')
This classmethod is the primary factory for creating a
fully populated AVG instance directly from a file
on disk. It handles the entire data loading pipeline.
Parameters:
path (str or pathlib.Path) – The filesystem path to the .avg file. Both legacy
(kind-1) and modern (kind-2) formats are supported.
verbose (bool, default False) – If True, log messages will be printed to the console
during the file loading and parsing process.
Returns:
A new, fully initialized instance of the AVG
class containing all the data and metadata from the
file.
This method is a convenient wrapper around the more general
read() method. It creates an instance of the class
and then immediately calls obj.read(path) to perform
the actual data loading and component population.
Examples
>>> frompycsamt.zongeimportAVG>>> # Load a modern AVG file>>> avg=AVG.from_file('data/avg/K2.avg',verbose=True)>>> print(avg.summary)
Export the complete dataset to a single xarray.Dataset.
This is the primary export method for creating a comprehensive,
multi-dimensional, and self-describing dataset. It aggregates
the primary data variables (resistivity, phase), the computed
impedance tensor (Z), and all available quality control (QC)
metrics into a single object.
Parameters:
include_qc (bool, default True) – If True, all available QC components (e.g., pc_emag,
s_phz) will be included as data variables in the
dataset.
include_z (bool, default True) – If True, the real and imaginary parts of the complex
impedance tensor (Z) and its propagated error will be
included as data variables.
Returns:
A dataset with coordinates (station,freq,comp) and
data variables for all included measurements and QC
metrics. The dataset’s attributes are populated from the
AVG file’s header.
Return type:
xarray.Dataset
Notes
This method provides a high-level, convenient way to get all
the processed data into a standard format for analysis and
plotting. It gracefully handles cases where optional
components (like certain QC metrics) are not available in the
source data by skipping them.
This is a convenience method that provides direct access to the
underlying tensor-shaping capabilities of the data components.
It delegates the call to the appropriate component (e.g., Z,
Resistivity) to reshape the selected variable into a NumPy
array.
Parameters:
var ({'z', 'z_real', 'z_imag', 'z_err', 'rho', 'phase'}, default 'z') – The specific data variable to export as a tensor.
station (int or float, optional) – If provided, the method returns a 3D tensor for only this
station. If None, it returns a 4D tensor containing
data for all stations.
agg (str or None, default 'mean') – The aggregation method to use if duplicate measurements
exist for the same station, frequency, and component.
fill_value (float, default np.nan) – The value to use for missing elements in the tensor grid.
sort_freq (bool, default True) – If True, the frequency axis of the tensor will be
sorted.
align ({'union', 'intersection'}, default 'union') – For multi-station tensors, determines how to handle
different frequency sets between stations.
Returns:
tensor (numpy.ndarray) – The data reshaped into a tensor. Shape is
(n_freq,2,2) for a single station or
(n_station,n_freq,2,2) for multiple stations.
freqs (numpy.ndarray) – The frequency axis for the tensor.
stations (numpy.ndarray) – The station axis for the tensor (empty for a single-
station request).
Read, parse, and populate components from a data source.
This is the primary data ingestion method for the class. It
is designed to be flexible, accepting various data sources
and orchestrating the entire process of parsing, transforming,
and populating the internal data components.
meta (mapping, optional) – An optional dictionary of metadata. This is primarily
used when source is a pd.DataFrame to provide
header-like information.
Returns:
self – The method returns the instance of the class, allowing
for convenient method chaining.
AvgFileError – If a file source cannot be reliably classified as either
legacy (kind-1) or modern (kind-2).
AvgDataError – If parsing fails due to malformed or missing data.
Notes
The read method encapsulates a complete data processing
pipeline:
File Handling: If a path is provided, it reads the
file and uses classify_avg_format() to
determine the file kind.
Transformation: If a legacy (kind-1) file is
detected, it automatically uses the
LegacyAVGBase transformer to
convert the data to a modern, structured format.
Standardization: If a raw pd.DataFrame is
provided, it is wrapped in an AVGFrame, which
triggers the automatic standardization of column names
to the internal canonical schema.
Component Population: The final, clean DataFrame and
metadata are passed to the DataInfo component, which
in turn populates all the individual data components
(e.g., Station, Z, Resistivity).
Examples
>>> frompycsamt.zongeimportAVG>>> # Read from a modern file>>> avg=AVG().read('data/avg/K2.avg')>>>>>> # Read from a legacy file (will be auto-transformed)>>> avg_legacy=AVG().read('data/avg/K1.avg')>>>>>> # Read from an in-memory DataFrame>>> importpandasaspd>>> df=pd.DataFrame({'Freq':[1024],'Resistivity':[100]})>>> avg_from_df=AVG().read(df)
Extends AVG with tensor components and analytical methods.
This class inherits from AVG and
provides a more specialized, analysis-focused interface. It is
designed for users who need to perform advanced geophysical
data processing and manipulation. It is designed for working
directly with specific tensor elements (e.g., z_xy, rho_yx).
The primary features of this class include:
Direct, property-based access to individual components of
the impedance, resistivity, and phase tensors as pandas
Series (e.g., z_xy, res_yx_err).
A suite of methods for advanced data processing, such as
tensor rotation, phase unwrapping, and statistical analysis.
In addition to providing direct access to tensor components,
this class includes methods for advanced data manipulation,
such as recalculating resistivity and phase from the impedance
tensor.
Variables:
z_yy (z_xx, z_xy, z_yx,) – The complex impedance for the respective tensor component.
z_yy_err (z_xx_err, z_xy_err, z_yx_err,) – The propagated error for the respective impedance component.
res_yy (res_xx, res_xy, res_yx,) – The apparent resistivity for the respective tensor component.
>>> frompycsamt.zongeimportAMTAVG>>> amt_avg=AMTAVG.from_file('data/avg/K2.avg')>>> # Rotate the data by 30 degrees clockwise>>> amt_avg.rotate(30)>>> # Access the newly rotated Z_xy component>>> z_xy_rotated=amt_avg.z_xy
This method calculates the apparent resistivity (\(\rho_a\))
and impedance phase (\(\phi\)) from the complex
impedance tensor (\(\mathbf{Z}\)). It also propagates
the errors from \(\mathbf{Z}\) to the new values.
Parameters:
todeg (bool, default False) – If True, the output impedance phase and its error
will be converted from milliradians to degrees.
Returns:
rho (pandas.Series) – The calculated apparent resistivity in ohm-meters.
phi (pandas.Series) – The calculated impedance phase in milliradians or
degrees.
rho_err (pandas.Series) – The propagated relative error in apparent resistivity
(in percent).
phi_err (pandas.Series) – The propagated error in impedance phase (in the same
units as phi).
This method updates the core DataFrame with new apparent
resistivity and impedance phase values. After updating, it
automatically re-initializes all data components, ensuring
that dependent values (like the impedance tensor Z) are
recalculated and the entire object state remains consistent.
Parameters:
rho (pandas.Series) – A Series containing the new apparent resistivity values.
The index must align with the internal DataFrame.
phi (pandas.Series) – A Series containing the new impedance phase values,
assumed to be in milliradians.
rho_err (pandas.Series, optional) – A Series containing the new relative error values for
resistivity (in percent).
phi_err (pandas.Series, optional) – A Series containing the new error values for phase (in
milliradians).
Notes
This is a powerful method for data manipulation, such as
applying static shifts or corrections. By calling
self.info.read at the end, it ensures that all changes
are propagated correctly throughout the entire data model.
Fetch a 3D tensor for a single station using xarray.
This method extracts the data for a single station from the
full multi-station dataset and returns it as a 3D tensor
in an xarray.DataArray.
Parameters:
station_id (int or float) – The unique identifier for the station to retrieve. This
must match one of the values in the ‘station’
coordinate of the dataset.
var ({'z', 'rho', 'phase'}, default 'z') – The specific data variable to retrieve.
- ‘z’: Complex impedance tensor.
- ‘rho’: Apparent resistivity tensor.
- ‘phase’: Impedance phase tensor.
Returns:
A 3D DataArray with dimensions (frequency,e,h)
containing the tensor data for the requested station.
Return type:
xarray.DataArray
Raises:
KeyError – If the specified station_id is not found in the
dataset’s station coordinates.
NotReadError – If this method is called before data has been loaded.
Notes
This method leverages the powerful label-based indexing of
xarray’s .sel() method. It first constructs the full
multi-station tensor in memory and then selects the slice
corresponding to the given station_id.
Examples
>>> frompycsamt.zongeimportAMTAVG>>> avg=AMTAVG.from_file('data/avg/K2.avg')>>> # Get the resistivity tensor for station 25.0>>> rho_tensor_25=avg.get_tensor_by_station(25.0,var='rho')>>> print(rho_tensor_25.shape)(28, 2, 2)
This method computes statistical metrics for repeated
measurements, mirroring the process of the original Zonge
AMTAVG program. It groups the data by station and frequency
and calculates the mean, standard deviation (σ), and
coefficient of variation (C-var) for key measurement
columns.
Parameters:
drop_on_failure (bool, default True) – If True, rows where statistical calculations
cannot be performed (e.g., only one measurement in a
group) will be excluded from the output DataFrame.
update_components (bool, default True) – If True, after calculating the statistics, this
method will automatically update the corresponding QC
components (e.g., self.pc_rho, self.s_phz) with the
newly computed values.
Returns:
A DataFrame containing the calculated statistical
metrics, including mean, standard deviation, and
coefficient of variation for rho, phase,
emag, and hmag.
The calculations are based on the formulas described in the
AMTAVG manual [1]_. The coefficient of variation (C-var),
which corresponds to the legacy % columns, is
calculated as:
\[C_{var} = 100 \cdot \frac{\sigma}{\bar{x}}\]
where \(\sigma\) is the standard deviation and
\(\bar{x}\) is the mean of the measurements.
Unwraps the impedance phase to correct for 2π jumps.
This method implements a phase referencing/unwrapping
algorithm, similar to the PHASEREF mode in the original
Zonge AMTAVG program. It corrects for phase wrapping by
ensuring that the absolute difference between consecutive
phase values (sorted by frequency) does not exceed π
radians (1000π milliradians).
Parameters:
todeg (bool, default False) – If True, the final unwrapped phase values in the
DataFrame will be converted from milliradians to
degrees.
update_components (bool, default True) – If True, the main DataFrame (self.info.df) is
updated in place with the unwrapped phase values, and
all data components are re-read to reflect this change.
Returns:
A new DataFrame with the unwrapped phase values. The
main self.info.df is also updated if
update_components is True.
The unwrapping is performed independently for each unique
station and component pair. The data is first sorted by
frequency to ensure the correct sequence for unwrapping.
The core of the algorithm uses numpy.unwrap(), which
operates in radians. The phase values, assumed to be in
milliradians, are converted to radians before unwrapping
and then converted back to the desired output unit.
Examples
>>> frompycsamt.zongeimportAMTAVG>>> avg=AMTAVG.from_file('data/avg/K2.avg')>>> # Unwrap phase and update the object in place>>> avg.unwrap_phase()>>> # Get the unwrapped phase for the xy component>>> unwrapped_phi_xy=avg.phase_xy
This method performs a 2D rotation of the impedance tensor
\(\mathbf{Z}\) for all measurements. The rotation is
applied in a clockwise-positive direction, which is standard
in many geophysical applications.
Parameters:
angle (float) – The rotation angle in degrees, where a positive angle
corresponds to a clockwise rotation of the coordinate
system.
update_components (bool, default True) – If True, the main DataFrame (self.info.df) is
updated in place with the new, rotated impedance values,
and all dependent components (like resistivity and
phase) are recalculated and updated.
Returns:
A new DataFrame containing the four rotated complex
impedance components (z_xx_rot, z_xy_rot,
z_yx_rot, z_yy_rot). The main self.info.df is
also updated if update_components is True.
where \(\mathbf{Z}\) is the original impedance tensor,
\(\mathbf{Z'}\) is the rotated tensor, \(\theta\)
is the rotation angle, and \(\mathbf{R}(\theta)\) is
the rotation matrix:
This operation updates the z_xx, z_xy, z_yx,
and z_yy components. If update_components is
True, the apparent resistivity and phase are then
recalculated from this new rotated impedance tensor.
This method computes the complex Tipper components (Tx, Ty)
by solving the linear relationship:
\(H_z = T_x H_x + T_y H_y\)
Parameters:
hz_data (pandas.Series) – A Series containing the complex vertical magnetic field
(\(H_z\)) data. The index of this Series must match
the index of the main DataFrame (self.info.df).
update_components (bool, default True) – If True, a new Tipper component will be created and
attached to the self.info object, populated with the
calculated Tx and Ty values.
Returns:
A new DataFrame containing the calculated complex Tipper
components, tx and ty.
The calculation is performed for each unique station and
frequency using a least-squares approach to solve for
\(T_x\) and \(T_y\). This requires at least two
linearly independent horizontal field measurements for a
stable solution.
This method requires the horizontal magnetic field data
(hmag, hphz) to be present in the dataset.