Monitoring#

Telemetry monitoring turns IoT packets into an operational survey status. It checks whether packets are arriving, whether packet acknowledgements are healthy, whether edge QC is accepting enough windows, and whether field thresholds for latency, packet gaps, battery, clock offset, required channels, and frequency coverage are being respected.

The example below uses synthetic telemetry packets for three stations on the L18 demo line. Synthetic packets are appropriate here because the monitor operates on live IoT messages, not EDI impedance files. The packet stream is deliberately mixed: one station is healthy, one has a rejected edge-QC window and an acknowledgement failure, and one has clock and frequency-band issues.

Create A Monitored Session#

Start with devices and a pycsamt.iot.MonitoringConfig. The config records the operational contract for the stream.

 1from pycsamt.iot import DeviceConfig, FieldSession, MonitoringConfig
 2
 3base_time = 1_700_000_000.0
 4devices = [
 5    DeviceConfig(
 6        "l18-node-01",
 7        station="001A",
 8        protocol="file",
 9        sample_rate_hz=512.0,
10        channels=["ex", "ey", "hx", "hy"],
11        role="recorder",
12    ),
13    DeviceConfig(
14        "l18-node-02",
15        station="002U",
16        protocol="file",
17        sample_rate_hz=512.0,
18        channels=["ex", "ey", "hx", "hy"],
19        role="recorder",
20    ),
21    DeviceConfig(
22        "l18-node-03",
23        station="003A",
24        protocol="file",
25        sample_rate_hz=512.0,
26        channels=["ex", "ey", "hx", "hy"],
27        role="recorder",
28    ),
29]
30
31config = MonitoringConfig(
32    method="amt",
33    expected_interval_s=60.0,
34    max_gap_s=120.0,
35    max_latency_s=10.0,
36    min_packet_success_rate=0.95,
37    min_edge_acceptance_rate=0.80,
38    min_battery_v=11.2,
39    max_clock_offset_ms=5.0,
40    required_channels=["ex", "ey", "hx", "hy"],
41    frequency_band_hz=(1.0, 1000.0),
42)
43
44session = FieldSession(
45    "WILLY-L18-MONITORING-DEMO",
46    devices=devices,
47    method="amt",
48    monitoring_config=config,
49)

Add Synthetic Telemetry Packets#

Each packet is a qc message with fields that the monitor knows how to enrich: station, method, channel list, accepted/rejected decision, acknowledgement status, latency, battery voltage, clock offset, and frequency band.

 1from pycsamt.iot import TelemetryPacket
 2
 3specs = [
 4    (devices[0], 0.0, True, True, 2.1, 12.4, 0.8,
 5     ["ex", "ey", "hx", "hy"], [1.0, 1000.0]),
 6    (devices[0], 60.0, True, True, 2.5, 12.3, 0.7,
 7     ["ex", "ey", "hx", "hy"], [1.0, 1000.0]),
 8    (devices[0], 120.0, True, True, 3.0, 12.2, 0.9,
 9     ["ex", "ey", "hx", "hy"], [1.0, 1000.0]),
10    (devices[1], 180.0, True, True, 4.2, 11.8, 1.6,
11     ["ex", "ey", "hx", "hy"], [1.0, 1000.0]),
12    (devices[1], 240.0, False, True, 13.5, 11.7, 2.0,
13     ["ex", "ey", "hx", "hy"], [1.0, 1000.0]),
14    (devices[1], 420.0, True, False, 9.8, 10.9, 3.2,
15     ["ex", "ey", "hx", "hy"], [1.0, 1000.0]),
16    (devices[2], 480.0, True, False, 8.5, 11.4, 7.5,
17     ["ex", "hx", "hy"], [1.0, 1000.0]),
18    (devices[2], 540.0, True, False, 6.0, 11.3, 8.1,
19     ["ex", "hx", "hy"], [0.2, 1200.0]),
20]
21
22for device, dt, accepted, ack_ok, latency, battery, clock, channels, band in specs:
23    session.add_packet(
24        TelemetryPacket.from_device(
25            device,
26            timestamp=base_time + dt,
27            survey_id=session.survey_id,
28            kind="qc",
29            payload={
30                "method": "amt",
31                "station": device.station,
32                "channels": channels,
33                "frequency_band_hz": band,
34                "accepted": accepted,
35                "decision": "accept" if accepted else "reject",
36                "ack_ok": ack_ok,
37                "latency_s": latency,
38                "battery_v": battery,
39                "clock_offset_ms": clock,
40            },
41        )
42    )

Packet Tables And Counts#

Use pycsamt.iot.packet_table() for raw packet inventory and pycsamt.iot.telemetry_summary() for packet counts by device/topic.

 1from pycsamt.iot import packet_table, telemetry_summary
 2
 3packets = packet_table(session.packets)
 4print(
 5    packets[
 6        ["device_id", "kind", "timestamp", "payload_keys"]
 7    ].head(4).to_string(index=False)
 8)
 9print()
10
11summary = telemetry_summary(session.packets)
12print(summary[["device_id", "topic", "n_packet"]].to_string(index=False))

Output:

  device_id kind    timestamp                                                                                           payload_keys
l18-node-01   qc 1700000000.0 accepted;ack_ok;battery_v;channels;clock_offset_ms;decision;frequency_band_hz;latency_s;method;station
l18-node-01   qc 1700000060.0 accepted;ack_ok;battery_v;channels;clock_offset_ms;decision;frequency_band_hz;latency_s;method;station
l18-node-01   qc 1700000120.0 accepted;ack_ok;battery_v;channels;clock_offset_ms;decision;frequency_band_hz;latency_s;method;station
l18-node-02   qc 1700000180.0 accepted;ack_ok;battery_v;channels;clock_offset_ms;decision;frequency_band_hz;latency_s;method;station

  device_id                                                 topic  n_packet
l18-node-01 pycsamt/WILLY-L18-MONITORING-DEMO/001A/l18-node-01/qc         3
l18-node-02 pycsamt/WILLY-L18-MONITORING-DEMO/002U/l18-node-02/qc         3
l18-node-03 pycsamt/WILLY-L18-MONITORING-DEMO/003A/l18-node-03/qc         2

Inspect Enriched Rows#

The monitor normalises payload fields into analysis-ready columns. This is the best view when you need to debug why a status became warning or critical.

 1from pycsamt.iot import TelemetryMonitor
 2
 3monitor = TelemetryMonitor(config)
 4enriched = monitor.table(session.packets, now=base_time + 600.0)
 5print(
 6    enriched[
 7        [
 8            "device_id",
 9            "station",
10            "edge_accepted",
11            "ack_ok",
12            "latency_s",
13            "battery_v",
14            "clock_offset_ms",
15            "frequency_min_hz",
16            "frequency_max_hz",
17        ]
18    ].head(6).to_string(index=False)
19)

Output:

  device_id station  edge_accepted  ack_ok  latency_s  battery_v  clock_offset_ms  frequency_min_hz  frequency_max_hz
l18-node-01    001A           True    True        2.1       12.4              0.8               1.0            1000.0
l18-node-01    001A           True    True        2.5       12.3              0.7               1.0            1000.0
l18-node-01    001A           True    True        3.0       12.2              0.9               1.0            1000.0
l18-node-02    002U           True    True        4.2       11.8              1.6               1.0            1000.0
l18-node-02    002U          False    True       13.5       11.7              2.0               1.0            1000.0
l18-node-02    002U           True   False        9.8       10.9              3.2               1.0            1000.0

Assess Stream Status#

The status level is one of ok, warning, critical, or no_data. Critical issues include packet success below threshold, edge-acceptance failure, low battery, high clock offset, method mismatch, or missing required channels.

 1from pycsamt.iot import assess_telemetry, monitoring_status_table
 2
 3status = monitor.assess(session.packets, now=base_time + 600.0)
 4status_df = monitoring_status_table(status)
 5print(
 6    status_df[
 7        [
 8            "level",
 9            "n_packet",
10            "packet_success_rate",
11            "edge_acceptance_rate",
12            "mean_latency_s",
13            "max_gap_s",
14            "battery_min_v",
15            "clock_offset_max_ms",
16            "issues",
17        ]
18    ].to_string(index=False)
19)
20
21status2 = assess_telemetry(
22    session.packets,
23    config=config,
24    now=base_time + 600.0,
25)
26print(f"Convenience wrapper level: {status2.level.value}")

Output:

   level  n_packet  packet_success_rate  edge_acceptance_rate  mean_latency_s  max_gap_s  battery_min_v  clock_offset_max_ms                                                                                                                                                                                     issues
critical         8                0.625                 0.875             6.2      180.0           10.9                  8.1 battery_below_threshold;clock_offset_above_threshold;frequency_outside_configured_band;packet_gap_above_threshold;packet_gap_exceeds_expected_interval;packet_success_rate_below_threshold
Convenience wrapper level: critical

Plot A Monitoring Audit#

The figure below is built from the enriched monitor table and status metrics. It shows which thresholds were violated without requiring the reader to inspect every packet by hand.

 1from pathlib import Path
 2
 3import matplotlib.pyplot as plt
 4import numpy as np
 5
 6out_dir = Path("docs/source/images/user_guide/iot")
 7out_dir.mkdir(parents=True, exist_ok=True)
 8
 9metrics = status.as_dict()
10fig, axes = plt.subplots(2, 2, figsize=(11.0, 7.4),
11                         constrained_layout=True)
12fig.suptitle(
13    "Telemetry monitoring audit: WILLY-L18-MONITORING-DEMO",
14    fontsize=14,
15)
16
17station_acceptance = (
18    enriched.groupby("station")["edge_accepted"].mean().sort_index()
19)
20colors = [
21    "#2ca25f" if value >= config.min_edge_acceptance_rate else "#de2d26"
22    for value in station_acceptance
23]
24axes[0, 0].bar(station_acceptance.index, station_acceptance.values,
25               color=colors)
26axes[0, 0].axhline(config.min_edge_acceptance_rate,
27                   ls="--", color="#de2d26")
28axes[0, 0].set_ylim(0, 1.05)
29axes[0, 0].set_ylabel("Acceptance rate")
30axes[0, 0].set_title("Edge acceptance by station")
31
32summary_metrics = {
33    "packet\nsuccess": metrics["packet_success_rate"],
34    "edge\nacceptance": metrics["edge_acceptance_rate"],
35}
36axes[0, 1].bar(list(summary_metrics), list(summary_metrics.values()),
37               color=["#de2d26", "#2ca25f"])
38axes[0, 1].axhline(config.min_packet_success_rate, ls="--",
39                   color="#756bb1", label="min packet success")
40axes[0, 1].axhline(config.min_edge_acceptance_rate, ls=":",
41                   color="#de2d26", label="min edge acceptance")
42axes[0, 1].set_ylim(0, 1.05)
43axes[0, 1].set_title("Stream-level rates")
44axes[0, 1].legend(loc="lower left", fontsize=8)
45
46packet_minutes = (
47    enriched["timestamp"] - enriched["timestamp"].min()
48) / 60.0
49packet_colors = [
50    "#2ca25f" if ok else "#de2d26"
51    for ok in enriched["ack_ok"]
52]
53axes[1, 0].scatter(
54    packet_minutes,
55    enriched["station"],
56    c=packet_colors,
57    s=70,
58    edgecolor="black",
59    linewidth=0.4,
60)
61axes[1, 0].set_xlabel("Minutes since first packet")
62axes[1, 0].set_ylabel("Station")
63axes[1, 0].set_title("Packet acknowledgements")
64
65ops = enriched.groupby("station").agg(
66    battery_min_v=("battery_v", "min"),
67    clock_offset_max_ms=(
68        "clock_offset_ms",
69        lambda s: float(np.max(np.abs(s))),
70    ),
71)
72x = np.arange(len(ops))
73axes[1, 1].bar(x - 0.18, ops["battery_min_v"], width=0.36,
74               label="battery V")
75axes[1, 1].bar(x + 0.18, ops["clock_offset_max_ms"], width=0.36,
76               label="clock offset ms")
77axes[1, 1].axhline(config.min_battery_v, ls="--",
78                   color="#de2d26", label="min battery")
79axes[1, 1].axhline(config.max_clock_offset_ms, ls=":",
80                   color="#756bb1", label="max clock offset")
81axes[1, 1].set_xticks(x, ops.index)
82axes[1, 1].set_title("Power and clock thresholds")
83axes[1, 1].legend(loc="upper right", fontsize=8)
84
85fig.savefig(out_dir / "user-guide-iot-monitoring-01.png", dpi=180)
../../_images/user-guide-iot-monitoring-01.png

Field Interpretation#

The stream is critical because packet acknowledgement success is below the configured threshold, one gap is too long, the minimum battery voltage falls below 11.2 V, the clock offset exceeds 5 ms, and one packet reports a frequency band outside the configured AMT band. The edge-acceptance rate is still above the stream-level threshold, but station 002U is visibly weaker than the others and should be reviewed.

In a live deployment, these status rows should be logged with the acquisition manifest. They explain why a station was accepted, repeated, or excluded before downstream impedance, dimensionality, or inversion workflows begin.