Measurement history and output files

History pickle

Measurement scripts store their respective Measure subclass, such as MSC or TEMCell, as a pickle. When that pickle is used as input to a subsequent measurement, the new measurement extends the same object and writes a new file. Preparatory measurements, reference data, EUT measurements, and evaluations therefore remain together.

The typical chain is:

previous history -> new measurement -> new evaluation -> new history pickle

The input file is not treated as a plain data export. It also contains the structures required to interpret the data. Applications should use mpylab.env.Measure.load_pickle_compat to load current and supported older mpylab pickles.

Measurement data models summarizes the supported measurement-data containers and their detailed references. The pickle also contains internal Python state; that state is not automatically a stable read interface.

Shared history state

Histories derived from Measure can contain the following persisted workflow information:

Shared workflow information

Attribute	Type	Meaning
ascmd	str or None	Human-readable historical description of the method call to resume. New program logic uses autosave_resume instead.
autosave_resume	Mapping or None	Structured resume instruction containing schema_version, measurement, method, description, and parameter_source.
autosave_info	Mapping, when present	Workflow-specific loop progress, such as the most recently completed frequency, tuner position, or measurement position. Its concrete structure belongs in the respective measurement-data reference.
graph_context_events	List of mappings	Detected external measurement-graph changes. An event contains timestamp, requested_context, mismatches, and action. action is "abort" or, after explicit operator approval, "reapply".

SearchPaths and file names can refer to the machine on which a measurement ran. They are provenance, but not portable data paths. Open log files, loggers, UI adapters, and interaction callbacks are removed before pickling and recreated when loading. Applications must not treat these runtime objects as part of the measurement-data contract.

Measurement acquisitions and graph snapshots

One description can contain more than one acquisition. This is intentional: changing a cable, instrument, correction table, or switching configuration does not require a new description, provided that the change remains traceable. Each affected description-level raw-data mapping therefore uses the following shared provenance fields for newly acquired data:

Description-local acquisition provenance

Key	Type	Meaning
mg	MGraph	Compatibility view of the most recently used graph. It is not the immutable evidence for earlier acquisitions.
measurement_graph_snapshots	Mapping	Deduplicated snapshots keyed by their sha256:... configuration_digest.
measurement_acquisitions	List of mappings	Ordered acquisition intervals. The list index is the compact acquisition identifier used by raw samples.
active_measurement_acquisition	int, optional	Acquisition currently represented by an incomplete autosave. It is removed when that interval is closed.

An item in measurement_acquisitions contains index, measurement, started_at_utc, finished_at_utc, status, snapshot_id, and run_configuration. Resume can additionally add resumed_at_utc or the legacy marker legacy_resume. Typical final states are complete, aborted, initialization_failed, and configuration_changed.

Each existing bundled raw-sample mapping receives only "acquisition": <int>. Follow that integer into measurement_acquisitions and then follow snapshot_id into measurement_graph_snapshots. Timestamps are deliberately recorded once per acquisition, not repeated for every scalar instrument value. A regular second invocation starts a new acquisition even when its snapshot is identical; an autosave resume reuses its incomplete acquisition only while the configuration digest is unchanged.

A graph snapshot stores the exact DOT text, resolved node INI files, external files referenced by INI FILE options, inline data blocks, and custom driver source files. It also records logical graph and condition mappings, resource hashes, source references, and search-path ambiguities. Absolute resolved paths help diagnose the original runtime but are not required to verify the embedded bytes. High-confidence credential fields and private-key material are rejected rather than persisted.

After MGraph.CreateDevices(), applications can capture a snapshot with MGraph.capture_configuration_snapshot(). The functions verify_measurement_graph_snapshot, compare_measurement_graph_snapshots, and extract_measurement_graph_snapshot in mpylab.tools.configuration_snapshot verify, compare, and reconstruct this evidence without initializing hardware. Extraction must be treated as source reconstruction; it does not restore an instrument’s physical state.

Autosave and resume

During a long measurement, mpylab writes an autosave pickle at configurable intervals. In addition to data already acquired, it stores structured information about the resume position. ascmd remains as a readable description of the call; program logic prefers the structured resume information. The current shared resume structure uses schema_version = 1. This version applies only to autosave_resume; it is not a global version of the complete pickle.

On the next start, the associated script detects the autosave file and can continue at the stored loop index. Frequencies or positions already completed are not measured again. After successful completion and creation of the final pickle, the autosave is normally removed. It may deliberately be retained for an incomplete immunity measurement.

An autosave is a complete history pickle containing an incomplete dataset. Missing later frequencies or positions are therefore normal, but existing entries must use the same structure as the final dataset. A final pickle is not necessarily complete merely because it can be loaded. Measurement and evaluation status must be checked as well.

Post-measurement snapshot

Many scripts additionally support after_measurement_pickle_file. This file is written after data acquisition but before or independently of the final evaluation. It is particularly useful when:

• an evaluation must be repeated without hardware access,

• evaluation parameters have been corrected,

• the measurement succeeded but a later step failed.

It does not replace the final history pickle. It records a deliberate intermediate state.

DAT and TSV files

rawdata_output_filename and processeddata_output_filename create human-readable DAT files. Raw files show acquired quantities; processed files show results derived from them. Units and uncertainties are retained according to the mpylab data format.

TSV files are used for tabular side products such as preflight results or machine-readable report metadata. DAT and TSV are suitable for comparison, version control, and further processing. The history pickle remains the more complete source because not every object relationship and item of metadata fits into flat tables.

TEM verification data documents the supported TEM-verification read paths, including the forward power used by later immunity measurements.

File names

Descriptions can become part of output file names. Scripts should use mpylab.tools.filenames.format_output_filename for this purpose. Some measurement scripts wrap this call in a local format_filename helper. The function combines the configured file-name pattern with sanitized, length-limited arguments. Applications should not use unchecked descriptions as complete file names.