Testing and commissioning

mpylab applications combine numerical evaluation, long stateful workflows, and real hardware. No single test type adequately covers all three. New features are therefore protected in several stages.

1. Pure logic

Test algorithms, unit conversions, parsers, and data structures without an instrument graph. These fast tests focus on:

  • limits and range boundaries;

  • units, uncertainty, and complex quantities;

  • empty, incomplete, and non-finite input;

  • deterministic serialization and return structures.

Mocks are useful here when producing one targeted failure state. They should not imitate the entire public API of an instrument.

2. Instrument contract

Each instrument base class has a common API contract. Test a specific driver against this contract: status codes, quantities, trigger order, channel isolation, timeout, and Quit.

The tested rules are summarized in common instrument-driver behavior.

Protocol tests use a controlled communication class and verify the actual commands sent and complete consumption of replies. They require no laboratory hardware.

3. Virtual configuration

Virtual drivers together with real DOT and INI files verify the interaction between configuration, MGraph, and measurement class. A configuration test should at least:

  1. load the graph;

  2. create every instrument;

  3. initialize the instruments;

  4. evaluate conditions for representative values;

  5. execute RF-off and Quit.

This catches incorrect driver names, channels, search paths, and mappings that an isolated unit test cannot see.

4. Virtual end-to-end run

A short run with a few frequencies and levels uses the regular script. It checks:

  • measurement and evaluation data in the final pickle;

  • raw data and output files;

  • log and preflight;

  • history references to preparatory measurements;

  • report modules where available.

Use temporary output directories. Log and pickle files must not remain in the repository root.

5. Autosave and resume

A resume test interrupts a virtual run after a defined point, loads the autosave through load_pickle_compat, and invokes the same measurement routine again. It proves that:

  • complete points are not measured again;

  • an incomplete point is repeated cleanly;

  • history is retained;

  • the final result matches an uninterrupted reference run;

  • the autosave is removed only after successful completion.

6. Safety and negative tests

Trigger protection functions deliberately. Cases include missing graph paths, ambiguous conditions, MAXIN, AmplifierProtect, unreachable leveling targets, operator abort, and errors during instrument communication.

The expected behavior is not always program termination. A leveler may stop at a protection limit and record the limited state. RF-off must still run.

7. Qt tests

Headless tests use:

QT_QPA_PLATFORM=offscreen \
    python -m pytest test/test_qt_ui_adapter.py

They verify thread lifetime, incremental progress, stop behavior, error display, and closing during an active operation. Timing-sensitive tests wait for signals or states instead of using unnecessarily long fixed sleeps.

8. Staged hardware test

Do not begin hardware testing with the complete measurement range:

  1. Run preflight without RF.

  2. Initialize and identify instruments individually.

  3. Verify RF-off and limit channels.

  4. Measure one frequency at a safe level.

  5. Run a small frequency range or a few points.

  6. Exercise autosave and resume in practice.

  7. Only then enable the complete workflow.

Templates below script/conf provide a HARDWARE_TEST.md for critical TEM workflows. Record local details and the first commissioning results there or in the configuration documentation.

Local tests and CI

The complete local test run corresponds to the core pipeline:

python -m pip install -e ".[dev]"
python -m coverage run -m pytest test
python -m coverage report

Build documentation separately with warnings treated as errors:

sphinx-build -W --keep-going -E -a \
    -b html docs/next/source docs/next/build/html

Coverage does not prove that a measurement workflow is tested correctly. Safety, resume, and traceability in particular require explicit behavioral assertions.