Testing and commissioning
=========================

mpylab applications combine numerical evaluation, long stateful workflows,
and real hardware. No single test type adequately covers all three. New
features are therefore protected in several stages.

1. Pure logic
-------------

Test algorithms, unit conversions, parsers, and data structures without an
instrument graph. These fast tests focus on:

* limits and range boundaries;
* units, uncertainty, and complex quantities;
* empty, incomplete, and non-finite input;
* deterministic serialization and return structures.

Mocks are useful here when producing one targeted failure state. They should
not imitate the entire public API of an instrument.

2. Instrument contract
-----------------------

Each instrument base class has a common API contract. Test a specific driver
against this contract: status codes, quantities, trigger order, channel
isolation, timeout, and ``Quit``.

The tested rules are summarized in :doc:`common instrument-driver behavior
<device-behavior>`.

Protocol tests use a controlled communication class and verify the actual
commands sent and complete consumption of replies. They require no laboratory
hardware.

3. Virtual configuration
------------------------

Virtual drivers together with real DOT and INI files verify the interaction
between configuration, MGraph, and measurement class. A configuration test
should at least:

#. load the graph;
#. create every instrument;
#. initialize the instruments;
#. evaluate conditions for representative values;
#. execute RF-off and ``Quit``.

This catches incorrect driver names, channels, search paths, and mappings
that an isolated unit test cannot see.

4. Virtual end-to-end run
-------------------------

A short run with a few frequencies and levels uses the regular script. It
checks:

* measurement and evaluation data in the final pickle;
* raw data and output files;
* log and preflight;
* history references to preparatory measurements;
* report modules where available.

Use temporary output directories. Log and pickle files must not remain in the
repository root.

5. Autosave and resume
----------------------

A resume test interrupts a virtual run after a defined point, loads the
autosave through ``load_pickle_compat``, and invokes the same measurement
routine again. It proves that:

* complete points are not measured again;
* an incomplete point is repeated cleanly;
* history is retained;
* the final result matches an uninterrupted reference run;
* the autosave is removed only after successful completion.

6. Safety and negative tests
----------------------------

Trigger protection functions deliberately. Cases include missing graph
paths, ambiguous conditions, ``MAXIN``, ``AmplifierProtect``, unreachable
leveling targets, operator abort, and errors during instrument communication.

The expected behavior is not always program termination. A leveler may stop
at a protection limit and record the limited state. RF-off must still run.

7. Qt tests
-----------

Headless tests use:

.. code-block:: console

   QT_QPA_PLATFORM=offscreen \
       python -m pytest test/test_qt_ui_adapter.py

They verify thread lifetime, incremental progress, stop behavior, error
display, and closing during an active operation. Timing-sensitive tests wait
for signals or states instead of using unnecessarily long fixed sleeps.

8. Staged hardware test
-----------------------

Do not begin hardware testing with the complete measurement range:

#. Run preflight without RF.
#. Initialize and identify instruments individually.
#. Verify RF-off and limit channels.
#. Measure one frequency at a safe level.
#. Run a small frequency range or a few points.
#. Exercise autosave and resume in practice.
#. Only then enable the complete workflow.

Templates below ``script/conf`` provide a ``HARDWARE_TEST.md`` for critical
TEM workflows. Record local details and the first commissioning results there
or in the configuration documentation.

Local tests and CI
------------------

The complete local test run corresponds to the core pipeline:

.. code-block:: console

   python -m pip install -e ".[dev]"
   python -m coverage run -m pytest test
   python -m coverage report

Build documentation separately with warnings treated as errors:

.. code-block:: console

   sphinx-build -W --keep-going -E -a \
       -b html docs/next/source docs/next/build/html

Coverage does not prove that a measurement workflow is tested correctly.
Safety, resume, and traceability in particular require explicit behavioral
assertions.