mpylab.tools.stats module

Statistical helper functions used by mpylab.

The module contains quantity-aware basic statistics, dB/linear averaging helpers, autocorrelation for ordered sequences, and Rayleigh distribution helpers used by MSC evaluations.

author:

Hans Georg Krauthäuser (main author)

license:

GPLv3 or higher

mpylab.tools.stats.CalcPsi(n, rho, eps=0.01)

Sample the distribution of a Pearson correlation coefficient.

Parameters:

  • n (int) – Number of paired observations used for the correlation coefficient.

  • rho (float) – Population correlation coefficient.

  • eps (float, optional) – Spacing of the sampled coefficient axis between -1 and 1.

Returns:

  • r (list of float) – Sampled correlation coefficients.

  • psi (list of float) – Normalized probability-density values at r.

  • cpsi (list of float) – Normalized cumulative probabilities at r.

Notes

The exact expression is used where numerically possible. An approximate normal distribution after Fisher transformation is used on overflow.

mpylab.tools.stats.CalcRho0(r, cpsi, alpha)

Interpolate correlation coefficients at cumulative probabilities.

Parameters:

  • r (array-like) – Correlation-coefficient samples.

  • cpsi (array-like) – Cumulative probabilities corresponding to r.

  • alpha (float or mutable sequence of float) – Probability or probabilities to interpolate. A supplied mutable sequence is sorted in place.

Returns:

Mapping from each requested probability to its interpolated correlation coefficient as a float.

Return type:

dict

mpylab.tools.stats.CalcSigma(lst, av=None)

Calculate the sample standard deviation of compatible values.

Parameters:

  • lst (sequence) – Numeric values or SCUQ quantities. At least two samples are required.

  • av (object, optional) – Precomputed arithmetic mean. If omitted, the mean is calculated from lst.

Returns:

Sample standard deviation using denominator N - 1. A quantity result retains the unit of the input values.

Return type:

object

mpylab.tools.stats.ECDF(seq: list[float]) → object

Construct an interpolated empirical cumulative distribution function.

Parameters:

seq (sequence of float) – Sample values used to form the empirical distribution.

Returns:

Interpolator mapping sorted sample values to ranks 1/N through 1. Evaluation outside the sample range produces NaN.

Return type:

scipy.interpolate.interp1d

class mpylab.tools.stats.RayleighDist(loc: float = 0, scale: float = 1)

Bases: object

Frozen Rayleigh distribution with commonly used summary values.

Parameters:

  • loc (float, optional) – Distribution location parameter.

  • scale (float, optional) – Distribution scale parameter.

mean, std, entropy, median, variance, skew, kurtosis, mode

Summary statistics obtained from the frozen SciPy distribution.

Type:

float

cdf(x: ArrayLike) → object

Evaluate the cumulative distribution function.

Parameters:

x (array-like) – Evaluation points.

Returns:

Cumulative probability at each evaluation point.

Return type:

float or numpy.ndarray

pdf(x: ArrayLike) → object

Evaluate the probability density function.

Parameters:

x (array-like) – Evaluation points.

Returns:

Probability density at each evaluation point.

Return type:

float or numpy.ndarray

rvs(size: int = 1, random_state: int | None | object = None) → ArrayLike

Draw random samples from the distribution.

Parameters:

  • size (int or tuple of int, optional) – Number or shape of samples to draw.

  • random_state (int or object, optional) – Seed or random-state object accepted by SciPy.

Returns:

Random sample or samples.

Return type:

float or numpy.ndarray

mpylab.tools.stats.autocorrelation(x, maxlag=None, cyclic=True)

Compute cyclic or finite-sequence autocorrelation coefficients.

Parameters:

  • x (collections.abc.MutableSequence) – Ordered numeric values or compatible SCUQ quantities. Other input types return None for compatibility with the historical API.

  • maxlag (int, optional) – Largest requested lag. The default is len(x).

  • cyclic (bool, optional) – If true, wrap shifted samples cyclically. If false, use only overlap and stop before fewer than two paired samples remain.

Returns:

Correlation coefficient for each available lag from zero through maxlag, or None when x is not a mutable sequence.

Return type:

list or None

Raises:

ValueError – If maxlag is negative.

mpylab.tools.stats.chi2_test(data: ArrayLike | Callable[[float], float], cdf: Callable[[float], float]) → float

Perform a histogram-based chi-square goodness-of-fit test.

Parameters:

  • data (array-like) – Observed sample values.

  • cdf (callable) – Reference cumulative distribution. It must accept the NumPy array of histogram bin edges.

Returns:

Test p-value using two fitted degrees of freedom.

Return type:

float

Raises:

ValueError – If the reference CDF assigns no probability mass to the observed range.

mpylab.tools.stats.correlation(x, y)

Compute the Pearson correlation coefficient of two sequences.

Parameters:

  • x (sequence) – Paired numeric values or compatible SCUQ quantities.

  • y (sequence) – Paired numeric values or compatible SCUQ quantities.

Returns:

Covariance divided by the product of the sample standard deviations.

Return type:

object

Raises:

ValueError – If either sample sequence has zero variance, for which the correlation coefficient is undefined.

mpylab.tools.stats.covariance(x, y)

Compute the unbiased sample covariance of two sequences.

Parameters:

  • x (sequence) – Paired numeric values or compatible SCUQ quantities.

  • y (sequence) – Paired numeric values or compatible SCUQ quantities.

Returns:

Sample covariance calculated with denominator N - 1.

Return type:

object

Raises:

RuntimeError – If the sequences have different lengths or contain fewer than two samples.

mpylab.tools.stats.first_autocorrelation_threshold_crossing(coefficients, threshold)

Interpolate the first lag where autocorrelation falls below a threshold.

Parameters:

  • coefficients (sequence of float) – Autocorrelation coefficients ordered by integer lag, beginning at lag zero.

  • threshold (float) – Correlation threshold.

Returns:

Linearly interpolated crossing lag, or None if the coefficients never fall below the threshold.

Return type:

float or None

Raises:

ValueError – If the input is empty, non-finite, or not one-dimensional.

mpylab.tools.stats.fit_rayleigh(data: ArrayLike) → tuple[float, float]

Fit a Rayleigh distribution by maximum likelihood.

Parameters:

data (array-like) – Observed sample values.

Returns:

  • loc (float) – Fitted location parameter.

  • scale (float) – Fitted scale parameter.

mpylab.tools.stats.get_dB_factors(dB, k, A=10)

Convert a linear coverage factor into asymmetric dB factors.

Parameters:

  • dB (float) – Standard uncertainty represented on the logarithmic scale.

  • k (float) – Coverage factor on the linear scale.

  • A (float, optional) – Logarithmic conversion factor, normally 10 for power quantities and 20 for amplitude quantities.

Returns:

Lower and upper coverage factors on the dB scale.

Return type:

tuple of float

mpylab.tools.stats.get_k_factor(*args, coverage=0.95, both_tails=True, dist=None, **kwargs)

Return the coverage factor of a SciPy probability distribution.

Parameters:

  • *args (object) – Positional shape parameters passed to the distribution.

  • coverage (float, optional) – Requested probability coverage.

  • both_tails (bool, optional) – Split the excluded probability equally between both tails. If false, only the upper tail is excluded.

  • dist (str, optional) – Name of a distribution in scipy.stats; defaults to "norm".

  • **kwargs (object) – Keyword shape parameters passed to the distribution.

Returns:

Quantile corresponding to the requested coverage.

Return type:

float

Raises:

AttributeError – If the distribution does not exist or has no ppf method.

mpylab.tools.stats.get_k_factor_norm(coverage=0.95, both_tails=True)

Return the normal-distribution coverage factor.

Parameters:

  • coverage (float, optional) – Requested probability coverage.

  • both_tails (bool, optional) – Whether to split excluded probability between both tails.

Returns:

Normal-distribution quantile for the requested coverage.

Return type:

float

mpylab.tools.stats.get_k_factor_rect(coverage=0.95, both_tails=True)

Return the rectangular-distribution coverage factor.

Parameters:

  • coverage (float, optional) – Requested probability coverage.

  • both_tails (bool, optional) – Whether to split excluded probability between both tails.

Returns:

Rectangular-distribution coverage factor.

Return type:

float

mpylab.tools.stats.get_k_factor_ushape(coverage=0.95, both_tails=True)

Return the U-shaped-distribution coverage factor.

Parameters:

  • coverage (float, optional) – Requested probability coverage.

  • both_tails (bool, optional) – Whether to split excluded probability between both tails.

Returns:

U-shaped-distribution coverage factor.

Return type:

float

mpylab.tools.stats.independent_sample_estimate(coefficients, sample_count, threshold)

Estimate independent cyclic samples from an autocorrelation crossing.

Parameters:

  • coefficients (sequence of float) – Autocorrelation coefficients ordered by integer lag.

  • sample_count (int) – Number of samples in the complete cyclic sequence.

  • threshold (float) – Correlation threshold defining statistical independence.

Returns:

sample_count divided by the interpolated crossing lag, or None if no crossing exists.

Return type:

float or None

Raises:

ValueError – If sample_count is not positive.

mpylab.tools.stats.ks_test(data: ArrayLike | Callable[[float], float], cdf: ArrayLike | Callable[[float], float]) → float

Perform a one-sample Kolmogorov-Smirnov goodness-of-fit test.

Parameters:

  • data (array-like or callable) – Observations or another input accepted by scipy.stats.kstest().

  • cdf (array-like or callable) – Reference cumulative distribution accepted by SciPy.

Returns:

Test p-value.

Return type:

float

mpylab.tools.stats.linstat_dB(dbvals, fac=10)

Calculate linear-domain statistics for logarithmic input values.

Parameters:

  • dbvals (sequence of float) – Values represented on a logarithmic scale.

  • fac (float, optional) – Logarithmic factor. Use 10 for power quantities and 20 for amplitude quantities.

Returns:

  • mean_dB (float) – Linear arithmetic mean converted back to the logarithmic scale.

  • upper_std (float) – Difference between the logarithmic mean and mean + sigma.

  • lower_std (float) – Difference between the logarithmic mean and mean - sigma. The lower linear bound is clipped to the smallest positive float32 value.

Notes

The upper and lower logarithmic deviations are generally asymmetric.

mpylab.tools.stats.linstat_lin(linvals)

Calculate arithmetic mean and sample standard deviation.

Parameters:

linvals (sequence) – Linear numeric values or compatible SCUQ quantities.

Returns:

  • mean (object) – Arithmetic mean.

  • standard_deviation (object) – Sample standard deviation using denominator N - 1.

mpylab.tools.stats.mean(x, zero=None)

Compute the arithmetic mean while preserving SCUQ quantities.

Parameters:

  • x (sequence) – Non-empty sequence of compatible numeric values or quantities.

  • zero (object, optional) – Additive identity passed to sum(). When omitted, the first element of x determines the result type through summation().

Returns:

Arithmetic mean in the type and unit implied by x.

Return type:

object

mpylab.tools.stats.mean_standard_uncertainty(values)

Return arithmetic mean and type-A standard uncertainty of the mean.

The input values are treated as independent samples of one measurand. For two or more samples the type-A contribution is the sample standard deviation divided by sqrt(N). A single sample has no observable type-A contribution and therefore returns zero uncertainty.

Parameters:

values (array-like of float) – Non-empty, finite, one-dimensional sample values.

Returns:

  • average (float) – Arithmetic mean of the samples.

  • standard_uncertainty (float) – Sample standard deviation divided by sqrt(N). A single sample returns zero.

Raises:

ValueError – If the input is empty, not one-dimensional, or contains a non-finite value.

mpylab.tools.stats.numeric_samples(values)

Convert numeric values or compatible SCUQ quantities to floats.

Parameters:

values (sequence) – One-dimensional numeric samples or SCUQ quantities. Quantity samples are converted to the unit of the first sample and represented by their expectation values.

Returns:

One-dimensional array of finite floating-point samples.

Return type:

numpy.ndarray

Raises:

  • ValueError – If the input is empty, not one-dimensional, or non-finite.

  • scuq.qexceptions.ConversionException – If quantity samples use incompatible units.

mpylab.tools.stats.stddev(x)

Compute the sample standard deviation of compatible values.

Parameters:

x (sequence) – Numeric values or compatible SCUQ quantities.

Returns:

Square root of variance().

Return type:

object

mpylab.tools.stats.summation(x)

Sum a non-empty sequence while preserving SCUQ quantities.

Parameters:

x (sequence) – Values to add. The first element supplies the additive type and unit.

Returns:

Sum of all elements, with the type and unit implied by x.

Return type:

object

Raises:

IndexError – If x is empty.

mpylab.tools.stats.test_for_rayleigh(ees: ArrayLike) → tuple[ArrayLike, ArrayLike, object, object, float, float]

Fit and test a zero-location Rayleigh model for observed values.

Parameters:

ees (array-like) – Observed non-negative numeric values or SCUQ quantities. Quantities must use compatible units and are evaluated in the unit of the first sample.

Returns:

  • hist (numpy.ndarray) – Observed histogram counts.

  • bins (numpy.ndarray) – Histogram bin edges.

  • empirical_cdf (scipy.interpolate.interp1d) – Interpolated empirical cumulative distribution.

  • fitted_distribution (scipy.stats rv_frozen) – Rayleigh distribution fitted with location fixed to zero.

  • chi2_pvalue (float) – Chi-square goodness-of-fit p-value.

  • ks_pvalue (float) – Kolmogorov-Smirnov goodness-of-fit p-value.

Raises:

ValueError – If fewer than three finite, non-negative samples are supplied.

mpylab.tools.stats.variance(x)

Compute the unbiased sample variance of compatible values.

Parameters:

x (sequence) – Numeric values or compatible SCUQ quantities.

Returns:

Sample variance calculated with denominator N - 1.

Return type:

object

Raises:

RuntimeError – If fewer than two samples are supplied.