Metrics (rojak.turbulence.metrics)

Binary classification and association metrics for verifying turbulence diagnostics

This module provides dask- and xarray-aware implementations of metrics used to verify binary/categorical turbulence forecasts against observed truth. Most functions accept dask.array.Array or dask-backed xarray.DataArray inputs so that metrics can be computed lazily over large, chunked datasets.

The functionality is broadly grouped into:

• Curve-based metrics summarising performance across all classification thresholds: binary_classification_curve(), received_operating_characteristic(), and area_under_curve().

• Point metrics computed from a 2x2 confusion matrix of true/false positives/negatives (see confusion_matrix()), such as accuracy(), f1_score(), matthews_corr_coeff(), critical_success_index(), gilbert_skill_score(), and true_skill_score().

• Association measures for (optionally multidimensional and/or stratified) binary variables, built on top of contingency_table() and ContingencyTable, such as sample_odds_ratio(), relative_risk(), their conditional/marginal/stratified counterparts, and odds_ratio_from_table() and relative_risk_from_table() for counts that are already tabulated, along with the multidimensional skill scores matthews_corr_coeff_multidim() and jaccard_index_multidim().

Functions

accuracy([truth, prediction, confuse_matrix])	Compute the Accuracy (ACC)
area_under_curve(x_values, y_values)	Area under the curve
binary_classification_curve(sorted_truth, ...)	Binary classification curve
binary_classification_rate_from_cumsum()	Binary classification curve with cumulative sum on labels
conditional_odds_ratio(effect_of, on_var, ...)	Sample odds ratio between two binary variables, conditional on one or more control variables
confusion_matrix(truth, prediction)	Compute the confusion matrix
contingency_table(x_var, y_var, *, sum_over)	Contingency Table for multidimensional arrays
critical_success_index([truth, prediction, ...])	Compute the Critical Success Index (CSI) or the Jaccard Similarity Coefficient Score
f1_score([truth, prediction, confuse_matrix])	Compute the F1 Score
gilbert_skill_score([truth, prediction, ...])	Compute the Gilbt Skill Score
jaccard_index_multidim(first_var, ...)	Jaccard Index or Critical Success Index for multidimensional data
marginal_odds_ratio(effect_of, on_var, ...)	Marginal (pooled) sample odds ratio between two binary variables, aggregated over one or more control variables
matthews_corr_coeff(*[, truth, prediction, ...])	Compute the Matthew's Correlation Coefficient
matthews_corr_coeff_multidim(first_var, ...)	Matthews Correlation Coefficient for multidimensional arrays
mean_absolute_error(truth, prediction)	Mean Absolute Error (MAE)
odds_ratio_from_table(table, use_log)	Compute the (log) sample odds ratio directly from a ContingencyTable
received_operating_characteristic(...[, ...])	Received operating characteristic or ROC curve
relative_risk(first_var, second_var, sum_over, *)	Relative risk of an outcome with respect to an exposure variable
relative_risk_from_table(table, use_log)	Compute the (log) relative risk directly from a ContingencyTable
sample_odds_ratio(first_var, second_var, ...)	Sample odds ratio between two binary variables
sensitivity(true_positive, false_negative)	Sensitivity statistical metric
specificity(true_negative, false_positive)	Specificity statistical metric :param true_negative: :param false_positive:
stratified_contingency_table(effect_of, ...)	Compute contingency tables stratified by one or more control variables
stratified_relative_risk(effect_of, on_var, ...)	Relative risk of an outcome with respect to an exposure variable, stratified by one or more control variables
true_skill_score([truth, prediction, ...])	True Skill Score (TSS) statistic

Classes

BinaryClassificationRateFromLabels(...)
BinaryClassificationResult(false_positives, ...)
ContingencyTable(n_00, n_11, n_01, n_10)