composite_observations_about_features#

rojak.atmosphere.feature_composite.composite_observations_about_features(feature: FeatureSource, observations: DataFrame, *, pairing: PairingStrategy, bins: BinSpec, controls: Sequence[ControlDesign] = (), return_pairs: Literal[False] = False, time_column: str = 'time', lat_column: str = 'latitude', lon_column: str = 'longitude', time_tolerance: timedelta64 | None = None) → Dataset[source]#
rojak.atmosphere.feature_composite.composite_observations_about_features(feature: FeatureSource, observations: DataFrame, *, pairing: PairingStrategy, bins: BinSpec, controls: Sequence[ControlDesign] = (), return_pairs: Literal[True], time_column: str = 'time', lat_column: str = 'latitude', lon_column: str = 'longitude', time_tolerance: timedelta64 | None = None) → tuple[Dataset, DataFrame, DataFrame]

Lazily count the observations about each centre of a feature, and about its controls, per time chunk.

Each observation is matched to the nearest time step of the feature’s fields, within time_tolerance, and paired with the centres at that time step by pairing. The pairs are counted in bins, in each centre’s km frame. For each control, the observations made observation_offset away from each centre’s time are paired with the same centre in the same way. A control is only counted for a centre if its observation time is within the time range of the fields.

Building the graph computes the number of observations per time chunk, once for the observations and once for all the controls together, so persist observations beforehand if it is expensive to compute. Nothing else is computed: with return_pairs=True, the tabulator and pairing also run once, eagerly, on an empty chunk to learn the columns and dtypes of the returned tables (see FeatureTabulator), which touches no real data. Compute the result, then derive statistics from it with impact_statistics() and bootstrap().

Parameters:

  • feature (FeatureSource) – The feature to composite about, e.g. lows.

  • observations (dask.dataframe.DataFrame) – Observations with a datetime column time_column, and latitudes and longitudes in degrees. Naive times are assumed to be in UTC. Longitudes may be in either [-180, 180) or [0, 360), independently of the feature’s fields.

  • pairing (PairingStrategy) – How observations are matched with centres.

  • bins (BinSpec) – The bins the pairs are counted in.

  • controls (Sequence[ControlDesign]) – The controls that estimate the counterfactual, each with a distinct label. Empty for none.

  • return_pairs (bool) – If True, also return the centres and the pairs as lazy dataframes.

  • time_column (str) – Name of the column of observations holding the time.

  • lat_column (str) – Name of the column of observations holding the latitude.

  • lon_column (str) – Name of the column of observations holding the longitude.

  • time_tolerance (numpy.timedelta64 | None) – Observations further than this from the nearest time step of the fields are dropped. None is half the median time step of the fields.

Returns:

The lazy counts, with the variables

  • feature_n_obs (chunk, *bins.dims): pairs per bin;

  • n_centres (chunk): number of centres. n_features, the number of distinct features (section 2), is deferred to a line feature (phase 6 or later): it always equals n_centres while every feature is POINT, so it is omitted rather than duplicated;

  • with controls, control_n_obs (chunk, control, *bins.dims): valid control pairs per bin, and n_controls (chunk, control): number of valid controls, with the control coordinate holding the labels in the order given;

and the attributes feature, geometry, pairing, bins, controls (the labels) and normalisation_unit (what one centre is). With return_pairs=True, a tuple of the counts, every centre (including those without pairs), and every pair of an observation with a centre (not those of the controls). All three share the chunk tasks, so compute them together, e.g. with dask.compute(), to run each once.

Return type:

xarray.Dataset | tuple[xarray.Dataset, dask.dataframe.DataFrame, dask.dataframe.DataFrame]

Raises:

  • TypeError – If feature is not a FeatureSource, pairing is not a PairingStrategy or bins is not a BinSpec.

  • ValueError – If pairing cannot handle feature, two controls share a label, or no observation (or no shifted observation of the controls) is within the time range of the feature’s fields.