pycontrails.models.humidity_scaling

Humidity scaling methodologies.

class pycontrails.models.humidity_scaling.ConstantHumidityScaling(met=None, params=None, **params_kwargs)

Bases: HumidityScaling

Scale specific humidity by applying a constant uniform scaling.

This scalar simply applies the transformation..

rhi -> rhi / rhi_adj

where rhi_adj is a constant specified by params or overridden by a variable or attribute on source in eval().

The convention to divide by rhi_adj instead of considering the more natural product rhi_adj * rhi is somewhat arbitrary. In short, rhi_adj can be thought of as the critical threshold for supersaturation.

References

default_params

alias of ConstantHumidityScalingParams

property description

Get description for instance.

downselect_met()

Downselect met domain to the max/min bounds of source.

Override this method if special handling is needed in met down-selection.

  • source must be defined before calling downselect_met().

  • This method copies and re-assigns met using met.copy() to avoid side-effects.

Raises:
classmethod ecmwf_met_variables()

Return an ECMWF-specific list of required meteorology variables.

Returns:

tuple[MetVariable] – List of ECMWF-specific variants of required variables

eval(source=None, **params)

Scale specific humidity values on source.

This method mutates the parameter source by modifying its “specific_humidity” variable and by attaching an “rhi” variable. Set model parameter copy_source=True to avoid mutating source.

Parameters:

  • source (GeoVectorDataset | MetDataset) – Data with variables “specific_humidity”, “air_temperature”, and any variables defined by scaler_specific_keys.

  • **params (Any) – Overwrite model parameters before eval

Returns:

GeoVectorDataset | MetDataset – Source data with updated “specific_humidity” and “rhi”. If source is GeoVectorDataset, “air_pressure” data is also attached.

See also

scale()

formula = 'rhi -> rhi / rhi_adj'
classmethod generic_met_variables()

Return a model-agnostic list of required meteorology variables.

Returns:

tuple[MetVariable] – List of model-agnostic variants of required variables

get_data_param(other, key, default=<object object>, *, set_attr=True)

Get data from other source-compatible object with default set by model parameter key.

Retrieves data with the following hierarchy:

  1. other.data[key]. Returns np.ndarray | xr.DataArray.

  2. other.attrs[key]

  3. params[key]

  4. default

In case 3., the value of params[key] is attached to other.attrs[key] unless set_attr is set to False.

Parameters:

  • key (str) – Key to retrieve

  • default (Any, optional) – Default value if key is not found.

  • set_attr (bool, optional) – If True (default), set source.attrs[key] to params[key] if found. This allows for better post model evaluation tracking.

Returns:

Any – Value(s) found for key in other data, other attrs, or model params

Raises:

KeyError – Raises KeyError if key is not found in any location and default is not provided.

See also

get_source_param, pycontrails.core.vector.GeoVectorDataset.get_data_or_attr

get_source_param(key, default=<object object>, *, set_attr=True)

Get source data with default set by parameter key.

Retrieves data with the following hierarchy:

  1. source.data[key]. Returns np.ndarray | xr.DataArray.

  2. source.attrs[key]

  3. params[key]

  4. default

In case 3., the value of params[key] is attached to source.attrs[key] unless set_attr is set to False.

Parameters:

  • key (str) – Key to retrieve

  • default (Any, optional) – Default value if key is not found.

  • set_attr (bool, optional) – If True (default), set source.attrs[key] to params[key] if found. This allows for better post model evaluation tracking.

Returns:

Any – Value(s) found for key in source data, source attrs, or model params

Raises:

KeyError – Raises KeyError if key is not found in any location and default is not provided.

See also

get_data_param, pycontrails.core.vector.GeoVectorDataset.get_data_or_attr

classmethod gfs_met_variables()

Return a GFS-specific list of required meteorology variables.

Returns:

tuple[MetVariable] – List of GFS-specific variants of required variables

property hash

Generate a unique hash for model instance.

Returns:

str – Unique hash for model instance (sha1)

property interp_kwargs

Shortcut to create interpolation arguments from params.

The output of this is useful for passing to interpolate_met().

Returns:

dict[str, Any] – Dictionary with keys

  • ”method”

  • ”bounds_error”

  • ”fill_value”

  • ”localize”

  • ”use_indices”

  • ”q_method”

as determined by params.

long_name = 'Constant humidity scaling'
met

Meteorology data

met_required = False

Require meteorology is not None on __init__()

met_variables

Required meteorology pressure level variables. Each element in the list is a MetVariable or a tuple[MetVariable]. If element is a tuple[MetVariable], the variable depends on the data source and the tuple must include entries for a model-agnostic variable, an ECMWF-specific variable, and a GFS-specific variable. Only one of the three variable in the tuple is required for model evaluation.

name = 'constant_scale'
optional_met_variables

Optional meteorology variables

params

Instantiated model parameters, in dictionary form

processed_met_variables

Set of required parameters if processing already complete on met input.

require_met()

Ensure that met is a MetDataset.

Returns:

MetDataset – Returns reference to met. This is helpful for type narrowing met when meteorology is required.

Raises:

ValueError – Raises when met is None.

require_source_type(type_)

Ensure that source is type_.

Returns:

_Source – Returns reference to source. This is helpful for type narrowing source to specific type(s).

Raises:

ValueError – Raises when source is not _type_.

scale(specific_humidity, air_temperature, air_pressure, **kwargs)

Compute scaled specific humidity and RHi.

See docstring for the implementing subclass for specific methodology.

Parameters:

  • specific_humidity (ArrayLike) – Unscaled specific relative humidity, [\(kg \ kg^{-1}\)]. Typically, this is interpolated meteorology data.

  • air_temperature (ArrayLike) – Air temperature, [\(K\)]. Typically, this is interpolated meteorology data.

  • air_pressure (ArrayLike) – Pressure, [\(Pa\)]

  • kwargs (ArrayLike) – Other keyword-only variables and model parameters used by the formula.

Returns:

  • specific_humidity (ArrayLike) – Scaled specific humidity.

  • rhi (ArrayLike) – Scaled relative humidity over ice.

See also

eval()

scaler_specific_keys = ('rhi_adj',)

Variables required in addition to specific_humidity, air_temperature, and air_pressure These are either ModelParams specific to scaling, or variables that should be extracted from eval() parameter source.

set_source(source=None)

Attach original or copy of input source to source.

Parameters:

source (MetDataset | GeoVectorDataset | Flight | Iterable[Flight] | None) – Parameter source passed in eval(). If None, an empty MetDataset with coordinates like met is set to source.

See also

eval

set_source_met(optional=False, variable=None)

Ensure or interpolate each required met_variables on source .

For each variable in met_variables, check source for data variable with the same name.

For GeoVectorDataset sources, try to interpolate met if variable does not exist.

For MetDataset sources, try to get data from met if variable does not exist.

Parameters:
Raises:
source

Data evaluated in model

property to_json

Get description for instance.

transfer_met_source_attrs(source=None)

Transfer met source metadata from met to source.

update_params(params=None, **params_kwargs)

Update model parameters on params.

Parameters:

  • params (dict[str, Any], optional) – Model parameters to update, as dictionary. Defaults to {}

  • **params_kwargs (Any) – Override keys in params with keyword arguments.

class pycontrails.models.humidity_scaling.ConstantHumidityScalingParams(copy_source=True, interpolation_method='linear', interpolation_bounds_error=False, interpolation_fill_value=nan, interpolation_localize=False, interpolation_use_indices=False, interpolation_q_method=None, verify_met=True, downselect_met=True, met_longitude_buffer=(0.0, 0.0), met_latitude_buffer=(0.0, 0.0), met_level_buffer=(0.0, 0.0), met_time_buffer=(np.timedelta64(0, 'h'), np.timedelta64(0, 'h')), rhi_adj=0.97)

Bases: ModelParams

Parameters for ConstantHumidityScaling.

as_dict()

Convert object to dictionary.

We use this method instead of dataclasses.asdict to use a shallow/unrecursive copy. This will return values as Any instead of dict.

Returns:

dict[str, Any] – Dictionary version of self.

copy_source = True

Copy input source data on eval

downselect_met = True

Downselect input MetDataset` to region around source.

interpolation_bounds_error = False

If True, points lying outside interpolation will raise an error

interpolation_fill_value = nan

Used for outside interpolation value if interpolation_bounds_error is False

interpolation_localize = False

Experimental. See pycontrails.core.interpolation.

interpolation_method = 'linear'

Interpolation method. Supported methods include “linear”, “nearest”, “slinear”, “cubic”, and “quintic”. See scipy.interpolate.RegularGridInterpolator for the description of each method. Not all methods are supported by all met grids. For example, the “cubic” method requires at least 4 points per dimension.

interpolation_q_method = None

Experimental. Alternative interpolation method to account for specific humidity lapse rate bias. Must be one of None, "cubic-spline", or "log-q-log-p". If None, no special interpolation is used for specific humidity. The "cubic-spline" method applies a custom stretching of the met interpolation table to account for the specific humidity lapse rate bias. The "log-q-log-p" method interpolates in the log of specific humidity and pressure, then converts back to specific humidity. Only used by models calling to interpolate_met().

interpolation_use_indices = False

Experimental. See pycontrails.core.interpolation.

met_latitude_buffer = (0.0, 0.0)

Met latitude buffer for input to Flight.downselect_met(), in WGS84 coordinates. Only applies when downselect_met is True.

met_level_buffer = (0.0, 0.0)

Met level buffer for input to Flight.downselect_met(), in [\(hPa\)]. Only applies when downselect_met is True.

met_longitude_buffer = (0.0, 0.0)

Met longitude buffer for input to Flight.downselect_met(), in WGS84 coordinates. Only applies when downselect_met is True.

met_time_buffer = (np.timedelta64(0,'h'), np.timedelta64(0,'h'))

Met time buffer for input to Flight.downselect_met() Only applies when downselect_met is True.

rhi_adj = 0.97

Scale specific humidity by dividing it with adjustment factor per [Schumann, 2012] eq. (9). Set to a constant 0.9 in [Schumann, 2012] to account for sub-scale variability of specific humidity. A value of 1.0 provides no scaling.

verify_met = True

Call _verify_met() on model instantiation.

class pycontrails.models.humidity_scaling.ExponentialBoostHumidityScaling(met=None, params=None, **params_kwargs)

Bases: HumidityScaling

Scale humidity by composing constant scaling with exponential boosting.

This formula composes the transformations

  1. constant scaling: rhi -> rhi / rhi_adj

  2. exponential boosting: rhi -> rhi ^ rhi_boost_exponent if rhi > 1

  3. clipping: rhi -> min(rhi, clip_upper)

where rhi_adj, rhi_boost_exponent, and clip_upper are model params.

References

See also

ExponentialBoostLatitudeCorrectionHumidityScaling

default_params

alias of ExponentialBoostHumidityScalingParams

property description

Get description for instance.

downselect_met()

Downselect met domain to the max/min bounds of source.

Override this method if special handling is needed in met down-selection.

  • source must be defined before calling downselect_met().

  • This method copies and re-assigns met using met.copy() to avoid side-effects.

Raises:
classmethod ecmwf_met_variables()

Return an ECMWF-specific list of required meteorology variables.

Returns:

tuple[MetVariable] – List of ECMWF-specific variants of required variables

eval(source=None, **params)

Scale specific humidity values on source.

This method mutates the parameter source by modifying its “specific_humidity” variable and by attaching an “rhi” variable. Set model parameter copy_source=True to avoid mutating source.

Parameters:

  • source (GeoVectorDataset | MetDataset) – Data with variables “specific_humidity”, “air_temperature”, and any variables defined by scaler_specific_keys.

  • **params (Any) – Overwrite model parameters before eval

Returns:

GeoVectorDataset | MetDataset – Source data with updated “specific_humidity” and “rhi”. If source is GeoVectorDataset, “air_pressure” data is also attached.

See also

scale()

formula = 'rhi -> (rhi / rhi_adj) ^ rhi_boost_exponent'
classmethod generic_met_variables()

Return a model-agnostic list of required meteorology variables.

Returns:

tuple[MetVariable] – List of model-agnostic variants of required variables

get_data_param(other, key, default=<object object>, *, set_attr=True)

Get data from other source-compatible object with default set by model parameter key.

Retrieves data with the following hierarchy:

  1. other.data[key]. Returns np.ndarray | xr.DataArray.

  2. other.attrs[key]

  3. params[key]

  4. default

In case 3., the value of params[key] is attached to other.attrs[key] unless set_attr is set to False.

Parameters:

  • key (str) – Key to retrieve

  • default (Any, optional) – Default value if key is not found.

  • set_attr (bool, optional) – If True (default), set source.attrs[key] to params[key] if found. This allows for better post model evaluation tracking.

Returns:

Any – Value(s) found for key in other data, other attrs, or model params

Raises:

KeyError – Raises KeyError if key is not found in any location and default is not provided.

See also

get_source_param, pycontrails.core.vector.GeoVectorDataset.get_data_or_attr

get_source_param(key, default=<object object>, *, set_attr=True)

Get source data with default set by parameter key.

Retrieves data with the following hierarchy:

  1. source.data[key]. Returns np.ndarray | xr.DataArray.

  2. source.attrs[key]

  3. params[key]

  4. default

In case 3., the value of params[key] is attached to source.attrs[key] unless set_attr is set to False.

Parameters:

  • key (str) – Key to retrieve

  • default (Any, optional) – Default value if key is not found.

  • set_attr (bool, optional) – If True (default), set source.attrs[key] to params[key] if found. This allows for better post model evaluation tracking.

Returns:

Any – Value(s) found for key in source data, source attrs, or model params

Raises:

KeyError – Raises KeyError if key is not found in any location and default is not provided.

See also

get_data_param, pycontrails.core.vector.GeoVectorDataset.get_data_or_attr

classmethod gfs_met_variables()

Return a GFS-specific list of required meteorology variables.

Returns:

tuple[MetVariable] – List of GFS-specific variants of required variables

property hash

Generate a unique hash for model instance.

Returns:

str – Unique hash for model instance (sha1)

property interp_kwargs

Shortcut to create interpolation arguments from params.

The output of this is useful for passing to interpolate_met().

Returns:

dict[str, Any] – Dictionary with keys

  • ”method”

  • ”bounds_error”

  • ”fill_value”

  • ”localize”

  • ”use_indices”

  • ”q_method”

as determined by params.

long_name = 'Constant humidity scaling composed with exponential boosting'
met

Meteorology data

met_required = False

Require meteorology is not None on __init__()

met_variables

Required meteorology pressure level variables. Each element in the list is a MetVariable or a tuple[MetVariable]. If element is a tuple[MetVariable], the variable depends on the data source and the tuple must include entries for a model-agnostic variable, an ECMWF-specific variable, and a GFS-specific variable. Only one of the three variable in the tuple is required for model evaluation.

name = 'exponential_boost'
optional_met_variables

Optional meteorology variables

params

Instantiated model parameters, in dictionary form

processed_met_variables

Set of required parameters if processing already complete on met input.

require_met()

Ensure that met is a MetDataset.

Returns:

MetDataset – Returns reference to met. This is helpful for type narrowing met when meteorology is required.

Raises:

ValueError – Raises when met is None.

require_source_type(type_)

Ensure that source is type_.

Returns:

_Source – Returns reference to source. This is helpful for type narrowing source to specific type(s).

Raises:

ValueError – Raises when source is not _type_.

scale(specific_humidity, air_temperature, air_pressure, **kwargs)

Compute scaled specific humidity and RHi.

See docstring for the implementing subclass for specific methodology.

Parameters:

  • specific_humidity (ArrayLike) – Unscaled specific relative humidity, [\(kg \ kg^{-1}\)]. Typically, this is interpolated meteorology data.

  • air_temperature (ArrayLike) – Air temperature, [\(K\)]. Typically, this is interpolated meteorology data.

  • air_pressure (ArrayLike) – Pressure, [\(Pa\)]

  • kwargs (ArrayLike) – Other keyword-only variables and model parameters used by the formula.

Returns:

  • specific_humidity (ArrayLike) – Scaled specific humidity.

  • rhi (ArrayLike) – Scaled relative humidity over ice.

See also

eval()

scaler_specific_keys = ('rhi_adj', 'rhi_boost_exponent', 'clip_upper')

Variables required in addition to specific_humidity, air_temperature, and air_pressure These are either ModelParams specific to scaling, or variables that should be extracted from eval() parameter source.

set_source(source=None)

Attach original or copy of input source to source.

Parameters:

source (MetDataset | GeoVectorDataset | Flight | Iterable[Flight] | None) – Parameter source passed in eval(). If None, an empty MetDataset with coordinates like met is set to source.

See also

eval

set_source_met(optional=False, variable=None)

Ensure or interpolate each required met_variables on source .

For each variable in met_variables, check source for data variable with the same name.

For GeoVectorDataset sources, try to interpolate met if variable does not exist.

For MetDataset sources, try to get data from met if variable does not exist.

Parameters:
Raises:
source

Data evaluated in model

property to_json

Get description for instance.

transfer_met_source_attrs(source=None)

Transfer met source metadata from met to source.

update_params(params=None, **params_kwargs)

Update model parameters on params.

Parameters:

  • params (dict[str, Any], optional) – Model parameters to update, as dictionary. Defaults to {}

  • **params_kwargs (Any) – Override keys in params with keyword arguments.

class pycontrails.models.humidity_scaling.ExponentialBoostHumidityScalingParams(copy_source=True, interpolation_method='linear', interpolation_bounds_error=False, interpolation_fill_value=nan, interpolation_localize=False, interpolation_use_indices=False, interpolation_q_method=None, verify_met=True, downselect_met=True, met_longitude_buffer=(0.0, 0.0), met_latitude_buffer=(0.0, 0.0), met_level_buffer=(0.0, 0.0), met_time_buffer=(np.timedelta64(0, 'h'), np.timedelta64(0, 'h')), rhi_adj=0.97, rhi_boost_exponent=1.7, clip_upper=1.7)

Bases: ConstantHumidityScalingParams

Parameters for ExponentialBoostHumidityScaling.

as_dict()

Convert object to dictionary.

We use this method instead of dataclasses.asdict to use a shallow/unrecursive copy. This will return values as Any instead of dict.

Returns:

dict[str, Any] – Dictionary version of self.

clip_upper = 1.7

Used to clip overinflated unrealistic RHi values.

copy_source = True

Copy input source data on eval

downselect_met = True

Downselect input MetDataset` to region around source.

interpolation_bounds_error = False

If True, points lying outside interpolation will raise an error

interpolation_fill_value = nan

Used for outside interpolation value if interpolation_bounds_error is False

interpolation_localize = False

Experimental. See pycontrails.core.interpolation.

interpolation_method = 'linear'

Interpolation method. Supported methods include “linear”, “nearest”, “slinear”, “cubic”, and “quintic”. See scipy.interpolate.RegularGridInterpolator for the description of each method. Not all methods are supported by all met grids. For example, the “cubic” method requires at least 4 points per dimension.

interpolation_q_method = None

Experimental. Alternative interpolation method to account for specific humidity lapse rate bias. Must be one of None, "cubic-spline", or "log-q-log-p". If None, no special interpolation is used for specific humidity. The "cubic-spline" method applies a custom stretching of the met interpolation table to account for the specific humidity lapse rate bias. The "log-q-log-p" method interpolates in the log of specific humidity and pressure, then converts back to specific humidity. Only used by models calling to interpolate_met().

interpolation_use_indices = False

Experimental. See pycontrails.core.interpolation.

met_latitude_buffer = (0.0, 0.0)

Met latitude buffer for input to Flight.downselect_met(), in WGS84 coordinates. Only applies when downselect_met is True.

met_level_buffer = (0.0, 0.0)

Met level buffer for input to Flight.downselect_met(), in [\(hPa\)]. Only applies when downselect_met is True.

met_longitude_buffer = (0.0, 0.0)

Met longitude buffer for input to Flight.downselect_met(), in WGS84 coordinates. Only applies when downselect_met is True.

met_time_buffer = (np.timedelta64(0,'h'), np.timedelta64(0,'h'))

Met time buffer for input to Flight.downselect_met() Only applies when downselect_met is True.

rhi_adj = 0.97

Scale specific humidity by dividing it with adjustment factor per [Schumann, 2012] eq. (9). Set to a constant 0.9 in [Schumann, 2012] to account for sub-scale variability of specific humidity. A value of 1.0 provides no scaling.

rhi_boost_exponent = 1.7

Boost RHi values exceeding 1 as described in [Teoh et al., 2022]. In eval(), this can be overridden by a keyword argument with the same name.

verify_met = True

Call _verify_met() on model instantiation.

class pycontrails.models.humidity_scaling.ExponentialBoostLatitudeCorrectionHumidityScaling(met=None, params=None, **params_kwargs)

Bases: HumidityScaling

Correct RHi values derived from ECMWF ERA5 HRES.

Unlike other RHi correction factors, this function applies a custom latitude-based term and has been tuned for global application.

This formula composes the transformations

  1. constant scaling: rhi -> rhi / rhi_adj

  2. exponential boosting: rhi -> rhi ^ rhi_boost_exponent if rhi > 1

  3. clipping: rhi -> min(rhi, rhi_max)

where rhi_adj and rhi_boost_exponent depend on latitude to minimize error between ERA5 HRES and IAGOS in-situ data.

For each waypoint, rhi_max ensures that the corrected RHi does not exceed the maximum value according to thermodynamics:

  • rhi_max = p_liq(T) / p_ice(T) for T > 235 K, (Pruppacher and Klett, 1997)

  • rhi_max = 1.67 + (1.45 - 1.67) * (T - 190.) / (235. - 190.) for T < 235 K (Karcher and Lohmann, 2002; Tompkins et al., 2007)

The RHi correction addresses the known limitations of the ERA5 HRES humidity fields, ensuring that the ISSR coverage area and RHi-distribution is consistent with in-situ measurements from the IAGOS dataset. Generally, the correction:

  1. reduces the ISSR coverage area near the equator,

  2. increases the ISSR coverage area at higher latitudes, and

  3. accounts for localized regions with very high ice supersaturation (RHi > 120%).

This methodology is an extension of Teoh et al. (2022) and has not yet been peer-reviewed/published.

The ERA5 HRES <> IAGOS fitting uses a sigmoid curve to capture significant changes in tropopause height at 20 - 50 degrees latitude.

The method eval() requires a latitude keyword argument.

References

  • [Teoh et al., 2022]

  • Kärcher, B. and Lohmann, U., 2002. A parameterization of cirrus cloud formation: Homogeneous freezing of supercooled aerosols. Journal of Geophysical Research: Atmospheres, 107(D2), pp.AAC-4.

  • Pruppacher, H.R. and Klett, J.D. (1997) Microphysics of Clouds and Precipitation. 2nd Edition, Kluwer Academic, Dordrecht, 954 p.

  • Tompkins, A.M., Gierens, K. and Rädel, G., 2007. Ice supersaturation in the ECMWF integrated forecast system. Quarterly Journal of the Royal Meteorological Society: A journal of the atmospheric sciences, applied meteorology and physical oceanography, 133(622), pp.53-63.

See also

ExponentialBoostHumidityScaling

default_params

alias of ExponentialBoostLatitudeCorrectionParams

property description

Get description for instance.

downselect_met()

Downselect met domain to the max/min bounds of source.

Override this method if special handling is needed in met down-selection.

  • source must be defined before calling downselect_met().

  • This method copies and re-assigns met using met.copy() to avoid side-effects.

Raises:
classmethod ecmwf_met_variables()

Return an ECMWF-specific list of required meteorology variables.

Returns:

tuple[MetVariable] – List of ECMWF-specific variants of required variables

eval(source=None, **params)

Scale specific humidity values on source.

This method mutates the parameter source by modifying its “specific_humidity” variable and by attaching an “rhi” variable. Set model parameter copy_source=True to avoid mutating source.

Parameters:

  • source (GeoVectorDataset | MetDataset) – Data with variables “specific_humidity”, “air_temperature”, and any variables defined by scaler_specific_keys.

  • **params (Any) – Overwrite model parameters before eval

Returns:

GeoVectorDataset | MetDataset – Source data with updated “specific_humidity” and “rhi”. If source is GeoVectorDataset, “air_pressure” data is also attached.

See also

scale()

formula = 'rhi -> (rhi / rhi_adj) ^ rhi_boost_exponent'
classmethod generic_met_variables()

Return a model-agnostic list of required meteorology variables.

Returns:

tuple[MetVariable] – List of model-agnostic variants of required variables

get_data_param(other, key, default=<object object>, *, set_attr=True)

Get data from other source-compatible object with default set by model parameter key.

Retrieves data with the following hierarchy:

  1. other.data[key]. Returns np.ndarray | xr.DataArray.

  2. other.attrs[key]

  3. params[key]

  4. default

In case 3., the value of params[key] is attached to other.attrs[key] unless set_attr is set to False.

Parameters:

  • key (str) – Key to retrieve

  • default (Any, optional) – Default value if key is not found.

  • set_attr (bool, optional) – If True (default), set source.attrs[key] to params[key] if found. This allows for better post model evaluation tracking.

Returns:

Any – Value(s) found for key in other data, other attrs, or model params

Raises:

KeyError – Raises KeyError if key is not found in any location and default is not provided.

See also

get_source_param, pycontrails.core.vector.GeoVectorDataset.get_data_or_attr

get_source_param(key, default=<object object>, *, set_attr=True)

Get source data with default set by parameter key.

Retrieves data with the following hierarchy:

  1. source.data[key]. Returns np.ndarray | xr.DataArray.

  2. source.attrs[key]

  3. params[key]

  4. default

In case 3., the value of params[key] is attached to source.attrs[key] unless set_attr is set to False.

Parameters:

  • key (str) – Key to retrieve

  • default (Any, optional) – Default value if key is not found.

  • set_attr (bool, optional) – If True (default), set source.attrs[key] to params[key] if found. This allows for better post model evaluation tracking.

Returns:

Any – Value(s) found for key in source data, source attrs, or model params

Raises:

KeyError – Raises KeyError if key is not found in any location and default is not provided.

See also

get_data_param, pycontrails.core.vector.GeoVectorDataset.get_data_or_attr

classmethod gfs_met_variables()

Return a GFS-specific list of required meteorology variables.

Returns:

tuple[MetVariable] – List of GFS-specific variants of required variables

property hash

Generate a unique hash for model instance.

Returns:

str – Unique hash for model instance (sha1)

property interp_kwargs

Shortcut to create interpolation arguments from params.

The output of this is useful for passing to interpolate_met().

Returns:

dict[str, Any] – Dictionary with keys

  • ”method”

  • ”bounds_error”

  • ”fill_value”

  • ”localize”

  • ”use_indices”

  • ”q_method”

as determined by params.

long_name = 'Latitude specific humidity scaling composed with exponential boosting'
met

Meteorology data

met_required = False

Require meteorology is not None on __init__()

met_variables

Required meteorology pressure level variables. Each element in the list is a MetVariable or a tuple[MetVariable]. If element is a tuple[MetVariable], the variable depends on the data source and the tuple must include entries for a model-agnostic variable, an ECMWF-specific variable, and a GFS-specific variable. Only one of the three variable in the tuple is required for model evaluation.

name = 'exponential_boost_latitude_customization'
optional_met_variables

Optional meteorology variables

params

Instantiated model parameters, in dictionary form

processed_met_variables

Set of required parameters if processing already complete on met input.

require_met()

Ensure that met is a MetDataset.

Returns:

MetDataset – Returns reference to met. This is helpful for type narrowing met when meteorology is required.

Raises:

ValueError – Raises when met is None.

require_source_type(type_)

Ensure that source is type_.

Returns:

_Source – Returns reference to source. This is helpful for type narrowing source to specific type(s).

Raises:

ValueError – Raises when source is not _type_.

scale(specific_humidity, air_temperature, air_pressure, **kwargs)

Compute scaled specific humidity and RHi.

See docstring for the implementing subclass for specific methodology.

Parameters:

  • specific_humidity (ArrayLike) – Unscaled specific relative humidity, [\(kg \ kg^{-1}\)]. Typically, this is interpolated meteorology data.

  • air_temperature (ArrayLike) – Air temperature, [\(K\)]. Typically, this is interpolated meteorology data.

  • air_pressure (ArrayLike) – Pressure, [\(Pa\)]

  • kwargs (ArrayLike) – Other keyword-only variables and model parameters used by the formula.

Returns:

  • specific_humidity (ArrayLike) – Scaled specific humidity.

  • rhi (ArrayLike) – Scaled relative humidity over ice.

See also

eval()

scaler_specific_keys = ('latitude',)

Variables required in addition to specific_humidity, air_temperature, and air_pressure These are either ModelParams specific to scaling, or variables that should be extracted from eval() parameter source.

set_source(source=None)

Attach original or copy of input source to source.

Parameters:

source (MetDataset | GeoVectorDataset | Flight | Iterable[Flight] | None) – Parameter source passed in eval(). If None, an empty MetDataset with coordinates like met is set to source.

See also

eval

set_source_met(optional=False, variable=None)

Ensure or interpolate each required met_variables on source .

For each variable in met_variables, check source for data variable with the same name.

For GeoVectorDataset sources, try to interpolate met if variable does not exist.

For MetDataset sources, try to get data from met if variable does not exist.

Parameters:
Raises:
source

Data evaluated in model

property to_json

Get description for instance.

transfer_met_source_attrs(source=None)

Transfer met source metadata from met to source.

update_params(params=None, **params_kwargs)

Update model parameters on params.

Parameters:

  • params (dict[str, Any], optional) – Model parameters to update, as dictionary. Defaults to {}

  • **params_kwargs (Any) – Override keys in params with keyword arguments.

class pycontrails.models.humidity_scaling.HistogramMatching(met=None, params=None, **params_kwargs)

Bases: HumidityScaling

Scale humidity by histogram matching to IAGOS RHi quantiles.

default_params

alias of HistogramMatchingParams

property description

Get description for instance.

downselect_met()

Downselect met domain to the max/min bounds of source.

Override this method if special handling is needed in met down-selection.

  • source must be defined before calling downselect_met().

  • This method copies and re-assigns met using met.copy() to avoid side-effects.

Raises:
classmethod ecmwf_met_variables()

Return an ECMWF-specific list of required meteorology variables.

Returns:

tuple[MetVariable] – List of ECMWF-specific variants of required variables

eval(source=None, **params)

Scale specific humidity values on source.

This method mutates the parameter source by modifying its “specific_humidity” variable and by attaching an “rhi” variable. Set model parameter copy_source=True to avoid mutating source.

Parameters:

  • source (GeoVectorDataset | MetDataset) – Data with variables “specific_humidity”, “air_temperature”, and any variables defined by scaler_specific_keys.

  • **params (Any) – Overwrite model parameters before eval

Returns:

GeoVectorDataset | MetDataset – Source data with updated “specific_humidity” and “rhi”. If source is GeoVectorDataset, “air_pressure” data is also attached.

See also

scale()

formula = 'era5_quantiles -> iagos_quantiles'
classmethod generic_met_variables()

Return a model-agnostic list of required meteorology variables.

Returns:

tuple[MetVariable] – List of model-agnostic variants of required variables

get_data_param(other, key, default=<object object>, *, set_attr=True)

Get data from other source-compatible object with default set by model parameter key.

Retrieves data with the following hierarchy:

  1. other.data[key]. Returns np.ndarray | xr.DataArray.

  2. other.attrs[key]

  3. params[key]

  4. default

In case 3., the value of params[key] is attached to other.attrs[key] unless set_attr is set to False.

Parameters:

  • key (str) – Key to retrieve

  • default (Any, optional) – Default value if key is not found.

  • set_attr (bool, optional) – If True (default), set source.attrs[key] to params[key] if found. This allows for better post model evaluation tracking.

Returns:

Any – Value(s) found for key in other data, other attrs, or model params

Raises:

KeyError – Raises KeyError if key is not found in any location and default is not provided.

See also

get_source_param, pycontrails.core.vector.GeoVectorDataset.get_data_or_attr

get_source_param(key, default=<object object>, *, set_attr=True)

Get source data with default set by parameter key.

Retrieves data with the following hierarchy:

  1. source.data[key]. Returns np.ndarray | xr.DataArray.

  2. source.attrs[key]

  3. params[key]

  4. default

In case 3., the value of params[key] is attached to source.attrs[key] unless set_attr is set to False.

Parameters:

  • key (str) – Key to retrieve

  • default (Any, optional) – Default value if key is not found.

  • set_attr (bool, optional) – If True (default), set source.attrs[key] to params[key] if found. This allows for better post model evaluation tracking.

Returns:

Any – Value(s) found for key in source data, source attrs, or model params

Raises:

KeyError – Raises KeyError if key is not found in any location and default is not provided.

See also

get_data_param, pycontrails.core.vector.GeoVectorDataset.get_data_or_attr

classmethod gfs_met_variables()

Return a GFS-specific list of required meteorology variables.

Returns:

tuple[MetVariable] – List of GFS-specific variants of required variables

property hash

Generate a unique hash for model instance.

Returns:

str – Unique hash for model instance (sha1)

property interp_kwargs

Shortcut to create interpolation arguments from params.

The output of this is useful for passing to interpolate_met().

Returns:

dict[str, Any] – Dictionary with keys

  • ”method”

  • ”bounds_error”

  • ”fill_value”

  • ”localize”

  • ”use_indices”

  • ”q_method”

as determined by params.

long_name = 'IAGOS RHi histogram matching'
met

Meteorology data

met_required = False

Require meteorology is not None on __init__()

met_variables

Required meteorology pressure level variables. Each element in the list is a MetVariable or a tuple[MetVariable]. If element is a tuple[MetVariable], the variable depends on the data source and the tuple must include entries for a model-agnostic variable, an ECMWF-specific variable, and a GFS-specific variable. Only one of the three variable in the tuple is required for model evaluation.

name = 'histogram_matching'
optional_met_variables

Optional meteorology variables

params

Instantiated model parameters, in dictionary form

processed_met_variables

Set of required parameters if processing already complete on met input.

require_met()

Ensure that met is a MetDataset.

Returns:

MetDataset – Returns reference to met. This is helpful for type narrowing met when meteorology is required.

Raises:

ValueError – Raises when met is None.

require_source_type(type_)

Ensure that source is type_.

Returns:

_Source – Returns reference to source. This is helpful for type narrowing source to specific type(s).

Raises:

ValueError – Raises when source is not _type_.

scale(specific_humidity, air_temperature, air_pressure, **kwargs)

Compute scaled specific humidity and RHi.

See docstring for the implementing subclass for specific methodology.

Parameters:

  • specific_humidity (ArrayLike) – Unscaled specific relative humidity, [\(kg \ kg^{-1}\)]. Typically, this is interpolated meteorology data.

  • air_temperature (ArrayLike) – Air temperature, [\(K\)]. Typically, this is interpolated meteorology data.

  • air_pressure (ArrayLike) – Pressure, [\(Pa\)]

  • kwargs (ArrayLike) – Other keyword-only variables and model parameters used by the formula.

Returns:

  • specific_humidity (ArrayLike) – Scaled specific humidity.

  • rhi (ArrayLike) – Scaled relative humidity over ice.

See also

eval()

scaler_specific_keys = ()

Variables required in addition to specific_humidity, air_temperature, and air_pressure These are either ModelParams specific to scaling, or variables that should be extracted from eval() parameter source.

set_source(source=None)

Attach original or copy of input source to source.

Parameters:

source (MetDataset | GeoVectorDataset | Flight | Iterable[Flight] | None) – Parameter source passed in eval(). If None, an empty MetDataset with coordinates like met is set to source.

See also

eval

set_source_met(optional=False, variable=None)

Ensure or interpolate each required met_variables on source .

For each variable in met_variables, check source for data variable with the same name.

For GeoVectorDataset sources, try to interpolate met if variable does not exist.

For MetDataset sources, try to get data from met if variable does not exist.

Parameters:
Raises:
source

Data evaluated in model

property to_json

Get description for instance.

transfer_met_source_attrs(source=None)

Transfer met source metadata from met to source.

update_params(params=None, **params_kwargs)

Update model parameters on params.

Parameters:

  • params (dict[str, Any], optional) – Model parameters to update, as dictionary. Defaults to {}

  • **params_kwargs (Any) – Override keys in params with keyword arguments.

class pycontrails.models.humidity_scaling.HistogramMatchingParams(copy_source=True, interpolation_method='linear', interpolation_bounds_error=False, interpolation_fill_value=nan, interpolation_localize=False, interpolation_use_indices=False, interpolation_q_method=None, verify_met=True, downselect_met=True, met_longitude_buffer=(0.0, 0.0), met_latitude_buffer=(0.0, 0.0), met_level_buffer=(0.0, 0.0), met_time_buffer=(np.timedelta64(0, 'h'), np.timedelta64(0, 'h')), product_type='reanalysis', level_type='pressure', member=None)

Bases: ModelParams

Parameters for HistogramMatching.

as_dict()

Convert object to dictionary.

We use this method instead of dataclasses.asdict to use a shallow/unrecursive copy. This will return values as Any instead of dict.

Returns:

dict[str, Any] – Dictionary version of self.

copy_source = True

Copy input source data on eval

downselect_met = True

Downselect input MetDataset` to region around source.

interpolation_bounds_error = False

If True, points lying outside interpolation will raise an error

interpolation_fill_value = nan

Used for outside interpolation value if interpolation_bounds_error is False

interpolation_localize = False

Experimental. See pycontrails.core.interpolation.

interpolation_method = 'linear'

Interpolation method. Supported methods include “linear”, “nearest”, “slinear”, “cubic”, and “quintic”. See scipy.interpolate.RegularGridInterpolator for the description of each method. Not all methods are supported by all met grids. For example, the “cubic” method requires at least 4 points per dimension.

interpolation_q_method = None

Experimental. Alternative interpolation method to account for specific humidity lapse rate bias. Must be one of None, "cubic-spline", or "log-q-log-p". If None, no special interpolation is used for specific humidity. The "cubic-spline" method applies a custom stretching of the met interpolation table to account for the specific humidity lapse rate bias. The "log-q-log-p" method interpolates in the log of specific humidity and pressure, then converts back to specific humidity. Only used by models calling to interpolate_met().

interpolation_use_indices = False

Experimental. See pycontrails.core.interpolation.

level_type = 'pressure'

The ERA5 vertical level type. Must be one of "pressure" or "model".

member = None

The ERA5 ensemble member to use. Must be in the range [0, 10). Only used if product_type is "ensemble_members".

met_latitude_buffer = (0.0, 0.0)

Met latitude buffer for input to Flight.downselect_met(), in WGS84 coordinates. Only applies when downselect_met is True.

met_level_buffer = (0.0, 0.0)

Met level buffer for input to Flight.downselect_met(), in [\(hPa\)]. Only applies when downselect_met is True.

met_longitude_buffer = (0.0, 0.0)

Met longitude buffer for input to Flight.downselect_met(), in WGS84 coordinates. Only applies when downselect_met is True.

met_time_buffer = (np.timedelta64(0,'h'), np.timedelta64(0,'h'))

Met time buffer for input to Flight.downselect_met() Only applies when downselect_met is True.

product_type = 'reanalysis'

The ERA5 product. Must be one of "reanalysis" or "ensemble_members".

verify_met = True

Call _verify_met() on model instantiation.

class pycontrails.models.humidity_scaling.HistogramMatchingWithEckel(met=None, params=None, **params_kwargs)

Bases: HumidityScaling

Scale humidity by histogram matching to IAGOS RHi quantiles.

This method also applies the Eckel scaling to the recalibrated RHi values.

Unlike other specific humidity scaling methods, this method requires met data and performs interpolation at evaluation time.

Warning

Experimental. This may change or be removed in a future release.

References

[Eckel and Walters, 1998]

default_params

alias of HistogramMatchingWithEckelParams

property description

Get description for instance.

downselect_met()

Downselect met domain to the max/min bounds of source.

Override this method if special handling is needed in met down-selection.

  • source must be defined before calling downselect_met().

  • This method copies and re-assigns met using met.copy() to avoid side-effects.

Raises:
classmethod ecmwf_met_variables()

Return an ECMWF-specific list of required meteorology variables.

Returns:

tuple[MetVariable] – List of ECMWF-specific variants of required variables

eval(source=None, **params)

Scale specific humidity by histogram matching to IAGOS RHi quantiles.

This method assumes source is equipped with the following variables:

  • air_temperature

  • specific_humidity: Humidity values for the params["member"] ERA5 ensemble member.

formula = 'era5_quantiles -> iagos_quantiles -> recalibrated_rhi'
classmethod generic_met_variables()

Return a model-agnostic list of required meteorology variables.

Returns:

tuple[MetVariable] – List of model-agnostic variants of required variables

get_data_param(other, key, default=<object object>, *, set_attr=True)

Get data from other source-compatible object with default set by model parameter key.

Retrieves data with the following hierarchy:

  1. other.data[key]. Returns np.ndarray | xr.DataArray.

  2. other.attrs[key]

  3. params[key]

  4. default

In case 3., the value of params[key] is attached to other.attrs[key] unless set_attr is set to False.

Parameters:

  • key (str) – Key to retrieve

  • default (Any, optional) – Default value if key is not found.

  • set_attr (bool, optional) – If True (default), set source.attrs[key] to params[key] if found. This allows for better post model evaluation tracking.

Returns:

Any – Value(s) found for key in other data, other attrs, or model params

Raises:

KeyError – Raises KeyError if key is not found in any location and default is not provided.

See also

get_source_param, pycontrails.core.vector.GeoVectorDataset.get_data_or_attr

get_source_param(key, default=<object object>, *, set_attr=True)

Get source data with default set by parameter key.

Retrieves data with the following hierarchy:

  1. source.data[key]. Returns np.ndarray | xr.DataArray.

  2. source.attrs[key]

  3. params[key]

  4. default

In case 3., the value of params[key] is attached to source.attrs[key] unless set_attr is set to False.

Parameters:

  • key (str) – Key to retrieve

  • default (Any, optional) – Default value if key is not found.

  • set_attr (bool, optional) – If True (default), set source.attrs[key] to params[key] if found. This allows for better post model evaluation tracking.

Returns:

Any – Value(s) found for key in source data, source attrs, or model params

Raises:

KeyError – Raises KeyError if key is not found in any location and default is not provided.

See also

get_data_param, pycontrails.core.vector.GeoVectorDataset.get_data_or_attr

classmethod gfs_met_variables()

Return a GFS-specific list of required meteorology variables.

Returns:

tuple[MetVariable] – List of GFS-specific variants of required variables

property hash

Generate a unique hash for model instance.

Returns:

str – Unique hash for model instance (sha1)

property interp_kwargs

Shortcut to create interpolation arguments from params.

The output of this is useful for passing to interpolate_met().

Returns:

dict[str, Any] – Dictionary with keys

  • ”method”

  • ”bounds_error”

  • ”fill_value”

  • ”localize”

  • ”use_indices”

  • ”q_method”

as determined by params.

long_name = 'IAGOS RHi histogram matching with Eckel scaling'
met

Meteorology data

met_required = False

Require meteorology is not None on __init__()

met_variables

Required meteorology pressure level variables. Each element in the list is a MetVariable or a tuple[MetVariable]. If element is a tuple[MetVariable], the variable depends on the data source and the tuple must include entries for a model-agnostic variable, an ECMWF-specific variable, and a GFS-specific variable. Only one of the three variable in the tuple is required for model evaluation.

n_members = 10
name = 'histogram_matching_with_eckel'
optional_met_variables

Optional meteorology variables

params

Instantiated model parameters, in dictionary form

processed_met_variables

Set of required parameters if processing already complete on met input.

require_met()

Ensure that met is a MetDataset.

Returns:

MetDataset – Returns reference to met. This is helpful for type narrowing met when meteorology is required.

Raises:

ValueError – Raises when met is None.

require_source_type(type_)

Ensure that source is type_.

Returns:

_Source – Returns reference to source. This is helpful for type narrowing source to specific type(s).

Raises:

ValueError – Raises when source is not _type_.

scale(specific_humidity, air_temperature, air_pressure, **kwargs)

Scale specific humidity values via histogram matching and Eckel scaling.

Unlike the method on the base class, the method assumes each of the input arrays are np.ndarray and not xr.DataArray objects.

Parameters:

  • specific_humidity (npt.NDArray[np.floating]) – A 2D array of specific humidity values for all ERA5 ensemble members. The shape of this array must be (n, 10), where n is the number of observations and 10 is the number of ERA5 ensemble members.

  • air_temperature (npt.NDArray[np.floating]) – A 1D array of air temperature values with shape (n,).

  • air_pressure (npt.NDArray[np.floating]) – A 1D array of air pressure values with shape (n,).

  • kwargs (Any) – Unused, kept for compatibility with the base class.

Returns:

  • specific_humidity (npt.NDArray[np.floating]) – The recalibrated specific humidity values. A 1D array with shape (n,).

  • rhi (npt.NDArray[np.floating]) – The recalibrated RHi values. A 1D array with shape (n,).

scaler_specific_keys = ()

Variables required in addition to specific_humidity, air_temperature, and air_pressure These are either ModelParams specific to scaling, or variables that should be extracted from eval() parameter source.

set_source(source=None)

Attach original or copy of input source to source.

Parameters:

source (MetDataset | GeoVectorDataset | Flight | Iterable[Flight] | None) – Parameter source passed in eval(). If None, an empty MetDataset with coordinates like met is set to source.

See also

eval

set_source_met(optional=False, variable=None)

Ensure or interpolate each required met_variables on source .

For each variable in met_variables, check source for data variable with the same name.

For GeoVectorDataset sources, try to interpolate met if variable does not exist.

For MetDataset sources, try to get data from met if variable does not exist.

Parameters:
Raises:
source

Data evaluated in model

property to_json

Get description for instance.

transfer_met_source_attrs(source=None)

Transfer met source metadata from met to source.

update_params(params=None, **params_kwargs)

Update model parameters on params.

Parameters:

  • params (dict[str, Any], optional) – Model parameters to update, as dictionary. Defaults to {}

  • **params_kwargs (Any) – Override keys in params with keyword arguments.

class pycontrails.models.humidity_scaling.HistogramMatchingWithEckelParams(copy_source=True, interpolation_method='linear', interpolation_bounds_error=False, interpolation_fill_value=nan, interpolation_localize=False, interpolation_use_indices=False, interpolation_q_method=None, verify_met=True, downselect_met=True, met_longitude_buffer=(0.0, 0.0), met_latitude_buffer=(0.0, 0.0), met_level_buffer=(0.0, 0.0), met_time_buffer=(np.timedelta64(0, 'h'), np.timedelta64(0, 'h')), ensemble_specific_humidity=None, member=None, log_applied=False)

Bases: ModelParams

Parameters for HistogramMatchingWithEckel.

Warning

Experimental. This may change or be removed in a future release.

as_dict()

Convert object to dictionary.

We use this method instead of dataclasses.asdict to use a shallow/unrecursive copy. This will return values as Any instead of dict.

Returns:

dict[str, Any] – Dictionary version of self.

copy_source = True

Copy input source data on eval

downselect_met = True

Downselect input MetDataset` to region around source.

ensemble_specific_humidity = None

A length-10 list of ERA5 ensemble members. Each element is a MetDataArray holding specific humidity values for a single ensemble member. If None, a ValueError will be raised at model instantiation time. The order of the list must be consistent with the order of the ERA5 ensemble members.

interpolation_bounds_error = False

If True, points lying outside interpolation will raise an error

interpolation_fill_value = nan

Used for outside interpolation value if interpolation_bounds_error is False

interpolation_localize = False

Experimental. See pycontrails.core.interpolation.

interpolation_method = 'linear'

Interpolation method. Supported methods include “linear”, “nearest”, “slinear”, “cubic”, and “quintic”. See scipy.interpolate.RegularGridInterpolator for the description of each method. Not all methods are supported by all met grids. For example, the “cubic” method requires at least 4 points per dimension.

interpolation_q_method = None

Experimental. Alternative interpolation method to account for specific humidity lapse rate bias. Must be one of None, "cubic-spline", or "log-q-log-p". If None, no special interpolation is used for specific humidity. The "cubic-spline" method applies a custom stretching of the met interpolation table to account for the specific humidity lapse rate bias. The "log-q-log-p" method interpolates in the log of specific humidity and pressure, then converts back to specific humidity. Only used by models calling to interpolate_met().

interpolation_use_indices = False

Experimental. See pycontrails.core.interpolation.

log_applied = False

If a log transform has already been applied to each member of ensemble_specific_humidity, set this to True.

member = None

The specific member used. Must be in the range [0, 10). If None, a ValueError will be raised at model instantiation time.

met_latitude_buffer = (0.0, 0.0)

Met latitude buffer for input to Flight.downselect_met(), in WGS84 coordinates. Only applies when downselect_met is True.

met_level_buffer = (0.0, 0.0)

Met level buffer for input to Flight.downselect_met(), in [\(hPa\)]. Only applies when downselect_met is True.

met_longitude_buffer = (0.0, 0.0)

Met longitude buffer for input to Flight.downselect_met(), in WGS84 coordinates. Only applies when downselect_met is True.

met_time_buffer = (np.timedelta64(0,'h'), np.timedelta64(0,'h'))

Met time buffer for input to Flight.downselect_met() Only applies when downselect_met is True.

verify_met = True

Call _verify_met() on model instantiation.

class pycontrails.models.humidity_scaling.HumidityScaling(met=None, params=None, **params_kwargs)

Bases: Model

Support for standardizing humidity scaling methodologies.

The method scale() or eval() should be called immediately after interpolation over meteorology data.

Added in version 0.27.0.

default_params

alias of ModelParams

property description

Get description for instance.

downselect_met()

Downselect met domain to the max/min bounds of source.

Override this method if special handling is needed in met down-selection.

  • source must be defined before calling downselect_met().

  • This method copies and re-assigns met using met.copy() to avoid side-effects.

Raises:
classmethod ecmwf_met_variables()

Return an ECMWF-specific list of required meteorology variables.

Returns:

tuple[MetVariable] – List of ECMWF-specific variants of required variables

eval(source=None, **params)

Scale specific humidity values on source.

This method mutates the parameter source by modifying its “specific_humidity” variable and by attaching an “rhi” variable. Set model parameter copy_source=True to avoid mutating source.

Parameters:

  • source (GeoVectorDataset | MetDataset) – Data with variables “specific_humidity”, “air_temperature”, and any variables defined by scaler_specific_keys.

  • **params (Any) – Overwrite model parameters before eval

Returns:

GeoVectorDataset | MetDataset – Source data with updated “specific_humidity” and “rhi”. If source is GeoVectorDataset, “air_pressure” data is also attached.

See also

scale()

abstract property formula

Serializable formula for humidity scaler.

classmethod generic_met_variables()

Return a model-agnostic list of required meteorology variables.

Returns:

tuple[MetVariable] – List of model-agnostic variants of required variables

get_data_param(other, key, default=<object object>, *, set_attr=True)

Get data from other source-compatible object with default set by model parameter key.

Retrieves data with the following hierarchy:

  1. other.data[key]. Returns np.ndarray | xr.DataArray.

  2. other.attrs[key]

  3. params[key]

  4. default

In case 3., the value of params[key] is attached to other.attrs[key] unless set_attr is set to False.

Parameters:

  • key (str) – Key to retrieve

  • default (Any, optional) – Default value if key is not found.

  • set_attr (bool, optional) – If True (default), set source.attrs[key] to params[key] if found. This allows for better post model evaluation tracking.

Returns:

Any – Value(s) found for key in other data, other attrs, or model params

Raises:

KeyError – Raises KeyError if key is not found in any location and default is not provided.

See also

get_source_param, pycontrails.core.vector.GeoVectorDataset.get_data_or_attr

get_source_param(key, default=<object object>, *, set_attr=True)

Get source data with default set by parameter key.

Retrieves data with the following hierarchy:

  1. source.data[key]. Returns np.ndarray | xr.DataArray.

  2. source.attrs[key]

  3. params[key]

  4. default

In case 3., the value of params[key] is attached to source.attrs[key] unless set_attr is set to False.

Parameters:

  • key (str) – Key to retrieve

  • default (Any, optional) – Default value if key is not found.

  • set_attr (bool, optional) – If True (default), set source.attrs[key] to params[key] if found. This allows for better post model evaluation tracking.

Returns:

Any – Value(s) found for key in source data, source attrs, or model params

Raises:

KeyError – Raises KeyError if key is not found in any location and default is not provided.

See also

get_data_param, pycontrails.core.vector.GeoVectorDataset.get_data_or_attr

classmethod gfs_met_variables()

Return a GFS-specific list of required meteorology variables.

Returns:

tuple[MetVariable] – List of GFS-specific variants of required variables

property hash

Generate a unique hash for model instance.

Returns:

str – Unique hash for model instance (sha1)

property interp_kwargs

Shortcut to create interpolation arguments from params.

The output of this is useful for passing to interpolate_met().

Returns:

dict[str, Any] – Dictionary with keys

  • ”method”

  • ”bounds_error”

  • ”fill_value”

  • ”localize”

  • ”use_indices”

  • ”q_method”

as determined by params.

abstract property long_name

Get long name descriptor, annotated on xr.DataArray outputs.

met

Meteorology data

met_required = False

Require meteorology is not None on __init__()

met_variables

Required meteorology pressure level variables. Each element in the list is a MetVariable or a tuple[MetVariable]. If element is a tuple[MetVariable], the variable depends on the data source and the tuple must include entries for a model-agnostic variable, an ECMWF-specific variable, and a GFS-specific variable. Only one of the three variable in the tuple is required for model evaluation.

abstract property name

class`Flight`.

Type:

Get model name for use as a data key in xr.DataArray or

optional_met_variables

Optional meteorology variables

params

Instantiated model parameters, in dictionary form

processed_met_variables

Set of required parameters if processing already complete on met input.

require_met()

Ensure that met is a MetDataset.

Returns:

MetDataset – Returns reference to met. This is helpful for type narrowing met when meteorology is required.

Raises:

ValueError – Raises when met is None.

require_source_type(type_)

Ensure that source is type_.

Returns:

_Source – Returns reference to source. This is helpful for type narrowing source to specific type(s).

Raises:

ValueError – Raises when source is not _type_.

abstract scale(specific_humidity, air_temperature, air_pressure, **kwargs)

Compute scaled specific humidity and RHi.

See docstring for the implementing subclass for specific methodology.

Parameters:

  • specific_humidity (ArrayLike) – Unscaled specific relative humidity, [\(kg \ kg^{-1}\)]. Typically, this is interpolated meteorology data.

  • air_temperature (ArrayLike) – Air temperature, [\(K\)]. Typically, this is interpolated meteorology data.

  • air_pressure (ArrayLike) – Pressure, [\(Pa\)]

  • kwargs (ArrayLike) – Other keyword-only variables and model parameters used by the formula.

Returns:

  • specific_humidity (ArrayLike) – Scaled specific humidity.

  • rhi (ArrayLike) – Scaled relative humidity over ice.

See also

eval()

scaler_specific_keys = ()

Variables required in addition to specific_humidity, air_temperature, and air_pressure These are either ModelParams specific to scaling, or variables that should be extracted from eval() parameter source.

set_source(source=None)

Attach original or copy of input source to source.

Parameters:

source (MetDataset | GeoVectorDataset | Flight | Iterable[Flight] | None) – Parameter source passed in eval(). If None, an empty MetDataset with coordinates like met is set to source.

See also

eval

set_source_met(optional=False, variable=None)

Ensure or interpolate each required met_variables on source .

For each variable in met_variables, check source for data variable with the same name.

For GeoVectorDataset sources, try to interpolate met if variable does not exist.

For MetDataset sources, try to get data from met if variable does not exist.

Parameters:
Raises:
source

Data evaluated in model

property to_json

Get description for instance.

transfer_met_source_attrs(source=None)

Transfer met source metadata from met to source.

update_params(params=None, **params_kwargs)

Update model parameters on params.

Parameters:

  • params (dict[str, Any], optional) – Model parameters to update, as dictionary. Defaults to {}

  • **params_kwargs (Any) – Override keys in params with keyword arguments.

class pycontrails.models.humidity_scaling.HumidityScalingByLevel(met=None, params=None, **params_kwargs)

Bases: HumidityScaling

Apply custom scaling to specific_humidity by pressure level.

This implements the original humidity scaling scheme suggested in [Schumann, 2012]. In particular, see eq. (9) and the surrounding text, quoted below.

Hence, the critical value RHic is usually taken different and below 100% in NWP models. In the ECMWF model, this value is..

RHic = 0.8, (9)

in the mid-troposphere, 1.0 in the stratosphere and follows a smooth transition with pressure altitude between these two values in the upper 20 % of the troposphere. For simplicity of further analysis, we divide the input value of q by RHic initially.

See ConstantHumidityScaling for the simple method described above.

The diagram below shows the piecewise-linear rhi_adj factor by level. In particular, rhi_adj is constant at the stratosphere and above, linearly changes from the mid-troposphere to the stratosphere, and is constant at the mid-troposphere and below.

             _________  stratosphere rhi_adj = 1.0
            /
           /
          /
_________/  mid-troposphere rhi_adj = 0.8

References

default_params

alias of HumidityScalingByLevelParams

property description

Get description for instance.

downselect_met()

Downselect met domain to the max/min bounds of source.

Override this method if special handling is needed in met down-selection.

  • source must be defined before calling downselect_met().

  • This method copies and re-assigns met using met.copy() to avoid side-effects.

Raises:
classmethod ecmwf_met_variables()

Return an ECMWF-specific list of required meteorology variables.

Returns:

tuple[MetVariable] – List of ECMWF-specific variants of required variables

eval(source=None, **params)

Scale specific humidity values on source.

This method mutates the parameter source by modifying its “specific_humidity” variable and by attaching an “rhi” variable. Set model parameter copy_source=True to avoid mutating source.

Parameters:

  • source (GeoVectorDataset | MetDataset) – Data with variables “specific_humidity”, “air_temperature”, and any variables defined by scaler_specific_keys.

  • **params (Any) – Overwrite model parameters before eval

Returns:

GeoVectorDataset | MetDataset – Source data with updated “specific_humidity” and “rhi”. If source is GeoVectorDataset, “air_pressure” data is also attached.

See also

scale()

formula = 'rhi -> rhi / rhi_adj'
classmethod generic_met_variables()

Return a model-agnostic list of required meteorology variables.

Returns:

tuple[MetVariable] – List of model-agnostic variants of required variables

get_data_param(other, key, default=<object object>, *, set_attr=True)

Get data from other source-compatible object with default set by model parameter key.

Retrieves data with the following hierarchy:

  1. other.data[key]. Returns np.ndarray | xr.DataArray.

  2. other.attrs[key]

  3. params[key]

  4. default

In case 3., the value of params[key] is attached to other.attrs[key] unless set_attr is set to False.

Parameters:

  • key (str) – Key to retrieve

  • default (Any, optional) – Default value if key is not found.

  • set_attr (bool, optional) – If True (default), set source.attrs[key] to params[key] if found. This allows for better post model evaluation tracking.

Returns:

Any – Value(s) found for key in other data, other attrs, or model params

Raises:

KeyError – Raises KeyError if key is not found in any location and default is not provided.

See also

get_source_param, pycontrails.core.vector.GeoVectorDataset.get_data_or_attr

get_source_param(key, default=<object object>, *, set_attr=True)

Get source data with default set by parameter key.

Retrieves data with the following hierarchy:

  1. source.data[key]. Returns np.ndarray | xr.DataArray.

  2. source.attrs[key]

  3. params[key]

  4. default

In case 3., the value of params[key] is attached to source.attrs[key] unless set_attr is set to False.

Parameters:

  • key (str) – Key to retrieve

  • default (Any, optional) – Default value if key is not found.

  • set_attr (bool, optional) – If True (default), set source.attrs[key] to params[key] if found. This allows for better post model evaluation tracking.

Returns:

Any – Value(s) found for key in source data, source attrs, or model params

Raises:

KeyError – Raises KeyError if key is not found in any location and default is not provided.

See also

get_data_param, pycontrails.core.vector.GeoVectorDataset.get_data_or_attr

classmethod gfs_met_variables()

Return a GFS-specific list of required meteorology variables.

Returns:

tuple[MetVariable] – List of GFS-specific variants of required variables

property hash

Generate a unique hash for model instance.

Returns:

str – Unique hash for model instance (sha1)

property interp_kwargs

Shortcut to create interpolation arguments from params.

The output of this is useful for passing to interpolate_met().

Returns:

dict[str, Any] – Dictionary with keys

  • ”method”

  • ”bounds_error”

  • ”fill_value”

  • ”localize”

  • ”use_indices”

  • ”q_method”

as determined by params.

long_name = 'Constant humidity scaling by level'
met

Meteorology data

met_required = False

Require meteorology is not None on __init__()

met_variables

Required meteorology pressure level variables. Each element in the list is a MetVariable or a tuple[MetVariable]. If element is a tuple[MetVariable], the variable depends on the data source and the tuple must include entries for a model-agnostic variable, an ECMWF-specific variable, and a GFS-specific variable. Only one of the three variable in the tuple is required for model evaluation.

name = 'constant_scale_by_level'
optional_met_variables

Optional meteorology variables

params

Instantiated model parameters, in dictionary form

processed_met_variables

Set of required parameters if processing already complete on met input.

require_met()

Ensure that met is a MetDataset.

Returns:

MetDataset – Returns reference to met. This is helpful for type narrowing met when meteorology is required.

Raises:

ValueError – Raises when met is None.

require_source_type(type_)

Ensure that source is type_.

Returns:

_Source – Returns reference to source. This is helpful for type narrowing source to specific type(s).

Raises:

ValueError – Raises when source is not _type_.

scale(specific_humidity, air_temperature, air_pressure, **kwargs)

Compute scaled specific humidity and RHi.

See docstring for the implementing subclass for specific methodology.

Parameters:

  • specific_humidity (ArrayLike) – Unscaled specific relative humidity, [\(kg \ kg^{-1}\)]. Typically, this is interpolated meteorology data.

  • air_temperature (ArrayLike) – Air temperature, [\(K\)]. Typically, this is interpolated meteorology data.

  • air_pressure (ArrayLike) – Pressure, [\(Pa\)]

  • kwargs (ArrayLike) – Other keyword-only variables and model parameters used by the formula.

Returns:

  • specific_humidity (ArrayLike) – Scaled specific humidity.

  • rhi (ArrayLike) – Scaled relative humidity over ice.

See also

eval()

scaler_specific_keys = ('rhi_adj_mid_troposphere', 'rhi_adj_stratosphere', 'mid_troposphere_threshold', 'stratosphere_threshold')

Variables required in addition to specific_humidity, air_temperature, and air_pressure These are either ModelParams specific to scaling, or variables that should be extracted from eval() parameter source.

set_source(source=None)

Attach original or copy of input source to source.

Parameters:

source (MetDataset | GeoVectorDataset | Flight | Iterable[Flight] | None) – Parameter source passed in eval(). If None, an empty MetDataset with coordinates like met is set to source.

See also

eval

set_source_met(optional=False, variable=None)

Ensure or interpolate each required met_variables on source .

For each variable in met_variables, check source for data variable with the same name.

For GeoVectorDataset sources, try to interpolate met if variable does not exist.

For MetDataset sources, try to get data from met if variable does not exist.

Parameters:
Raises:
source

Data evaluated in model

property to_json

Get description for instance.

transfer_met_source_attrs(source=None)

Transfer met source metadata from met to source.

update_params(params=None, **params_kwargs)

Update model parameters on params.

Parameters:

  • params (dict[str, Any], optional) – Model parameters to update, as dictionary. Defaults to {}

  • **params_kwargs (Any) – Override keys in params with keyword arguments.

class pycontrails.models.humidity_scaling.HumidityScalingByLevelParams(copy_source=True, interpolation_method='linear', interpolation_bounds_error=False, interpolation_fill_value=nan, interpolation_localize=False, interpolation_use_indices=False, interpolation_q_method=None, verify_met=True, downselect_met=True, met_longitude_buffer=(0.0, 0.0), met_latitude_buffer=(0.0, 0.0), met_level_buffer=(0.0, 0.0), met_time_buffer=(np.timedelta64(0, 'h'), np.timedelta64(0, 'h')), rhi_adj_mid_troposphere=0.8, rhi_adj_stratosphere=1.0, mid_troposphere_threshold=0.8, stratosphere_threshold=1.0)

Bases: ModelParams

Parameters for HumidityScalingByLevel.

as_dict()

Convert object to dictionary.

We use this method instead of dataclasses.asdict to use a shallow/unrecursive copy. This will return values as Any instead of dict.

Returns:

dict[str, Any] – Dictionary version of self.

copy_source = True

Copy input source data on eval

downselect_met = True

Downselect input MetDataset` to region around source.

interpolation_bounds_error = False

If True, points lying outside interpolation will raise an error

interpolation_fill_value = nan

Used for outside interpolation value if interpolation_bounds_error is False

interpolation_localize = False

Experimental. See pycontrails.core.interpolation.

interpolation_method = 'linear'

Interpolation method. Supported methods include “linear”, “nearest”, “slinear”, “cubic”, and “quintic”. See scipy.interpolate.RegularGridInterpolator for the description of each method. Not all methods are supported by all met grids. For example, the “cubic” method requires at least 4 points per dimension.

interpolation_q_method = None

Experimental. Alternative interpolation method to account for specific humidity lapse rate bias. Must be one of None, "cubic-spline", or "log-q-log-p". If None, no special interpolation is used for specific humidity. The "cubic-spline" method applies a custom stretching of the met interpolation table to account for the specific humidity lapse rate bias. The "log-q-log-p" method interpolates in the log of specific humidity and pressure, then converts back to specific humidity. Only used by models calling to interpolate_met().

interpolation_use_indices = False

Experimental. See pycontrails.core.interpolation.

met_latitude_buffer = (0.0, 0.0)

Met latitude buffer for input to Flight.downselect_met(), in WGS84 coordinates. Only applies when downselect_met is True.

met_level_buffer = (0.0, 0.0)

Met level buffer for input to Flight.downselect_met(), in [\(hPa\)]. Only applies when downselect_met is True.

met_longitude_buffer = (0.0, 0.0)

Met longitude buffer for input to Flight.downselect_met(), in WGS84 coordinates. Only applies when downselect_met is True.

met_time_buffer = (np.timedelta64(0,'h'), np.timedelta64(0,'h'))

Met time buffer for input to Flight.downselect_met() Only applies when downselect_met is True.

mid_troposphere_threshold = 0.8

Adjustment factor for mid-troposphere humidity scaling. Default value of 0.8 taken from [Schumann, 2012].

rhi_adj_mid_troposphere = 0.8

Fraction of troposphere for mid-troposphere humidity scaling. Default value suggested in [Schumann, 2012].

rhi_adj_stratosphere = 1.0

Fraction of troposphere for stratosphere humidity scaling. Default value suggested in [Schumann, 2012].

stratosphere_threshold = 1.0

Adjustment factor for stratosphere humidity scaling. Default value of 1.0 taken from [Schumann, 2012].

verify_met = True

Call _verify_met() on model instantiation.

pycontrails.models.humidity_scaling.eckel_scaling(ensemble_mean_rhi, ensemble_member_rhi, q_method)

Apply Eckel scaling to the given RHi values.

Parameters:

  • ensemble_mean_rhi (npt.NDArray[np.floating]) – The ensemble mean RHi values. This should be a 1D array with the same shape as ensemble_member_rhi.

  • ensemble_member_rhi (npt.NDArray[np.floating]) – The RHi values for a single ensemble member.

  • q_method ({None, "cubic-spline", "log-q-log-p"}) – The interpolation method.

Returns:

npt.NDArray[np.floating] – The scaled RHi values. Values are manually clipped at 0 to ensure only non-negative values are returned.

References

[Eckel and Walters, 1998]

pycontrails.models.humidity_scaling.histogram_matching(era5_rhi, product_type, level_type, member, q_method)

Map ERA5-derived RHi to its corresponding IAGOS quantile via histogram matching.

This matching is performed on a single ERA5 ensemble member.

Parameters:

  • era5_rhi (ArrayLike) – ERA5-derived RHi values for the given ensemble member.

  • product_type ({"reanalysis", "ensemble_members"}) – The ERA5 product type.

  • level_type ({"pressure", "model"}) – Select whether to perform quantile mapping based on quantiles from pressure- or model-level ERA5 data. Selecting level_type == "model" when product_type == "ensemble_members" will produce a warning and change product_type to "reanalysis".

  • member (int | None) – The ERA5 ensemble member to use. Must be in the range [0, 10). Only used if product_type == "ensemble_members".

  • q_method ({None, "cubic-spline", "log-q-log-p"}) – The interpolation method.

Returns:

npt.NDArray[np.floating] – The IAGOS quantiles corresponding to the ERA5-derived RHi values. Returned as a numpy array with the same shape and dtype as era5_rhi.

pycontrails.models.humidity_scaling.histogram_matching_all_members(era5_rhi_all_members, member, q_method)

Recalibrate ERA5-derived RHi values to IAGOS quantiles by histogram matching.

This recalibration requires values for all ERA5 ensemble members. Currently, the number of ensemble members is hard-coded to 10.

Parameters:

  • era5_rhi_all_members (npt.NDArray[np.floating]) – ERA5-derived RHi values for all ensemble members. This array should have shape (n, 10).

  • member (int) – The ERA5 ensemble member to use. Must be in the range [0, 10).

  • q_method ({None, "cubic-spline", "log-q-log-p"}) – The interpolation method.

Returns:

  • ensemble_mean_rhi (npt.NDArray[np.floating]) – The mean RHi values after histogram matching over all ensemble members. This is an array of shape (n,).

  • ensemble_member_rhi (npt.NDArray[np.floating]) – The RHi values after histogram matching for the given ensemble member. This is an array of shape (n,).