pycontrails.models.humidity_scaling

Humidity scaling methodologies.

class pycontrails.models.humidity_scaling.ConstantHumidityScaling(met=None, params=None, **params_kwargs)

Bases: HumidityScaling

Scale specific humidity by applying a constant uniform scaling.

This scalar simply applies the transformation..

rhi -> rhi / rhi_adj

where rhi_adj is a constant specified by params or overridden by a variable or attribute on source in eval().

The convention to divide by rhi_adj instead of considering the more natural product rhi_adj * rhi is somewhat arbitrary. In short, rhi_adj can be thought of as the critical threshold for supersaturation.

References

default_params

alias of ConstantHumidityScalingParams

formula = 'rhi -> rhi / rhi_adj'
long_name = 'Constant humidity scaling'
met

Meteorology data

name = 'constant_scale'
params

Instantiated model parameters, in dictionary form

scale(specific_humidity, air_temperature, air_pressure, **kwargs)

Compute scaled specific humidity and RHi.

See docstring for the implementing subclass for specific methodology.

Parameters:

  • specific_humidity (ArrayLike) – Unscaled specific relative humidity, [\(kg \ kg^{-1}\)]. Typically, this is interpolated meteorology data.

  • air_temperature (ArrayLike) – Air temperature, [\(K\)]. Typically, this is interpolated meteorology data.

  • air_pressure (ArrayLike) – Pressure, [\(Pa\)]

  • kwargs (ArrayLike) – Other keyword-only variables and model parameters used by the formula.

Returns:

  • specific_humidity (ArrayLike) – Scaled specific humidity.

  • rhi (ArrayLike) – Scaled relative humidity over ice.

See also

eval()

scaler_specific_keys = ('rhi_adj',)

Variables required in addition to specific_humidity, air_temperature, and air_pressure These are either ModelParams specific to scaling, or variables that should be extracted from eval() parameter source.

source

Data evaluated in model

class pycontrails.models.humidity_scaling.ConstantHumidityScalingParams(copy_source=True, interpolation_method='linear', interpolation_bounds_error=False, interpolation_fill_value=nan, interpolation_localize=False, interpolation_use_indices=False, interpolation_q_method=None, verify_met=True, downselect_met=True, met_longitude_buffer=(0.0, 0.0), met_latitude_buffer=(0.0, 0.0), met_level_buffer=(0.0, 0.0), met_time_buffer=(numpy.timedelta64(0, 'h'), numpy.timedelta64(0, 'h')), rhi_adj=0.97)

Bases: ModelParams

Parameters for ConstantHumidityScaling.

rhi_adj = 0.97

Scale specific humidity by dividing it with adjustment factor per [Schumann, 2012] eq. (9). Set to a constant 0.9 in [Schumann, 2012] to account for sub-scale variability of specific humidity. A value of 1.0 provides no scaling.

class pycontrails.models.humidity_scaling.ExponentialBoostHumidityScaling(met=None, params=None, **params_kwargs)

Bases: HumidityScaling

Scale humidity by composing constant scaling with exponential boosting.

This formula composes the transformations

  1. constant scaling: rhi -> rhi / rhi_adj

  2. exponential boosting: rhi -> rhi ^ rhi_boost_exponent if rhi > 1

  3. clipping: rhi -> min(rhi, clip_upper)

where rhi_adj, rhi_boost_exponent, and clip_upper are model params.

References

See also

ExponentialBoostLatitudeCorrectionHumidityScaling

default_params

alias of ExponentialBoostHumidityScalingParams

formula = 'rhi -> (rhi / rhi_adj) ^ rhi_boost_exponent'
long_name = 'Constant humidity scaling composed with exponential boosting'
met

Meteorology data

name = 'exponential_boost'
params

Instantiated model parameters, in dictionary form

scale(specific_humidity, air_temperature, air_pressure, **kwargs)

Compute scaled specific humidity and RHi.

See docstring for the implementing subclass for specific methodology.

Parameters:

  • specific_humidity (ArrayLike) – Unscaled specific relative humidity, [\(kg \ kg^{-1}\)]. Typically, this is interpolated meteorology data.

  • air_temperature (ArrayLike) – Air temperature, [\(K\)]. Typically, this is interpolated meteorology data.

  • air_pressure (ArrayLike) – Pressure, [\(Pa\)]

  • kwargs (ArrayLike) – Other keyword-only variables and model parameters used by the formula.

Returns:

  • specific_humidity (ArrayLike) – Scaled specific humidity.

  • rhi (ArrayLike) – Scaled relative humidity over ice.

See also

eval()

scaler_specific_keys = ('rhi_adj', 'rhi_boost_exponent', 'clip_upper')

Variables required in addition to specific_humidity, air_temperature, and air_pressure These are either ModelParams specific to scaling, or variables that should be extracted from eval() parameter source.

source

Data evaluated in model

class pycontrails.models.humidity_scaling.ExponentialBoostHumidityScalingParams(copy_source=True, interpolation_method='linear', interpolation_bounds_error=False, interpolation_fill_value=nan, interpolation_localize=False, interpolation_use_indices=False, interpolation_q_method=None, verify_met=True, downselect_met=True, met_longitude_buffer=(0.0, 0.0), met_latitude_buffer=(0.0, 0.0), met_level_buffer=(0.0, 0.0), met_time_buffer=(numpy.timedelta64(0, 'h'), numpy.timedelta64(0, 'h')), rhi_adj=0.97, rhi_boost_exponent=1.7, clip_upper=1.7)

Bases: ConstantHumidityScalingParams

Parameters for ExponentialBoostHumidityScaling.

clip_upper = 1.7

Used to clip overinflated unrealistic RHi values.

rhi_boost_exponent = 1.7

Boost RHi values exceeding 1 as described in [Teoh et al., 2022]. In eval(), this can be overridden by a keyword argument with the same name.

class pycontrails.models.humidity_scaling.ExponentialBoostLatitudeCorrectionHumidityScaling(met=None, params=None, **params_kwargs)

Bases: HumidityScaling

Correct RHi values derived from ECMWF ERA5 HRES.

Unlike other RHi correction factors, this function applies a custom latitude-based term and has been tuned for global application.

This formula composes the transformations

  1. constant scaling: rhi -> rhi / rhi_adj

  2. exponential boosting: rhi -> rhi ^ rhi_boost_exponent if rhi > 1

  3. clipping: rhi -> min(rhi, rhi_max)

where rhi_adj and rhi_boost_exponent depend on latitude to minimize error between ERA5 HRES and IAGOS in-situ data.

For each waypoint, rhi_max ensures that the corrected RHi does not exceed the maximum value according to thermodynamics:

  • rhi_max = p_liq(T) / p_ice(T) for T > 235 K, (Pruppacher and Klett, 1997)

  • rhi_max = 1.67 + (1.45 - 1.67) * (T - 190.) / (235. - 190.) for T < 235 K (Karcher and Lohmann, 2002; Tompkins et al., 2007)

The RHi correction addresses the known limitations of the ERA5 HRES humidity fields, ensuring that the ISSR coverage area and RHi-distribution is consistent with in-situ measurements from the IAGOS dataset. Generally, the correction:

  1. reduces the ISSR coverage area near the equator,

  2. increases the ISSR coverage area at higher latitudes, and

  3. accounts for localized regions with very high ice supersaturation (RHi > 120%).

This methodology is an extension of Teoh et al. (2022) and has not yet been peer-reviewed/published.

The ERA5 HRES <> IAGOS fitting uses a sigmoid curve to capture significant changes in tropopause height at 20 - 50 degrees latitude.

The method eval() requires a latitude keyword argument.

References

  • [Teoh et al., 2022]

  • Kärcher, B. and Lohmann, U., 2002. A parameterization of cirrus cloud formation: Homogeneous freezing of supercooled aerosols. Journal of Geophysical Research: Atmospheres, 107(D2), pp.AAC-4.

  • Pruppacher, H.R. and Klett, J.D. (1997) Microphysics of Clouds and Precipitation. 2nd Edition, Kluwer Academic, Dordrecht, 954 p.

  • Tompkins, A.M., Gierens, K. and Rädel, G., 2007. Ice supersaturation in the ECMWF integrated forecast system. Quarterly Journal of the Royal Meteorological Society: A journal of the atmospheric sciences, applied meteorology and physical oceanography, 133(622), pp.53-63.

See also

ExponentialBoostHumidityScaling

default_params

alias of ModelParams

formula = 'rhi -> (rhi / rhi_adj) ^ rhi_boost_exponent'
long_name = 'Latitude specific humidity scaling composed with exponential boosting'
met

Meteorology data

name = 'exponential_boost_latitude_customization'
params

Instantiated model parameters, in dictionary form

scale(specific_humidity, air_temperature, air_pressure, **kwargs)

Compute scaled specific humidity and RHi.

See docstring for the implementing subclass for specific methodology.

Parameters:

  • specific_humidity (ArrayLike) – Unscaled specific relative humidity, [\(kg \ kg^{-1}\)]. Typically, this is interpolated meteorology data.

  • air_temperature (ArrayLike) – Air temperature, [\(K\)]. Typically, this is interpolated meteorology data.

  • air_pressure (ArrayLike) – Pressure, [\(Pa\)]

  • kwargs (ArrayLike) – Other keyword-only variables and model parameters used by the formula.

Returns:

  • specific_humidity (ArrayLike) – Scaled specific humidity.

  • rhi (ArrayLike) – Scaled relative humidity over ice.

See also

eval()

scaler_specific_keys = ('latitude',)

Variables required in addition to specific_humidity, air_temperature, and air_pressure These are either ModelParams specific to scaling, or variables that should be extracted from eval() parameter source.

source

Data evaluated in model

class pycontrails.models.humidity_scaling.HistogramMatching(met=None, params=None, **params_kwargs)

Bases: HumidityScaling

Scale humidity by histogram matching to IAGOS RHi quantiles.

default_params

alias of HistogramMatchingParams

formula = 'era5_quantiles -> iagos_quantiles'
long_name = 'IAGOS RHi histogram matching'
met

Meteorology data

name = 'histogram_matching'
params

Instantiated model parameters, in dictionary form

scale(specific_humidity, air_temperature, air_pressure, **kwargs)

Compute scaled specific humidity and RHi.

See docstring for the implementing subclass for specific methodology.

Parameters:

  • specific_humidity (ArrayLike) – Unscaled specific relative humidity, [\(kg \ kg^{-1}\)]. Typically, this is interpolated meteorology data.

  • air_temperature (ArrayLike) – Air temperature, [\(K\)]. Typically, this is interpolated meteorology data.

  • air_pressure (ArrayLike) – Pressure, [\(Pa\)]

  • kwargs (ArrayLike) – Other keyword-only variables and model parameters used by the formula.

Returns:

  • specific_humidity (ArrayLike) – Scaled specific humidity.

  • rhi (ArrayLike) – Scaled relative humidity over ice.

See also

eval()

source

Data evaluated in model

class pycontrails.models.humidity_scaling.HistogramMatchingParams(copy_source=True, interpolation_method='linear', interpolation_bounds_error=False, interpolation_fill_value=nan, interpolation_localize=False, interpolation_use_indices=False, interpolation_q_method=None, verify_met=True, downselect_met=True, met_longitude_buffer=(0.0, 0.0), met_latitude_buffer=(0.0, 0.0), met_level_buffer=(0.0, 0.0), met_time_buffer=(numpy.timedelta64(0, 'h'), numpy.timedelta64(0, 'h')), product_type='reanalysis', member=None)

Bases: ModelParams

Parameters for HistogramMatching.

member = None

The ERA5 ensemble member to use. Must be in the range [0, 10). Only used if product_type is "ensemble_members".

product_type = 'reanalysis'

The ERA5 product. Must be one of "reanalysis" or "ensemble_members".

class pycontrails.models.humidity_scaling.HistogramMatchingWithEckel(met=None, params=None, **params_kwargs)

Bases: HumidityScaling

Scale humidity by histogram matching to IAGOS RHi quantiles.

This method also applies the Eckel scaling to the recalibrated RHi values.

Unlike other specific humidity scaling methods, this method requires met data and performs interpolation at evaluation time.

Warning

Experimental. This may change or be removed in a future release.

References

[Eckel and Walters, 1998]

default_params

alias of HistogramMatchingWithEckelParams

eval(source=None, **params)

Scale specific humidity by histogram matching to IAGOS RHi quantiles.

This method assumes source is equipped with the following variables:

  • air_temperature

  • specific_humidity: Humidity values for the params["member"] ERA5 ensemble member.

formula = 'era5_quantiles -> iagos_quantiles -> recalibrated_rhi'
long_name = 'IAGOS RHi histogram matching with Eckel scaling'
met

Meteorology data

n_members = 10
name = 'histogram_matching_with_eckel'
params

Instantiated model parameters, in dictionary form

scale(specific_humidity, air_temperature, air_pressure, **kwargs)

Scale specific humidity values via histogram matching and Eckel scaling.

Unlike the method on the base class, the method assumes each of the input arrays are np.ndarray and not xr.DataArray objects.

Parameters:

  • specific_humidity (npt.NDArray[np.float64]) – A 2D array of specific humidity values for all ERA5 ensemble members. The shape of this array must be (n, 10), where n is the number of observations and 10 is the number of ERA5 ensemble members.

  • air_temperature (npt.NDArray[np.float64]) – A 1D array of air temperature values with shape (n,).

  • air_pressure (npt.NDArray[np.float64]) – A 1D array of air pressure values with shape (n,).

  • kwargs (Any) – Unused, kept for compatibility with the base class.

Returns:

  • specific_humidity (npt.NDArray[np.float64]) – The recalibrated specific humidity values. A 1D array with shape (n,).

  • rhi (npt.NDArray[np.float64]) – The recalibrated RHi values. A 1D array with shape (n,).

source

Data evaluated in model

class pycontrails.models.humidity_scaling.HistogramMatchingWithEckelParams(copy_source=True, interpolation_method='linear', interpolation_bounds_error=False, interpolation_fill_value=nan, interpolation_localize=False, interpolation_use_indices=False, interpolation_q_method=None, verify_met=True, downselect_met=True, met_longitude_buffer=(0.0, 0.0), met_latitude_buffer=(0.0, 0.0), met_level_buffer=(0.0, 0.0), met_time_buffer=(numpy.timedelta64(0, 'h'), numpy.timedelta64(0, 'h')), ensemble_specific_humidity=None, member=None, log_applied=False)

Bases: ModelParams

Parameters for HistogramMatchingWithEckel.

Warning

Experimental. This may change or be removed in a future release.

ensemble_specific_humidity = None

A length-10 list of ERA5 ensemble members. Each element is a MetDataArray holding specific humidity values for a single ensemble member. If None, a ValueError will be raised at model instantiation time. The order of the list must be consistent with the order of the ERA5 ensemble members.

log_applied = False

If a log transform has already been applied to each member of ensemble_specific_humidity, set this to True.

member = None

The specific member used. Must be in the range [0, 10). If None, a ValueError will be raised at model instantiation time.

class pycontrails.models.humidity_scaling.HumidityScaling(met=None, params=None, **params_kwargs)

Bases: Model

Support for standardizing humidity scaling methodologies.

The method scale() or eval() should be called immediately after interpolation over meteorology data.

Added in version 0.27.0.

property description

Get description for instance.

eval(source=None, **params)

Scale specific humidity values on source.

This method mutates the parameter source by modifying its “specific_humidity” variable and by attaching an “rhi” variable. Set model parameter copy_source=True to avoid mutating source.

Parameters:

  • source (GeoVectorDataset | MetDataset) – Data with variables “specific_humidity”, “air_temperature”, and any variables defined by scaler_specific_keys.

  • **params (Any) – Overwrite model parameters before eval

Returns:

GeoVectorDataset | MetDataset – Source data with updated “specific_humidity” and “rhi”. If source is GeoVectorDataset, “air_pressure” data is also attached.

See also

scale()

abstract property formula

Serializable formula for humidity scaler.

met

Meteorology data

params

Instantiated model parameters, in dictionary form

abstract scale(specific_humidity, air_temperature, air_pressure, **kwargs)

Compute scaled specific humidity and RHi.

See docstring for the implementing subclass for specific methodology.

Parameters:

  • specific_humidity (ArrayLike) – Unscaled specific relative humidity, [\(kg \ kg^{-1}\)]. Typically, this is interpolated meteorology data.

  • air_temperature (ArrayLike) – Air temperature, [\(K\)]. Typically, this is interpolated meteorology data.

  • air_pressure (ArrayLike) – Pressure, [\(Pa\)]

  • kwargs (ArrayLike) – Other keyword-only variables and model parameters used by the formula.

Returns:

  • specific_humidity (ArrayLike) – Scaled specific humidity.

  • rhi (ArrayLike) – Scaled relative humidity over ice.

See also

eval()

scaler_specific_keys = ()

Variables required in addition to specific_humidity, air_temperature, and air_pressure These are either ModelParams specific to scaling, or variables that should be extracted from eval() parameter source.

source

Data evaluated in model

property to_json

Get description for instance.

class pycontrails.models.humidity_scaling.HumidityScalingByLevel(met=None, params=None, **params_kwargs)

Bases: HumidityScaling

Apply custom scaling to specific_humidity by pressure level.

This implements the original humidity scaling scheme suggested in [Schumann, 2012]. In particular, see eq. (9) and the surrounding text, quoted below.

Hence, the critical value RHic is usually taken different and below 100% in NWP models. In the ECMWF model, this value is..

RHic = 0.8, (9)

in the mid-troposphere, 1.0 in the stratosphere and follows a smooth transition with pressure altitude between these two values in the upper 20 % of the troposphere. For simplicity of further analysis, we divide the input value of q by RHic initially.

See ConstantHumidityScaling for the simple method described above.

The diagram below shows the piecewise-linear rhi_adj factor by level. In particular, rhi_adj is constant at the stratosphere and above, linearly changes from the mid-troposphere to the stratosphere, and is constant at the mid-troposphere and below.

             _________  stratosphere rhi_adj = 1.0
            /
           /
          /
_________/  mid-troposphere rhi_adj = 0.8

References

default_params

alias of HumidityScalingByLevelParams

formula = 'rhi -> rhi / rhi_adj'
long_name = 'Constant humidity scaling by level'
met

Meteorology data

name = 'constant_scale_by_level'
params

Instantiated model parameters, in dictionary form

scale(specific_humidity, air_temperature, air_pressure, **kwargs)

Compute scaled specific humidity and RHi.

See docstring for the implementing subclass for specific methodology.

Parameters:

  • specific_humidity (ArrayLike) – Unscaled specific relative humidity, [\(kg \ kg^{-1}\)]. Typically, this is interpolated meteorology data.

  • air_temperature (ArrayLike) – Air temperature, [\(K\)]. Typically, this is interpolated meteorology data.

  • air_pressure (ArrayLike) – Pressure, [\(Pa\)]

  • kwargs (ArrayLike) – Other keyword-only variables and model parameters used by the formula.

Returns:

  • specific_humidity (ArrayLike) – Scaled specific humidity.

  • rhi (ArrayLike) – Scaled relative humidity over ice.

See also

eval()

scaler_specific_keys = ('rhi_adj_mid_troposphere', 'rhi_adj_stratosphere', 'mid_troposphere_threshold', 'stratosphere_threshold')

Variables required in addition to specific_humidity, air_temperature, and air_pressure These are either ModelParams specific to scaling, or variables that should be extracted from eval() parameter source.

source

Data evaluated in model

class pycontrails.models.humidity_scaling.HumidityScalingByLevelParams(copy_source=True, interpolation_method='linear', interpolation_bounds_error=False, interpolation_fill_value=nan, interpolation_localize=False, interpolation_use_indices=False, interpolation_q_method=None, verify_met=True, downselect_met=True, met_longitude_buffer=(0.0, 0.0), met_latitude_buffer=(0.0, 0.0), met_level_buffer=(0.0, 0.0), met_time_buffer=(numpy.timedelta64(0, 'h'), numpy.timedelta64(0, 'h')), rhi_adj_mid_troposphere=0.8, rhi_adj_stratosphere=1.0, mid_troposphere_threshold=0.8, stratosphere_threshold=1.0)

Bases: ModelParams

Parameters for HumidityScalingByLevel.

mid_troposphere_threshold = 0.8

Adjustment factor for mid-troposphere humidity scaling. Default value of 0.8 taken from [Schumann, 2012].

rhi_adj_mid_troposphere = 0.8

Fraction of troposphere for mid-troposphere humidity scaling. Default value suggested in [Schumann, 2012].

rhi_adj_stratosphere = 1.0

Fraction of troposphere for stratosphere humidity scaling. Default value suggested in [Schumann, 2012].

stratosphere_threshold = 1.0

Adjustment factor for stratosphere humidity scaling. Default value of 1.0 taken from [Schumann, 2012].

pycontrails.models.humidity_scaling.eckel_scaling(ensemble_mean_rhi, ensemble_member_rhi, q_method)

Apply Eckel scaling to the given RHi values.

Parameters:

  • ensemble_mean_rhi (npt.NDArray[np.float64]) – The ensemble mean RHi values. This should be a 1D array with the same shape as ensemble_member_rhi.

  • ensemble_member_rhi (npt.NDArray[np.float64]) – The RHi values for a single ensemble member.

  • q_method ({None, "cubic-spline", "log-q-log-p"}) – The interpolation method.

Returns:

npt.NDArray[np.float64] – The scaled RHi values. Values are manually clipped at 0 to ensure only non-negative values are returned.

References

[Eckel and Walters, 1998]

pycontrails.models.humidity_scaling.histogram_matching(era5_rhi, product_type, member, q_method)

Map ERA5-derived RHi to it’s corresponding IAGOS quantile via histogram matching.

This matching is performed on a single ERA5 ensemble member.

Parameters:

  • era5_rhi (ArrayLike) – ERA5-derived RHi values for the given ensemble member.

  • product_type ({"reanalysis", "ensemble_members"}) – The ERA5 product type.

  • member (int | None) – The ERA5 ensemble member to use. Must be in the range [0, 10). Only used if product_type == "ensemble_members".

  • q_method ({None, "cubic-spline", "log-q-log-p"}) – The interpolation method.

Returns:

npt.NDArray[np.float64] – The IAGOS quantiles corresponding to the ERA5-derived RHi values. Returned as a numpy array with the same shape and dtype as era5_rhi.

pycontrails.models.humidity_scaling.histogram_matching_all_members(era5_rhi_all_members, member, q_method)

Recalibrate ERA5-derived RHi values to IAGOS quantiles by histogram matching.

This recalibration requires values for all ERA5 ensemble members. Currently, the number of ensemble members is hard-coded to 10.

Parameters:

  • era5_rhi_all_members (npt.NDArray[np.float64]) – ERA5-derived RHi values for all ensemble members. This array should have shape (n, 10).

  • member (int) – The ERA5 ensemble member to use. Must be in the range [0, 10).

  • q_method ({None, "cubic-spline", "log-q-log-p"}) – The interpolation method.

Returns:

  • ensemble_mean_rhi (npt.NDArray[np.float64]) – The mean RHi values after histogram matching over all ensemble members. This is an array of shape (n,).

  • ensemble_member_rhi (npt.NDArray[np.float64]) – The RHi values after histogram matching for the given ensemble member. This is an array of shape (n,).