pycontrails.datalib.ecmwf.hres_model_level

Model-level HRES data access from the ECMWF operational archive.

This module supports

  • Retrieving model-level HRES data by submitting MARS requests through the ECMWF API.

  • Processing retrieved model-level files to produce netCDF files on target pressure levels.

  • Local caching of processed netCDF files.

  • Opening processed and cached files as a pycontrails.MetDataset object.

Classes

HRESModelLevel(time, variables[, ...])

Class to support model-level HRES data access, download, and organization.

class pycontrails.datalib.ecmwf.hres_model_level.HRESModelLevel(time, variables, pressure_levels=None, timestep_freq=None, grid=None, forecast_time=None, model_levels=None, cachestore=<object object>, cache_download=False, url=None, key=None, email=None)

Bases: ECMWFAPI

Class to support model-level HRES data access, download, and organization.

The interface is similar to pycontrails.datalib.ecmwf.HRES, which downloads pressure-level data with much lower vertical resolution and single-level data. Note, however, that only a subset of the pressure-level data available through the operational archive is available as model-level data. As a consequence, this interface only supports access to nominal HRES forecasts (corresponding to stream = "oper" and field_type = "fc" in pycontrails.datalib.ecmwf.HRES) initialized at 00z and 12z.

Requires account with ECMWF and API key.

API credentials can be set in local ~/.ecmwfapirc file:

{
    "url": "https://api.ecmwf.int/v1",
    "email": "<email>",
    "key": "<key>"
}

Credentials can also be provided directly in url, key, and email keyword args.

See ecmwf-api-client documentation for more information.

Parameters:
  • time (metsource.TimeInput) – The time range for data retrieval, either a single datetime or (start, end) datetime range. Input must be datetime-like or tuple of datetime-like (datetime.datetime, pandas.Timestamp, numpy.datetime64) specifying the (start, end) of the date range, inclusive. All times will be downloaded in a single NetCDF file, which ensures that exactly one request is submitted per file on tape accessed. If forecast_time is unspecified, the forecast time will be assumed to be the nearest synoptic hour available in the operational archive (00 or 12). All subsequent times will be downloaded for relative to forecast_time.

  • variables (metsource.VariableInput) – Variable name (i.e. “t”, “air_temperature”, [“air_temperature, specific_humidity”])

  • pressure_levels (metsource.PressureLevelInput, optional) – Pressure levels for data, in hPa (mbar). To download surface-level parameters, use pycontrails.datalib.ecmwf.HRES. Defaults to pressure levels that match model levels at a nominal surface pressure.

  • timestep_freq (str, optional) – Manually set the timestep interval within the bounds defined by time. Supports any string that can be passed to pandas.date_range(freq=...). By default, this is set to the highest frequency that can supported the requested time range (“1h” out to 96 hours, “3h” out to 144 hours, and “6h” out to 240 hours)

  • grid (float, optional) – Specify latitude/longitude grid spacing in data. By default, this is set to 0.1.

  • forecast_time (DatetimeLike, optional) – Specify forecast by initialization time. By default, set to the most recent forecast that includes the requested time range.

  • levels (list[int], optional) – Specify ECMWF model levels to include in MARS requests. By default, this is set to include all model levels.

  • cachestore (CacheStore | None, optional) – Cache data store for staging processed netCDF files. Defaults to pycontrails.core.cache.DiskCacheStore. If None, cache is turned off.

  • cache_download (bool, optional) – If True, cache downloaded NetCDF files rather than storing them in a temporary file. By default, False.

  • url (str) – Override ecmwf-api-client url

  • key (str) – Override ecmwf-api-client key

  • email (str) – Override ecmwf-api-client email

create_cachepath(t)

Return cachepath to local HRES data file based on datetime.

This uniquely defines a cached data file with class parameters.

Parameters:

t (datetime | pd.Timestamp) – Datetime of datafile

Returns:

str – Path to local HRES data file

download_dataset(times)

Download data from data source for input times.

Parameters:

times (list[datetime]) – List of datetimes to download a store in cache

get_forecast_steps(times)

Convert list of times to list of forecast steps.

Parameters:

times (list[datetime]) – Times to convert to forecast steps

Returns:

list[int] – Forecast step at each time

grid

Lat / Lon grid spacing

mars_request(times)

Generate MARS request for specific list of times.

Parameters:

times (list[datetime]) – Times included in MARS request.

Returns:

str – MARS request for submission to ECMWF API.

open_metdataset(dataset=None, xr_kwargs=None, **kwargs)

Open MetDataset from data source.

This method should download / load any required datafiles and returns a MetDataset of the multi-file dataset opened by xarray.

Parameters:
  • dataset (xr.Dataset | None, optional) – Input xr.Dataset loaded manually. The dataset must have the same format as the original data source API or files.

  • xr_kwargs (dict[str, Any] | None, optional) – Dictionary of keyword arguments passed into xarray.open_mfdataset() when opening files. Examples include “chunks”, “engine”, “parallel”, etc. Ignored if dataset is input.

  • **kwargs (Any) – Keyword arguments passed through directly into MetDataset constructor.

Returns:

MetDataset – Meteorology dataset

paths

Path to local source files to load. Set to the paths of files cached in cachestore if no paths input is provided on init.

property pressure_level_variables

ECMWF pressure level parameters available on model levels.

Returns:

list[MetVariable] – List of MetVariable available in datasource

pressure_levels

List of pressure levels. Set to [-1] for data without level coordinate. Use parse_pressure_levels() to handle PressureLevelInput.

set_metadata(ds)

Set met source metadata on ds.attrs.

This is called within the open_metdataset() method to set metadata on the returned MetDataset instance.

Parameters:

ds (xr.Dataset | MetDataset) – Dataset to set metadata on. Mutated in place.

property single_level_variables

ECMWF single-level parameters available on model levels.

Returns:

list[MetVariable] – Always returns an empty list. To access single-level variables, use pycontrails.datalib.ecmwf.HRES.

property step_offset

Difference between forecast_time and first timestep.

Returns:

int – Number of steps to offset in order to retrieve data starting from input time.

property steps

Forecast steps from forecast_time corresponding within input time.

Returns:

list[int] – List of forecast steps relative to forecast_time

timesteps

List of individual timesteps from data source derived from time Use parse_time() to handle TimeInput.

variables

Variables requested from data source Use parse_variables() to handle VariableInput.