pycontrails.datalib.dwd.ICON¶

class pycontrails.datalib.dwd.ICON(time, variables, pressure_levels=None, domain='global', timestep_freq=None, grid=None, forecast_time=None, model_levels=None, show_progress=False, cachestore=<object object>, cache_download=False)¶

Bases: MetDataSource

Class to support ICON data access, download, and organization.

Access is credential-free via the DWD Open Data Server.

The current operational version of ICON uses a single-moment microphysics scheme that underestimates humidity in ice-supersaturated regions. Documentation for this datalib will be updated when the double-moment microphysics scheme currently under development becomes operational.

The DWD provides ICON forecasts on three domains: a global domain (~13 km resolution), a higher-resolution Europe domain (~7 km), and a high-resolution Germany domain (~2 km). Global forecasts are initialized every 6 hours and Europe and Germany forecasts are initialized every 3 hours. The Open Data Server does not provide a long-term forecast archive. Data for each forecast cycle is typically available for about 24 hours.

Global ICON forecasts are provided on an icosahedral grid and must be remapped to a latitude-longitude grid. This datalib currently supports nearest-neighbor remapping only. Nearest-neighbor remapping preserves extreme values in forecast fields but can alter the locations where they occur. Additional remapping methods may be added in future releases.

The forecast horizon depends on the domain and forecast initialization time:

ICON forecast horizon¶
Forecast	Global (00, 12)	Global (06, 18)	Europe (00, 06, 12, 18)	Europe (03, 09, 15, 21)	Germany (all forecasts)
Horizon (hourly forecast)	+78h	+78h	+78h	+30h	+48h
Horizon (extended forecast)	+180h	+120h	+120h	+48h	N/A
Extended forecast timestep	3h	3h	3h	6h	N/A

This datalib currently supports only those variables required to run Cocip and CocipGrid. Please contact the pycontrails developers to request support for additional variables.

Parameters:

time (metsource.TimeInput) – The time range for data retrieval, either a single datetime or (start, end) datetime range. Input must be datetime-like or tuple of datetime-like specifying the (start, end) of the date range, inclusive.
variables (metsource.VariableInput) – Variable name (e.g., “t”, “air_temperature”, [“air_temperature, specific_humidity”])
pressure_levels (metsource.PressureLevelInput | None, optional) – Pressure levels for processed data, in hPa (mbar). To download single-level parameters, set to -1. Defaults to pressure levels that match standard flight levels between FL200 and FL500.
domain (str, optional) – Forecast domain. Must be one of ‘global’ (global domain with ~13 km resolution, default), ‘europe’ (European domain nested inside the global domain with ~7 km resolution), or ‘germany’ (regional domain centered on Germany with ~2.2 km resolution).
timestep_freq (str | timedelta | None, optional) – Manually set the timestep interval within the bounds defined by time. Supports any value that can be passed to pandas.date_range(freq=...). By default, this is set to the highest frequency that can supported the requested time range on the requested domain.
grid (float | None, optional) – Latitude/longitude grid resolution. Used only when domain is ‘global’, in which case data must be remapped from ICON’s native icosahedral grid. Fields from the European nest and the regional Germany forecast are provided on regular latitude-longitude grids, so no interpolation is required and this parameter is ignored. If no value is provided when domain is ‘global’, data will be interpolated to a 0.25 degree grid, which provides spatial resolution comparable to ICON’s native grid at midlatitude. A warning is issued if a value is provided when domain is set to ‘europe’ or ‘germany’.
forecast_time (DatetimeLike | None, optional) – Specify forecast by initialization time. By default, set to the most recent forecast that includes the requested time range. This is the most recent multiple of 3 hours (00z, 03z, 06z, etc) when domain is ‘europe’ or ‘germany’ and the most recent multiple of 6 hours (00z, 06z, etc) when domain is ‘global’.
model_levels (list[int] | None, optional) – Specify ICON model levels to include in downloads from the Open Data Server. By default, this is set to include all model levels.
show_progress (bool, optional) – Show progress while downloading and processing ICON data. Disabled by default.
cachestore (cache.CacheStore, optional) – Cache data store for staging processed netCDF files. Defaults to pycontrails.core.cache.DiskCacheStore. If None, cache is turned off.
cache_download (bool, optional) – If True, cache downloaded GRIB files rather than storing them in a temporary file. By default, False.

See also

pycontrails.datalib.dwd.ods.list_forecasts(): list available forecast cycles
pycontrails.datalib.dwd.ods.list_forecast_steps(): list available forecast steps

__init__(time, variables, pressure_levels=None, domain='global', timestep_freq=None, grid=None, forecast_time=None, model_levels=None, show_progress=False, cachestore=<object object>, cache_download=False)¶

Methods

`__init__`(time, variables[, pressure_levels, ...])
`cache_dataset`(dataset)	Cache data from data source.
`create_cachepath`(t)	Return cachepath to local data file based on datetime.
`download`(**xr_kwargs)	Confirm all data files are downloaded and available locally in the `cachestore`.
`download_dataset`(times)	Download data from data source for input times.
`get_forecast_step`(time)	Convert time to forecast steps.
`is_datafile_cached`(t, **xr_kwargs)	Check datafile defined by datetime for variables and pressure levels in class.
`list_timesteps_cached`(**xr_kwargs)	Get a list of data files available locally in the `cachestore`.
`list_timesteps_not_cached`(**xr_kwargs)	Get a list of data files not available locally in the `cachestore`.
`open_dataset`(disk_paths, **xr_kwargs)	Open multi-file dataset in xarray.
`open_metdataset`([dataset, xr_kwargs])	Open MetDataset from data source.
`rpaths`(time)	Get list of remote paths for download.
`set_metadata`(ds)	Set met source metadata on `ds.attrs`.

Attributes

`cache_download`	Whether to save raw data files to `cachestore` for reuse
`cachestore`	Cache store for intermediates while processing data source If None, cache is turned off.
`domain`	ICON forecast domain
`forecast_time`	Forecast cycle start time
`model_levels`	Model levels included when downloading raw data files
`show_progress`	Whether to show progress bar while downloading and processing data
`dataset`	MetDataset 'dataset' attribute.
`grid`	Lat / Lon grid spacing
`hash`	Generate a unique hash for this datasource.
`is_single_level`	Return True if the datasource is single level data.
`paths`	Path to local source files to load.
`pressure_level_variables`	Available pressure-level variables.
`pressure_levels`	List of pressure levels.
`single_level_variables`	Available single-level variables.
`steps`	Forecast steps corresponding to input `time`.
`supported_pressure_levels`	Pressure levels available from datasource.
`supported_variables`	Parameters available from data source.
`timesteps`	List of individual timesteps from data source derived from `time` Use `parse_time()` to handle `TimeInput`.
`variable_shortnames`	Return a list of variable short names.
`variable_standardnames`	Return a list of variable standard names.
`variables`	Variables requested from data source Use `parse_variables()` to handle `VariableInput`.

cache_dataset(dataset)¶

Cache data from data source.

Parameters:: dataset (xarray.Dataset) – Dataset loaded from remote API or local files. The dataset must have the same format as the original data source API or files.

cache_download¶: Whether to save raw data files to cachestore for reuse

cachestore¶: Cache store for intermediates while processing data source If None, cache is turned off.

create_cachepath(t)¶

Return cachepath to local data file based on datetime.

Parameters:: t (datetime) – Datetime of datafile
Returns:: str – Path to cached data file

property dataset¶

MetDataset ‘dataset’ attribute.

Returns:: str – One of “ICON”, “ICON-EU”, or “ICON-D2”

domain¶: ICON forecast domain

download(**xr_kwargs)¶

Confirm all data files are downloaded and available locally in the cachestore.

Parameters:: **xr_kwargs – Passed into xarray.open_dataset() via is_datafile_cached().

download_dataset(times)¶

Download data from data source for input times.

Parameters:: times (list[datetime]) – List of datetimes to download a store in cache

forecast_time¶: Forecast cycle start time

get_forecast_step(time)¶

Convert time to forecast steps.

Parameters:: times (datetime) – Time to convert to forecast steps
Returns:: int – Forecast step at given time

grid¶: Lat / Lon grid spacing

property hash¶

Generate a unique hash for this datasource.

Returns:: str – Unique hash for met instance (sha1)

is_datafile_cached(t, **xr_kwargs)¶

Check datafile defined by datetime for variables and pressure levels in class.

If using a cloud cache store (i.e. cache.GCPCacheStore), this is where the datafile will be mirrored to a local file for access.

Parameters:

t (datetime) – Datetime of datafile
**xr_kwargs (Any) – Additional kwargs passed directly to xarray.open_mfdataset() when opening files. By default, the following values are used if not specified:
- chunks: {“time”: 1}
- engine: “netcdf4”
- parallel: False

Returns:

bool – True if data file exists for datetime with all variables and pressure levels, False otherwise

property is_single_level¶: Return True if the datasource is single level data.

Added in version 0.50.0.

list_timesteps_cached(**xr_kwargs)¶

Get a list of data files available locally in the cachestore.

Parameters:: **xr_kwargs – Passed into xarray.open_dataset() via is_datafile_cached().

list_timesteps_not_cached(**xr_kwargs)¶

Get a list of data files not available locally in the cachestore.

Parameters:: **xr_kwargs – Passed into xarray.open_dataset() via is_datafile_cached().

model_levels¶: Model levels included when downloading raw data files

open_dataset(disk_paths, **xr_kwargs)¶

Open multi-file dataset in xarray.

Parameters:

disk_paths (str | list[str] | pathlib.Path | list[pathlib.Path]) – list of string paths to local files to open
**xr_kwargs (Any) – Additional kwargs passed directly to xarray.open_mfdataset() when opening files. By default, the following values are used if not specified:
- chunks: {“time”: 1}
- engine: “netcdf4”
- parallel: False
- lock: False

Returns:

xarray.Dataset – Open xarray dataset

open_metdataset(dataset=None, xr_kwargs=None, **kwargs)¶

Open MetDataset from data source.

This method should download / load any required datafiles and returns a MetDataset of the multi-file dataset opened by xarray.

Parameters:

dataset (xr.Dataset | None, optional) – Input xr.Dataset loaded manually. The dataset must have the same format as the original data source API or files.
xr_kwargs (dict[str, Any] | None, optional) – Dictionary of keyword arguments passed into xarray.open_mfdataset() when opening files. Examples include “chunks”, “engine”, “parallel”, etc. Ignored if dataset is input.
**kwargs (Any) – Keyword arguments passed through directly into MetDataset constructor.

Returns:

MetDataset – Meteorology dataset