pycontrails.datalib.ukmo.ukmo¶

UK Met Office (UKMO) Unified Model (UM) data access.

This module supports

Retrieving global deterministic 10 km UM forecasts from the public met-office-atmospheric-model-data S3 bucket.
Selecting pressure levels and deriving specific humidity from relative humidity.
Local caching of processed forecasts as netCDF files.
Opening processed and cached files as a pycontrails.MetDataset.

Warning

This datalib currently supports only air temperature, relative humidity, and specific humidity. Therefore, it cannot be used to run Cocip or CocipGrid without additional external data. It does provide the fields required by ISSR, SAC, and PCR.

Forecasts are provided on a small set of standard pressure levels rather than on model levels. Near cruise altitudes the levels are spaced 25 hPa apart, which is roughly 1,900 to 3,200 ft. This vertical resolution is coarse compared to model-level data, so thin ice-supersaturated layers are not resolved.

Module Attributes

PROVIDER

MetDataset.attrs values set by UM10.set_metadata().

Classes

UM10(time, *[, variables, pressure_levels, ...])

Class to support UK Met Office UM global deterministic 10 km forecast data access.

Exceptions

UM10DataNotFoundError

Raised when requested data could not be fetched from the S3 archive.

pycontrails.datalib.ukmo.ukmo.PROVIDER = 'UKMO'¶: MetDataset.attrs values set by UM10.set_metadata().

class pycontrails.datalib.ukmo.ukmo.UM10(time, *, variables=None, pressure_levels=None, timestep_freq=None, grid=None, forecast_time=None, lead_hours=None, cachestore=<object object>)¶

Bases: MetDataSource

Class to support UK Met Office UM global deterministic 10 km forecast data access.

Access is credential-free. Data is read anonymously from the met-office-atmospheric-model-data S3 bucket (eu-west-2), which requires the optional dependency s3fs (pip install pycontrails[ukmo]).

The Met Office Unified Model (UM) is the Met Office’s operational numerical weather prediction model. This datalib provides the global deterministic configuration. Fields are provided on a regular latitude-longitude grid with 2560 x 1920 points (0.140625 degrees in longitude and 0.09375 degrees in latitude, approximately 10 km at mid-latitudes). Longitude runs from -179.93 to 179.93 degrees and latitude from -89.95 to 89.95 degrees.

The bucket holds a rolling two year archive of forecasts, as well as the latest forecasts. At the time of writing, the earliest available forecast was initialized at 2024-10-14 18Z. Forecasts are initialized every 6 hours (00, 06, 12, and 18 UTC). The forecast horizon and time step depend on the initialization time:

UM forecast horizon¶
Forecast	Global (00, 12)	Global (06, 18)
Horizon (hourly forecast)	+54h	+54h
Horizon (3-hourly forecast)	+144h	+69h
Horizon (6-hourly forecast)	+168h	N/A

The 06 and 18 UTC forecasts ended at +66h rather than +69h before 2026-01-21 18Z.

The archive provides fields on 37 pressure levels between 1000 hPa and 0.4 hPa (temperature) and on 33 pressure levels between 1000 hPa and 10 hPa (relative humidity). This datalib supports the 33 levels between 1000 hPa and 10 hPa that both fields share (see supported_pressure_levels). Near cruise altitude the pressure levels are 300, 275, 250, 225, 200, 175, and 150 hPa (approximately 30,000 to 44,600 ft). The spacing between these levels varies from about 1,900 ft to 3,200 ft. This is coarse compared to model-level data.

This datalib currently supports only air temperature, relative humidity, and specific humidity (derived from temperature and relative humidity). These fields are sufficient to run ISSR, SAC, and PCR, but this datalib does not provide winds and so cannot be used to run Cocip or CocipGrid. The archive includes other fields (for example geopotential height, wind speed and direction, and vertical velocity on pressure levels). Please contact the pycontrails developers to request support for additional variables.

Fields are served on the native latitude-longitude grid. This datalib does not support regridding. Users who want forecasts at a different resolution must regrid the returned MetDataset themselves.

Parameters:

time (metsource.TimeInput) – The time range for data retrieval, either a single datetime or (start, end) datetime range. Input must be datetime-like or tuple of datetime-like specifying the (start, end) of the date range, inclusive.
variables (metsource.VariableInput, optional) – Requested variables. Defaults to [met_var.AirTemperature, met_var.SpecificHumidity]. Only the requested variables are cached. specific_humidity is derived from temperature and relative humidity, so requesting it downloads both of those fields. Requesting only air_temperature or relative_humidity downloads only that field.
pressure_levels (metsource.PressureLevelInput, optional) – Requested pressure levels in hPa. Must be in supported_pressure_levels. Defaults to 300, 275, 250, 225, 200, 175, and 150 hPa (approximately 30,000 to 44,600 ft). Each requested level and variable is held in memory as a full global field while a forecast hour is processed, so requesting many levels uses a lot of memory.
timestep_freq (str | timedelta | None, optional) – Manually set the timestep interval within the bounds defined by time. Supports any value that can be passed to pandas.date_range(freq=...). By default, this is set to the highest frequency that the forecast supports over the requested time range: hourly if the time range ends within 54 hours of the start of the forecast, then 3-hourly, then (for 00 and 12 UTC forecasts) 6-hourly. If lead_hours is set, this defaults to 1 hour for lead_hours up to 48 and to 6 hours for longer leads, because the archive publishes only some hours at longer leads.
grid (float, optional) – Not supported. This datalib does not support regridding. A non-None value is ignored with a warning.
forecast_time (DatetimeLike | None, optional) – Specify the forecast by initialization time. Must be 00, 06, 12, or 18 UTC. By default, set to the forecast with the shortest lead times that covers the requested time range: the most recent forecast initialized at or before the start of the time range, or an older one if it is needed to reach the end of the time range. All data come from this single forecast. A ValueError is raised if no single forecast covers the time range.
lead_hours (int, optional) – Fetch data with a lead time of at least this many hours, instead of from a single forecast. Each timestep comes from the most recent forecast initialized at least lead_hours earlier that reaches the timestep, so the time range may be longer than any single forecast and consecutive timesteps may come from different forecasts. Because forecasts are initialized every 6 hours, the lead time of a timestep is less than lead_hours + 6 hours (less than lead_hours + 12 hours beyond a lead of 69 hours, which only the 00 and 12 UTC forecasts reach). Timesteps whose lead time is not published in the archive (see the forecast horizon table) raise a ValueError. lead_hours=0 uses the most recent forecast at each timestep. Cannot be combined with forecast_time. By default, None.
cachestore (cache.CacheStore, optional) – Cache store for fetched, processed hourly data. Defaults to pycontrails.core.cache.DiskCacheStore. Other cache stores, such as cache.GCPCacheStore(bucket=...), can be used to read from a pre-populated archive, but hours missing from it can only be fetched live with a DiskCacheStore. A cache store is required: a cachestore of None raises a ValueError.

cache_dataset(dataset)¶: Write processed hourly data to cachestore.

cachestore¶: Cache store for intermediates while processing data source If None, cache is turned off.

create_cachepath(t)¶

Return the cache path for the processed hourly data covering t.

Hashes the forecast initialization time, the valid time, the requested pressure levels, and the requested variables, so two differently-configured instances sharing a cachestore can’t collide on the same path.

Returns:: str – Cache path for t.
Raises:: ValueError – If cachestore is None.

download(**xr_kwargs)¶

Confirm all data files are downloaded and available locally in the cachestore.

Parameters:: **xr_kwargs – Passed into xarray.open_dataset() via is_datafile_cached().

download_dataset(times)¶

Fetch missing hours directly from the public Met Office S3 archive.

Only called (via the inherited download()) with hours that is_datafile_cached() has already determined are missing from cachestore.

Writing fetched hours to the cache is only supported for a DiskCacheStore. Other cache stores, such as a GCPCacheStore, can only provide hours already in them.

Fails fast: before downloading anything, every object needed for times is checked against a listing of the archive, and a single UM10DataNotFoundError naming every missing hour is raised if any are absent. Each hour is roughly 370 MB to download, so a request for data that does not exist is reported straight away instead of after the available hours have been downloaded. If an object nonetheless disappears between the check and its download, UM10DataNotFoundError is raised at that hour. Hours already downloaded are cached and are not fetched again by a repeated call.

Raises:

NotImplementedError – If cachestore is not a DiskCacheStore.
UM10DataNotFoundError – If any requested hour is not in the S3 archive.
ValueError – If a time in times is one the forecast doesn’t publish (see s3.lead_for_valid_time()). This is a caller error, not a data availability issue, and so is not wrapped as UM10DataNotFoundError.

forecast_time¶: Forecast initialization time, or None if fetching at a fixed lead_hours

grid¶: Lat / Lon grid spacing

property hash¶

Generate a unique hash for this datasource.

Returns:: str – Unique hash for met instance (sha1)

is_datafile_cached(t, **xr_kwargs)¶

Check datafile defined by datetime for variables and pressure levels in class.

If using a cloud cache store (i.e. GCPCacheStore), this is where the datafile will be mirrored to a local file for access.

Parameters:

t (datetime) – Datetime of datafile
**xr_kwargs (Any) – Additional kwargs passed directly to xarray.open_mfdataset() when opening files. By default, the following values are used if not specified:
- chunks: {“time”: 1}
- engine: “netcdf4”
- parallel: False

Returns:

bool – True if data file exists for datetime with all variables and pressure levels, False otherwise

property is_single_level¶: Return True if the datasource is single level data.

Added in version 0.50.0.

lead_hours¶

list_timesteps_cached(**xr_kwargs)¶

Get a list of data files available locally in the cachestore.

Parameters:: **xr_kwargs – Passed into xarray.open_dataset() via is_datafile_cached().

list_timesteps_not_cached(**xr_kwargs)¶

Get a list of data files not available locally in the cachestore.

Parameters:: **xr_kwargs – Passed into xarray.open_dataset() via is_datafile_cached().

open_dataset(disk_paths, **xr_kwargs)¶

Open multi-file dataset in xarray.

Parameters:

disk_paths (str | list[str] | pathlib.Path | list[pathlib.Path]) – list of string paths to local files to open
**xr_kwargs (Any) – Additional kwargs passed directly to xarray.open_mfdataset() when opening files. By default, the following values are used if not specified:
- chunks: {“time”: 1}
- engine: “netcdf4”
- parallel: False
- lock: False

Returns:

xarray.Dataset – Open xarray dataset

open_metdataset(dataset=None, xr_kwargs=None, **kwargs)¶

Open MetDataset from data source.

This method should download / load any required datafiles and returns a MetDataset of the multi-file dataset opened by xarray.

Parameters:

dataset (xr.Dataset | None, optional) – Input Dataset loaded manually. The dataset must have the same format as the original data source API or files.
xr_kwargs (dict[str, Any] | None, optional) – Dictionary of keyword arguments passed into xarray.open_mfdataset() when opening files. Examples include “chunks”, “engine”, “parallel”, etc. Ignored if dataset is input.
**kwargs (Any) – Keyword arguments passed through directly into MetDataset constructor.

Returns:

MetDataset – Meteorology dataset