Set up a Cache store¶

The DiskCacheStore and GCPCacheStore module enables connections to backend storage sources for caching large files.

By default, pycontrails uses the DiskCacheStore which stores files in the user cache directory or the current hard drive.

[1]:

from pycontrails import DiskCacheStore, GCPCacheStore

Disk Cache¶

Store and retrieve files from the local hard drive.

[2]:

disk = DiskCacheStore()

[3]:

disk.exists("test/cache/path.nc")

[3]:

False

[4]:

# Total cache size, in MB
disk.size

[4]:

23709.123237

Requires [gcp] optional dependencies:
$ pip install pycontrails[gcp]

Store and retrieve files from Google Cloud Storage.

[5]:

gcp = GCPCacheStore(bucket="contrails-301217-unit-test", cache_dir="test/objects")

[6]:

gcp.bucket

[6]:

'contrails-301217-unit-test'

[7]:

gcp.path("path.nc")

[7]:

'test/objects/path.nc'

[8]:

gcp.gs_path("met/ecmwf/some-path.nc")

[8]:

'gs://contrails-301217-unit-test/test/objects/met/ecmwf/some-path.nc'

[ ]:

gcp.exists("some-path.nc")

False

By default, GCP cache is read only. Pass read_only=False to constructor to allow writing

[9]:

gcp = GCPCacheStore(bucket="contrails-301217-unit-test", cache_dir="test/objects", read_only=False)

[ ]:

gcp.put("cache.ipynb", "cache.ipynb")

'cache.ipynb'