Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

Commit f923fc5

Browse files
negin513pre-commit-ci[bot]jhammanscottyhq
authored
Zarr tutorial (#323)
* adding the links to all indexing materials * typo fix + remove a redundant example. * initial commits and contents * data storage * updates and cloud storage * updates and cloud storage * additional resources * updates * more organized * sharding * sharding added * zarr added * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * minor updates to zarr tutorial * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ensure zarr notebook is rendered, add cell metadata for large outputs * remove accidentaly advaced-indexing edit * use data subfolder for io notebook as in zarr notebook * fix cat and tree paths, toggle output * link to cmip6 notebook at end --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Joseph Hamman <joe@earthmover.io> Co-authored-by: Scott Henderson <3924836+scottyhq@users.noreply.github.com>
1 parent 2c896d9 commit f923fc5

File tree

4 files changed

+732
-31
lines changed

4 files changed

+732
-31
lines changed

‎_toc.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@ parts:
4444
- file: intermediate/indexing/boolean-masking-indexing.ipynb
4545
- file: intermediate/hierarchical_computation.ipynb
4646
- file: intermediate/xarray_and_dask
47+
- file: intermediate/intro-to-zarr.ipynb
4748
- file: intermediate/xarray_ecosystem
4849
- file: intermediate/hvplot
4950
- file: intermediate/remote_data/index

‎fundamentals/01.1_io.ipynb‎

Lines changed: 44 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -51,9 +51,27 @@
5151
]
5252
},
5353
{
54-
"cell_type": "markdown",
54+
"cell_type": "code",
55+
"execution_count": null,
5556
"id": "2",
5657
"metadata": {},
58+
"outputs": [],
59+
"source": [
60+
"# Ensure we start with a clean directory for the tutorial\n",
61+
"import pathlib\n",
62+
"import shutil\n",
63+
"\n",
64+
"datadir = pathlib.Path('../data/io-tutorial')\n",
65+
"if datadir.exists():\n",
66+
" shutil.rmtree(datadir)\n",
67+
"else:\n",
68+
" datadir.mkdir()"
69+
]
70+
},
71+
{
72+
"cell_type": "markdown",
73+
"id": "3",
74+
"metadata": {},
5775
"source": [
5876
"The constructor of `Dataset` takes three parameters:\n",
5977
"\n",
@@ -66,7 +84,7 @@
6684
{
6785
"cell_type": "code",
6886
"execution_count": null,
69-
"id": "3",
87+
"id": "4",
7088
"metadata": {},
7189
"outputs": [],
7290
"source": [
@@ -94,16 +112,16 @@
94112
")\n",
95113
"\n",
96114
"# write datasets\n",
97-
"ds1.to_netcdf(\"ds1.nc\")\n",
98-
"ds2.to_netcdf(\"ds2.nc\")\n",
115+
"ds1.to_netcdf(datadir / \"ds1.nc\")\n",
116+
"ds2.to_netcdf(datadir / \"ds2.nc\")\n",
99117
"\n",
100118
"# write dataarray\n",
101-
"ds1.a.to_netcdf(\"da1.nc\")"
119+
"ds1.a.to_netcdf(datadir / \"da1.nc\")"
102120
]
103121
},
104122
{
105123
"cell_type": "markdown",
106-
"id": "4",
124+
"id": "5",
107125
"metadata": {},
108126
"source": [
109127
"Reading those files is just as simple:\n"
@@ -112,26 +130,26 @@
112130
{
113131
"cell_type": "code",
114132
"execution_count": null,
115-
"id": "5",
133+
"id": "6",
116134
"metadata": {},
117135
"outputs": [],
118136
"source": [
119-
"xr.open_dataset(\"ds1.nc\")"
137+
"xr.open_dataset(datadir / \"ds1.nc\")"
120138
]
121139
},
122140
{
123141
"cell_type": "code",
124142
"execution_count": null,
125-
"id": "6",
143+
"id": "7",
126144
"metadata": {},
127145
"outputs": [],
128146
"source": [
129-
"xr.open_dataarray(\"da1.nc\")"
147+
"xr.open_dataarray(datadir / \"da1.nc\")"
130148
]
131149
},
132150
{
133151
"cell_type": "markdown",
134-
"id": "7",
152+
"id": "8",
135153
"metadata": {},
136154
"source": [
137155
"<img src=\"https://zarr.readthedocs.io/en/stable/_static/logo1.png\" align=\"right\" width=\"20%\">\n",
@@ -151,16 +169,16 @@
151169
{
152170
"cell_type": "code",
153171
"execution_count": null,
154-
"id": "8",
172+
"id": "9",
155173
"metadata": {},
156174
"outputs": [],
157175
"source": [
158-
"ds1.to_zarr(\"ds1.zarr\", mode=\"w\")"
176+
"ds1.to_zarr(datadir / \"ds1.zarr\", mode=\"w\")"
159177
]
160178
},
161179
{
162180
"cell_type": "markdown",
163-
"id": "9",
181+
"id": "10",
164182
"metadata": {},
165183
"source": [
166184
"We can then read the created file with:\n"
@@ -169,16 +187,16 @@
169187
{
170188
"cell_type": "code",
171189
"execution_count": null,
172-
"id": "10",
190+
"id": "11",
173191
"metadata": {},
174192
"outputs": [],
175193
"source": [
176-
"xr.open_zarr(\"ds1.zarr\", chunks=None)"
194+
"xr.open_zarr(datadir / \"ds1.zarr\", chunks=None)"
177195
]
178196
},
179197
{
180198
"cell_type": "markdown",
181-
"id": "11",
199+
"id": "12",
182200
"metadata": {},
183201
"source": [
184202
"setting the `chunks` parameter to `None` avoids `dask` (more on that in a later\n",
@@ -187,7 +205,7 @@
187205
},
188206
{
189207
"cell_type": "markdown",
190-
"id": "12",
208+
"id": "13",
191209
"metadata": {},
192210
"source": [
193211
"**tip:** You can write to any dictionary-like (`MutableMapping`) interface:"
@@ -196,7 +214,7 @@
196214
{
197215
"cell_type": "code",
198216
"execution_count": null,
199-
"id": "13",
217+
"id": "14",
200218
"metadata": {},
201219
"outputs": [],
202220
"source": [
@@ -207,7 +225,7 @@
207225
},
208226
{
209227
"cell_type": "markdown",
210-
"id": "14",
228+
"id": "15",
211229
"metadata": {},
212230
"source": [
213231
"## Raster files using rioxarray\n",
@@ -220,7 +238,7 @@
220238
{
221239
"cell_type": "code",
222240
"execution_count": null,
223-
"id": "15",
241+
"id": "16",
224242
"metadata": {},
225243
"outputs": [],
226244
"source": [
@@ -241,16 +259,16 @@
241259
{
242260
"cell_type": "code",
243261
"execution_count": null,
244-
"id": "16",
262+
"id": "17",
245263
"metadata": {},
246264
"outputs": [],
247265
"source": [
248-
"da.rio.to_raster('ds1_a.tiff')"
266+
"da.rio.to_raster(datadir / 'ds1_a.tiff')"
249267
]
250268
},
251269
{
252270
"cell_type": "markdown",
253-
"id": "17",
271+
"id": "18",
254272
"metadata": {},
255273
"source": [
256274
"NOTE: you can now load this file into GIS tools like [QGIS](https://www.qgis.org)! Or open back into Xarray:"
@@ -259,11 +277,11 @@
259277
{
260278
"cell_type": "code",
261279
"execution_count": null,
262-
"id": "18",
280+
"id": "19",
263281
"metadata": {},
264282
"outputs": [],
265283
"source": [
266-
"DA = xr.open_dataarray('ds1_a.tiff', engine='rasterio')\n",
284+
"DA = xr.open_dataarray(datadir / 'ds1_a.tiff', engine='rasterio')\n",
267285
"DA.rio.crs"
268286
]
269287
}
@@ -279,11 +297,6 @@
279297
"name": "python",
280298
"nbconvert_exporter": "python",
281299
"pygments_lexer": "ipython3"
282-
},
283-
"vscode": {
284-
"interpreter": {
285-
"hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
286-
}
287300
}
288301
},
289302
"nbformat": 4,

0 commit comments

Comments
(0)

AltStyle によって変換されたページ (->オリジナル) /