Read multiple netCDF4 using Python 3

Question

I need to read a multiple netCDF4 files using python 3. The files are in different folder named 2019, 2018 with months and days. And the files have the same variables I use to read and plot it.

I can do that just for one file. How I can read all files and variable IR in all files in all folder?

path='/home/all/2019/03/25/Mmult_2.nc'
nc = netCDF4.Dataset('/home/all/2019/03/25/Mmult_2.nc')
IR=nc.variables['IR'][:]
plt.pcolormesh(IR)
plt.colorbar()
plt.show()

Answer 1

You can use the glob.glob module to search all the nc-files in your sub-folder like this (taken from https://stackoverflow.com/questions/14798220/how-can-i-search-sub-folders-using-glob-glob-module):

import glob
list_of_paths = glob.glob('C:/Users/sam/Desktop/myfolder/**/*.nc', recursive=True)

Then, if there is a dimension in your netcdfs that is unlimited, you can open all the files in one line like this:

import netCDF4
nc = netCDF4.MFDataset(list_of_paths)

After this you will probably want to be carefull, because doing:

IR=nc.variables['IR'][:]

will load the 'IR'-data from ALL your nc-files into your memory at once, which could crash you computer. So it's probably better to load small parts at a time by replacing [:] with, for example, [0:2].

• Sarah Tohami: thanks u but what mean [0:2] my problem that i have a simple application wxpython when i have button this button for select file in directory netcdf file i want to open so after open i want plot it so i will try ... but please what mean [0:2] ? exist method to open the file i want and plot it ?
• Sarah Tohami: i have question we can use loop for and loop for for variables keys ?
• Sarah Tohami: that is imformations about my variable IR in file netCDF4._netCDF4.Variable'> int16 IR_108(ny, nx) Offset : radiance = offset + CN x slope: -10.456819486590666 Slope : radiance = offset + CN x slope: 0.2050356762076601 unlimited dimensions: current shape = (3712, 3712) filling on
• Sarah Tohami: when i do : import numpy as np import netCDF4 import matplotlib.pyplot as plt import glob list_of_paths = glob.glob('/home/all/2019/03/25/*.nc', recursive=True)nc = netCDF4.MFDataset(list_of_paths) IR=nc.variables['IR'][0:1] i had error like : name 'nc' is not defined In [15]: nc = netCDF4.MFDataset(list_of_paths) --------------------------------------------------------------------------- OSError: dataset /homm/all/msg04/all/2019/03/25/Mmulti_201903252000.nc does not have a aggregation dimension
• Bert Coerver: ok then you'll have to use "netCDF4.Dataset" instead of "netCDF4.MFDataset" and load a single nc-file at a time. So you need to make a for-loop on the list_of_paths

Answer 2

I would recommend xarray. http://xarray.pydata.org/en/stable/io.html#reading-multi-file-datasets

import xarray
xarray.open_mfdataset('my/files/*.nc', parallel=True)