Making GrADS control files on Discover

[cbblanke]

Anybody have a script to make a GrADS control file for NetCDF or Grib output files on Discover? I used to be able to do this (for grib) but now I am getting an error "cannot run wgrib".

[dmocko]

Hi @cbblanke

Do you have "wgrib" in your $PATH? What happens when you run "which grads"?

You have two options to add "wgrib" to your $PATH on Discover:

1. Add "/discover/nobackup/projects/gmao/share/dasilva/opengrads/Contents" to your PATH environmental variable.

2. Run "module add opengrads/2.1.a2.oga.2"

To generate a GrADS control file for GRIB-1 data: /home/dmocko/bin/grib2ctl.pl
To generate a GrADS control file for GRIB-2 data: /home/dmocko/bin/g2ctl.pl

To generate a GrADS "*.xdf" file for netCDF data, I don't have a script. Instead, I just edit one myself by following this:
http://cola.gmu.edu/grads/gadoc/SDFdescriptorfile.html

Then I use "xdfopen [filename].xdf" to open in GrADS.

For some sample files, look for the "*.xdf" files in this directory:
/discover/nobackup/projects/lis/MET_FORCING/PRINCETON/V2.2_Classic

The temperature "*.xdf" file is:

DSET ^%y4/tas_3hourly_%y4-%y4.nc
TITLE PRINCETON forcing V2.2
DTYPE netcdf
OPTIONS template
TDEF time 195776 LINEAR 00Z01jan1948 3hr

Note that not all aspects of a data descriptor file are needed. GrADS fills in the rest once you get the file open.

If you have any specific datasets and are having issues, just let us know.

[cbblanke]

Thanks. I tried making an xdf descriptor file lis.xdf in /home/cbblanke/nobackup/runz/bigafrica/netcdf. I am running LIS with NetCDF output. Here is the descriptor file.

dset ^SURFACEMODEL/%y4/%y4%m2/LIS_HIST_%y4%m2%d2%h2%n2.d01.nc
options template
dtype netcdf
undef 9.999E+20
title lis
tdef time 30 linear 00Z01OCT2016 24hr

And here is the error message.

xdfopga-> xdfopen lis.xdf
Scanning Descriptor File: lis.xdf
read_metadata: gaopfn failed (rc=-88888)
gadsdf: Couldn't ingest SDF metadata.
SDF Descriptor file lis.xdf was not successfully opened & parsed.

[dmocko]

@karsenau is correct that for LIS output datasets, you do also need to add the lat/lon information to the .xdf file.

Some netCDF files don't need these lines, but some do. It really does depend on the output of "ncdump -h".

But this particular error is because your "dset" path is not pointing to any netCDF file. You are using "3 level hierarchy" for your LIS output, which sort files into directories by "YYYYMM". However, your ".xdf" file is looking for files in a "4 level hierarchy", which is "YYYY/YYYYMM".

Change your dset line to:
dset ^SURFACEMODEL/%y4%m2/LIS_HIST_%y4%m2%d2%h2%n2.d01.nc

"ncdump" on your first output file shows:
ncdump -h SURFACEMODEL/200006/LIS_HIST_200006020000.d01.nc

netcdf LIS_HIST_200006020000.d01 {
dimensions:
        east_west = 2351 ;
        north_south = 2231 ;
        time = 1 ;
        SoilMoist_profiles = 4 ;
[......................]
// global attributes:
                :missing_value = -9999.f ;
                :NUM_SOIL_LAYERS = 4 ;
                :SOIL_LAYER_THICKNESSES = 10.f, 30.f, 60.f, 100.f ;
                :title = "LIS land surface model output" ;
                :institution = "NASA GSFC" ;
                :source = "" ;
                :history = "created on date: 2022-08-10T11:06:50.360" ;
                :references = "Kumar_etal_EMS_2006, Peters-Lidard_etal_ISSE_2007" ;
                :conventions = "CF-1.6" ;
                :comment = "website: http://lis.gsfc.nasa.gov/" ;
                :MAP_PROJECTION = "EQUIDISTANT CYLINDRICAL" ;
                :SOUTH_WEST_CORNER_LAT = -34.895f ;
                :SOUTH_WEST_CORNER_LON = -17.505f ;
                :DX = 0.03f ;
                :DY = 0.03f ;

A "lis.xdf" file that will work is:

dset ^SURFACEMODEL/%y4%m2/LIS_HIST_%y4%m2%d2%h2%n2.d01.nc
options template
dtype netcdf
undef -9999.0
title lis
xdef          east_west 2351 linear    -17.505   0.03
ydef        north_south 2231 linear    -34.895   0.03
zdef SoilMoist_profiles    4 levels  0.1 0.4 1.0 2.0
tdef               time 5982 linear 00Z02jun2000 1dy

1. The "undef" in the ".xdf" file is the "missing_value" global attribute from "ncdump".
2. The second column of the xdef/ydef/zdef/tdef lines are the names of the dimensions from "ncdump".
3. The third column of these lines are the number of the dimensions from "ncdump". Since you are using "options template", I counted the total number of LIS_HIST output files, and put that number (5982) in the tdef row.
4. The lower left lat/on start values (fifth column of the xdef/ydef lines) are the "SOUTH_WEST_CORNER" values from "ncdump".
5. The sixth column of the xdef/ydef lines are from "DX" and "DY" from "ncdump".
6. The number of "zdef" levels (4) is from "NUM_SOIL_LAYERS" from "ncdump".
7. The bottom of the 4 soil layers (0.1 0.4 1.0 2.0) in meters is calculated from "SOIL_LAYER_THICKNESSES" from "ncdump" (which is in centimeters). Note that you could just as easily put (10.0 30.0 60.0 100.0) here or any other 4 values, but just for convenience, it might be good to let GrADS know the layer structure for your plots. It's really up to you.
8. The start date time "00Z02jun2000" is from the first output file". I used "1dy" for the time interval, but "24hr" would also work.

[cbblanke]

OK, that is perfect, thanks!

[cbblanke]

What do you do if you have multiple variables (e.g. soil moisture and temperature, or volumetric and relative soil moisture) with 4 layers?

[dmocko]

Good question. The latter case (of volumetric and relative soil moisture) should have the same "SoilMoist_" dimension in the name of the variable from "ncdump". So, it should open both variables.

For soil moisture vs. temperature, you can only have one variable on the "zdef" line. So, what I always do is just create a parallel ".xdf" file, and then open it in GrADS as file 2. If I'm plotting soil temperature, I have to remember to use file 2 instead.

The soiltemp ".xdf" would be the same as the regular ".xdf" file, except for one line:

zdef SoilMoist_profiles 4 levels 0.1 0.4 1.0 2.0

[karsenau]

Hi @cbblanke:

For the LIS-based netcdf files, you will also need to include the lat and lon information found in a typical "ncdump -h" and with one of the LIS netcdf files. You can find all the details on how to include those additional details you need, here:

http://cola.gmu.edu/grads/gadoc/SDFdescriptorfile.html

[karsenau]

And you include your "SoilMoist_profiles = 4" levels, in the "zdef" level

[cbblanke]

1. Where can I find "ncdump" on Discover?
2. I tried using the same xdef and ydef that I get for the grib data, and I get an error message:

ga-> xdfopen lis.xdf
Scanning Descriptor File: lis.xdf
gadxdf error: Missing or invalid dimension size.
--> The invalid description file record is:
--> xdef 2351 linear -17.505 0.03
The data file was not opened.

[dmocko]

What module are you using on Discover? It should add the "bin" directory of your netCDF library, which will contain the "ncdump" binary.

I am working on a full response to your issues with the .xdf file.

[dmocko]

Specifically, these lines in your module file will add the "bin" directory to your path:

set def_lis_netcdf /discover/nobackup/projects/lis/libs/...[whichever compiler version you are using]

prepend-path PATH "$def_lis_netcdf/bin:$def_lis_eccodes/bin"

[cbblanke]

I had module load lisf_7_gnu_11.2.0_impi_2021.4.0 in my LIS execution script but not my login environment. Fixed it.