add 9-deg ocean+ice file generation, specification of ATM resolutions via namelist

[DeniseWorthen]

DESCRIPTION OF CHANGES:

One or more paragraphs describing the problem, solution, and required changes.

TESTS CONDUCTED:

If there are changes to the build or source code, the tests below must be conducted. Contact a repository manager if you need assistance.

• Compile branch on all Tier 1 machines using Intel (Orion, Jet, Hera, Hercules and WCOSS2).
• Compile branch on Hera using GNU.
• Compile branch in 'Debug' mode on WCOSS2.
• Run unit tests locally on any Tier 1 machine.
• Run the cpld_gridgen test locally on all Tier 1 machines.

Describe any additional tests performed.

DEPENDENCIES:

Add any links to pending PRs that are required prior to merging this PR. For example:

ufs-community/UFS_UTILS/pull/<pr_number>

DOCUMENTATION:

If this PR is contributing new capabilities that need to be documented, please also include updates to the RST files in the docs/source directory as supporting material.

ISSUE:

Fixes #1004

CONTRIBUTORS (optional):

If others have contributed to this work aside from the PR author, list them here

[DeniseWorthen]

@GeorgeGayno-NOAA I've been able to use the fix file location on Hera to generate all the mapped ocean masks for C12 thru C3072. Generation of the C3072 extends the time of the RT to more than an hour though. This is because the RegridWeight call happens in serial (even if the code were modified to run on more than 1 PET).

One note: All the mapped ocean masks currently in the fix directory (ie, /scratch1/NCEPDEV/global/glopara/fix/orog/20240917/C1152/ocean_mask/025) were not generated by me. I believe they were generated by Sanath, using his locally modified version of the cpld-gridgen. This must be the case, because until #993, only "matched" combinations of atm and ocean resolutions were produced. I've checked what this PR is creating vs those ocean_mask files and there are B4B differences, but I do not believe them to be consequential (O~10-14).

[GeorgeGayno-NOAA]

To reduce the RT wall clock time, do we need to test every combination? Can we just do a subset? If I set up the test as follows:

# TEST_NAME | ATMRESLIST

  mx025 | 1152
  mx050 | 768
  mx100 | 192,384
  mx500 | 48,96
  mx900 | 12,24

Everything runs in about 24 minutes:

run_025.log:real        17m40.759s
run_050.log:real        5m27.939s
run_100.log:real        1m27.381s
run_500.log:real        0m7.415s
run_900.log:real        0m4.461s

[GeorgeGayno-NOAA]

Another way to speed things up? Could the mx??? tests be run in parallel? Most UFS_UTILS tests run that way. For example, see the chgres_cube driver scripts. All tests are submitted to the queue. Then, there is a job that waits for all tests to complete, then prints a summary.

[DeniseWorthen]

@GeorgeGayno-NOA Regarding running in parallel, I'm still relying on the intial rt.sh functionality that Minsuk put in. I don't really have the time to better develop a RT testing framework for cpld-gridgen. Do you have someone on your side who could take that on?

In terms of what to generate, there are two really issues I think. One, is a generating a sufficient baseline to test PRs against---that doesn't have to be every combination. A second is the production of "things" needed to test various workflows (like the post-weights). Right now, we're populating the fix files w/ the results of the baseline RTs, which produces the cpld fix files required for any combination up to C1152. That gives workflows a simple place to grab anything they might need (eg post weights for C192mx025). I don't know if that is the right system though. Should GW be asked to generate what they need themselves?

The simplest solution is to leave C3072 off and maintain what we currently have--the RTs would run in the same time

mx025 | 48,96,192,384,768,1152 
(same for 050,100)
mx500 | 12,24
mx900 | 12,24

[GeorgeGayno-NOAA]

@GeorgeGayno-NOA Regarding running in parallel, I'm still relying on the intial rt.sh functionality that Minsuk put in. I don't really have the time to better develop a RT testing framework for cpld-gridgen. Do you have someone on your side who could take that on?

I was just curious it that is feasible. Maybe I can find time to work on it.

In terms of what to generate, there are two really issues I think. One, is a generating a sufficient baseline to test PRs against---that doesn't have to be every combination. A second is the production of "things" needed to test various workflows (like the post-weights). Right now, we're populating the fix files w/ the results of the baseline RTs, which produces the cpld fix files required for any combination up to C1152. That gives workflows a simple place to grab anything they might need (eg post weights for C192mx025). I don't know if that is the right system though. Should GW be asked to generate what they need themselves?

The simplest solution is to leave C3072 off and maintain what we currently have--the RTs would run in the same time

mx025 | 48,96,192,384,768,1152 
(same for 050,100)
mx500 | 12,24
mx900 | 12,24

I have no problem asking people to generate what they need. i.e., the rt.conf file would be setup for the regression testing and users can adjust it for their particular project. But let's just keep what we have for now. Can you baseline what you have above?

[DeniseWorthen]

@GeorgeGayno-NOAA Sure, I will create baselines. Are the new C12 and C24 fix available everywhere (they weren't on hercules when I looked earlier today).