Add efcs and epos to ufs_hybatm xml (#3192) #3193

RussTreadon-NOAA · 2024-12-27T14:02:17Z

Description

g-w PR #3185 inadvertently removed metatasks enkfgdas_fcst and enkfgdas_epmn from the ufs_hybatmDA xml. This PR modifies workflow/applications/gfs_cycled.py so that these metatasks are once again in the ufs_hybatmDA xml.

Resolves: #3192

Type of change

Bug fix (fixes something broken)

Change characteristics

Does this change require a documentation update? NO
Does this change require an update to any of the following submodules? NO

How has this been tested?

clone and install bugfix/ufs_hybatmDA on Dogwood
set up g-w CI C96C48_ufs_hybatmDA by executing $HOMEgfs/workflow/create_experiment.py
open C96C48_ufs_hybatmDA xml
confirm that metatasks enkfgdas_fcst and enkfgdas_epmn are defined

Checklist

My code follows the style guidelines of this project
I have performed a self-review of my own code
My changes generate no new warnings
New and existing tests pass with my changes
This change is covered by an existing CI test or a new one has been added

aerorahul

looks good to me.

WalterKolczynski-NOAA · 2024-12-27T19:17:16Z

I think the earc dependencies will need to be added back as well.

RussTreadon-NOAA · 2024-12-27T22:07:17Z

@WalterKolczynski-NOAA , feel free add the earc changes to my branch. Not sure if this will work, but give it a try. If it fails, point me at your earc changes and I'll add them. Alternatively, you can open a new issue and PR and pull my changes into your PR.

WalterKolczynski-NOAA · 2024-12-27T22:32:04Z

I'm on leave until Jan 2, but you can see what #3185 took out of gfs_tasks.py and put it back in.

RussTreadon-NOAA · 2024-12-27T22:47:31Z

No rush. We can wait until after everyone returns from leave.

RussTreadon-NOAA · 2024-12-28T19:25:37Z

Limited g-w CI on Hera

Ran g-w CI for C48mx500_3DVarAOWCDA, C96C48_hybatmaerosnowDA, and C96C48_ufs_hybatmDA on Hera from an installation of g-w bugfix/ufs_hybatmDA at 5acc46c.

All jobs in C48mx500_3DVarAOWCDA and C96C48_ufs_hybatmDA ran to completion.

The 20211220 18Z C96C48_hybatmaerosnowDA enkfgdas_earc01 failed with the error message

  File "/scratch1/NCEPDEV/da/Russ.Treadon/git/global-workflow/test/parm/archive/enkf_restartb_grp.yaml.j2", line 41, in template
    {% endfor %}
^^^^^^^^^^^^^^^^^
jinja2.exceptions.TemplateSyntaxError: Encountered unknown tag 'endfor'. You probably made a nesting mistake. Jinja is expecting this tag, but currently looking for 'elif' or 'else' or 'endif'. The innermost block that needs to be closed is 'if'.
+ JGDAS_ENKF_ARCHIVE[1]: postamble JGDAS_ENKF_ARCHIVE 1735397434 1

The line 41 {% endfor %} is one line before the end of enkf_restartb_grp.yaml.j2

        {% endfor %}
        {% endfor %}
        {% endif %}

This is not correct. The template needs to end with {% endfor %} to close the top-level loop over members

nkf_restartb_grp:
    name: "ENKF_RESTARTB_GRP"
    target: "{{ ATARDIR }}/{{ cycle_YMDH }}/{{ RUN }}_restartb_grp{{ ENSGRP }}.tar"
    required:
        {% for mem in range(first_group_mem, last_group_mem + 1) %}

Make the following change to parm/archive/enkf_restartb_grp.yaml.j2

@@ -38,5 +38,5 @@ enkf_restartb_grp:
         - "{{ COMIN_ATMOS_RESTART_MEM | relpath(ROTDIR) }}/{{ r_prefix }}.coupler.res"
         - "{{ COMIN_ATMOS_RESTART_MEM | relpath(ROTDIR) }}/{{ r_prefix }}.fv_core.res.nc"
         {% endfor %}
-        {% endfor %}
         {% endif %}
+        {% endfor %}

With this change in place, the 20211220 18Z enkfgdas_earc01 successfully ran to completion.

…EMC#3192)

TravisElless-NOAA · 2024-12-30T15:47:05Z

@RussTreadon-NOAA Apologies for the inconvenience my PR errors caused and thank you for your efforts to correct them.

One suggestion: instead of adding the lines to the top half of the do_jediatmens if statement, we can just pull the lines #3185 added in the bottom half of the if statement outside of the if statement all together.

The earc dependencies are still fine and won't have to be updated. Changes to the dependencies in #3185 focused on the requirements between the early- and late-cycle. This PR focuses on restoring only late-cycle jobs for the UFS DA configuration and maintains that the UFS DA will still have the fcst and pos jobs turned off for the early-cycle.

)

RussTreadon-NOAA · 2025-01-02T11:59:17Z

Requested changes have been made. Once everyone is OK with the changes in this PR, we should run run g-w CI on Hera to ensure no unintended consequences.

RussTreadon-NOAA · 2025-01-02T16:46:55Z

Thank you @TravisElless-NOAA . I'll start g-w CI on Hera once I receive confirmation (approval) from the g-w team that the changes in this PR are OK with them. We don't want to run CI multiple times. Hence my waiting.

aerorahul

lgtm

RussTreadon-NOAA · 2025-01-02T17:06:54Z

Thank you @aerorahul for the approval and triggering Hera and Hercules CI.

emcbot · 2025-01-02T22:19:52Z

CI Passed on Hercules in Build# 1
Built and ran in directory /work2/noaa/global/CI/HERCULES/3193


Experiment C48_ATM_e93036d4 Completed 2 Cycles: *SUCCESS* at Thu Jan  2 12:46:27 CST 2025
Experiment C48mx500_3DVarAOWCDA_e93036d4 Completed 2 Cycles: *SUCCESS* at Thu Jan  2 12:52:30 CST 2025
Experiment C48mx500_hybAOWCDA_e93036d4 Completed 2 Cycles: *SUCCESS* at Thu Jan  2 12:58:39 CST 2025
Experiment C96C48_hybatmDA_e93036d4 Completed 3 Cycles: *SUCCESS* at Thu Jan  2 13:53:20 CST 2025
Experiment C96_S2SWA_gefs_replay_ics_e93036d4 Completed 1 Cycles: *SUCCESS* at Thu Jan  2 13:53:42 CST 2025
Experiment C96_atm3DVar_e93036d4 Completed 3 Cycles: *SUCCESS* at Thu Jan  2 13:59:12 CST 2025
Experiment C48_S2SW_e93036d4 Completed 2 Cycles: *SUCCESS* at Thu Jan  2 14:18:09 CST 2025
Experiment C48_S2SWA_gefs_e93036d4 Completed 1 Cycles: *SUCCESS* at Thu Jan  2 16:19:42 CST 2025

emcbot · 2025-01-02T22:20:16Z

CI Passed on Hera in Build# 2
Built and ran in directory /scratch1/NCEPDEV/global/CI/3193


Experiment C48mx500_3DVarAOWCDA_e93036d4 Completed 2 Cycles: *SUCCESS* at Thu Jan  2 19:19:31 UTC 2025
Experiment C48mx500_hybAOWCDA_e93036d4 Completed 2 Cycles: *SUCCESS* at Thu Jan  2 19:19:31 UTC 2025
Experiment C48_ATM_e93036d4 Completed 2 Cycles: *SUCCESS* at Thu Jan  2 19:25:46 UTC 2025
Experiment C96_S2SWA_gefs_replay_ics_e93036d4 Completed 1 Cycles: *SUCCESS* at Thu Jan  2 19:33:19 UTC 2025
Experiment C96C48_hybatmaerosnowDA_e93036d4 Completed 3 Cycles: *SUCCESS* at Thu Jan  2 20:42:56 UTC 2025
Experiment C96C48_hybatmDA_e93036d4 Completed 3 Cycles: *SUCCESS* at Thu Jan  2 20:42:57 UTC 2025
Experiment C96_atm3DVar_e93036d4 Completed 3 Cycles: *SUCCESS* at Thu Jan  2 20:52:16 UTC 2025
Experiment C96C48_ufs_hybatmDA_e93036d4 Completed 3 Cycles: *SUCCESS* at Thu Jan  2 21:29:12 UTC 2025
Experiment C48_S2SW_e93036d4 Completed 2 Cycles: *SUCCESS* at Thu Jan  2 21:36:54 UTC 2025
Experiment C48_S2SWA_gefs_e93036d4 Completed 1 Cycles: *SUCCESS* at Thu Jan  2 22:14:57 UTC 2025

RussTreadon-NOAA · 2025-01-02T23:50:38Z

@WalterKolczynski-NOAA : Hera and Hercules g-w CI pass. Is this sufficient or do we also need to run g-w CI on WCOSS2 (Dogwood)?

RussTreadon-NOAA · 2025-01-03T01:27:00Z

Thank you @aerorahul

* develop: Ensure OCNRES and ICERES have 3 digits in the archive script (NOAA-EMC#3199) Set runtime shell requirements within Jenkins Pipeline (NOAA-EMC#3171) Add efcs and epos to ufs_hybatm xml (NOAA-EMC#3192) (NOAA-EMC#3193) Fix GEFS and SFS compile flags in build_all.sh (NOAA-EMC#3197) Remove early-cycle EnKF forecast (NOAA-EMC#3185) Fix mod_icec bug in atmos_prod (NOAA-EMC#3167) Create compute build option (NOAA-EMC#3186) Support global-workflow using Rocky 8 on CSPs (NOAA-EMC#2998)

add efcs and epos in ufs_hybatm xml (NOAA-EMC#3192)

5acc46c

RussTreadon-NOAA requested review from WalterKolczynski-NOAA, aerorahul and DavidHuber-NOAA as code owners December 27, 2024 14:02

RussTreadon-NOAA self-assigned this Dec 27, 2024

RussTreadon-NOAA requested a review from TravisElless-NOAA December 27, 2024 14:02

aerorahul previously approved these changes Dec 27, 2024

View reviewed changes

correct misplaced endfor in enkf_restartb_grp archive template (NOAA-…

61cf3e1

…EMC#3192)

RussTreadon-NOAA dismissed aerorahul’s stale review via 61cf3e1 December 28, 2024 19:31

simplify addition of efcs and epos to do_jediatmens block (NOAA-EMC#3192

e93036d

)

TravisElless-NOAA approved these changes Jan 2, 2025

View reviewed changes

aerorahul approved these changes Jan 2, 2025

View reviewed changes

aerorahul added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules labels Jan 2, 2025

emcbot added CI-Hercules-Running **Bot use only** CI testing on Hercules for this PR is in-progress CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress and removed CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules labels Jan 2, 2025

emcbot added CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully and removed CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Jan 2, 2025

Merge branch 'develop' into bugfix/ufs_hybatmDA

29f1acf

aerorahul merged commit da9d717 into NOAA-EMC:develop Jan 3, 2025
5 checks passed

AntonMFernando-NOAA mentioned this pull request Jan 3, 2025

Remove multi-grid wave support #3188

Merged

16 tasks

RussTreadon-NOAA deleted the bugfix/ufs_hybatmDA branch January 6, 2025 20:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add efcs and epos to ufs_hybatm xml (#3192) #3193

Add efcs and epos to ufs_hybatm xml (#3192) #3193

RussTreadon-NOAA commented Dec 27, 2024 •

edited

Loading

aerorahul left a comment

WalterKolczynski-NOAA commented Dec 27, 2024

RussTreadon-NOAA commented Dec 27, 2024

WalterKolczynski-NOAA commented Dec 27, 2024

RussTreadon-NOAA commented Dec 27, 2024

RussTreadon-NOAA commented Dec 28, 2024

TravisElless-NOAA commented Dec 30, 2024

RussTreadon-NOAA commented Jan 2, 2025

RussTreadon-NOAA commented Jan 2, 2025

aerorahul left a comment

RussTreadon-NOAA commented Jan 2, 2025

emcbot commented Jan 2, 2025

emcbot commented Jan 2, 2025

RussTreadon-NOAA commented Jan 2, 2025

RussTreadon-NOAA commented Jan 3, 2025

Add efcs and epos to ufs_hybatm xml (#3192) #3193

Add efcs and epos to ufs_hybatm xml (#3192) #3193

Conversation

RussTreadon-NOAA commented Dec 27, 2024 • edited Loading

Description

Type of change

Change characteristics

How has this been tested?

Checklist

aerorahul left a comment

Choose a reason for hiding this comment

WalterKolczynski-NOAA commented Dec 27, 2024

RussTreadon-NOAA commented Dec 27, 2024

WalterKolczynski-NOAA commented Dec 27, 2024

RussTreadon-NOAA commented Dec 27, 2024

RussTreadon-NOAA commented Dec 28, 2024

TravisElless-NOAA commented Dec 30, 2024

RussTreadon-NOAA commented Jan 2, 2025

RussTreadon-NOAA commented Jan 2, 2025

aerorahul left a comment

Choose a reason for hiding this comment

RussTreadon-NOAA commented Jan 2, 2025

emcbot commented Jan 2, 2025

emcbot commented Jan 2, 2025

RussTreadon-NOAA commented Jan 2, 2025

RussTreadon-NOAA commented Jan 3, 2025

RussTreadon-NOAA commented Dec 27, 2024 •

edited

Loading