Alternative Fortran parsing implementation

[mwaxmonsky]

Description

Currently there are several chunks of regular expressions in parse_fortran_file.py and parse_checkers.py that work with our current input files that parse key details of the Fortran code without being a full fledged parser that generates an AST.

There are currently several comments documenting cases the regular expressions don't support and I suspect as we open up the CCPP to the greater community, there will either be more modifications/additions that we will need to added and without any restrictions on the Fortran code being parsed, these will get very complicated fairly quickly.

Solution

This may not be an issue or a priority today but I would like to open the discussion on our options and whether we begin discussing/documenting the Fortran code format that the CCPP will support with our regular expressions or we start evaluating third party libraries such as https://github.com/stfc/fparser/tree/master which is maintained by the UK Science and Technology Facilities Council (https://www.scd.stfc.ac.uk/Pages/About-Us.aspx) or https://github.com/rose-compiler/rose from LLNL (albeit this is a little more heavy handed considering it does far more than just parsing). We have reached out the fortls team to see if they support this as well and will update when we hear back if they have this functionality as well.

Alternatives (optional)

There is also the option to dump the internal syntax tree from gfortran directly
using https://gcc.gnu.org/onlinedocs/gfortran/Developer-Options.html#index-fdump-fortran-original (example :https://godbolt.org/z/TjrKzj9Wz):

integer function square(x)
    implicit none
    integer, intent(in) :: x
    square = x * x
end function square

and calling gfortran produces:

$ gfortran -g -fdump-fortran-original
Namespace: A-Z: (UNKNOWN 0)
procedure name = square
  symtree: 'square'      || symbol: 'square'       
    type spec : (INTEGER 4)
    attributes: (PROCEDURE  FUNCTION)
    result: square
    Formal arglist: x
  symtree: 'x'           || symbol: 'x'            
    type spec : (INTEGER 4)
    attributes: (VARIABLE  DUMMY(IN))

  code:
  ASSIGN square:square (* square:x square:x)

But then we need to have a parser of GNUs internal AST printed to stdout which brings us back to square one unfortunately.

Related to (optional)

N\A

[gold2718]

I think this is a great idea and worth exploring (again). When I first started working on capgen (more than 6 years ago (!)) I wanted to just grab a parser and start from there. I don't remember if I looked at Rose but did look at some LLVM-based projects. In the end, I did not want to have to build something just to run the CCPP Framework. I also looked at a Python-based Fortran parser but it was written by someone who did not support it at all and I found that code too hard to work with. I did not know about the fparser project but it was really new back then anyway and probably not ready for prime time.

I still think it is important to not up the requirements for running the framework so to me, a Python-based tool seems best.

[mwaxmonsky]

Thanks all!

@gold2718 I was definitely leaning towards the fparser project as well since a simple pip install into a virtual environment is far easier than the other approaches. Plus it's nice that it's already in Python. 😄

[climbfuji]

It should also be trivial to include fparser in spack-stack, in which case 90% of our current users don't need to worry about it at all. But we need to make sure that whatever solution we choose can get approval for installation on FISMA high platforms. We should do this before we even start implementing it in ccpp-framework.

[mwaxmonsky]

Thanks for the heads up @climbfuji, do you know who we can talk to or what the process is to start the conversation on that front?

[climbfuji]

We should open an issue in spack-stack for adding fparser as a dependency. That will automatically alert EMC folks and start the process. If you open an "install request" issue, you'll see that there is an entire section devoted to WCOSS2 (which is a FISMA high system): https://github.com/JCSDA/spack-stack/issues/new?assignees=Hang-Lei-NOAA%2CAlexanderRichert-NOAA%2Cjkbk2004%2Cclimbfuji&labels=install&projects=&template=install_request.yml&title=%5BINSTALL%5D%3A+

[mwaxmonsky]

Would it be worthwhile to create a proof of concept before we start asking for adding a spack package to the stack? It's already a spack package so it might not be the worst idea to ask to add it even if we move to a different solution.