Use pooch for testdata management, modernize

[Zeitsperre]

Overview

Changes:

• Refactored the testing setup to make use of the clisops testing utilities and pytest fixtures.
• Removed gitpython: mini-esgf-data is now managed by the clisops setup code.
• Gave the package the birdhouse-rook name in preparation for a release of this library on PyPI
• Modified the pre-commit and linting steps to rely solely on ruff (faster, more modern).
• Moved the requirement files into the pyprojec.toml file.
• Sorted imports and removed lots of dead code.

Related Issue / Discussion

Some of these changes are being made to ensure that projects using the mini-esgf-data repository can make use of the new layout using pooch (see the restructuring/v1 branches). Once this has been done, we can start using a versioned approach to test data collection and management (with pooch and the stratus class from clisops).

I've also renamed the library to birdhouse-rook (rook was taken on PyPI). This is no different from a few other birdhouse projects that have been added to PyPI (birdhouse-raven, birdhouse-finch). This name does not impact the existing module name (still rook), so no changes will be needed when using it as a library.

Additional Information

@cehbrecht

My changes seem to have affected some of the configuration defaults. My theory is that there are some hard-coded filepaths that are causing the FileNotFound errors. If you could point me in the right direction, I would be willing to make the final adjustments here.

It also appears that the db code is fully deprecated. We will need to update the SQL handling here to be able to support modern daops and pandas.

[cehbrecht]

My changes seem to have affected some of the configuration defaults. My theory is that there are some hard-coded filepaths that are causing the FileNotFound errors. If you could point me in the right direction, I would be willing to make the final adjustments here.

@Zeitsperre the rook tests are using a generated roocs.ini file pointing to the directory with the test data. This should always overwrite the defaults in roocs_utils:

rook/tests/common.py

Line 22 in f9d0638

def write_roocs_cfg():

[cehbrecht]

It also appears that the db code is fully deprecated. We will need to update the SQL handling here to be able to support modern daops and pandas.

Still using pandas < 2.0 and sqlalchemy < 2.0. I probably also need to update the intake catalogs which we are using. And the CDS needs to update as well. Maybe we can handle this in separate PR series.

[Zeitsperre]

@cehbrecht Agreed. I think that some of the work in here can be reused, but there's a lot to sort out first with the underlying libraries. My priorities are to get roocs-utils into CLISOPS and then clisops.core into its own library.

Once the dust has settled, it'll be much easier to bring this service up to speed.