refactor: Python Inspector Overhaul Part 1 - Buildsystem migration #210

heliocastro · 2025-03-27T21:29:49Z

Migration to Astral Uv to modernize projcet buildsystem
Use Ruff and Mypy as sole unique linters. This allows to replace old code style checkers and centralize configuration only on pyproject.toml
Add pre-commit capability
Update README to Github markdown and reflect buildsystem changes
Add new CI testing and publishing

pombredanne

@heliocastro Thanks for the push, but there are way too many changes at once mixed in one PR here. This is too hard to review and impossible to merge as it is. We would need to have smaller PRs focused on one major change at a time, like introducing one tool at a time not many at once.

pombredanne · 2025-03-27T22:05:07Z

.github/workflows/testing.yml

+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.10", "3.11", "3.12", "3.13"]


Why drop the supported Python 3.9 version?

For this one, i expected that we will not be able to fully release in a month or so, and Python 3.9 is EOL soon.
But i reverted to use 3.9 as well.

pombredanne · 2025-03-27T22:09:08Z

.pre-commit-config.yaml

@@ -0,0 +1,30 @@
+default_language_version:


Introducing pre-commit should be its own PR

This is not mandatory and is not turned on by default, so no harm.
I really appreciate if we keep it to follow the README.

pombredanne · 2025-03-27T22:12:50Z

.github/workflows/testing.yml

@@ -0,0 +1,27 @@
+name: Run Pytest with Astral Uv


We run tests using a common azure pipelines and we are not using GH workflows for running tests. The base script would need to be updated in the https://github.com/aboutcode-org/skeleton first and ensure that this works fine.
At this stage all tests are failing also. See https://dev.azure.com/nexB/python-inspector/_build/results?buildId=15693&view=results

And there we run on linux, windows and mac in all cases.

I know, i know.
But now i'm on chicken and egg problem. I can't change skeleton without be approved and not break other projects, and at same time i need to see tests working.

Having the tests on Github cause no harm, so we can keep it.
If i have an easy way to update the skeleton without break the rest of the projects and i can effective use on the PR, would be fine.

pombredanne · 2025-03-27T22:15:34Z

etc/scripts/test_utils_pip_compatibility_tags.py

@@ -30,7 +30,7 @@

 import pytest

-import utils_pip_compatibility_tags


Please do not modify the code in etc/script, this is not code that is part of python-inspector but utility code from the skeleton. Furthermore, this is not a python module so do not add an __init__.py Note that these tests are not running in this repo with our pytest config, by design.

Yes, but without that, tests fail.

Removed init files and used pythonpath direct on pytest

pombredanne · 2025-03-27T22:16:40Z

tests/data/default-url-expected.json

@@ -6,7 +6,7 @@
    "options": [
      "--json <file>",
      "--operating-system linux",
-      "--python-version 38",
+      "--python-version 39",


Please do not change the Python default version in this PR, this should happen elsewhere, in another PR.

Actually, as i dropped support for 3.8, this is actually correct

pombredanne · 2025-03-27T22:42:38Z

uv.lock

+[[package]]
+name = "aboutcode-toolkit"
+version = "11.0.0"
+source = { registry = "https://jfrog.hub.vwgroup.com/artifactory/api/pypi/optima-pypi-virtual/simple" }


This is unlikely something anyone can use publicly?

No no, was wrong registry on lock generation. Fixed already

pombredanne · 2025-03-27T22:44:43Z

pyproject.toml

@@ -1,52 +1,227 @@
 [build-system]
-requires = ["setuptools >= 50", "wheel", "setuptools_scm[toml] >= 6"]
-build-backend = "setuptools.build_meta"
+requires = ["hatchling"]


I would rather not change the build backend mixed with other things. Make it its own PR.

Moved back to setuptools.

pombredanne · 2025-03-27T22:46:06Z

pyproject.toml

+    "Topic :: Software Development :: Libraries :: Python Modules",
+]
+dependencies = [
+    "aiofiles>=24.1.0",


Why introduce new dependencies in this PR? I cannot see where they would be used. I'd like to see something simple like a basic conversion from setup to pyproject with no other unrelated changes, otherwise this is hard to review

This came from the different branch, i removed all unnecessary extra deps

pombredanne · 2025-03-27T22:47:41Z

pyproject.toml


 [tool.pytest.ini_options]
-norecursedirs = [


These are needed. Why remove these?

Instead of doing huge multiple exclusions, used only testpath for the two required directories

pombredanne · 2025-03-27T22:49:07Z

pyproject.toml

-python_classes = "Test"
-python_functions = "test"
-
-addopts = [


These options are needed.

Added back some opts, the python_classed and python_functions are not needed as default from pytest

heliocastro · 2025-03-28T08:20:56Z

@heliocastro Thanks for the push, but there are way too many changes at once mixed in one PR here. This is too hard to review and impossible to merge as it is. We would need to have smaller PRs focused on one major change at a time, like introducing one tool at a time not many at once.

This one is already minimal.

We will need to do some compromise here, we're talking in substantial changes. This is the minimal required to get the base changed and then we can do simple changes.

But please, bear with me on this one.

Replace setup tools and configure to unified Uv with pyproject based setup. Set minimum default python to 3.9 as python 3.8 is EOL. Project file pyproject.toml now replaced all linters for a unified solution using ruff and mypy and integrated pytest configurations. Signed-off-by: Helio Chissini de Castro <helio.chissini.de.castro@cariad.technology>

- Regenerated tests with REGEN_TEST_FIXTURES - Updated tests to proper import internal code - Added testing Ci check for Github Signed-off-by: Helio Chissini de Castro <helio.chissini.de.castro@cariad.technology>

Signed-off-by: Helio Chissini de Castro <helio.chissini.de.castro@cariad.technology>

heliocastro force-pushed the refactor/cariad_uv branch 5 times, most recently from fadbe3b to 42e0bb7 Compare March 27, 2025 22:21

pombredanne reviewed Mar 27, 2025

View reviewed changes

heliocastro force-pushed the refactor/cariad_uv branch 4 times, most recently from 1101214 to a4ed493 Compare March 28, 2025 08:17

heliocastro force-pushed the refactor/cariad_uv branch 7 times, most recently from 3cb7e5c to c8c83ea Compare March 28, 2025 12:09

heliocastro force-pushed the refactor/cariad_uv branch from c8c83ea to b871c68 Compare March 28, 2025 12:19

heliocastro added 2 commits March 28, 2025 13:57

refactor: Update tests

ec10f85

- Regenerated tests with REGEN_TEST_FIXTURES - Updated tests to proper import internal code - Added testing Ci check for Github Signed-off-by: Helio Chissini de Castro <helio.chissini.de.castro@cariad.technology>

fix: Adjust code block for Readme

4796257

Signed-off-by: Helio Chissini de Castro <helio.chissini.de.castro@cariad.technology>

heliocastro force-pushed the refactor/cariad_uv branch from b871c68 to 4796257 Compare March 28, 2025 12:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: Python Inspector Overhaul Part 1 - Buildsystem migration #210

refactor: Python Inspector Overhaul Part 1 - Buildsystem migration #210

heliocastro commented Mar 27, 2025 •

edited by mjherzog

Loading

pombredanne left a comment •

edited by mjherzog

Loading

pombredanne Mar 27, 2025

heliocastro Mar 28, 2025

pombredanne Mar 27, 2025

heliocastro Mar 28, 2025

pombredanne Mar 27, 2025

heliocastro Mar 28, 2025

pombredanne Mar 27, 2025

heliocastro Mar 28, 2025

heliocastro Mar 28, 2025

pombredanne Mar 27, 2025

heliocastro Mar 28, 2025

heliocastro Mar 28, 2025

pombredanne Mar 27, 2025

heliocastro Mar 28, 2025

pombredanne Mar 27, 2025

heliocastro Mar 28, 2025

pombredanne Mar 27, 2025

heliocastro Mar 28, 2025

pombredanne Mar 27, 2025

heliocastro Mar 28, 2025 •

edited

Loading

pombredanne Mar 27, 2025

heliocastro Mar 28, 2025

heliocastro commented Mar 28, 2025

		@@ -30,7 +30,7 @@

		import pytest

		import utils_pip_compatibility_tags


		[tool.pytest.ini_options]
		norecursedirs = [

refactor: Python Inspector Overhaul Part 1 - Buildsystem migration #210

Are you sure you want to change the base?

refactor: Python Inspector Overhaul Part 1 - Buildsystem migration #210

Conversation

heliocastro commented Mar 27, 2025 • edited by mjherzog Loading

pombredanne left a comment • edited by mjherzog Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

heliocastro Mar 28, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

heliocastro commented Mar 28, 2025

heliocastro commented Mar 27, 2025 •

edited by mjherzog

Loading

pombredanne left a comment •

edited by mjherzog

Loading

heliocastro Mar 28, 2025 •

edited

Loading