Improve Kedro onboarding experience by making uv primary tool in our documentation and READMEs

[astrojuanlu]

Discussed in #4453

We are seeing strong user signals that uv is gaining traction in the community:

• Questions related to setting up Kedro with uv (see original discussion)
• Lots of people mentioning uv in the 1.0 survey (final data TBC)
• Adjacent products adopting it to great success
  • MLflow adopted uv in the 2.2.0 version to manage their model lifecycle https://github.com/mlflow/mlflow/releases/tag/v2.20.0
  • meltano SDK also migrated from Poetry to uv https://sdk.meltano.com/en/v0.44.2/guides/migrate-to-uv.html
  • Cookiecutter Data Science 2.1 added support for uv Proposal to add UV as dependency manager option drivendataorg/cookiecutter-data-science#403
• In a bit less than a year uv surpassed Hatch, PDM, Pipenv in monthly downloads, and is only behind Poetry https://clickpy.clickhouse.com/dashboard/uv and this only accounts for PyPI downloads! Looks like they have ~10s of millions of downloads on GitHub releases (see example)

By leveraging uv, we can tell our users to run kedro new with uvx (like we would with pipx) and therefore softening or mitigating #681 , which was identified as one of the most tricky onboarding problems when designing the VS Code walkthrough kedro-org/vscode-kedro#170

The idea here is to make uv the primary workflow tool in our documentation, replacing the current mix of pip + venv + conda. This includes

• uvx kedro new for project creation (no more "double venv", see Run kedro new without creating a new directory #681)
• uv run kedro run, no more venv activation/deactivation

(We can retain a passing mention of conda in the introduction just to make sure folks are aware it can be used)

[datajoely]

The world has changed a great deal since Kedro was initially designed ~2017, documented and released to the world in ~2019.

When it was first built internally, making sure our people were usingg venvs (via conda at the time), black, isort and a bunch of other utilities were cutting edge now. Pretty much all of those approaches have been made redundant trough the work of Charlie Marsh (the creator of uv and ruff) alone 😂 .

The case against sticking with pip:

• It is getting to the point that Kedro looks outdated not using uv anymore.
• The pain of bundling cookiecutter will be removed when users can just use uvx ephemerally.
• The double venv journey on the docs, the very first introduction in the docs, has always been silly and we're now at the point you can't argue it's even a necessary evil.
• First impressions installing Kedro + dataset dependencies massively improved through speed.
• If we started building Kedro today, this wouldn't even be an argument. It's clearly the right horse to back, after decades of people arguing over poeatry, pyenv etc. the choice is now simple.

I'm of the opinion this is a nonnegotiable for 1.0.0

[astrojuanlu]

@merelcht flags an important point: let's do this in a way that it doesn't seem like Kedro is coupled with uv, so that folks can continue using it with other tools if so they desire.

[astrojuanlu]

Early user evidence from the 1.0 survey:

Are there any missing features or improvements you would like to see in Kedro 1.0?: Yes (please describe what you'd like to see in Kedro 1.0)

• Offline project templates, default integration with uv for project mgmt
• mono repo, nested kedro repositories, (not necessary using uv workspaces, just nested environments)
• Les small bugs here and there. Improved guidelines for custom datasets. Better multiprocessing support. Full uv support

What would make Kedro 1.0 a success for you?

• Get rid of some of the "bandaid cures" that we currently have (logging not being the greatest, especially wrt kedro-airflow). Make uv the default project deps tool, or at least make it easier (i.e., roll kedro-init stuff into kedro proper).

[astrojuanlu]

And last: to clarify, the idea is to improve the Kedro onboarding thanks to uv, rather than just adding 1 more tool recommendation (our users are smart enough already to know they can uv pip install kedro)

[datajoely]

I actually am completely in favour of coupling to uv, here are some examples of where the industry and our comparators are going:

https://docs.dagster.io/guides/labs/dg
https://docs.marimo.io/guides/package_reproducibility/?h=uv

[datajoely]

Since uv is highly standards compliant coupling also doesn't have the same risk as ever pitching the same thing with poetry or the like

[datajoely]

Also on the monorepo point I saw this recently

https://chrismati.cz/posts/uv-pex-monorepo/

[astrojuanlu]

More evidence: uv jumped almost straight to "Adopt" in the Tech Radar https://www.thoughtworks.com/radar/tools/summary/uv

[datajoely]

#4648