Improving docs about available arguments in hooks

[maxschulz-COL]

Description

I am currently teaching myself hooks (love them), but I find the docs not ideal in terms of variable access.

Documentation page (if applicable)

https://docs.kedro.org/en/stable/hooks/introduction.html

Take the following code example:

@hook_spec
def after_catalog_created(
    self,
    catalog: DataCatalog,
    conf_catalog: Dict[str, Any],
    conf_creds: Dict[str, Any],
    save_version: str,
    load_versions: Dict[str, str],
) -> None:
    pass

On this page, I quickly understand that I can have access to those variables, although they could be described a bit better here, but I roughly get what they are.

I was then unable to find the accessible arguments for all other hooks. In the end I went to the specs.py in the source code, and checked myself, but that was quite a bother.

Improvement

So my improvement suggestion would be to link the above described list of hooks to their actual specs, and maybe have a few more simple examples. If this is already described somewhere, I was unable to find it.

[noklam]

Totally agree, this was a bit of pain to me too. It's actually more like API docs so you can find the info here: https://docs.kedro.org/en/stable/api/kedro.framework.hooks.specs.html#module-kedro.framework.hooks.specs

Though I would think that people often look for hooks spec so it should be in a single page that is easy to search. Because of how it's structure, it's spread across different class, and thus different page in the API docs.

[noklam]

^ I believe there should be some way to re-organise those API docs automatically with sphinx

[maxschulz-COL]

Indeed, as soon as I posted the issue I realised it must be somewhere in API docs, but at the very least I would link from the intro page to them!

But you are right, on one page would also be great, because while in source code, I was able to search for e.g. session_id to see which hooks have access to that!

[noklam]

If you would like to raise a PR, feel free to go ahead.

[rashidakanchwala]

@astrojuanlu , please let us know if the priority of this needs to change.

[maxschulz-COL]

What may also help is just widely extending available args in hooks - basically make all arguments that can be available in a hook available. Not sure if that's the case, but I seem to be missing always one argument in every hook :P

[astrojuanlu]

I'm not sure I understood @maxschulz-COL but just in case, remember that in pluggy you don't need to declare all arguments of a hook implementation https://pluggy.readthedocs.io/en/stable/#opt-in-arguments

@astrojuanlu , please let us know if the priority of this needs to change.

This is still Medium priority for us but anybody is welcome to send a PR 🙏🏼

[maxschulz-COL]

I'm not sure I understood @maxschulz-COL but just in case, remember that in pluggy you don't need to declare all arguments of a hook implementation https://pluggy.readthedocs.io/en/stable/#opt-in-arguments

@astrojuanlu , please let us know if the priority of this needs to change.

This is still Medium priority for us but anybody is welcome to send a PR 🙏🏼

I know, but it felt that I was always missing the argument I needed, so unless specific things are not available at a certain time, why not allow the maximum of arguments for all hooks, precisely because you don't need to declare all

[astrojuanlu]

Ah got it. So the request is to pass everything in every hook.

I don't know if there are any lifecycle constraints to do so. This would be a separate feature request in any case.

[datajoely]

+1 , long overdue, also the slightly unusual behaviour that not all arguments have to be provided should be covered too