about logging

[daviddoji]

If I understood correctly from yesterday's discussion, your suggestions for having a log-like file (metadata as you call it) is that after one has figure out the parameters (or partial results) that have to be tuned for the analysis (tunning steps), you could run exactly the same steps in a different dataset for the same experiment.

If that assumption is correct, I can see that to do a "systematic" analysis of different dataset, one would need to run steps that are not neccessary for the final result (tunning steps, as described before).

Would you consider the possibility of having a "playground" notebook that could save those "partial" results and, once you're happy with them, execute a "master" script/notebook that will use those saved "tunning" parameters?

I have implemented this idea previously using nbparemeterise and users were quite happy with it.

[rettigl]

That's something I did not know, but does not really solve the problem that we want to document the actions on the data within the data item
Here are some approaches:
https://stackoverflow.com/questions/5103735/better-way-to-log-method-calls-in-python

[daviddoji]

Thanks for the link. It doesn't seem to be complicated to implement the logger but it can easily become a rabbit hole.
And not only one would have to implement the logger functionality, but also a parser that transforms those logs into something useful for the user afterwards.

The idea I'm proposing is way much simpler, as one can generate an html or pdf document with the execution of the script, once you're happy with the results you have previously obtained using a "playground" template.

Honestly, either I'm missing some details here or I still can't see the benefits of getting all the actions logged when one can easily get the output of the analysis in a more user-friendly way.

From a reproducibility point of view, having a fix kernel and the same input parameters, the output must be the same. That's the approach followed by the 'calibration' team at XFEL.

Why don't we start with the simplest approach?

[zain-sohail]

I agree with your idea and simplicity, David.

The idea is to publish the data, as well as all other "meta" data on zenodo for example. For the experimental metadata, there is already a mpes-nexus format. What we would like is to include however we process the data within this structure as well (just an nexus h5 file). If you have a way we could include the notebook there, perhaps that could work too.

[steinnymir]

This notebook based approach looks interesting, and I can see that users might very much like this.
The reason why we are insisting so much on logging the processing steps in hte final data container is due to many discussions we have had across the community. The idea of using notebooks was proposed but we found it leaves still too much room for the user to make "mistakes". Logging the analysis steps without user intervention was deemed as a better way to ensure reproducibility, even when the analysis was performed by different users.

[daviddoji]

The use of the notebook approach is a sort of logging.
A user only has to define parameters and the run the notebook. It does automatically generate an html where all the steps are transparent and, if you need to change anything, you just re-execute the notebook and a new html file is generated.