Skip to content

Commit

Permalink
Merge branch 'main' into feature/multiclass_confusion_matrix
Browse files Browse the repository at this point in the history
# Conflicts:
#	docs/tutorials/performance_estimation/multiclass_performance_estimation.rst
  • Loading branch information
nnansters committed Nov 9, 2023
2 parents 287e49a + dd20ef7 commit 22f47a8
Show file tree
Hide file tree
Showing 58 changed files with 12,621 additions and 156 deletions.
2 changes: 1 addition & 1 deletion .bumpversion.cfg
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 0.9.0
current_version = 0.9.1
commit = True
tag = True

Expand Down
1 change: 1 addition & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,4 @@ repos:
- types-python-dateutil
- types-click
- sqlmodel
- types-requests
19 changes: 19 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,25 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.9.1] - 2023-07-12

### Changed

- Updated Mendable client library version to deal with styling overrides in the RTD documentation theme
- Removed superfluous limits for confidence bands in the CBPE class (these are present in the metric classes instead)
- Threshold value limiting behaviour (e.g. overriding a value and emitting a warning) will be triggered not only when
the value crosses the threshold but also when it is equal to the threshold value. This is because we interpret the
threshold as a theoretical maximum.

### Added

- Added a new example notebook walking through a full use case using the NYC Green Taxi dataset, based on the blog of [@santiviquez](https://github.com/santiviquez)

### Fixed

- Fixed broken Docker container build due to changes in public Poetry installation procedure
- Fixed broken image source link in the README, thanks [@NeoKish](https://github.com/NeoKish)!

## [0.9.0] - 2023-06-26

### Changed
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ RUN apt-get update && \
build-essential

# Install Poetry - respects $POETRY_VERSION & $POETRY_HOME
RUN curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py | python
RUN curl -sSL https://install.python-poetry.org | python3 - --version $POETRY_VERSION
ENV PATH="$POETRY_HOME/bin:$PATH"

# Import our project files
Expand Down
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,15 +71,15 @@ Allowing you to have the following benefits:
| 🔬 **[Technical reference]** | Monitor the performance of your ML models. |
| 🔎 **[Blog]** | Thoughts on post-deployment data science from the NannyML team. |
| 📬 **[Newsletter]** | All things post-deployment data science. Subscribe to see the latest papers and blogs. |
| 💎 **[New in v0.9.0]** | New features, bug fixes. |
| 💎 **[New in v0.9.1]** | New features, bug fixes. |
| 🧑‍💻 **[Contribute]** | How to contribute to the NannyML project and codebase. |
| <img src="https://raw.githubusercontent.com/NannyML/nannyml/main/media/slack.png" height='15'> **[Join slack]** | Need help with your specific use case? Say hi on slack! |

[nannyml 101]: https://nannyml.readthedocs.io/en/stable/
[performance estimation]: https://nannyml.readthedocs.io/en/stable/how_it_works/performance_estimation.html
[key concepts]: https://nannyml.readthedocs.io/en/stable/glossary.html
[technical reference]: https://nannyml.readthedocs.io/en/stable/nannyml/modules.html
[new in v0.9.0]: https://github.com/NannyML/nannyml/releases/latest/
[new in v0.9.1]: https://github.com/NannyML/nannyml/releases/latest/
[real world example]: https://nannyml.readthedocs.io/en/stable/examples/california_housing.html
[blog]: https://www.nannyml.com/blog
[newsletter]: https://mailchi.mp/022c62281d13/postdeploymentnewsletter
Expand All @@ -100,7 +100,7 @@ NannyML can also **track the realised performance** of your machine learning mod

To detect **multivariate feature drift** NannyML uses [PCA-based data reconstruction](https://nannyml.readthedocs.io/en/main/how_it_works/data_reconstruction.html). Changes in the resulting reconstruction error are monitored over time and data drift alerts are logged when the reconstruction error in a certain period exceeds a threshold. This threshold is calculated based on the reconstruction error observed in the reference period.

<p><img src="https://raw.githubusercontent.com/NannyML/nannyml/main/docs/_static/drift-guide-multivariate.svg"></p>
<p><img src="https://raw.githubusercontent.com/NannyML/nannyml/main/docs/_static/butterfly-multivariate-drift.svg"></p>

NannyML utilises statistical tests to detect **univariate feature drift**. We have just added a bunch of new univariate tests including Jensen-Shannon Distance and L-Infinity Distance, check out the [comprehensive list](https://nannyml.readthedocs.io/en/stable/how_it_works/univariate_drift_detection.html#methods-for-continuous-features). The results of these tests are tracked over time, properly corrected to counteract multiplicity and overlayed on the temporal feature distributions. (It is also possible to visualise the test-statistics over time, to get a notion of the drift magnitude.)

Expand Down Expand Up @@ -264,11 +264,11 @@ Curious what we are working on next? Have a look at our [roadmap](https://bit.ly

To cite NannyML in academic papers, please use the following BibTeX entry.

### Version 0.9.0
### Version 0.9.1

```
@misc{nannyml,
title = {{N}anny{ML} (release 0.9.0)},
title = {{N}anny{ML} (release 0.9.1)},
howpublished = {\url{https://github.com/NannyML/nannyml}},
month = mar,
year = 2023,
Expand Down
1 change: 1 addition & 0 deletions docs/_static/example_green_taxi_all_udc.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docs/_static/example_green_taxi_dle.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docs/_static/example_green_taxi_dle_vs_realized.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 22f47a8

Please sign in to comment.