Skip to content

Commit

Permalink
updating MCM docs
Browse files Browse the repository at this point in the history
  • Loading branch information
nikml committed Jun 27, 2023
1 parent 247dfe7 commit f49202c
Show file tree
Hide file tree
Showing 7 changed files with 59 additions and 8,440 deletions.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
.. _multiclass-confusion-matrix-calculation:

========================================================================================
===================================================================
Calculating Confusion Matrix Elements for Multiclass Classification
========================================================================================
===================================================================

This tutorial explains how to use NannyML to calculate the :term:`confusion matrix<Confusion Matrix>` for multiclass classification
models.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
.. _multiclass-standard-metric-calculation:

================================================================
Monitoring Realized Performance for Multiclass Classification
================================================================
======================================================================
Calculating Standard Performance Metrics for Multiclass Classification
======================================================================

.. note::
The following example uses :term:`timestamps<Timestamp>`.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
.. _multiclass-performance-estimation:

================================================
====================================================
Estimating Performance for Multiclass Classification
================================================
====================================================

We currently support the following **standard** metrics for multiclass classification performance estimation:

Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
.. _multiclass-confusion-matrix-estimation:

========================================================================================
==================================================================
Estimating Confusion Matrix Elements for Multiclass Classification
========================================================================================
==================================================================

This tutorial explains how to use NannyML to estimate the :term:`confusion matrix<Confusion Matrix>` for multiclass classification
models in the absence of target data. To find out how CBPE estimates performance, read the :ref:`explanation of Confidence-based
Performance Estimation<performance-estimation-deep-dive>`.
Expand Down Expand Up @@ -162,8 +163,6 @@ What's next
-----------

The :ref:`Data Drift<data-drift>` functionality can help us to understand whether data drift is causing the performance problem.

You can learn more about the Confidence Based Performance Estimation and its limitations in the
:ref:`How it Works page<performance-estimation-deep-dive>`.


When the target values become available we can use
:ref:`realized performance calculation<multiclass-confusion-matrix-calculation>` to
:ref:`compare realized and estimated confusion matrix results<compare_estimated_and_realized_performance>`.
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
.. _multiclass_standard-metric-estimation:

====================================================
Estimating Performance for Multiclass Classification
====================================================
=====================================================================
Estimating Standard Performance Metrics for Multiclass Classification
=====================================================================

This tutorial explains how to use NannyML to estimate the performance of binary classification
models in the absence of target data. To find out how :class:`~nannyml.performance_estimation.confidence_based.cbpe.CBPE` estimates performance, read the :ref:`explanation of Confidence-based
Expand Down Expand Up @@ -140,7 +140,6 @@ What's next
-----------

The :ref:`Data Drift<data-drift>` functionality can help us to understand whether data drift is causing the performance problem.
When the target results become available they can be :ref:`compared with the estimated results<compare_estimated_and_realized_performance>`.

You can learn more about the Confidence Based Performance Estimation and its limitations in the
:ref:`How it Works page<performance-estimation-deep-dive>`.
When the target values become available we can use
:ref:`realized performance calculation<multiclass-standard-metric-calculation>` to
:ref:`compare realized and estimated confusion matrix results<compare_estimated_and_realized_performance>`.

0 comments on commit f49202c

Please sign in to comment.