Confusion Matrix as Sankey Diagram.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "00becc18-4b83-4e03-b4ca-461866ca9d8e",
   "metadata": {},
   "source": [
    "<p style=\"text-align:center\"><font size=\"12\">Confusion Matrix as Sankey Diagram</font></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "385531a1-231d-4361-a7ad-d04c21e5d8d5",
   "metadata": {},
   "source": [
    "In machine learning a confusion matrix is a kind of a table that is used to understand how well our classification model predictions perform, typically a supervised learning. It helps us a lot in understanding the model behavior and interpreting the results.  \n",
    "\n",
    "Usually, a confusion matrix is displayed as raw numbers in an array. Very often we visualize a confusion matrix by plotting it as a heatmap. But there is also another, more elegant and interactive way. In this notebook we will describe step-by-step the process of creating an interactive Sankey confusion matrix using Plotly. \n",
    "\n",
    "These are the main features of a Sankey confusion matrix:\n",
    "- **Nodes:** The source nodes are positioned on the left, representing the *actual* class labels. The target nodes are on the right, representing the *predicted* class labels.\n",
    "- **Node size:** the size of each node is proportional to the number of samples belonging to that specific class, offering insights into the class distribution.\n",
    "- **Links:** the links between nodes show the flow of samples during the classification process. The width is proportional to the number of samples classified correctly or incorrectly between the respective class labels.\n",
    "- **Tooltips:** - hovering over the nodes and links provides additional information, displaying the  numerical and textual representation of the confusion matrix.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11c97899-509a-4cbf-be5d-6bb34d266c6c",
   "metadata": {},
   "source": [
    "# Dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "4c3c7183-9c33-4eec-9a57-c8743a3eacff",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:18.939897Z",
     "iopub.status.busy": "2023-07-21T00:15:18.939645Z",
     "iopub.status.idle": "2023-07-21T00:15:19.883379Z",
     "shell.execute_reply": "2023-07-21T00:15:19.882018Z",
     "shell.execute_reply.started": "2023-07-21T00:15:18.939878Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "pd.set_option('display.max_colwidth', None)\n",
    "\n",
    "import os\n",
    "\n",
    "# Classification metrics\n",
    "from sklearn.metrics import confusion_matrix "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0204ea3f-11a2-4d28-ae60-5b50186d41d8",
   "metadata": {},
   "source": [
    "**Plotly dependencies**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "821aae51-4ed4-4b85-bca8-d87679ba0620",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:19.885069Z",
     "iopub.status.busy": "2023-07-21T00:15:19.884730Z",
     "iopub.status.idle": "2023-07-21T00:15:19.998073Z",
     "shell.execute_reply": "2023-07-21T00:15:19.997044Z",
     "shell.execute_reply.started": "2023-07-21T00:15:19.885050Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from plotly import graph_objects as go\n",
    "\n",
    "# Set the appropriate renderer in Jupyter Lab to allow Plotly displays figure correctly\n",
    "# Set the default renderer explicitly as iframe \n",
    "import plotly.io as pio\n",
    "pio.renderers.default = 'iframe' \n",
    "\n",
    "# If multiple notebooks are using 'iframe', set different 'html_directory' for each notebook\n",
    "iframe_renderer = pio.renderers['iframe']\n",
    "iframe_renderer.html_directory='iframe_figures_n1'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "075d4b00-ffac-44c2-93f0-e76bec944fdd",
   "metadata": {},
   "source": [
    "## Helper Functions"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ce21dea-c82f-43eb-ba1e-d50b00a7212f",
   "metadata": {},
   "source": [
    "To help us with visualizations, we will import the module `metrics_utilities.py`. It is a collection of several helper functions for confusion matrix visualization.\n",
    "\n",
    "The function developed in this notebook will be added to this module.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "39741c4e-14a9-4952-9c5c-cc5d0d3be5d3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:19.999173Z",
     "iopub.status.busy": "2023-07-21T00:15:19.998864Z",
     "iopub.status.idle": "2023-07-21T00:15:20.476877Z",
     "shell.execute_reply": "2023-07-21T00:15:20.475886Z",
     "shell.execute_reply.started": "2023-07-21T00:15:19.999154Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Import the script from different folder\n",
    "import sys  \n",
    "sys.path.append('./scripts')\n",
    "\n",
    "import metrics_utilities as mu"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e160d8b-b5b5-4e23-866b-aeaaf9e18c6c",
   "metadata": {},
   "source": [
    "# Preparing Data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8109eef-af49-439b-8aa5-9c15f49877a4",
   "metadata": {},
   "source": [
    "The data used in this notebook is a result from one of my previous projects - [Bank-Churn-Prediction](https://github.com/zunicd/Bank-Churn-Prediction). <br>\n",
    "I saved the true (actual) labels and predictions in `.npy` format and in the next two cells we will load them."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec4618e6-4f12-48ce-8c13-469a5a0cd953",
   "metadata": {},
   "source": [
    "## Load Test True (Actual) Labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "080111b0-7bd9-4e2a-9e10-fa42bc187745",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.479256Z",
     "iopub.status.busy": "2023-07-21T00:15:20.478284Z",
     "iopub.status.idle": "2023-07-21T00:15:20.491357Z",
     "shell.execute_reply": "2023-07-21T00:15:20.490537Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.479237Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# True labels\n",
    "y_test = np.load('./data/y_test.npy')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a905c76-e922-49f5-913d-6ab07dae4403",
   "metadata": {},
   "source": [
    "## Load Predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a92e612b-5ca6-42ca-8fbf-a70e949e23e6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.492885Z",
     "iopub.status.busy": "2023-07-21T00:15:20.492588Z",
     "iopub.status.idle": "2023-07-21T00:15:20.544161Z",
     "shell.execute_reply": "2023-07-21T00:15:20.543525Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.492863Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Prepare predictions for our models\n",
    "pred_dt = np.load('./data/pred_dt.npy')\n",
    "pred_dl = np.load('./data/pred_dl.npy')\n",
    "pred_knn = np.load('./data/pred_knn.npy')\n",
    "pred_lr = np.load('./data/pred_lr.npy')\n",
    "pred_rf = np.load('./data/pred_rf.npy')\n",
    "pred_svm = np.load('./data/pred_svm.npy')\n",
    "pred_xgb = np.load('./data/pred_xgb.npy')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a5dd6b2-fa50-49e6-a351-d901fd401e5a",
   "metadata": {},
   "source": [
    "## Names of Classes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e1d5bf0-b2de-4fb7-ad02-90a8d572e526",
   "metadata": {},
   "source": [
    "The `target_names` variable holds names of our classes. It will be used later for displaying evaluation results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "91b44d00-6d3d-41c2-8c35-abe7d71d8db2",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.545742Z",
     "iopub.status.busy": "2023-07-21T00:15:20.545548Z",
     "iopub.status.idle": "2023-07-21T00:15:20.549051Z",
     "shell.execute_reply": "2023-07-21T00:15:20.548217Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.545725Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Names of our classes\n",
    "target_names = ['Stays', 'Exits']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d59c95a3-e39c-42ee-9fd2-6023262e063d",
   "metadata": {},
   "source": [
    "# Confusion Matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c118dac-1568-426c-84e2-06c68b4275d2",
   "metadata": {},
   "source": [
    "## Axes Convention"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "537788b1-b714-48cc-93cf-2722e48e4a37",
   "metadata": {},
   "source": [
    "In the literature, we can find two variants for representing the samples in a confusion matrix:\n",
    "1. each row of the matrix represents samples in an *actual* class, and each column represents samples in a *predicted* class\n",
    "2. in the other variant, this arrangement is reversed,with each row representing samples in the *predicted* class and each column representing samples in the *actual* class.\n",
    "\n",
    "In this notebook we will use the first variant, where *actual* labels are on the horizontal axes and *predicted* labels on the vertical axes. Let us consider a binary classification problem with two classes: `0` (*Negative*) and `1` (*Positive*). The confusion matrix would be:\n",
    "\n",
    "<img src=\"images/cm_convention_table.png\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf5eb13f-53d2-40c4-98e8-16e619296b50",
   "metadata": {},
   "source": [
    "## Why Normalized Confusion Matrix?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa85c758-bbb6-4909-9547-4e51d52393bd",
   "metadata": {},
   "source": [
    "As real-life data is frequently imbalanced, utilizing a confusion matrix without normalization can potentially lead to misleading or incorrect conclusions.  \n",
    "\n",
    "In our Sankey diagram we will include values from both, unnormalized and normalized, matrices.\n",
    "\n",
    "The simplest way to display a confusion matrix is as raw numbers in an array."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d4a47fb-9751-4614-bdb1-4122cca87291",
   "metadata": {},
   "source": [
    "**Unnormalized Confusion Matrix**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "61eda4a1-8274-4b49-97d1-6d36734d6952",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.551525Z",
     "iopub.status.busy": "2023-07-21T00:15:20.550658Z",
     "iopub.status.idle": "2023-07-21T00:15:20.559647Z",
     "shell.execute_reply": "2023-07-21T00:15:20.558602Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.551482Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1979  410]\n",
      " [ 198  413]]\n"
     ]
    }
   ],
   "source": [
    "# Confusion matrix\n",
    "cm = confusion_matrix(y_test, pred_dt)\n",
    "print(cm)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91894982-9b5a-468b-aff8-084ae432cf25",
   "metadata": {},
   "source": [
    "**Normalized Confusion Matrix**\n",
    "\n",
    "Few points to know:\n",
    "1. to calculate normalized version, divide each row element by the sum of the entire row\n",
    "2. each row represents the total number of true (actual) values for each class label \n",
    "3. the normalized matrix shows the percentage of predictions made by the model for each class with respect to the corresponding *actual* (*true*) label."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "fba587c7-4414-4f51-8ad7-3fc2567b7eeb",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.560778Z",
     "iopub.status.busy": "2023-07-21T00:15:20.560547Z",
     "iopub.status.idle": "2023-07-21T00:15:20.565588Z",
     "shell.execute_reply": "2023-07-21T00:15:20.564796Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.560761Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.83 0.17]\n",
      " [0.32 0.68]]\n"
     ]
    }
   ],
   "source": [
    "# Normalized confusion matrix\n",
    "cmn = np.around(cm / cm.sum(axis=1)[:, np.newaxis], 2)\n",
    "print(cmn)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "898e13ba-fa33-42ee-8a96-62bdab9a8b4b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T19:09:33.915202Z",
     "iopub.status.busy": "2023-07-13T19:09:33.914849Z",
     "iopub.status.idle": "2023-07-13T19:09:33.920709Z",
     "shell.execute_reply": "2023-07-13T19:09:33.919872Z",
     "shell.execute_reply.started": "2023-07-13T19:09:33.915163Z"
    },
    "tags": []
   },
   "source": [
    "# Sankey Diagram"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5acaae30-029b-4844-8f25-428d2a385bdd",
   "metadata": {},
   "source": [
    "A [Sankey Diagram](https://en.wikipedia.org/wiki/Sankey_diagram) is a visual tool used to illustrate the transfer of energy, money, materials, or the flow of any isolated system or process.  \n",
    "It provides a clear depiction of flows and their quantities, showcasing the proportion of values transferring from one set to another. In a Sankey Diagram, the interconnected elements are referred to as nodes, and the connections between them (flows) are known as links.\n",
    "\n",
    "To understand Sankey diagrams, it is fundamental to become familiar with the key terminology:\n",
    "\n",
    "- **Source:** This represents the starting node or the origin of the flow.\n",
    "- **Target:** It refers to the node to which the source connects, representing the destination of the flow.\n",
    "- **Value:** This corresponds to the connection flow volume, denoted by a numerical value, and determines the thickness of the lines that connect the nodes in the Sankey diagram.\n",
    "- **Label:** The label refers to the name or description of the node, helping to identify and understand the elements represented in the diagram."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d5c49ed-9de1-4042-911b-f26af3dd0f4a",
   "metadata": {},
   "source": [
    "## Create Dataframe for Sankey"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7eead03-d295-42e3-b6cc-e97378b5f21b",
   "metadata": {},
   "source": [
    "To create a Sankey diagram, first we have to organize our data. We will use Pandas DataFrame to prepare and store the data.  \n",
    "We wiil split the process in several steps:\n",
    "- base dataframe - using all data from our confusion matrix\n",
    "- node labels - create node labels list and dictionary of node labels indices\n",
    "- final dataframe - add columns for normalized matrix, color and text to display when hovering over the links\n",
    "- mapping node labels to integers\n",
    "- prepare text that we want to print in bold font"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4fe87d3b-0152-448e-a02e-df210f44993d",
   "metadata": {},
   "source": [
    "## Base Dataframe and Node Labels"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "403c8a87-1c1f-4bab-bf09-583b6f621f40",
   "metadata": {},
   "source": [
    "#### Confusion Matrix &rarr; DataFrame"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c983777-45b8-44c4-a8b5-facff435f596",
   "metadata": {},
   "source": [
    "Create the DataFrame from the confusion matrix, using previously defined `target_names` as row and column names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "bd8966f5-4c4c-486d-9d14-f9fcc839e084",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.566611Z",
     "iopub.status.busy": "2023-07-21T00:15:20.566358Z",
     "iopub.status.idle": "2023-07-21T00:15:20.580342Z",
     "shell.execute_reply": "2023-07-21T00:15:20.579428Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.566593Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Stays</th>\n",
       "      <th>Exits</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Stays</th>\n",
       "      <td>1979</td>\n",
       "      <td>410</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Exits</th>\n",
       "      <td>198</td>\n",
       "      <td>413</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       Stays  Exits\n",
       "Stays   1979    410\n",
       "Exits    198    413"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create dataframe\n",
    "df = pd.DataFrame(cm, columns=target_names, index=target_names)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed41db33-59d3-406c-b9f3-d41e921eb8e6",
   "metadata": {},
   "source": [
    "#### The Goal"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e07ce12-253c-4f4b-85ae-9a4ed268959f",
   "metadata": {},
   "source": [
    "A Sankey diagram requires three data columns — one for the \"From\" column (*source nodes*), one for the \"To\" column (*target nodes*), and one for the values (flow quantity) corresponding to each pairing (*link*). "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1186f2bb-d07a-42cc-a92b-507012083cfa",
   "metadata": {},
   "source": [
    "We need to transform this base dataframe to the following dataframe:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6507e7f1-845d-4486-b8e8-7c9a951b091c",
   "metadata": {},
   "source": [
    "```\n",
    "                 actual\t        predicted\t  samples\n",
    "        0\tACTUAL Stays\tPREDICTED Stays\t    1979\n",
    "        1\tACTUAL Stays\tPREDICTED Exits\t     410\n",
    "        2\tACTUAL Exits\tPREDICTED Stays\t     198\n",
    "        3\tACTUAL Exits\tPREDICTED Exits\t     413\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf9b867a-345c-4b7a-83fa-dfcd76b385bb",
   "metadata": {},
   "source": [
    "For our Sankey diagram  \n",
    "\n",
    "- column `actual` represents Sankey *source nodes*  \n",
    "- column `predicted` represents Sankey *target nodes* \n",
    "- column `samples`represents flow quantities for Sankey *links*.\n",
    "\n",
    "Later we will add more columns to improve interpretability of our Sankey confusion matrix."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "880edeee-7c42-40d2-b893-b5044bbdaecd",
   "metadata": {},
   "source": [
    "#### Name the Axes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "750f05c0-8a21-43c7-8dde-7b38e3623359",
   "metadata": {},
   "source": [
    "Let's name the row axis to **ACTUAL** and the column axis to **PREDICTED**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "08ddc8b4-4d65-41b0-b84e-ff6a5127292e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.583323Z",
     "iopub.status.busy": "2023-07-21T00:15:20.583070Z",
     "iopub.status.idle": "2023-07-21T00:15:20.590184Z",
     "shell.execute_reply": "2023-07-21T00:15:20.589383Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.583306Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>PREDICTED</th>\n",
       "      <th>Stays</th>\n",
       "      <th>Exits</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ACTUAL</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Stays</th>\n",
       "      <td>1979</td>\n",
       "      <td>410</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Exits</th>\n",
       "      <td>198</td>\n",
       "      <td>413</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "PREDICTED  Stays  Exits\n",
       "ACTUAL                 \n",
       "Stays       1979    410\n",
       "Exits        198    413"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Axes naming\n",
    "df = df.rename_axis(index='ACTUAL', columns='PREDICTED')\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "202da707-7444-4e8e-9417-7989a5a65e07",
   "metadata": {},
   "source": [
    "#### Update Columns"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a44ee94-c3c2-443d-ba5f-27f3da911a8a",
   "metadata": {},
   "source": [
    "Let's append axes names to labels of rows and columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "e06781fb-c09d-461e-b31d-47d3d2a95eec",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.591032Z",
     "iopub.status.busy": "2023-07-21T00:15:20.590816Z",
     "iopub.status.idle": "2023-07-21T00:15:20.597638Z",
     "shell.execute_reply": "2023-07-21T00:15:20.596739Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.591015Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['ACTUAL Stays', 'ACTUAL Exits']"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[f'ACTUAL {s}' for s in target_names]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "0676c417-7512-4163-9e2e-3fdbfa1cfebb",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.599931Z",
     "iopub.status.busy": "2023-07-21T00:15:20.598779Z",
     "iopub.status.idle": "2023-07-21T00:15:20.606200Z",
     "shell.execute_reply": "2023-07-21T00:15:20.605425Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.599897Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              PREDICTED Stays  PREDICTED Exits\n",
      "ACTUAL Stays             1979              410\n",
      "ACTUAL Exits              198              413\n"
     ]
    }
   ],
   "source": [
    "# Set new labels for rows and columns\n",
    "df = df.set_axis([f'ACTUAL {s}' for s in target_names], axis=0)\n",
    "df = df.set_axis([f'PREDICTED {s}' for s in target_names], axis=1)\n",
    "print(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62e85424-fdca-4536-9494-d39aba64202d",
   "metadata": {},
   "source": [
    "We can do the same in one line.  \n",
    "We will create a dataframe from the confusion matrix using the new labels for rows and columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "b96c9a91-26cb-46ba-adae-36bb89065800",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.607036Z",
     "iopub.status.busy": "2023-07-21T00:15:20.606829Z",
     "iopub.status.idle": "2023-07-21T00:15:20.615311Z",
     "shell.execute_reply": "2023-07-21T00:15:20.614520Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.607020Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              PREDICTED Stays  PREDICTED Exits\n",
      "ACTUAL Stays             1979              410\n",
      "ACTUAL Exits              198              413\n"
     ]
    }
   ],
   "source": [
    "df = pd.DataFrame(cm, columns=[f'PREDICTED {s}' for s in target_names], index=[f'ACTUAL {s}' for s in target_names])\n",
    "print(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5bf24161-ff01-4709-864c-70e17e3803d5",
   "metadata": {},
   "source": [
    "#### Node Labels"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df82f25b-0756-43af-b73a-398caa3f1b91",
   "metadata": {},
   "source": [
    "**IMPORTANT:** Sankeys only take integers for node and target values.  \n",
    "We will do this transformation a little bit later. \n",
    "And for now let's prepare data for that.  \n",
    "\n",
    "First we will create a list of node labels and then a dictionary of their indices."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d474b65-70d8-4ac4-a406-c42945315f0b",
   "metadata": {},
   "source": [
    "##### List of Node Labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "01312679-5a76-47d4-a1dc-cb1413af5e50",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.616331Z",
     "iopub.status.busy": "2023-07-21T00:15:20.616148Z",
     "iopub.status.idle": "2023-07-21T00:15:20.626223Z",
     "shell.execute_reply": "2023-07-21T00:15:20.624964Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.616315Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['PREDICTED Stays', 'PREDICTED Exits']"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# column labels --> Sankey target nodes\n",
    "cl = df.columns.values.tolist()\n",
    "cl"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "b1f4cbc5-687c-4be2-9c16-a084661b0485",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.627818Z",
     "iopub.status.busy": "2023-07-21T00:15:20.627417Z",
     "iopub.status.idle": "2023-07-21T00:15:20.635667Z",
     "shell.execute_reply": "2023-07-21T00:15:20.634745Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.627786Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['ACTUAL Stays', 'ACTUAL Exits']"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# row labels --> Sankey source nodes\n",
    "rl = df.index.values.tolist()\n",
    "rl"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "5251f37f-5a39-4f6b-bc2e-2be49cf0d8ef",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.637797Z",
     "iopub.status.busy": "2023-07-21T00:15:20.636765Z",
     "iopub.status.idle": "2023-07-21T00:15:20.646856Z",
     "shell.execute_reply": "2023-07-21T00:15:20.645787Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.637753Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['ACTUAL Stays', 'ACTUAL Exits', 'PREDICTED Stays', 'PREDICTED Exits']"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "node_labels = rl + cl\n",
    "node_labels"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb33b242-af52-416e-931b-a8c5f618863b",
   "metadata": {},
   "source": [
    "##### Indices for Node Labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "a699c5a1-99cf-41d5-b31c-a4901b677b1c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.648736Z",
     "iopub.status.busy": "2023-07-21T00:15:20.648269Z",
     "iopub.status.idle": "2023-07-21T00:15:20.655184Z",
     "shell.execute_reply": "2023-07-21T00:15:20.654350Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.648703Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ACTUAL Stays': 0,\n",
       " 'ACTUAL Exits': 1,\n",
       " 'PREDICTED Stays': 2,\n",
       " 'PREDICTED Exits': 3}"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create dictionary with node labels indices\n",
    "node_labels_inds = {label:ind for ind, label in enumerate(node_labels)}\n",
    "node_labels_inds"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f77678bc-c6cf-4fc2-838a-70c0a055da77",
   "metadata": {},
   "source": [
    "#### Reshape DataFrame"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d48e6c5f-6b46-4c71-838b-52a96e453e9d",
   "metadata": {},
   "source": [
    "For Sankey diagram we need to plot flows from source nodes to target nodes. The flows are the numbers of samples being correctly or incorrectly classified.\n",
    "- Our **source nodes**: 'ACTUAL Stays', 'ACTUAL Exits'\n",
    "- Our **target nodes**: 'PREDICTED Stays', 'PREDICTED Exits'\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8a58c40-2204-40ef-ad41-caa51bf75327",
   "metadata": {},
   "source": [
    "The new reshaped dataframe will have 2x2=4 rows, 4 combinations. Each row is one flow:\n",
    "    \n",
    "ACTUAL Stays &rarr; PREDICTED Stays  =  # of Stays correctly classified  \n",
    "ACTUAL Stays &rarr; PREDICTED Exits  =  # of Stays incorrectly classified  \n",
    "ACTUAL Exits &rarr; PREDICTED Stays  =  # of Exits incorrectly classified  \n",
    "ACTUAL Exits &rarr; PREDICTED Exits  =  # of Exits correctly classified\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0338ee0-88fa-45c6-95e5-ae4467aec883",
   "metadata": {},
   "source": [
    "To acomplish this we will use Pandas funcitions:  \n",
    "\n",
    "- stack() - stack the columns to rows, it returns a series with two levels MultiIndex\n",
    "- reset_index() - reset the multilevel index to default one, and the original index gets converted to columns\n",
    "- rename() - rename new columns\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "d0ddebf0-ea21-4c29-b93c-df02ec542e00",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.657045Z",
     "iopub.status.busy": "2023-07-21T00:15:20.656083Z",
     "iopub.status.idle": "2023-07-21T00:15:20.667549Z",
     "shell.execute_reply": "2023-07-21T00:15:20.666561Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.657022Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>actual</th>\n",
       "      <th>predicted</th>\n",
       "      <th>samples</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ACTUAL Stays</td>\n",
       "      <td>PREDICTED Stays</td>\n",
       "      <td>1979</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ACTUAL Stays</td>\n",
       "      <td>PREDICTED Exits</td>\n",
       "      <td>410</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ACTUAL Exits</td>\n",
       "      <td>PREDICTED Stays</td>\n",
       "      <td>198</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ACTUAL Exits</td>\n",
       "      <td>PREDICTED Exits</td>\n",
       "      <td>413</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         actual        predicted  samples\n",
       "0  ACTUAL Stays  PREDICTED Stays     1979\n",
       "1  ACTUAL Stays  PREDICTED Exits      410\n",
       "2  ACTUAL Exits  PREDICTED Stays      198\n",
       "3  ACTUAL Exits  PREDICTED Exits      413"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Reshape dataframe\n",
    "df = df.stack().reset_index()\n",
    "df.rename(columns={0:'samples', 'level_0':'actual', 'level_1':'predicted'}, inplace=True)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cc2412f6-7046-4dc6-b22f-71305467690d",
   "metadata": {},
   "source": [
    "## Final DataFrame"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f7afaea-e1f6-41de-838e-577e00410ae5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-14T22:54:15.955750Z",
     "iopub.status.busy": "2023-07-14T22:54:15.955466Z",
     "iopub.status.idle": "2023-07-14T22:54:15.961803Z",
     "shell.execute_reply": "2023-07-14T22:54:15.960649Z",
     "shell.execute_reply.started": "2023-07-14T22:54:15.955733Z"
    },
    "tags": []
   },
   "source": [
    "### Normalized Confusion Matrix Column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "3a4617b9-a26d-4422-9431-9d23d52706a0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.668793Z",
     "iopub.status.busy": "2023-07-21T00:15:20.668517Z",
     "iopub.status.idle": "2023-07-21T00:15:20.673810Z",
     "shell.execute_reply": "2023-07-21T00:15:20.672412Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.668775Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.83 0.17]\n",
      " [0.32 0.68]]\n"
     ]
    }
   ],
   "source": [
    "# Normalized confusion matrix\n",
    "cmn = np.around(cm / cm.sum(axis=1)[:, np.newaxis], 2)\n",
    "print(cmn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "306163e8-7250-4834-8424-af818bcc56d7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.676552Z",
     "iopub.status.busy": "2023-07-21T00:15:20.675736Z",
     "iopub.status.idle": "2023-07-21T00:15:20.688420Z",
     "shell.execute_reply": "2023-07-21T00:15:20.687076Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.676507Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>actual</th>\n",
       "      <th>predicted</th>\n",
       "      <th>samples</th>\n",
       "      <th>norm_samples</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ACTUAL Stays</td>\n",
       "      <td>PREDICTED Stays</td>\n",
       "      <td>1979</td>\n",
       "      <td>0.83</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ACTUAL Stays</td>\n",
       "      <td>PREDICTED Exits</td>\n",
       "      <td>410</td>\n",
       "      <td>0.17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ACTUAL Exits</td>\n",
       "      <td>PREDICTED Stays</td>\n",
       "      <td>198</td>\n",
       "      <td>0.32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ACTUAL Exits</td>\n",
       "      <td>PREDICTED Exits</td>\n",
       "      <td>413</td>\n",
       "      <td>0.68</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         actual        predicted  samples  norm_samples\n",
       "0  ACTUAL Stays  PREDICTED Stays     1979          0.83\n",
       "1  ACTUAL Stays  PREDICTED Exits      410          0.17\n",
       "2  ACTUAL Exits  PREDICTED Stays      198          0.32\n",
       "3  ACTUAL Exits  PREDICTED Exits      413          0.68"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Flatten normmalized confusion matrix and add as a new column\n",
    "df['norm_samples'] = cmn.ravel()\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d787c126-d55d-4fc0-b9b6-b71ccfe80ba5",
   "metadata": {},
   "source": [
    "### Add New Columns `color` and `link_hover_text`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c6aef67-c864-4f29-9510-99b30ef0c4ce",
   "metadata": {},
   "source": [
    "The ink color is determioned based on classification result (correct or incorrect)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "f0bda4cd-a9f9-40c4-8886-c8e3e3b5bb5f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.690883Z",
     "iopub.status.busy": "2023-07-21T00:15:20.690067Z",
     "iopub.status.idle": "2023-07-21T00:15:20.696203Z",
     "shell.execute_reply": "2023-07-21T00:15:20.694761Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.690848Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "incorrect_red = \"rgba(205, 92, 92, 0.8)\"\n",
    "correct_green = \"rgba(144, 238, 144, 0.8)\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2080bb4-bb94-44e2-9352-9ebf67cb4c9f",
   "metadata": {},
   "source": [
    "Create a helper function to add columns `color`, and `link_hover_text` for text to be displayed when hovering over the Sankey links."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "1d681dfa-fb34-49d7-b34b-da2bbd44ee27",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.697959Z",
     "iopub.status.busy": "2023-07-21T00:15:20.697602Z",
     "iopub.status.idle": "2023-07-21T00:15:20.703900Z",
     "shell.execute_reply": "2023-07-21T00:15:20.703041Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.697931Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# 'color' - link color based on classification result (correct or incorrect)\n",
    "# 'link_hover_text' - text for hovering over connecting links of sankey diagram\n",
    "def new_columns(row):\n",
    "    source_1 = ''.join(row.actual.split()[1:])\n",
    "    target_1 = ''.join(row.predicted.split()[1:])\n",
    "    # Correct classification\n",
    "    if source_1 == target_1:\n",
    "        row['color'] = correct_green\n",
    "        row['link_hover_text'] = f\"{row.samples} ({row.norm_samples:.0%}) {source_1} samples correctly classified as {target_1}\"\n",
    "    # Incorrect classification\n",
    "    else:\n",
    "        row['color'] = incorrect_red\n",
    "        row['link_hover_text'] = f\"{row.samples} ({row.norm_samples:.0%}) {source_1} samples incorrectly classified as {target_1}\"\n",
    "    return row"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7249740a-037b-4f25-9fb8-a8c8f12d869b",
   "metadata": {},
   "source": [
    "Finalize the DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "612cf871-b91d-4f99-951c-0442717146cb",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.704923Z",
     "iopub.status.busy": "2023-07-21T00:15:20.704700Z",
     "iopub.status.idle": "2023-07-21T00:15:20.719876Z",
     "shell.execute_reply": "2023-07-21T00:15:20.719081Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.704905Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>actual</th>\n",
       "      <th>predicted</th>\n",
       "      <th>samples</th>\n",
       "      <th>norm_samples</th>\n",
       "      <th>color</th>\n",
       "      <th>link_hover_text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ACTUAL Stays</td>\n",
       "      <td>PREDICTED Stays</td>\n",
       "      <td>1979</td>\n",
       "      <td>0.83</td>\n",
       "      <td>rgba(144, 238, 144, 0.8)</td>\n",
       "      <td>1979 (83%) Stays samples correctly classified as Stays</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ACTUAL Stays</td>\n",
       "      <td>PREDICTED Exits</td>\n",
       "      <td>410</td>\n",
       "      <td>0.17</td>\n",
       "      <td>rgba(205, 92, 92, 0.8)</td>\n",
       "      <td>410 (17%) Stays samples incorrectly classified as Exits</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ACTUAL Exits</td>\n",
       "      <td>PREDICTED Stays</td>\n",
       "      <td>198</td>\n",
       "      <td>0.32</td>\n",
       "      <td>rgba(205, 92, 92, 0.8)</td>\n",
       "      <td>198 (32%) Exits samples incorrectly classified as Stays</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ACTUAL Exits</td>\n",
       "      <td>PREDICTED Exits</td>\n",
       "      <td>413</td>\n",
       "      <td>0.68</td>\n",
       "      <td>rgba(144, 238, 144, 0.8)</td>\n",
       "      <td>413 (68%) Exits samples correctly classified as Exits</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         actual        predicted  samples  norm_samples   \n",
       "0  ACTUAL Stays  PREDICTED Stays     1979          0.83  \\\n",
       "1  ACTUAL Stays  PREDICTED Exits      410          0.17   \n",
       "2  ACTUAL Exits  PREDICTED Stays      198          0.32   \n",
       "3  ACTUAL Exits  PREDICTED Exits      413          0.68   \n",
       "\n",
       "                      color   \n",
       "0  rgba(144, 238, 144, 0.8)  \\\n",
       "1    rgba(205, 92, 92, 0.8)   \n",
       "2    rgba(205, 92, 92, 0.8)   \n",
       "3  rgba(144, 238, 144, 0.8)   \n",
       "\n",
       "                                           link_hover_text  \n",
       "0   1979 (83%) Stays samples correctly classified as Stays  \n",
       "1  410 (17%) Stays samples incorrectly classified as Exits  \n",
       "2  198 (32%) Exits samples incorrectly classified as Stays  \n",
       "3    413 (68%) Exits samples correctly classified as Exits  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Apply heper function\n",
    "df = df.apply(lambda x: new_columns(x), axis=1)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c566ec8a-d1b8-42dc-b8b9-8ccc15370bbd",
   "metadata": {},
   "source": [
    "### Map Node Labels to Integers"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82b7d859-78af-4dd2-ad27-b681e7fc5dc6",
   "metadata": {},
   "source": [
    "Map node label columns (`actual`, `predicted`) to integers due to Sankey requirements."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "db165615-7133-42cf-851d-f197ba3a0c70",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.720868Z",
     "iopub.status.busy": "2023-07-21T00:15:20.720663Z",
     "iopub.status.idle": "2023-07-21T00:15:20.728869Z",
     "shell.execute_reply": "2023-07-21T00:15:20.727540Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.720850Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ACTUAL Stays': 0,\n",
       " 'ACTUAL Exits': 1,\n",
       " 'PREDICTED Stays': 2,\n",
       " 'PREDICTED Exits': 3}"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "node_labels_inds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "e000b0a4-5111-4aae-9c65-a1375dd0d2b4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.730979Z",
     "iopub.status.busy": "2023-07-21T00:15:20.730197Z",
     "iopub.status.idle": "2023-07-21T00:15:20.743283Z",
     "shell.execute_reply": "2023-07-21T00:15:20.742403Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.730942Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>actual</th>\n",
       "      <th>predicted</th>\n",
       "      <th>samples</th>\n",
       "      <th>norm_samples</th>\n",
       "      <th>color</th>\n",
       "      <th>link_hover_text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1979</td>\n",
       "      <td>0.83</td>\n",
       "      <td>rgba(144, 238, 144, 0.8)</td>\n",
       "      <td>1979 (83%) Stays samples correctly classified as Stays</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>410</td>\n",
       "      <td>0.17</td>\n",
       "      <td>rgba(205, 92, 92, 0.8)</td>\n",
       "      <td>410 (17%) Stays samples incorrectly classified as Exits</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>198</td>\n",
       "      <td>0.32</td>\n",
       "      <td>rgba(205, 92, 92, 0.8)</td>\n",
       "      <td>198 (32%) Exits samples incorrectly classified as Stays</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>413</td>\n",
       "      <td>0.68</td>\n",
       "      <td>rgba(144, 238, 144, 0.8)</td>\n",
       "      <td>413 (68%) Exits samples correctly classified as Exits</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   actual  predicted  samples  norm_samples                     color   \n",
       "0       0          2     1979          0.83  rgba(144, 238, 144, 0.8)  \\\n",
       "1       0          3      410          0.17    rgba(205, 92, 92, 0.8)   \n",
       "2       1          2      198          0.32    rgba(205, 92, 92, 0.8)   \n",
       "3       1          3      413          0.68  rgba(144, 238, 144, 0.8)   \n",
       "\n",
       "                                           link_hover_text  \n",
       "0   1979 (83%) Stays samples correctly classified as Stays  \n",
       "1  410 (17%) Stays samples incorrectly classified as Exits  \n",
       "2  198 (32%) Exits samples incorrectly classified as Stays  \n",
       "3    413 (68%) Exits samples correctly classified as Exits  "
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# using replace for multiple columns\n",
    "df = df.replace({'actual':node_labels_inds, 'predicted':node_labels_inds})\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "83fda399-7a28-493f-b8fd-496d2001a840",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.744315Z",
     "iopub.status.busy": "2023-07-21T00:15:20.744112Z",
     "iopub.status.idle": "2023-07-21T00:15:20.749722Z",
     "shell.execute_reply": "2023-07-21T00:15:20.748644Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.744299Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# using assign + apply + lambda\n",
    "# dft.assign(actual    = dft.actual.apply(lambda x: node_labels_inds[x]),\n",
    "#            predicted = dft.predicted.apply(lambda x: node_labels_inds[x]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "17969e6f-a8d4-4ab4-a8c0-58b7f9cf1667",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.751498Z",
     "iopub.status.busy": "2023-07-21T00:15:20.750909Z",
     "iopub.status.idle": "2023-07-21T00:15:20.759557Z",
     "shell.execute_reply": "2023-07-21T00:15:20.758249Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.751469Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Using assign + map\n",
    "# dft.assign(actual    = dft.actual.map(node_labels_indices),\n",
    "#             predicted = dft.predicted.map(node_labels_indices))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b09d74c8-c984-4f21-b1e7-aab6cf8b4e41",
   "metadata": {},
   "source": [
    "### Bold Printing in Plotly"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57bf1e51-06da-4481-a0a4-d754f3c63ad1",
   "metadata": {},
   "source": [
    "Prepare data for bold printing of some words in Plotly."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f887a4cd-ef93-4450-a5ef-002c623660c3",
   "metadata": {},
   "source": [
    "#### Node Labels"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5cda8fd-c36a-4bf3-94e8-4397617859d5",
   "metadata": {},
   "source": [
    "We want to print class names (2nd word in a string) in bold font.  \n",
    "We will use the HTML `<b>` tag for that.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "41b28f30-fbc1-4256-9c3e-283081e5c10c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.766150Z",
     "iopub.status.busy": "2023-07-21T00:15:20.765504Z",
     "iopub.status.idle": "2023-07-21T00:15:20.772144Z",
     "shell.execute_reply": "2023-07-21T00:15:20.771102Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.766122Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['ACTUAL Stays', 'ACTUAL Exits', 'PREDICTED Stays', 'PREDICTED Exits']"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "node_labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "6011c486-dd0d-40e9-9b79-78ea11570415",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.773422Z",
     "iopub.status.busy": "2023-07-21T00:15:20.773175Z",
     "iopub.status.idle": "2023-07-21T00:15:20.778001Z",
     "shell.execute_reply": "2023-07-21T00:15:20.777085Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.773403Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['ACTUAL <b>Stays</b>', 'ACTUAL <b>Exits</b>', 'PREDICTED <b>Stays</b>', 'PREDICTED <b>Exits</b>']\n"
     ]
    }
   ],
   "source": [
    "node_labels = [f'{ls[0]} <b>{ls[1]}</b>' for ls in [l.split() for l in node_labels]]\n",
    "print(node_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d0b3524-1d98-4104-a16f-82140ef2b37b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-07T19:14:34.018318Z",
     "iopub.status.busy": "2023-07-07T19:14:34.017592Z",
     "iopub.status.idle": "2023-07-07T19:14:34.028438Z",
     "shell.execute_reply": "2023-07-07T19:14:34.026664Z",
     "shell.execute_reply.started": "2023-07-07T19:14:34.018282Z"
    }
   },
   "source": [
    "#### Hovering Text"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "504f5fea-a7f0-4ff9-a9f2-8110736d1bff",
   "metadata": {},
   "source": [
    "Printing class names in bold font."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "e7ed011e-2e86-492e-95d2-7cc2b6451484",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.779099Z",
     "iopub.status.busy": "2023-07-21T00:15:20.778872Z",
     "iopub.status.idle": "2023-07-21T00:15:20.791262Z",
     "shell.execute_reply": "2023-07-21T00:15:20.790544Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.779083Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>actual</th>\n",
       "      <th>predicted</th>\n",
       "      <th>samples</th>\n",
       "      <th>norm_samples</th>\n",
       "      <th>color</th>\n",
       "      <th>link_hover_text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1979</td>\n",
       "      <td>0.83</td>\n",
       "      <td>rgba(144, 238, 144, 0.8)</td>\n",
       "      <td>1979 (83%) &lt;b&gt;Stays&lt;/b&gt; samples correctly classified as &lt;b&gt;Stays&lt;/b&gt;</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>410</td>\n",
       "      <td>0.17</td>\n",
       "      <td>rgba(205, 92, 92, 0.8)</td>\n",
       "      <td>410 (17%) &lt;b&gt;Stays&lt;/b&gt; samples incorrectly classified as &lt;b&gt;Exits&lt;/b&gt;</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>198</td>\n",
       "      <td>0.32</td>\n",
       "      <td>rgba(205, 92, 92, 0.8)</td>\n",
       "      <td>198 (32%) &lt;b&gt;Exits&lt;/b&gt; samples incorrectly classified as &lt;b&gt;Stays&lt;/b&gt;</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>413</td>\n",
       "      <td>0.68</td>\n",
       "      <td>rgba(144, 238, 144, 0.8)</td>\n",
       "      <td>413 (68%) &lt;b&gt;Exits&lt;/b&gt; samples correctly classified as &lt;b&gt;Exits&lt;/b&gt;</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   actual  predicted  samples  norm_samples                     color   \n",
       "0       0          2     1979          0.83  rgba(144, 238, 144, 0.8)  \\\n",
       "1       0          3      410          0.17    rgba(205, 92, 92, 0.8)   \n",
       "2       1          2      198          0.32    rgba(205, 92, 92, 0.8)   \n",
       "3       1          3      413          0.68  rgba(144, 238, 144, 0.8)   \n",
       "\n",
       "                                                         link_hover_text  \n",
       "0   1979 (83%) <b>Stays</b> samples correctly classified as <b>Stays</b>  \n",
       "1  410 (17%) <b>Stays</b> samples incorrectly classified as <b>Exits</b>  \n",
       "2  198 (32%) <b>Exits</b> samples incorrectly classified as <b>Stays</b>  \n",
       "3    413 (68%) <b>Exits</b> samples correctly classified as <b>Exits</b>  "
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['link_hover_text'] = [f'{\" \".join(ls[0:2])} <b>{ls[2]}</b> {\" \".join(ls[3:-1])} <b>{ls[-1]}</b>' for ls in [l.split() for l in df['link_hover_text']]]\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7c41258-ed9d-4e1e-a027-bc21545efd03",
   "metadata": {},
   "source": [
    "## Plotting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "bd3cb3cb-28d7-4d38-80a7-894c08e5eaea",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.792575Z",
     "iopub.status.busy": "2023-07-21T00:15:20.792353Z",
     "iopub.status.idle": "2023-07-21T00:15:20.876515Z",
     "shell.execute_reply": "2023-07-21T00:15:20.875634Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.792559Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<iframe\n",
       "    scrolling=\"no\"\n",
       "    width=\"620px\"\n",
       "    height=\"520\"\n",
       "    src=\"iframe_figures_n1/figure_31.html\"\n",
       "    frameborder=\"0\"\n",
       "    allowfullscreen\n",
       "></iframe>\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = go.Figure(data=[go.Sankey(\n",
    "    \n",
    "node = dict(\n",
    "    pad = 30,\n",
    "    thickness = 20,\n",
    "    line = dict(color = \"gray\", width = 1.0),\n",
    "    label = node_labels,\n",
    "    hovertemplate = \"%{label} has total %{value:d} samples<extra></extra>\"\n",
    "    ),\n",
    "link = dict(\n",
    "    source = df.actual, \n",
    "    target = df.predicted,\n",
    "    value = df.samples,\n",
    "    color = df.color,\n",
    "    customdata = df['link_hover_text'], \n",
    "    hovertemplate = \"%{customdata}<extra></extra>\"  \n",
    "))])\n",
    "\n",
    "title = f'Decision Tree'\n",
    "\n",
    "fig.update_layout(\n",
    "    # hovermode = 'x',\n",
    "    title = {\n",
    "    'text': title,\n",
    "    'x':0.5,\n",
    "    },\n",
    "    # paper_bgcolor = '#51504f',\n",
    "    font_size = 15,\n",
    "    # font_color = 'white',\n",
    "    width = 600,\n",
    "    height = 500\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f75a939-a105-46c4-88f6-04eb552f1d0b",
   "metadata": {},
   "source": [
    "### How to interpret our Sankey confusion matrix?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a62f36a-275d-4156-93b3-45e90d2db57c",
   "metadata": {},
   "source": [
    "To interpret our Sankey confusion matrix, with **Stays** representing the negative class and **Exits** representing the positive class, take note of the following key elements:\n",
    "\n",
    "- **Nodes:** The source nodes (\"ACTUAL **Stays**\" and \"ACTUAL **Exits**\" ) are positioned on the left, representing the *actual* class labels. The target nodes (\"PREDICTED **Stays**\" and \"PREDICTED **Exits**\") are on the right, representing the *predicted* class labels.\n",
    "- **Node Size:** The size of each node is proportional to the number of samples belonging to that specific class, offering insights into the class distribution.\n",
    "- **Links:** The links between nodes show the flow of samples during classification. Wider links indicate a higher number of samples classified correctly (shown in green) or incorrectly (depicted in red) between the respective class labels.\n",
    "- **Tooltips:** Hovering over the nodes and links provides additional information, displaying the numerical and textual representation of the confusion matrix. This interactive feature enhances the understanding of classification outcomes and allows for a more detailed examination of the model's performance."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf15806f-6f16-4707-8480-ea5c7459ef63",
   "metadata": {},
   "source": [
    "## Define Function"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3799fe1e-7a3a-4a55-aaef-daf91c3c58e0",
   "metadata": {},
   "source": [
    "Let's now collect all these together into a function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "30b200f3-c08c-497f-b897-b975c254c19e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.878296Z",
     "iopub.status.busy": "2023-07-21T00:15:20.877989Z",
     "iopub.status.idle": "2023-07-21T00:15:20.891877Z",
     "shell.execute_reply": "2023-07-21T00:15:20.890426Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.878275Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def plot_cm_sankey(model_name, y_test, y_pred, target_names=None):\n",
    "    \"\"\" Plot confusion matrix with Sankey diagram \n",
    "\n",
    "    Args:\n",
    "        model_name: name of the model\n",
    "        y_test: test target variable\n",
    "        y_pred: prediction\n",
    "        target_names: list of class names\n",
    "\n",
    "    Returns:\n",
    "        Plot Sankey diagram of confusion matrix\n",
    "    \"\"\" \n",
    "    \n",
    "    # Calculate confusion matrix\n",
    "    cm = confusion_matrix(y_test, y_pred)\n",
    "    \n",
    "    # If class labels not passed, create dummy class labels\n",
    "    if target_names == None: \n",
    "        target_names = []\n",
    "    if not len(target_names):\n",
    "        target_names = [f'class-{i+1}' for i in range(len(cm))]\n",
    "    \n",
    "    # Prepare dataframe with parameters for Sankey\n",
    "    def prepare_df_for_sankey(cm, target_names):\n",
    "        # create a dataframe\n",
    "        df = pd.DataFrame(cm, columns=[f'PREDICTED {s}' for s in target_names], index=[f'ACTUAL {s}' for s in target_names])\n",
    "        \n",
    "        # Create list of node labels\n",
    "        # target nodes = column labels (PREDICTED ...)\n",
    "        cl = df.columns.values.tolist()\n",
    "        # source nodes = row (index) labels (ACTUAL ...)\n",
    "        rl = df.index.values.tolist()\n",
    "        node_labels = rl + cl\n",
    "        \n",
    "        # Create dictionary with indices for node labels\n",
    "        node_labels_inds = {label:ind for ind, label in enumerate(node_labels)}\n",
    "        \n",
    "        # Stack label from column to row, output is Series\n",
    "        # Reset index to get DataFrame and rename columns\n",
    "        df = df.stack().reset_index()\n",
    "        df.rename(columns={0:'samples', 'level_0':'actual', 'level_1':'predicted'}, inplace=True)\n",
    "        \n",
    "        \"\"\"\n",
    "               actual\t       predicted\t  samples\n",
    "        0\tACTUAL Stays\tPREDICTED Stays\t    1979\n",
    "        1\tACTUAL Stays\tPREDICTED Exits\t     410\n",
    "        2\tACTUAL Exits\tPREDICTED Stays\t     198\n",
    "        3\tACTUAL Exits\tPREDICTED Exits\t     413\n",
    "        \"\"\"\n",
    "\n",
    "        # Normalized confusion matrix\n",
    "        cmn = np.around(cm / cm.sum(axis=1)[:, np.newaxis], 2)\n",
    "        # Add a column with normalized values of samples\n",
    "        df['norm_samples'] = cmn.ravel()\n",
    "        \n",
    "        # Helper function to add new columns: color and link_hover_text \n",
    "        # 'color' - link color based on classification result (correct or incorrect)        \n",
    "        incorrect_red = \"rgba(205, 92, 92, 0.8)\"\n",
    "        correct_green = \"rgba(144, 238, 144, 0.8)\"\n",
    "        # # 'link_hover_text' - text for hovering on connecting links of sankey diagram\n",
    "        \n",
    "        def new_columns(row):\n",
    "            source_1 = ''.join(row.actual.split()[1:])\n",
    "            target_1 = ''.join(row.predicted.split()[1:])\n",
    "            # Correct classification\n",
    "            if source_1 == target_1:\n",
    "                row['color'] = correct_green\n",
    "                row['link_hover_text'] = f\"{row.samples} ({row.norm_samples:.0%}) {source_1} samples correctly classified as {target_1}\"\n",
    "            # Incorrect classification\n",
    "            else:\n",
    "                row['color'] = incorrect_red\n",
    "                row['link_hover_text'] = f\"{row.samples} ({row.norm_samples:.0%}) {source_1} samples incorrectly classified as {target_1}\"\n",
    "            return row\n",
    "\n",
    "        # Apply \"new_columns\" function\n",
    "        df = df.apply(lambda x: new_columns(x), axis=1)\n",
    "        \n",
    "        # Sankey only takes integers for node and target values,\n",
    "        #  so we need to map node label columns (actual, predicted) to numbers\n",
    "        # Using replace for multiple columns\n",
    "        df = df.replace({'actual':node_labels_inds, 'predicted':node_labels_inds})\n",
    "               \n",
    "        return df, node_labels\n",
    "    \n",
    "    \n",
    "    # Plotting confusion matrix as Sankey diagram\n",
    "    # Get dataframe and node labels\n",
    "    df, node_labels = prepare_df_for_sankey(cm, target_names)\n",
    "    \n",
    "    # Prepare for bold printing of some words in Plotly\n",
    "    node_labels = [f'{ls[0]} <b>{ls[1]}</b>' for ls in [l.split() for l in node_labels]]\n",
    "    df['link_hover_text'] = [f'{\" \".join(ls[0:2])} <b>{ls[2]}</b> {\" \".join(ls[3:-1])} <b>{ls[-1]}</b>' for ls in [l.split() for l in df['link_hover_text']]]\n",
    "    \n",
    "\n",
    "    fig = go.Figure(data=[go.Sankey(    \n",
    "        node = dict(\n",
    "        pad = 50,\n",
    "        thickness = 30,\n",
    "        line = dict(color = \"gray\", width = 1.0),\n",
    "        label = node_labels,\n",
    "        hovertemplate = \"%{label} has total %{value:d} samples<extra></extra>\"\n",
    "        ),\n",
    "    link = dict(\n",
    "        source = df.actual, \n",
    "        target = df.predicted,\n",
    "        value = df.samples,\n",
    "        color = df.color,\n",
    "        customdata = df['link_hover_text'], \n",
    "        hovertemplate = \"%{customdata}<extra></extra>\"  \n",
    "    ))])\n",
    "    \n",
    "    margins = {'l': 25, 'r': 25, 't': 70, 'b': 25}\n",
    "    \n",
    "    fig.update_layout(\n",
    "        title = {\n",
    "        'text': f'<b>{model_name}</b>',\n",
    "        'x':0.5,\n",
    "        },\n",
    "        font_size = 15,\n",
    "        width = 625,\n",
    "        height = 500,\n",
    "        #paper_bgcolor = '#d3d3d3',\n",
    "        # paper_bgcolor = 'white',\n",
    "        # plot_bgcolor = 'black',\n",
    "        margin = margins,\n",
    "    )\n",
    "    \n",
    "    return fig"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93091c01-c15d-4b10-ae65-91a83d4d9ce1",
   "metadata": {},
   "source": [
    "### Run Function"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f64d5467-ed42-4c3a-998e-a6c49b27f6b9",
   "metadata": {},
   "source": [
    "Let's test the function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "5083ad56-aee0-4476-9219-8db797ea0dc6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.894644Z",
     "iopub.status.busy": "2023-07-21T00:15:20.893758Z",
     "iopub.status.idle": "2023-07-21T00:15:20.957723Z",
     "shell.execute_reply": "2023-07-21T00:15:20.956833Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.894603Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<iframe\n",
       "    scrolling=\"no\"\n",
       "    width=\"645px\"\n",
       "    height=\"520\"\n",
       "    src=\"iframe_figures_n1/figure_33.html\"\n",
       "    frameborder=\"0\"\n",
       "    allowfullscreen\n",
       "></iframe>\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_cm_sankey('Decision Tree', y_test, pred_dt, target_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c9062be-101f-4229-ab6e-dddcb38f9328",
   "metadata": {},
   "source": [
    "Copy the function to the module `metrics_utilities.py` and reload the kernel. After running all required cells above, run the following cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "b42a3ae6-990c-438b-8ffb-9cbc3a490a78",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:20.959158Z",
     "iopub.status.busy": "2023-07-21T00:15:20.958960Z",
     "iopub.status.idle": "2023-07-21T00:15:21.003124Z",
     "shell.execute_reply": "2023-07-21T00:15:21.002074Z",
     "shell.execute_reply.started": "2023-07-21T00:15:20.959143Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<iframe\n",
       "    scrolling=\"no\"\n",
       "    width=\"645px\"\n",
       "    height=\"520\"\n",
       "    src=\"iframe_figures_n1/figure_34.html\"\n",
       "    frameborder=\"0\"\n",
       "    allowfullscreen\n",
       "></iframe>\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "mu.plot_cm_sankey('Decision Tree', y_test, pred_dt, target_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fcbc0984-124b-4295-ad7a-dd2eb979cf5b",
   "metadata": {},
   "source": [
    "Hover over the diagram to get more information about the confusion matrix."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fed8b488-2138-4896-b1a5-24de6f636dc1",
   "metadata": {},
   "source": [
    "### 3x3 Confusion Matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1eab518c-b4ac-4a83-8279-b478f3fd0aa4",
   "metadata": {},
   "source": [
    "Let's see how this works for an 3x3 confusion matrix.\n",
    "\n",
    "We will use data from my project [T2D-Predictions](https://github.com/zunicd/T2D-Predictions)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "06ff78f8-69d5-4e4e-b1d4-d21c2b6ef0be",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:21.004734Z",
     "iopub.status.busy": "2023-07-21T00:15:21.004197Z",
     "iopub.status.idle": "2023-07-21T00:15:21.015368Z",
     "shell.execute_reply": "2023-07-21T00:15:21.014529Z",
     "shell.execute_reply.started": "2023-07-21T00:15:21.004715Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Actual (True) labels\n",
    "t2d_y_test = np.load('./data/t2d_y_test.npy')\n",
    "# Prediction from random forest model\n",
    "t2d_pred_rf = np.load('./data/t2d_pred_rf.npy')\n",
    "# Classes\n",
    "t2d_classes = ['no_diabetes', 'pre_diabetes', 'diabetes']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "76651cfa-c8c1-456d-9240-99988ace7e42",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-21T00:15:21.016688Z",
     "iopub.status.busy": "2023-07-21T00:15:21.016217Z",
     "iopub.status.idle": "2023-07-21T00:15:21.064813Z",
     "shell.execute_reply": "2023-07-21T00:15:21.063973Z",
     "shell.execute_reply.started": "2023-07-21T00:15:21.016669Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<iframe\n",
       "    scrolling=\"no\"\n",
       "    width=\"645px\"\n",
       "    height=\"520\"\n",
       "    src=\"iframe_figures_n1/figure_36.html\"\n",
       "    frameborder=\"0\"\n",
       "    allowfullscreen\n",
       "></iframe>\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "mu.plot_cm_sankey('Random Forest', t2d_y_test, t2d_pred_rf, t2d_classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02b2206e-aa84-4028-a659-e616e293ceb7",
   "metadata": {},
   "source": [
    "Hover over the diagram to get more information about the confusion matrix."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5562aebe-701c-407a-9ecc-d36a41e64688",
   "metadata": {},
   "source": [
    "**NOTE 1:**  \n",
    "\n",
    "- Depending on the values in the confusion matrix, especially in the case of multi-class classification, Plotly might display the target nodes (PREDICTED labels) in a different order compared to the source nodes (ACTUAL labels).\n",
    "\n",
    "- However, rest assured that despite the potential rearrangement of nodes, all the connected links in the Sankey Diagram will accurately represent the correct width (values) as per the confusion matrix. The flow volumes will be appropriately maintained, enabling a precise representation of the classification outcomes between the actual and predicted labels."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4bb7104c-73f7-4d08-b03e-aa38af66c7bb",
   "metadata": {},
   "source": [
    "<br>  \n",
    "\n",
    "**NOTE 2 - for Jupyter Lab users:**  \n",
    "\n",
    "-  Set the appropriate renderer in Jupyter Lab to allow Plotly displays figure correctly\n",
    "- As suggested in the [Plotly documentation](https://plotly.com/python/renderers/#interactive-renderers), you might set the default renderer explicitly as iframe by adding following lines into your codes:\n",
    "\n",
    "```python\n",
    "import plotly.io as pio\n",
    "pio.renderers.default = 'iframe'\n",
    "```\n",
    "\n",
    "- This renderer writes figures out as standalone HTML files and then displays [iframe](https://www.w3schools.com/html/html_iframe.asp) elements that reference these files\n",
    "- They are stored in a subdirectory named *iframe_figures*\n",
    "- The names of the files are given based on the execution number of the notebook cell that produced the figure\n",
    "-  Storing multiple notebooks using an iframe renderer in the same directory, could result in notebooks overwriting each other's figures\n",
    "- To avoid this, add the following lines in each notebook to set different names for the `html_directory`. in this notebook we named it *iframe_figures_n2*\n",
    "\n",
    "\n",
    "```python\n",
    "iframe_renderer = pio.renderers['iframe']\n",
    "iframe_renderer.html_directory='iframe_figures_n1'\n",
    "```\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "omdena_p310",
   "language": "python",
   "name": "conda-env-omdena_p310-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.10"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}