Cw1_w1750131_CosminAlexandruStefanescu.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "a1f8bee8ea2570bec4dab351d9c08c0398d89683"
   },
   "source": [
    "# Introduction\n",
    "* In this notebook, we will learn text mining together step by step. \n",
    "* **Lets start with why we need to learn text mining?**\n",
    "    * Text is everywhere like books, facebook or twitter.\n",
    "    * Text data is growing each passing day. The amount of text data will be approximately 40 zettabytes(10^21) in 2 years. \n",
    "    * We learn text mining to analize the sentiment of the text, the topic and to find, classify and extract information.\n",
    "<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: numpy in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (1.21.5)\n",
      "Requirement already satisfied: pandas in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (1.3.5)\n",
      "Requirement already satisfied: pytz>=2017.3 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from pandas) (2021.3)\n",
      "Requirement already satisfied: numpy>=1.17.3 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from pandas) (1.21.5)\n",
      "Requirement already satisfied: python-dateutil>=2.7.3 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from pandas) (2.8.2)\n",
      "Requirement already satisfied: six>=1.5 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from python-dateutil>=2.7.3->pandas) (1.16.0)\n"
     ]
    }
   ],
   "source": [
    "!pip install numpy\n",
    "!pip install pandas\n",
    "import numpy as np # linear algebra\n",
    "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "230fd1e4857bd46e4e99054cb02bed9a7193ecb8"
   },
   "source": [
    "<a id=\"0\"></a> <br>\n",
    "## Basic Text Mining Methods\n",
    " * The text can be sentences, strings, words, characters and large documents.\n",
    " * Now lets create a sentence to understand basics of text mining methods.\n",
    " * Our sentece is \"no woman no cry\" from Bob Marley."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "274b69a23a2d274c7b45212f37fb952633fd0be2"
   },
   "source": [
    "<a id=\"1\"></a> <br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "_uuid": "4aaf5f3cdfe386125fabfb0abb58bd0b0ff0413e",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "length of text:  15\n",
      "Splitted text:  ['No', 'woman', 'no', 'cry']\n"
     ]
    }
   ],
   "source": [
    "# lets create a text\n",
    "text = \"No woman no cry\"\n",
    "\n",
    "# length of text ( includes spaces)\n",
    "print(\"length of text: \",len(text))\n",
    "\n",
    "# split the text\n",
    "splitted_text = text.split() # default split methods splits text according to spaces\n",
    "print(\"Splitted text: \",splitted_text)    # splitted_text is a list of the words from \"text\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "0410821d3cddccc7eb4a6d2e59606130e50f6088"
   },
   "source": [
    "<a id=\"2\"></a> <br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "_uuid": "97d799c7dacd503ff314ef66861d985b1db194e8",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Words which have more than 3 letter:  ['woman', 'cry']\n",
      "Capitalized words:  ['No']\n",
      "words end with o:  ['No', 'no']\n",
      "words start with w:  ['woman']\n"
     ]
    }
   ],
   "source": [
    "# find specific words with list comprehension method\n",
    "specific_words = [word for word in splitted_text if(len(word)>2)]\n",
    "print(\"Words which have more than 3 letter: \",specific_words)\n",
    "\n",
    "# find capitalized words with istitle() method \n",
    "capital_words = [ word for word in splitted_text if word.istitle()]\n",
    "print(\"Capitalized words: \",capital_words)\n",
    "\n",
    "# find words that end with \"o\" using the: endswith() method\n",
    "words_end_with_o =  [word for word in splitted_text if word.endswith(\"o\")]\n",
    "print(\"words end with o: \",words_end_with_o) \n",
    "\n",
    "# find words that start with \"w\" using the: startswith() method\n",
    "words_start_with_w = [word for word in splitted_text if word.startswith(\"w\")]\n",
    "print(\"words start with w: \",words_start_with_w) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "9c7349dcd176633090a431e7a3f760e547f66dc9"
   },
   "source": [
    "<a id=\"3\"></a> <br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "_uuid": "5de3084612dc757b017147d1e73868f552585451",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "unique words:  {'no', 'No', 'cry', 'woman'}\n",
      "unique words:  {'no', 'cry', 'woman'}\n"
     ]
    }
   ],
   "source": [
    "# unique with set() method\n",
    "print(\"unique words: \",set(splitted_text))  # The word \"no\" occurs twice becausec one starts with a capital letter and the other doesn't\n",
    "\n",
    "# make all letters lowercase with lower() method\n",
    "lowercase_text = [word.lower() for word in splitted_text]\n",
    "\n",
    "# then find uniques again with set() method\n",
    "print(\"unique words: \",set(lowercase_text))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "277d130e66c1e0ff842c8bdaa59f9510b8a7b05a"
   },
   "source": [
    "<a id=\"4\"></a> <br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "_uuid": "b78ee7bd5103514ca8714cdb5df92e9960064a87",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Is w a letter in the word 'woman': True\n",
      "Is word uppercase: True\n",
      "Is word lowercase: True\n",
      "Is word made of by digits:  True\n",
      "00000000No cry:  No cry\n",
      "Find particular letter from the beggining:  1\n",
      "Find particular letter from back:  8\n",
      "Replace o with 3  N3 cry n3\n",
      "Each letter:  ['N', 'o', ' ', 'c', 'r', 'y']\n"
     ]
    }
   ],
   "source": [
    "# chech which words include or not a particular substring or letter\n",
    "print(\"Is w a letter in the word 'woman':\", \"w\" in \"woman\")\n",
    "\n",
    "# check which words are upper case or lower case\n",
    "print(\"Is word uppercase:\", \"WOMAN\".isupper())\n",
    "print(\"Is word lowercase:\", \"cry\".islower())\n",
    "\n",
    "# check which words are made of digits\n",
    "print(\"Is word made of by digits: \",\"12345\".isdigit())\n",
    "\n",
    "# get rid of space characters like spaces and tabs or from unwanted letters with strip() method\n",
    "print(\"00000000No cry: \",\"00000000No cry\".strip(\"0\"))\n",
    "\n",
    "# find particular letter from the beggining \n",
    "print(\"Find particular letter from the beggining: \",\"No cry no\".find(\"o\"))  # at index 1\n",
    "\n",
    "# find particular letter from the end  rfind = reverse find\n",
    "print(\"Find particular letter from back: \",\"No cry no\".rfind(\"o\"))  # at index 8\n",
    "\n",
    "# replace a letter with another letter\n",
    "print(\"Replace o with 3 \", \"No cry no\".replace(\"o\",\"3\"))\n",
    "\n",
    "# find each letter/character and store them in list\n",
    "print(\"Each letter: \",list(\"No cry\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "_uuid": "1352213c2e932e92c6e53b37809d925210aecefa",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Split text:  ['', '', '', '', 'Be', 'fair', 'and', 'tolerant', '', '', '', '']\n",
      "Cleaned text:  ['Be', 'fair', 'and', 'tolerant']\n"
     ]
    }
   ],
   "source": [
    "# Cleaning text\n",
    "text1 = \"    Be fair and tolerant    \"\n",
    "print(\"Split text: \",text1.split(\" \"))   # as you can see there are unnecessary spaces in the list\n",
    "\n",
    "# get rid of these unnecassary spaces with strip() method then split\n",
    "print(\"Cleaned text: \",text1.strip().split(\" \"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "373c53e00457b0929e12d550ef1f680b62c4d2ff"
   },
   "source": [
    "<a id=\"5\"></a> <br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "_uuid": "3d857c1704e668528b980ef65b70d532f958ed0c",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "﻿The Project Gutenberg EBook of The Gospel of Buddha, by Paul Carus\n",
      "\n",
      "Length of text:  580941\n",
      "Number of lines:  13220\n"
     ]
    }
   ],
   "source": [
    "# reading files line by line\n",
    "f = open(\"data/35895-0.txt\",\"r\",encoding='utf-8', errors='ignore')\n",
    "\n",
    "# read first line\n",
    "print(f.readline())\n",
    "\n",
    "# length of text\n",
    "text3=f.read()\n",
    "print(\"Length of text: \",len(text3))\n",
    "\n",
    "# Number of lines with splitlines() method\n",
    "lines = text3.splitlines()\n",
    "print(\"Number of lines: \",len(lines))\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "_uuid": "9120ded28f063e7d164909a8a8c8af4802c8353d",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>text</th>\n",
       "      <th>favorited</th>\n",
       "      <th>favoriteCount</th>\n",
       "      <th>replyToSN</th>\n",
       "      <th>created</th>\n",
       "      <th>truncated</th>\n",
       "      <th>replyToSID</th>\n",
       "      <th>id</th>\n",
       "      <th>replyToUID</th>\n",
       "      <th>statusSource</th>\n",
       "      <th>screenName</th>\n",
       "      <th>retweetCount</th>\n",
       "      <th>isRetweet</th>\n",
       "      <th>retweeted</th>\n",
       "      <th>longitude</th>\n",
       "      <th>latitude</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>âThe case for learned index structuresâ - ...</td>\n",
       "      <td>False</td>\n",
       "      <td>29</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2017-12-11 16:21:59</td>\n",
       "      <td>True</td>\n",
       "      <td>NaN</td>\n",
       "      <td>940255351378624512</td>\n",
       "      <td>NaN</td>\n",
       "      <td>&lt;a href=\"http://twitter.com/download/iphone\" r...</td>\n",
       "      <td>benhamner</td>\n",
       "      <td>10</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>@Smerity Lock you in a black box with a window...</td>\n",
       "      <td>False</td>\n",
       "      <td>11</td>\n",
       "      <td>Smerity</td>\n",
       "      <td>2017-12-10 04:16:21</td>\n",
       "      <td>True</td>\n",
       "      <td>9.396588e+17</td>\n",
       "      <td>939710351305670656</td>\n",
       "      <td>15363432.0</td>\n",
       "      <td>&lt;a href=\"http://twitter.com/download/iphone\" r...</td>\n",
       "      <td>benhamner</td>\n",
       "      <td>0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>What policy outcomes are you aiming to achieve...</td>\n",
       "      <td>False</td>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2017-12-09 22:47:22</td>\n",
       "      <td>True</td>\n",
       "      <td>NaN</td>\n",
       "      <td>939627560631132160</td>\n",
       "      <td>NaN</td>\n",
       "      <td>&lt;a href=\"http://twitter.com/download/iphone\" r...</td>\n",
       "      <td>benhamner</td>\n",
       "      <td>0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>âMachine learning for systems and systems fo...</td>\n",
       "      <td>False</td>\n",
       "      <td>34</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2017-12-09 22:40:48</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>939625906775003136</td>\n",
       "      <td>NaN</td>\n",
       "      <td>&lt;a href=\"http://twitter.com/download/iphone\" r...</td>\n",
       "      <td>benhamner</td>\n",
       "      <td>21</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>From the number of talks @goodfellow_ianâs g...</td>\n",
       "      <td>True</td>\n",
       "      <td>180</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2017-12-09 21:59:28</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>939615505165402112</td>\n",
       "      <td>NaN</td>\n",
       "      <td>&lt;a href=\"http://twitter.com/download/iphone\" r...</td>\n",
       "      <td>benhamner</td>\n",
       "      <td>10</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                text  favorited  \\\n",
       "0  âThe case for learned index structuresâ - ...      False   \n",
       "1  @Smerity Lock you in a black box with a window...      False   \n",
       "2  What policy outcomes are you aiming to achieve...      False   \n",
       "3  âMachine learning for systems and systems fo...      False   \n",
       "4  From the number of talks @goodfellow_ianâs g...       True   \n",
       "\n",
       "   favoriteCount replyToSN              created  truncated    replyToSID  \\\n",
       "0             29       NaN  2017-12-11 16:21:59       True           NaN   \n",
       "1             11   Smerity  2017-12-10 04:16:21       True  9.396588e+17   \n",
       "2              5       NaN  2017-12-09 22:47:22       True           NaN   \n",
       "3             34       NaN  2017-12-09 22:40:48      False           NaN   \n",
       "4            180       NaN  2017-12-09 21:59:28      False           NaN   \n",
       "\n",
       "                   id  replyToUID  \\\n",
       "0  940255351378624512         NaN   \n",
       "1  939710351305670656  15363432.0   \n",
       "2  939627560631132160         NaN   \n",
       "3  939625906775003136         NaN   \n",
       "4  939615505165402112         NaN   \n",
       "\n",
       "                                        statusSource screenName  retweetCount  \\\n",
       "0  <a href=\"http://twitter.com/download/iphone\" r...  benhamner            10   \n",
       "1  <a href=\"http://twitter.com/download/iphone\" r...  benhamner             0   \n",
       "2  <a href=\"http://twitter.com/download/iphone\" r...  benhamner             0   \n",
       "3  <a href=\"http://twitter.com/download/iphone\" r...  benhamner            21   \n",
       "4  <a href=\"http://twitter.com/download/iphone\" r...  benhamner            10   \n",
       "\n",
       "   isRetweet  retweeted  longitude  latitude  \n",
       "0      False      False        NaN       NaN  \n",
       "1      False      False        NaN       NaN  \n",
       "2      False      False        NaN       NaN  \n",
       "3      False      False        NaN       NaN  \n",
       "4      False      False        NaN       NaN  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# read data\n",
    "data = pd.read_csv(r\"data/benhamner.csv\", encoding='latin-1')\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "_uuid": "59c2c50ae9bc0e78831a69efcd4be876109fbe45",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "In his tweets, the rate of occuring of the word 'love' is:  0.0136986301369863\n",
      "@Smerity Lock you in a black box with a window for showing numbers and 10 switches to flip. Randomly dripfeed you câ¦ https://t.co/cBmm2SSoRI\n"
     ]
    }
   ],
   "source": [
    "# find which entries contain the word 'Love'\n",
    "print(\"In his tweets, the rate of occuring of the word 'love' is: \",sum(data.text.str.contains('love'))/len(data))\n",
    "# text\n",
    "text = data.text[1]\n",
    "print(text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "aa0e587f8148155c401577e804c3a85f76f5e18c"
   },
   "source": [
    "<a id=\"6\"></a> <br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "_uuid": "9e7ca63da940c23e9d4098f4d8a9a489c77e0651",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "callouts:  ['@Smerity']\n"
     ]
    }
   ],
   "source": [
    "# find regular expression on text\n",
    "# import regular expression package\n",
    "import re\n",
    "# find callouts that starts with @\n",
    "callouts = [word for word in text.split(\" \") if re.search(\"@[A-Za-z0-9_]+\",word)]\n",
    "print(\"callouts: \",callouts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "b76990b47417e0b8ed2687849763518d54131855"
   },
   "source": [
    "* Lets look at this @[A-Za-z0-9_]+ expression more detailed\n",
    "    * @: we say that our searched word start with @\n",
    "    * [A-Za-z0-9_]: @ is followed by upper or lower case letters, digits or underscore\n",
    "    * +: there can be more than one @. So with \"+\" sign we say that the words which start with @ can be occured more than one times."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "_uuid": "6a9bed92566f8f89a528d6dae7fa868ce96029e4",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "callouts:  ['@Smerity']\n"
     ]
    }
   ],
   "source": [
    "# continue finding regular expressions\n",
    "# [A-Za-z0-9_] =\\w\n",
    "# We will use \"\\w\" to find callouts and our result will be same because \\w matches with [A-Za-z0-9_] \n",
    "callouts1 = [word for word in text.split(\" \") if re.search(\"@\\w+\",word)]\n",
    "print(\"callouts: \",callouts1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "264778d4ff52535ef72c9e15e93c3810fb7d387a"
   },
   "source": [
    "<a id=\"7\"></a> <br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "_uuid": "76085e59be9195f2b74b5969ee38719bee33859e",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['w', 'w', 'w', 'w', 'w']\n",
      "['@', 'S', 'm', 'e', 'r', 'i', 't', 'y', ' ', 'L', 'o', 'c', 'k', ' ', 'y', 'o', 'u', ' ', 'i', 'n', ' ', 'a', ' ', 'b', 'l', 'a', 'c', 'k', ' ', 'b', 'o', 'x', ' ', 'i', 't', 'h', ' ', 'a', ' ', 'i', 'n', 'd', 'o', ' ', 'f', 'o', 'r', ' ', 's', 'h', 'o', 'i', 'n', 'g', ' ', 'n', 'u', 'm', 'b', 'e', 'r', 's', ' ', 'a', 'n', 'd', ' ', '1', '0', ' ', 's', 'i', 't', 'c', 'h', 'e', 's', ' ', 't', 'o', ' ', 'f', 'l', 'i', 'p', '.', ' ', 'R', 'a', 'n', 'd', 'o', 'm', 'l', 'y', ' ', 'd', 'r', 'i', 'p', 'f', 'e', 'e', 'd', ' ', 'y', 'o', 'u', ' ', 'c', 'â', '\\x80', '¦', ' ', 'h', 't', 't', 'p', 's', ':', '/', '/', 't', '.', 'c', 'o', '/', 'c', 'B', 'm', 'm', '2', 'S', 'S', 'o', 'R', 'I']\n"
     ]
    }
   ],
   "source": [
    "# find specific characters like \"w\"\n",
    "print(re.findall(r\"[w]\",text))\n",
    "# \"w\"ith, \"w\"indo\"w\", sho\"w\"ing, s\"w\"itches \n",
    "\n",
    "# do not find specific character like \"w\". We will use \"^\" symbol\n",
    "print(re.findall(r\"[^w]\",text))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "1b4d0b3090c0c508fd9896f1b7040c135236a204"
   },
   "source": [
    "* Lets look at regular expressions for date\n",
    "* We will use \"\\d{1,2}[/-]\\d{1,2}[/-]\\d{1,4}\" expression to find dates\n",
    "    * d{1,2}: first number can be 1 or 2 digit\n",
    "    * [/-]: between digits there can be \"/\" or \"-\" symbols\n",
    "    * d{1,4}: last number can be between 1 and 4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "_uuid": "53dc78ef29b7f087ed73fa18150a568e1e678860",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['15-10-2000', '09/10/2005', '15-05-1999', '05/05/99', '05/05/199', '05/05/9']"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Regular expressions for Dates\n",
    "date = \"15-10-2000\\n09/10/2005\\n15-05-1999\\n05/05/99\\n\\n05/05/199\\n\\n05/05/9\"\n",
    "re.findall(r\"\\d{1,2}[/-]\\d{1,2}[/-]\\d{1,4}\",date)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "8e75a144ddd3ec6596c2f52a89bf7702df302feb"
   },
   "source": [
    "<a id=\"8\"></a> <br>\n",
    "## Natural Language Processing (NLP)\n",
    "* Natural language is any language that is used by people in everyday life, like English or Spanish;\n",
    "* Natural language processing is any computation and manipulation of a language to get an information about what the words mean and how the sentences are contructed;\n",
    "* We will use natural language tool kit that is open source library in python;\n",
    "    * It supports most of the NLP tasks\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "_uuid": "de1458d10de38b7af7ded327d61be9225ddd56d3",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: nltk in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (3.6.7)\n",
      "Requirement already satisfied: tqdm in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from nltk) (4.62.3)\n",
      "Requirement already satisfied: regex>=2021.8.3 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from nltk) (2021.11.10)\n",
      "Requirement already satisfied: joblib in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from nltk) (1.1.0)\n",
      "Requirement already satisfied: click in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from nltk) (8.0.3)\n",
      "Requirement already satisfied: colorama in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from click->nltk) (0.4.4)\n",
      "Requirement already satisfied: importlib-metadata in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from click->nltk) (4.8.2)\n",
      "Requirement already satisfied: typing-extensions>=3.6.4 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from importlib-metadata->click->nltk) (3.10.0.2)\n",
      "Requirement already satisfied: zipp>=0.5 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from importlib-metadata->click->nltk) (3.6.0)\n",
      "number of words:  23\n",
      "number of unique words:  21\n",
      "first 5 unique words:  ['@Smerity', 'window', 'a', 'https://t.co/cBmm2SSoRI', 'câ\\x80¦']\n",
      "frequency of words:  <FreqDist with 21 samples and 23 outcomes>\n",
      "words in text:  dict_keys(['@Smerity', 'Lock', 'you', 'in', 'a', 'black', 'box', 'with', 'window', 'for', 'showing', 'numbers', 'and', '10', 'switches', 'to', 'flip.', 'Randomly', 'dripfeed', 'câ\\x80¦', 'https://t.co/cBmm2SSoRI'])\n",
      "the word box is occured how many times: 1\n"
     ]
    }
   ],
   "source": [
    "!pip install nltk\n",
    "# import natural language tool kit\n",
    "import nltk as nlp\n",
    "\n",
    "\n",
    "# counting the words\n",
    "text = data.text[1]\n",
    "splitted = text.split(\" \")\n",
    "print(\"number of words: \",len(splitted))\n",
    "\n",
    "# counting the number of unique words\n",
    "text = data.text[1]\n",
    "print(\"number of unique words: \",len(set(splitted)))\n",
    "\n",
    "# print first five unique words\n",
    "print(\"first 5 unique words: \",list(set(splitted))[:5])\n",
    "\n",
    "# frequency of words \n",
    "dist = nlp.FreqDist(splitted)\n",
    "print(\"frequency of words: \",dist)\n",
    "\n",
    "# look at keys in the dictionary\n",
    "print(\"words in text: \",dist.keys())\n",
    "\n",
    "# count how many time a particalar value occurs. Lets look at \"box\"\n",
    "print(\"the word box is occured how many times:\",dist[\"box\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "d033fe87cec8adb5d7b77b3c438000aae72163d1"
   },
   "source": [
    "<a id=\"9\"></a> <br>\n",
    "## Normalization and Stemming words\n",
    "* Normalization refers to different forms of the same word like have and having\n",
    "* Stemming is finding a root of the words like having => have\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "_uuid": "fc58b5041d135fafc5a2bff10e3cd72e4d84c895",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "normalized words:  ['task', 'tasked', 'tasks', 'tasking']\n",
      "roots of task Tasked tasks tasking:  ['task', 'task', 'task', 'task']\n"
     ]
    }
   ],
   "source": [
    "# normalization\n",
    "words = \"task Tasked tasks tasking\"\n",
    "words_list = words.lower().split(\" \")\n",
    "print(\"normalized words: \",words_list)\n",
    "\n",
    "# stemming\n",
    "porter_stemmer = nlp.PorterStemmer()\n",
    "roots = [porter_stemmer.stem(each) for each in words_list]\n",
    "print(\"roots of task Tasked tasks tasking: \",roots)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "ff8bbe08d0b42ca72cb1e47e626527bad07d7d7a"
   },
   "source": [
    "<a id=\"10\"></a> <br>\n",
    "## Lemmatization\n",
    "* It is also stemming but resulting stems are all valid words"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package wordnet to\n",
      "[nltk_data]     C:\\Users\\ascos\\AppData\\Roaming\\nltk_data...\n",
      "[nltk_data]   Package wordnet is already up-to-date!\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import nltk\n",
    "nltk.download('wordnet')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package omw-1.4 to\n",
      "[nltk_data]     C:\\Users\\ascos\\AppData\\Roaming\\nltk_data...\n",
      "[nltk_data]   Package omw-1.4 is already up-to-date!\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    " nltk.download('omw-1.4')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "_uuid": "902265ac9ba0a6d1ef27dfed933eb46ca8f5434c",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "result of stemming:  ['univers', 'recognit', 'becom', 'be', 'happen']\n",
      "result of lemmatization:  ['Universal', 'recognition', 'Become', 'being', 'happened']\n"
     ]
    }
   ],
   "source": [
    "# stemming\n",
    "stemming_word_list = [\"Universal\",\"recognition\",\"Become\",\"being\",\"happened\"]\n",
    "porter_stemmer = nlp.PorterStemmer()\n",
    "roots = [porter_stemmer.stem(each) for each in stemming_word_list]\n",
    "print(\"result of stemming: \",roots)\n",
    "\n",
    "# lemmatization\n",
    "lemma = nlp.WordNetLemmatizer()\n",
    "lemma_roots = [lemma.lemmatize(each) for each in stemming_word_list]\n",
    "print(\"result of lemmatization: \",lemma_roots)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "ac7a394d97e0fa295e052733fd77608db20d3d41"
   },
   "source": [
    "<a id=\"11\"></a> <br>\n",
    "## Tokenization\n",
    "* Splitting a sentece into words(tokes)\n",
    "* Learn tokenize with nltk"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package punkt to\n",
      "[nltk_data]     C:\\Users\\ascos\\AppData\\Roaming\\nltk_data...\n",
      "[nltk_data]   Package punkt is already up-to-date!\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import nltk\n",
    "nltk.download('punkt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "_uuid": "29582aada7b17bba4f23ab76641b7252e8b95368",
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "split the sentece:  ['You’re', 'in', 'the', 'right', 'place!']\n",
      "tokenize with nltk:  ['You', '’', 're', 'in', 'the', 'right', 'place', '!']\n"
     ]
    }
   ],
   "source": [
    "text_t = \"You’re in the right place!\"\n",
    "print(\"split the sentece: \", text_t.split(\" \"))  # 5 words\n",
    "\n",
    "# tokenization with nltk\n",
    "print(\"tokenize with nltk: \",nlp.word_tokenize(text_t))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "35df7948a79bf3b774e2f8dcd61b209a38540bfe"
   },
   "source": [
    "<a id=\"12\"></a> <br>\n",
    "# Text Classification\n",
    "*  Classify male and female according to their tweets(description)\n",
    "* import twitter data set from \"Twitter User Gender Classification\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "_uuid": "33ffff53a5a166b92fc43f8252aaee1bd9074ce4",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# %% import data\n",
    "data = pd.read_csv(r\"data/gender-classifier-DFE-791531.csv\",encoding='latin1')\n",
    "\n",
    "# concat gender and description\n",
    "data = pd.concat([data.gender,data.description],axis=1)\n",
    "\n",
    "# drop nan values\n",
    "data.dropna(inplace=True,axis=0)\n",
    "\n",
    "# convert genders from female and male to 1 and 0 respectively\n",
    "data.gender = [1 if each == \"female\" else 0 for each in data.gender] "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "_uuid": "47eb761ea96d61cf38372073cbf57cc38bb839c5",
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package stopwords to\n",
      "[nltk_data]     C:\\Users\\ascos\\AppData\\Roaming\\nltk_data...\n",
      "[nltk_data]   Package stopwords is already up-to-date!\n"
     ]
    }
   ],
   "source": [
    "# import re # regular expression library\n",
    "# # %% remove words that are not important as: \"a, the, that, and, in\" \n",
    "# import nltk\n",
    "import nltk as nlp\n",
    "nltk.download(\"stopwords\")  # stopwords = (irrelavent words)\n",
    "from nltk.corpus import stopwords \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "review_list=list(data[\"description\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "_uuid": "144258d2fc56c53d68a76b10042572063c82b521",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: scikit-learn in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (1.0.2)\n",
      "Requirement already satisfied: numpy>=1.14.6 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from scikit-learn) (1.21.5)\n",
      "Requirement already satisfied: threadpoolctl>=2.0.0 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from scikit-learn) (3.0.0)\n",
      "Requirement already satisfied: joblib>=0.11 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from scikit-learn) (1.1.0)\n",
      "Requirement already satisfied: scipy>=1.1.0 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from scikit-learn) (1.7.3)\n",
      "Most used 150 words: ['10', '14', '15', '16', '17', '18', '19', '20', '2015', '__', 'account', 'art', 'artist', 'author', 'based', 'beautiful', 'best', 'better', 'big', 'blog', 'blogger', 'book', 'books', 'born', 'business', 'check', 'city', 'com', 'community', 'continuous', 'dad', 'day', 'design', 'designer', 'digital', 'director', 'don', 'editor', 'email', 'enthusiast', 'events', 'family', 'fan', 'fashion', 'father', 'follow', 'food', 'football', 'founder', 'free', 'friend', 'friends', 'fun', 'game', 'gamer', 'games', 'girl', 'gmail', 'god', 'good', 'got', 'great', 'happy', 'hard', 'health', 'heart', 'help', 'high', 'home', 'http', 'https', 'husband', 'ig', 'im', 'instagram', 'internet', 'just', 'know', 'latest', 'life', 'like', 'little', 'live', 'living', 'll', 'love', 'lover', 'loves', 'make', 'man', 'marketing', 'media', 'member', 'mind', 'mom', 'music', 'need', 'new', 'news', 'official', 'old', 'online', 'owner', 'page', 'people', 'personal', 'photographer', 'play', 'politics', 'price', 'professional', 'proud', 'radio', 'real', 'right', 'say', 'sc', 'school', 'snapchat', 'social', 'sports', 'state', 'student', 'stuff', 'talk', 'team', 'tech', 'technology', 'things', 'time', 'travel', 'tv', 'tweet', 'tweets', 'twitter', 'uk', 'university', 'updates', 'video', 'views', 'want', 'way', 'wife', 'work', 'world', 'writer', 'year', 'years', 'ªá', 'êû']\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages\\sklearn\\utils\\deprecation.py:87: FutureWarning: Function get_feature_names is deprecated; get_feature_names is deprecated in 1.0 and will be removed in 1.2. Please use get_feature_names_out instead.\n",
      "  warnings.warn(msg, category=FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "# %% creating a bag of model-words\n",
    "!pip install scikit-learn\n",
    "from sklearn.feature_extraction.text import CountVectorizer  # for bag of words \n",
    "max_features = 150 # max_features dimension reduction \n",
    "count_vectorizer = CountVectorizer(stop_words = \"english\",max_features = max_features)  \n",
    "\n",
    "sparce_matrix = count_vectorizer.fit_transform(review_list).toarray() \n",
    "\n",
    "print(\"Most used {} words: {}\".format(max_features,count_vectorizer.get_feature_names()))\n",
    "\n",
    "y = data.iloc[:,0].values "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "_uuid": "627fd0ca8b9c9ebf9966d1bebf307243ca3b92e1",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# train test split\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "x_train, x_test, y_train, y_test = train_test_split(sparce_matrix,y,test_size = 0.1,random_state = 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "d30dc0c6970ff75b329ad12ca6cd117a513e9e1a"
   },
   "source": [
    "* As a classifier we use RandomForrest."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "_uuid": "636af2788fbbf7323fa5311607fcb5fcd5c6a9d3",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestClassifier()"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# %% naive bayes classifier\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "rf = RandomForestClassifier()\n",
    "rf.fit(sparce_matrix,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "_uuid": "610196f8904d8e86540388836a38203d68fe7cf7",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# %% predict\n",
    "y_pred = rf.predict(x_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "78.00369685767097"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score\n",
    "accuracy_score(y_test, y_pred)*100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "_uuid": "a19eecab94635b192fece6c996d101b1906ccde5",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[968,  56],\n",
       "       [301, 298]], dtype=int64)"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#%%\n",
    "from sklearn.metrics import confusion_matrix\n",
    "cm = confusion_matrix(y_test,y_pred)\n",
    "cm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "_uuid": "4291363543ad57bab64708e1fe7460fd00f34dd9",
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: matplotlib in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (3.5.1)\n",
      "Requirement already satisfied: packaging>=20.0 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from matplotlib) (21.3)\n",
      "Requirement already satisfied: numpy>=1.17 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from matplotlib) (1.21.5)\n",
      "Requirement already satisfied: cycler>=0.10 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from matplotlib) (0.11.0)\n",
      "Requirement already satisfied: fonttools>=4.22.0 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from matplotlib) (4.28.5)\n",
      "Requirement already satisfied: pillow>=6.2.0 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from matplotlib) (9.0.0)\n",
      "Requirement already satisfied: python-dateutil>=2.7 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from matplotlib) (2.8.2)\n",
      "Requirement already satisfied: kiwisolver>=1.0.1 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from matplotlib) (1.3.2)\n",
      "Requirement already satisfied: pyparsing>=2.2.1 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from matplotlib) (3.0.4)\n",
      "Requirement already satisfied: six>=1.5 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from python-dateutil>=2.7->matplotlib) (1.16.0)\n",
      "Requirement already satisfied: seaborn in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (0.11.2)\n",
      "Requirement already satisfied: pandas>=0.23 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from seaborn) (1.3.5)\n",
      "Requirement already satisfied: matplotlib>=2.2 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from seaborn) (3.5.1)\n",
      "Requirement already satisfied: numpy>=1.15 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from seaborn) (1.21.5)\n",
      "Requirement already satisfied: scipy>=1.0 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from seaborn) (1.7.3)\n",
      "Requirement already satisfied: cycler>=0.10 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from matplotlib>=2.2->seaborn) (0.11.0)\n",
      "Requirement already satisfied: packaging>=20.0 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from matplotlib>=2.2->seaborn) (21.3)\n",
      "Requirement already satisfied: pyparsing>=2.2.1 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from matplotlib>=2.2->seaborn) (3.0.4)\n",
      "Requirement already satisfied: fonttools>=4.22.0 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from matplotlib>=2.2->seaborn) (4.28.5)\n",
      "Requirement already satisfied: pillow>=6.2.0 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from matplotlib>=2.2->seaborn) (9.0.0)\n",
      "Requirement already satisfied: python-dateutil>=2.7 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from matplotlib>=2.2->seaborn) (2.8.2)\n",
      "Requirement already satisfied: kiwisolver>=1.0.1 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from matplotlib>=2.2->seaborn) (1.3.2)\n",
      "Requirement already satisfied: pytz>=2017.3 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from pandas>=0.23->seaborn) (2021.3)\n",
      "Requirement already satisfied: six>=1.5 in c:\\users\\ascos\\anaconda3\\envs\\py37\\lib\\site-packages (from python-dateutil>=2.7->matplotlib>=2.2->seaborn) (1.16.0)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUIAAAE9CAYAAAB+2WuIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAfTklEQVR4nO3dd5wV1fnH8c+zjSbdBZEuEhRRAYGAFUUsRMXelSQkqDG24M/YorHG9tMEf7EQ0WBEVBQFFQUVsSRiQdAIiGBlKYpIkyLs7vP7487iBbZxndl778737WtezMydmXOuVx6fM+fMGXN3RETiLCfdFRARSTcFQhGJPQVCEYk9BUIRiT0FQhGJPQVCEYm9vHRXoEJmGtcjkg7ulsppm779LKW/s/k77pJSeWHK3EAIbFr2abqrICnKL+xEXv7O6a6GpKA41RNLS8KsRo3K6EAoIlnES9Ndg5QpEIpIOEoVCEUk5lwZoYjEnjJCEYk9ZYQiEnvqNRaR2MvijFBPlohI7CkjFJFwqLNEROJOw2dERJQRikjsKSMUkdjT8BkRiT1lhCISe7pHKCKxp4xQRGJPGaGIxJ27OktEJO7UNBaR2FPTWERiTxmhiMSeBlSLSOwpIxSR2Mvie4SamFVEYk8ZoYiEQ01jEYm9LG4aKxCKSDgUCEUk7vSInYiIMkIRiT11lohI7CkjFJHYU0YoIrGnjFBEYk8ZoYjEnjJCEYk9BUIRiT01jUUk9pQRikjsKSMUkdjL4oxQE7OKSOwpIxSRcKhpLCKxp6axiMReaWlqSzWY2SVmNtvMPjKzsWZW18w6mtnbZrbAzB43s4Lg2DrB9oLg8w5VXV+BUETC4Z7aUgUzaw1cCPRy925ALnAqcCtwl7vvCqwAhganDAVWBPvvCo6rlAKhiIQjwoyQxG28emaWB9QHlgCHAE8Gn48Gjg3WBwfbBJ8PMDOr7OIKhCISjogCobsvAu4AviIRAFcBM4CV7l4cHFYEtA7WWwMLg3OLg+ObV1aGAqGIhMNLU1rMbJiZvZe0DEu+rJk1JZHldQR2BhoAR4RZdfUai0g4Uuw1dveRwMhKDjkU+NzdlwGY2XhgP6CJmeUFWV8bYFFw/CKgLVAUNKUbA8srq4MyQhEJR0SdJSSaxH3NrH5wr28AMAd4FTgxOGYIMCFYnxhsE3w+1b3ygpQRikg4IhpH6O5vm9mTwPtAMTCTRAb5PPCYmd0Y7BsVnDIK+JeZLQC+I9HDXCkFQhEJR4QDqt39WuDarXZ/BvQp59gNwEnbc30FQhEJhx6xE5G489Jq3e/LSAqEIhKOLH7WOPJAaGb1gHbuPi/qskQkjbK4aRzp8BkzOxqYBbwYbHc3s4lRlikiaVLqqS0ZIOpxhH8m0auzEsDdZ5EYHS4ikjGibhpvcvdVWz3vnBn/CxCRcOkeYYVmm9npQK6ZdSYxlc5/Ii5TRNJBgbBCFwBXAT8AY4HJwA0Rl1nj/vXEMzw18UXcnROPOYKzTjkOgDHjJvDY+OfIycnhwH37MPz8oWwqLubav/yVuZ98SnFJCcccMYDfnn3KNtcsWryU/7n2FlauWk3XLp255ZpLyc/Pr+mvFjsLPpnOmu+/p6SklOLiYvr2GwTA+b/7Feed90tKSkp44YVXuPyKm7Y59/DD+nPnndeTm5PDgw+N5bbb/17T1U+v6j0ul5EiDYTuvo5EILwqynLSaf5nX/DUxBcZ+8Bfyc/L59zhV3PQfj9n6dfLePXN6Tw1+u8UFBSwfMVKAKZMfYONmzbx9L/uZf2GDQw+4xwGDexP61Ytt7juXfc+yFmnHMugQ/tz3W1389Rzkzn1uKPS8A3j59CBJ7F8+YrN2/0P2pdjjj6cnvsMZOPGjRQWbjujU05ODiP+dhNHDDqNoqIlTH9rEs8+N4W5c+fXZNXTSxnhlszsWSq5F+jux0RRbjp89sVC9tyjC/Xq1gWgV/c9efm1fzP74/kMPfNkCgoKAGjetAkAZsb6DRsoLi7hhx82kp+fzw4N6m9xTXfn7RkfcOu1fwRg8KBDuWfUIwqEaXLOOWdz2+1/Z+PGjQAsW7btRCZ9evfg00+/4PPPvwLgiScmcMzRh8csECoj3NodEV034+y6S3tGjBzNylWrqVOngDfeepc9duvMF18tYsYHHzFi5GjqFOQz/Pe/Yc/duzDw4P2Z+sZbHDz4dDZs+IHLLhxG40YNt7jmylWrabhDA/LycgFoWbgj35Tzl0/C5+68MGks7s4//vEID4waQ+fOu7D//n244frLEr/ZH2/gvRkfbHHezq13YmHR4s3bRYuW0Kd3j5qufnpl8TjCSAKhu78WxXUzUacO7fj1GScx7JKrqFe3Ll0670JOTg4lJSWsXr2GR0fexUdzP+HSP/2FF8c9xH/nzCM3J4epE8awes33DDnvUvr26kHb1q3S/VUEOOjg41i8eCmFhc158YXHmDdvAXl5uTRt2oR99z+a3r26M/bR++jcpV+6q5p5sjgjjHpAdWcze9LM5pjZZ2VLJcdvnqm2slkaM80JRx/OEw/ezeh7bqdRw4Z0aNeGli125NCD9sPM2LNrF8yMFStXMemlaezXtxf5eXk0b9qE7nt1ZfbHWzafmjRuxJrv11JcXALA18u+pUU596UkfIsXLwUSzd8JE16gd+/uLCpawjPPvADAu+/NorS0lB13bLbleYuW0rbNzpu327RutflaceGlpSktmSDqAdUPAfeSmEPsYOBh4JGKDnb3ke7ey917DavooAxU1hGyZOk3vPLavxk0sD+HHNCPd95PNJ+++KqITcXFNG3SmFYtC3knaFatW7+BD2d/TMf2bbe4npnRp+deTJn2BgATJr3MIQcoA4la/fr12GGHBpvXBx56ELNnz2PCxMn0778vAJ0770JBQQHffvvdFue++94sdt21Ix06tCU/P5+TTx7Ms89NqfHvkFZZ/GRJ1MNn6rn7K2Zm7v4l8GczmwFcE3G5NeqSK29k5erV5OXlcdXw39Go4Q4cf9RhXH3zXRx75rnk5+dx89XDMTNOO/5orr75TgafcQ6Oc+ygw+iya+Jhm/OG/4nrLr+YFoXNueS8X/M/197C3SMfZvefdeL4ow5L87es/Vq2LOTJcYm5PfPycnnssWeYPGUa+fn5PPCP/2XWzFfYuHETvx56MQCtWrVk5H23c/TgsykpKeGii69m0vOPkpuTwz9HP86cOZ+k8dukQRbfI7QqZrD+aRc3+w+wP4lX6k0l8S6BW9y9SzVO9k3LPo2sbhKt/MJO5OXvXPWBknGKNy6q9NWXFVl7/RkpBZMG14xJqbwwRZ0RXkTiHaQXkhhIfQg/vktARGqTDLnfl4qoB1S/G6x+D/wqyrJEJM0y5H5fKqIaUF3pVFu1aUC1iASy+B5hVBlhPxJvmh8LvA2k/R6AiERMGeE2dgIGAqcBp5N47d5Yd58dUXkikmaZMiYwFZGMI3T3End/0d2HAH2BBcA0M/t9FOWJiPwUkXWWmFkd4BckssIOwAjg6ajKE5E0U9N4S2b2MNANmARc5+4fRVGOiGQQBcJtnAmsJTGO8MKkqfoNcHdvFFG5IpIu6jXekrtH/QyziGQaZYQiEneuQCgisadAKCKxl8XjCBUIRSQcyghFJPYUCEUk7qKc2zRqCoQiEg5lhCISewqEIhJ3GkcoIqJAKCKxl73DCBUIRSQcahqLiGRxINQsMSISe8oIRSQcukcoInGne4QiIsoIRSTulBGKiCgjFJG4y+J3NykQikhIFAhFJO6UEYqIKBCKSNxlc0aoR+xEJBRemtpSHWbWxMyeNLOPzWyumfUzs2Zm9pKZzQ/+bBoca2Y2wswWmNmHZtazqusrEIpIKKIMhMDfgBfdfTdgb2AucDnwirt3Bl4JtgGOBDoHyzDg3qourkAoIuFwS22pgpk1Bg4ERgG4+0Z3XwkMBkYHh40Gjg3WBwMPe8J0oImZtaqsDAVCEQlFhBlhR2AZ8JCZzTSzB8ysAdDS3ZcExywFWgbrrYGFSecXBfsqpEAoIqHwUktpMbNhZvZe0jJsq0vnAT2Be929B7CWH5vBibIT7xJN+Rk/9RqLSChS7TV295HAyEoOKQKK3P3tYPtJEoHwazNr5e5LgqbvN8Hni4C2See3CfZVSBmhiGQ0d18KLDSzLsGuAcAcYCIwJNg3BJgQrE8Ezg56j/sCq5Ka0OVSRigiofBqdHz8BBcAY8ysAPgM+BWJRO4JMxsKfAmcHBw7CRgELADWBcdWSoFQREIR5YBqd58F9CrnowHlHOvA+dtzfQVCEQmFl0aaEUZKgVBEQuHZOy+rAqGIhEMZoYjEngKhiMSemsYiEnvKCEUk9iIeRxgpBUIRCUU2T8yqQCgioShVRigicaemsYjEnjpLRCT2snn4TJXTcAVT2ZxpZtcE2+3MrE/0VRORbJLqxKyZoDrzEd4D9ANOC7bXAH+PrEYikpVK3VJaMkF1msY/d/eeZjYTwN1XBHOCiYjUCtUJhJvMLJfgfQBmVkhWv9NeRKKQzb3G1WkajwCeBlqY2U3Am8DNkdZKRLKOe2pLJqgyI3T3MWY2g8RMsAYc6+5zI6+ZiGSVTLnfl4oqA6GZtSMx7/+zyfvc/asoKyYi2SWbm8bVuUf4PIn7gwbUJfGy5XnAHhHWS0SyTKY0c1NRnabxnsnbZtYT+F1kNRKRrFSrm8Zbc/f3zeznUVRma/mFnWqiGIlI8abF6a6C1KBa3TQ2sz8kbeYAPYEa+S/8vPYn1UQxEoF7vxzHOe1PTHc1JAX3p3hebc8IGyatF5O4Z/hUNNURkWyVxbcIKw+EwUDqhu5+aQ3VR0SyVK3MCM0sz92LzWy/mqyQiGSn2nqP8B0S9wNnmdlEYBywtuxDdx8fcd1EJItk83O31blHWBdYDhzCj+MJHVAgFJHNnNqZEbYIeow/4scAWCab74uKSARKszgqVBYIc4EdoNwwn8VfWUSiUFpLM8Il7n59jdVERLJaNjeNK5uGK3u/lYjIdqgsIxxQY7UQkaxXK3uN3f27mqyIiGS3bG4a63WeIhKKWpkRiohsDwVCEYk9NY1FJPYy5F3tKVEgFJFQ1NYB1SIi1ZbNj5spEIpIKNRZIiKxV2pqGotIzKlpLCKxp6axiMSehs+ISOxp+IyIxJ7uEYpI7GVz07iyiVlFRGJBGaGIhCKbe42VEYpIKDzFpTrMLNfMZprZc8F2RzN728wWmNnjZlYQ7K8TbC8IPu9QnesrEIpIKEottaWaLgLmJm3fCtzl7rsCK4Chwf6hwIpg/13BcVVSIBSRUJSmuFTFzNoAvwAeCLYNOAR4MjhkNHBssD442Cb4fEBwfKUUCEUkFFEFQuCvwGVJhzcHVrp7cbBdBLQO1lsDCwGCz1cFx1dKgVBEQuGW2mJmw8zsvaRlWNk1zewo4Bt3nxFl3dVrLCKhSLXX2N1HAiMr+Hg/4BgzGwTUBRoBfwOamFlekPW1ARYFxy8C2gJFZpYHNAaWV1UHZYQiEooomsbufoW7t3H3DsCpwFR3PwN4FTgxOGwIMCFYnxhsE3w+1d2r7JxWIBSRUEQ5fKYcfwT+YGYLSNwDHBXsHwU0D/b/Abi8OhdT01hEQhH1I3buPg2YFqx/BvQp55gNwEnbe20FQhEJRTY/WaJAKCKhUCAUkdjTNFwiEnvZPA2XAqGIhEJNYxGJPTWNRST2SrM4FGpAtYjEnjJCEQmF7hGKSOxlb8NYgVBEQqKMUERiT+MIRST2srnXWIFQREKRvWEw4uEzlnCmmV0TbLczs22mzhGR7BfhO0siF/U4wnuAfsBpwfYa4O8RlykiaVCKp7Rkgqibxj93955mNhPA3VeUvYhZRGqXzAhpqYk6EG4ys1yCf0dmVkjmZMMiEqJs/osddSAcATwNtDCzm0i8TOXqiMsUkTTIlGZuKiINhO4+xsxmAAMAA45197lRliki6ZG9YTCiQGhmzZI2vwHGJn/m7t9FUa6IpI+axtuaQeJ/EMljzcu2HdglonJFJE08i3PCSAKhu3eM4roikrmUEVbCzJoCnYG6Zfvc/fWoyxWRmqXOkgqY2W+Ai4A2wCygL/AWcEiU5YqIbI+oM8KLgN7AdHc/2Mx2A26OuMwal1cnn+GPX0denTxycnOZ+cJ0nrtrHM3bFDL0/y6mQZOGfPXRZ/zzkrsp2VTCrn1256RrhtB6t/aMuuCvzHzh7XKv265bR86+43zy6xYw+9WZPHHdQzX8zWq/pq2a86s7f0/DHZuAO2+MfZmpD02ize7tOeOmYdSpX5flRd8w6uIRbPh+PTl5uZx967m022MXcvJymD7+NV6855ltrtu8TQt+m/TbP3jJ3ZRsKq7x71eTsjcfjP4Ruw3uvgHAzOq4+8dAl4jLrHHFP2zir6dfx01HXsZNgy6j60Hd6dijM8ddfiZTRz3Ptf0vZN2qtex3SiIR/m7xtzx86T28O+HNSq972o2/ZcwV93Nt/wtp0XEn9ujfvQa+TbyUFJcw7saHuW7gJdxy3JX0P+twWu3ahrNuOZfxt47h+iOGM3PyOxw27BgA9hnUj7yCfK4/Yjg3HfVHDjh9IM3bFG5z3eMvP4OXRz3Hn/pfwNpV32/+7WuzbH7ELupAWGRmTYBngJfMbALwZcRlpsUP634AIDcvl9y8XNydLvvuwfuTpgMw/alp7H1YbwC+K1rGoo+/wr3i/wgaFTahbsN6fD5zfuL88a9vPl/Cs3rZShbO/hyAH9ZuYMmni2iyUzNadtyZ+W/PAWDumx/S48i+wRlOnXp1yMnNoaBuASUbi1m/Zv02191t325Jv/1rdI/Bb5fNky5EPaD6uGD1z2b2KtAYeDHKMtPFcowrnruVwvY78dq/JrPsy69Zt3odpSWJn3rlku9o0rJZFVf5UZOdmrFyyfLN2yuXLN+u82X7NW9TSLuuHfl81nwWz1/I3of15oMp77LPoH40a9UcgBmTprP3wN7c9s4/KKhXwLgbRrNu1fdbXKdB04Zb/PYrYvLbZfPwmcjfYmdmTc1sLxIzzxQB3aIuMx281Ll50GVc2e9cOuzdiZ067ZzuKsl2qFO/LufceylPXP8QG75fz+jL7qH/mYdz5bO3UneHuhQH9/c67r0rpSWlXPbzYVx1wPkc+puj2bFtizTXPjMoI6yAmd0A/BL4jB+/s1NBr7GZDQOGAdwfZcUitH71Oj55aza79PwZ9RvVJyc3h9KSUpq0asbKr6v/QM3Kpd/RJMhCAJq0ar5d50v15eTlcs59w3nnmTeYOfkdAL7+dDF/O/tGAFp0bEW3g/cBoM/g/Zn92ixKi0tYs3w1n874mPZ7deLbhd9svt7aFWu2+O2bxuS3U0ZYsZOBTu5+kLsfHCwV3jV295Hu3svdew2LuGJh2qFZQ+o1qg9Afp18dt9/L5YsWMS8t2bTc1Di3lLfE/rzwZT3qn3N1ctWsmHNejr26Jw4//gDt+t8qb6zbz2PpQsW8fKo5zbva9i8EQBmxqDfn8DrY6YAiY6u3fZNNGoK6tWhY4+fsfTTRdtcc8vf/iA+mPJu1F8j7ZQRVuwjoAmJ541rrcYtmjLkf8/HcnLIyTFmPP8WH019n6Xzixh698UcPfxUFs7+nP88MRWA9nt14pz7L6V+4wbsOWAfjrrkZG44bDgAV066jZsHXQbA2D89wJA7fpcYPjNtFrOnzUzbd6ytOvXajX4nHETR3C+5etLtADxz26O06NiK/mcdDsDMye/wn3GvAjDt4ckMuf13XDvlTjDjrXGvsujjrwD4/UNX8K8/3seqb1Yw/pZH+M3dlzB4+GksnP05/w5++9qstJLOv0xnlfVc/uSLm/UCJpAIiD+U7Xf3Y6pxsp/X/qTI6ibRuvfLcZzT/sR0V0NScP8X41J6H92Z7Y9PKZg88uX4tL//LuqMcDRwK/BfMicLFpEIZMqYwFREHQjXufuIiMsQkQyQzZ0lUQfCN8zsL8BEtmwavx9xuSJSw7K5yRd1IOwR/Nk3aV+Fw2dEJHupaVwBdz84yuuLSObI5qZx1C94b2lmo8zshWC7q5kNjbJMEUmPbB5HGPWA6n8Ck4Gy580+AS6OuEwRSQN3T2nJBFEHwh3d/QmCwO/uxUBJxGWKiGyXqDtL1ppZc358wXtfYFXEZYpIGqizpGJ/IDF0ppOZ/RsoJPGSdxGpZTLlfl8qonqvcTt3/8rd3zezg0jMSm3APHffFEWZIpJe2dxrHFVG+AzQM1h/3N1PiKgcEckQahpvK/khar3MXSQGMqUHOBVRBUKvYF1EaindI9zW3ma2mkRmWC9YJ9h2d28UUbkikia6R7gVd8+N4roikrl0j1BEYi+b7xFG/hY7EYmHqF7wbmZtzexVM5tjZrPN7KJgfzMze8nM5gd/Ng32m5mNMLMFZvahmfWsvAQFQhEJiaf4TzUUA8PdvSuJKf3ON7OuwOXAK+7eGXgl2AY4EugcLMOAe6sqQIFQREJR6p7SUhV3X1I2mbO7rwHmAq2BwSReB0Lw57HB+mDgYU+YDjQxs1aVlaFAKCKh8BSX7WFmHUhM+Pw20NLdlwQfLQVaBuutgYVJpxUF+yqkQCgioUj1HqGZDTOz95KWcl9rbmY7AE8BF7v76uTPPNFTk3JvjXqNRSQUqQ6fcfeRwMjKjjGzfBJBcIy7jw92f21mrdx9SdD0LXt/+iKgbdLpbYJ9FVJGKCKhiGpiVjMzYBQw193vTPpoIjAkWB9C4h3qZfvPDnqP+wKrkprQ5VJGKCKZbj/gLOC/ZjYr2HclcAvwRPD6jy+Bk4PPJgGDgAXAOuBXVRWgQCgioYjqyRJ3f5MtJ3JJNqCc4x04f3vKUCAUkVDoWWMRib1sfsROgVBEQqFJF0Qk9pQRikjsKSMUkdhTZ4mIxF51JlDIVAqEIhIKZYQiEnvKCEUk9pQRikjsKSMUkdhTRigisaeMUERiTxmhiMSee2m6q5AyzVAtIrGnjFBEQqFnjUUk9jT7jIjEnjJCEYk9ZYQiEnsaRygisadxhCISe2oai0jsqbNERGJPGaGIxJ46S0Qk9pQRikjs6R6hiMSeMkIRiT3dIxSR2NOAahGJPWWEIhJ72XyPUDNUi0jsKSMUkVDoHqGIxF42N40VCEUkFAqEIhJ72RsGwbI5imczMxvm7iPTXQ9JjX6/2kW9xukzLN0VkJ9Ev18tokAoIrGnQCgisadAmD66v5Td9PvVIuosEZHYU0YoIrGnQBgiM3MzeyRpO8/MlpnZc1Wc17+qYyQ8ZlZiZrOSlg4RlvWFme0Y1fUlHBpQHa61QDczq+fu64GBwKI010m2td7du6e7EpI5lBGGbxLwi2D9NGBs2Qdm1sfM3jKzmWb2HzPrsvXJZtbAzB40s3eC4wbXUL1jzcz2MbPXzGyGmU02s1bB/mlmdpeZvWdmc82st5mNN7P5ZnZj0vnPBOfONrNyxxia2ZnB7zrLzO43s9ya+n5SOQXC8D0GnGpmdYG9gLeTPvsYOMDdewDXADeXc/5VwFR37wMcDNxuZg0irnPc1EtqFj9tZvnA3cCJ7r4P8CBwU9LxG929F3AfMAE4H+gG/NLMmgfH/Do4txdwYdJ+AMxsd+AUYL8gGy0BzojuK8r2UNM4ZO7+YXDP6TQS2WGyxsBoM+tM4tHM/HIucRhwjJldGmzXBdoBc6OpcSxt0TQ2s24kAttLZgaQCyxJOn5i8Od/gdnuviQ47zOgLbCcRPA7LjiuLdA52F9mALAP8G5QRj3gm1C/laRMgTAaE4E7gP5AcmZwA/Cqux8XBMtp5ZxrwAnuPi/iOsqPjESA61fB5z8Ef5YmrZdt55lZf+BQoJ+7rzOzaST+B7Z1GaPd/YqwKi3hUdM4Gg8C17n7f7fa35gfO09+WcG5k4ELLEgbzKxHJDWUZPOAQjPrB2Bm+Wa2x3ac3xhYEQTB3YC+5RzzCnCimbUIymhmZu1/asUlHAqEEXD3IncfUc5HtwF/MbOZVJyN30Ciyfyhmc0OtiVC7r4ROBG41cw+AGYB+27HJV4kkRnOBW4BppdTxhzgamCKmX0IvAS0+olVl5DoyRIRiT1lhCISewqEIhJ7CoQiEnsKhCISewqEIhJ7CoQxljQLy0dmNs7M6v+Ea/3TzE4M1h8ws66VHNvfzLZneErZeZrJRSKhQBhv6929u7t3AzYC5yZ/aGYpPXnk7r8Jxs1VpD/bN05PJFIKhFLmDWDXIFt7w8wmAnPMLNfMbjezd83sQzM7B8AS/s/M5pnZy0CLsgsFM7b0CtaPMLP3zewDM3sleLTwXOCSIBs9wMwKzeypoIx3zWy/4NzmZjYlmNHlARKPqYmETs8aS1nmdySJJyQAegLd3P3zYEqpVe7e28zqAP82sylAD6AL0BVoCcwh8Whh8nULgX8ABwbXaubu35nZfcD37n5HcNyjwF3u/qaZtSPxmOHuwLXAm+5+vZn9Ahga6b8IiS0FwnirZ2azgvU3gFEkmqzvuPvnwf7DgL3K7v+ReK62M3AgMNbdS4DFZja1nOv3BV4vu5a7f1dBPQ4FugaPVwM0MrMdgjKOD8593sxWpPY1RSqnQBhv28zUHASjtcm7gAvcffJWxw0KsR45QF9331BOXUQip3uEUpXJwHnB5KWY2c+CiWJfB04J7iG2IjGJ7NamAweaWcfg3GbB/jVAw6TjpgAXlG2YWfdg9XXg9GDfkUDTsL6USDIFQqnKAyTu/71vZh8B95NoSTwNzA8+exh4a+sT3X0ZMAwYH8zq8njw0bPAcWWdJcCFQK+gM2YOP/ZeX0cikM4m0UT+KqLvKDGn2WdEJPaUEYpI7CkQikjsKRCKSOwpEIpI7CkQikjsKRCKSOwpEIpI7CkQikjs/T+qqTnU1q3p9QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 360x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "<Figure size 432x288 with 0 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "!pip install matplotlib\n",
    "!pip install seaborn\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "f,ax = plt.subplots(figsize=(5, 5))\n",
    "g=sns.heatmap(cm, annot=True, linewidths=0.5,linecolor=\"red\", fmt= '.1f',ax=ax)\n",
    "g.set_xticklabels(['Male','Female'])\n",
    "g.set_yticklabels(['Male','Female'])\n",
    "plt.ylabel(\"True\")\n",
    "plt.xlabel(\"Predicted\")\n",
    "plt.show()\n",
    "plt.savefig('graph.png')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}