Project 7 - Natural Language Processing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <center><span style=\"font-family: Arial; font-weight:bold;font-size:1.9em;\">Project - Natural Language Processing</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **Importing all the necessary Libraries**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-02-09T15:19:39.168839Z",
     "iopub.status.busy": "2024-02-09T15:19:39.168252Z",
     "iopub.status.idle": "2024-02-09T15:19:39.177924Z",
     "shell.execute_reply": "2024-02-09T15:19:39.176887Z",
     "shell.execute_reply.started": "2024-02-09T15:19:39.168812Z"
    }
   },
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import plotly.express as px\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.image as mpimg\n",
    "import seaborn as sns\n",
    "%matplotlib inline\n",
    "\n",
    "import tensorflow as tf\n",
    "from tensorflow.keras.optimizers import Adam\n",
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.preprocessing.text import Tokenizer\n",
    "from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
    "from tensorflow.keras.layers import Embedding, LSTM, Dense, Bidirectional, Dropout\n",
    "from sklearn.metrics import classification_report"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <center><span style=\"font-family: Arial; font-weight:bold;font-size:1.9em;\">Part - A</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**`DOMAIN`:** Digital content and entertainment industry\n",
    "\n",
    "**`CONTEXT`:** The objective of this project is to build a text classification model that analyses the customer's sentiments\n",
    "based on their reviews in the IMDB database. The model uses a complex deep learning model to build an embedding layer\n",
    "followed by a classification algorithm to analyse the sentiment of the customers.\n",
    "\n",
    "**`DATA DESCRIPTION`:** The Dataset of 50,000 movie reviews from IMDB, labelled by sentiment (positive/negative). Reviews\n",
    "have been preprocessed, and each review is encoded as a sequence of word indexes (integers). For convenience, the\n",
    "words are indexed by their frequency in the dataset, meaning the for that has index 1 is the most frequent word. Use the\n",
    "first 20 words from each review to speed up training, using a max vocabulary size of 10,000. As a convention, \"0\" does\n",
    "not stand for a specific word, but instead is used to encode any unknown word.\n",
    "\n",
    "**`PROJECT OBJECTIVE`:** To Build a sequential NLP classifier which can use input text parameters to determine the customer\n",
    "sentiments."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Import and analyse the data set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-02-09T15:20:00.712639Z",
     "iopub.status.busy": "2024-02-09T15:20:00.712040Z",
     "iopub.status.idle": "2024-02-09T15:20:05.989405Z",
     "shell.execute_reply": "2024-02-09T15:20:05.988485Z",
     "shell.execute_reply.started": "2024-02-09T15:20:00.712608Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/imdb.npz\n",
      "17464789/17464789 [==============================] - 0s 0us/step\n",
      "Training data shape: (25000,)\n",
      "Training labels shape: (25000,)\n",
      "Test data shape: (25000,)\n",
      "Test labels shape: (25000,)\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.keras.datasets import imdb\n",
    "\n",
    "(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000)\n",
    "\n",
    "print(\"Training data shape:\", train_data.shape)\n",
    "print(\"Training labels shape:\", train_labels.shape)\n",
    "print(\"Test data shape:\", test_data.shape)\n",
    "print(\"Test labels shape:\", test_labels.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Perform relevant sequence adding on the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-02-09T15:20:05.991316Z",
     "iopub.status.busy": "2024-02-09T15:20:05.991031Z",
     "iopub.status.idle": "2024-02-09T15:20:06.830384Z",
     "shell.execute_reply": "2024-02-09T15:20:06.829452Z",
     "shell.execute_reply.started": "2024-02-09T15:20:05.991293Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Padded training data shape: (25000, 200)\n",
      "Padded test data shape: (25000, 200)\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
    "\n",
    "# Set the maximum length of sequences\n",
    "max_length = 200\n",
    "\n",
    "# Pad sequences for both train and test datasets\n",
    "padded_train_data = pad_sequences(train_data, maxlen=max_length, padding='post')\n",
    "padded_test_data = pad_sequences(test_data, maxlen=max_length, padding='post')\n",
    "\n",
    "# Display the shape of padded datasets\n",
    "print(\"Padded training data shape:\", padded_train_data.shape)\n",
    "print(\"Padded test data shape:\", padded_test_data.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, I have to padded the dataset i.e. train and test dataset with the exact same length."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Print shape of features and labels, Print value of any one feature and it's label"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-02-09T15:20:09.047611Z",
     "iopub.status.busy": "2024-02-09T15:20:09.046908Z",
     "iopub.status.idle": "2024-02-09T15:20:09.054770Z",
     "shell.execute_reply": "2024-02-09T15:20:09.053759Z",
     "shell.execute_reply.started": "2024-02-09T15:20:09.047556Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of padded train data: (25000, 200)\n",
      "Shape of train labels: (25000,)\n",
      "Shape of padded test data: (25000, 200)\n",
      "Shape of test labels: (25000,)\n",
      "\n",
      "Example Feature (Sequence):\n",
      "[   5   25  100   43  838  112   50  670    2    9   35  480  284    5\n",
      "  150    4  172  112  167    2  336  385   39    4  172 4536 1111   17\n",
      "  546   38   13  447    4  192   50   16    6  147 2025   19   14   22\n",
      "    4 1920 4613  469    4   22   71   87   12   16   43  530   38   76\n",
      "   15   13 1247    4   22   17  515   17   12   16  626   18    2    5\n",
      "   62  386   12    8  316    8  106    5    4 2223 5244   16  480   66\n",
      " 3785   33    4  130   12   16   38  619    5   25  124   51   36  135\n",
      "   48   25 1415   33    6   22   12  215   28   77   52    5   14  407\n",
      "   16   82    2    8    4  107  117 5952   15  256    4    2    7 3766\n",
      "    5  723   36   71   43  530  476   26  400  317   46    7    4    2\n",
      " 1029   13  104   88    4  381   15  297   98   32 2071   56   26  141\n",
      "    6  194 7486   18    4  226   22   21  134  476   26  480    5  144\n",
      "   30 5535   18   51   36   28  224   92   25  104    4  226   65   16\n",
      "   38 1334   88   12   16  283    5   16 4472  113  103   32   15   16\n",
      " 5345   19  178   32]\n",
      "\n",
      "Corresponding Label:\n",
      "1\n"
     ]
    }
   ],
   "source": [
    "# Print the shape of features and labels\n",
    "print(\"Shape of padded train data:\", padded_train_data.shape)\n",
    "print(\"Shape of train labels:\", train_labels.shape)\n",
    "print(\"Shape of padded test data:\", padded_test_data.shape)\n",
    "print(\"Shape of test labels:\", test_labels.shape)\n",
    "\n",
    "# Print the value of one feature and its corresponding label\n",
    "sample_index = 0  # Changable/variable value to inspect different samples\n",
    "print(\"\\nExample Feature (Sequence):\")\n",
    "print(padded_train_data[sample_index])\n",
    "print(\"\\nCorresponding Label:\")\n",
    "print(train_labels[sample_index])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Decode the feature value to get original sentence"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-02-09T15:20:11.032052Z",
     "iopub.status.busy": "2024-02-09T15:20:11.031697Z",
     "iopub.status.idle": "2024-02-09T15:20:11.236088Z",
     "shell.execute_reply": "2024-02-09T15:20:11.234957Z",
     "shell.execute_reply.started": "2024-02-09T15:20:11.032026Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/imdb_word_index.json\n",
      "1641221/1641221 [==============================] - 0s 0us/step\n",
      "\n",
      "Decoded Sequence:\n",
      "and you could just imagine being there robert ? is an amazing actor and now the same being director ? father came from the same scottish island as myself so i loved the fact there was a real connection with this film the witty remarks throughout the film were great it was just brilliant so much that i bought the film as soon as it was released for ? and would recommend it to everyone to watch and the fly fishing was amazing really cried at the end it was so sad and you know what they say if you cry at a film it must have been good and this definitely was also ? to the two little boy's that played the ? of norman and paul they were just brilliant children are often left out of the ? list i think because the stars that play them all grown up are such a big profile for the whole film but these children are amazing and should be praised for what they have done don't you think the whole story was so lovely because it was true and was someone's life after all that was shared with us all\n"
     ]
    }
   ],
   "source": [
    "# Reverse word index to map integer indices to words\n",
    "word_index = imdb.get_word_index()\n",
    "\n",
    "# Reverse the word index mapping to map integer indices to words\n",
    "reverse_word_index = dict([(value, key) for (key, value) in word_index.items()])\n",
    "\n",
    "# Decode the first sequence in the padded train data\n",
    "decoded_sequence = ' '.join([reverse_word_index.get(i - 3, '?') for i in padded_train_data[sample_index]])\n",
    "\n",
    "print(\"\\nDecoded Sequence:\")\n",
    "print(decoded_sequence)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* We first obtain the word index mapping using imdb.get_word_index().\n",
    "* We then reverse the word index mapping to create a dictionary mapping word indices to words.\n",
    "* Next, we define a function decode_review(sequence) that takes a sequence of word indices as input and returns the corresponding original sentence by mapping each index back to its word using the reverse word index mapping.\n",
    "* Finally, we decode the feature value (padded_train_data[sample_index]) into the original sentence using the decode_review function and print the decoded sentence."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. Design, train, tune and test a sequential model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-02-09T15:20:41.526746Z",
     "iopub.status.busy": "2024-02-09T15:20:41.526120Z",
     "iopub.status.idle": "2024-02-09T15:22:17.980213Z",
     "shell.execute_reply": "2024-02-09T15:22:17.979266Z",
     "shell.execute_reply.started": "2024-02-09T15:20:41.526714Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_1\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " embedding_1 (Embedding)     (None, 200, 128)          1280000   \n",
      "                                                                 \n",
      " bidirectional_2 (Bidirecti  (None, 200, 128)          98816     \n",
      " onal)                                                           \n",
      "                                                                 \n",
      " bidirectional_3 (Bidirecti  (None, 64)                41216     \n",
      " onal)                                                           \n",
      "                                                                 \n",
      " dense_2 (Dense)             (None, 64)                4160      \n",
      "                                                                 \n",
      " dropout_1 (Dropout)         (None, 64)                0         \n",
      "                                                                 \n",
      " dense_3 (Dense)             (None, 1)                 65        \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 1424257 (5.43 MB)\n",
      "Trainable params: 1424257 (5.43 MB)\n",
      "Non-trainable params: 0 (0.00 Byte)\n",
      "_________________________________________________________________\n",
      "None\n",
      "\n",
      "Epoch 1/5\n",
      "196/196 [==============================] - 30s 119ms/step - loss: 0.4944 - accuracy: 0.7477 - val_loss: 0.3146 - val_accuracy: 0.8707\n",
      "Epoch 2/5\n",
      "196/196 [==============================] - 15s 77ms/step - loss: 0.2310 - accuracy: 0.9091 - val_loss: 0.3001 - val_accuracy: 0.8806\n",
      "Epoch 3/5\n",
      "196/196 [==============================] - 12s 60ms/step - loss: 0.1457 - accuracy: 0.9481 - val_loss: 0.2989 - val_accuracy: 0.8759\n",
      "Epoch 4/5\n",
      "196/196 [==============================] - 10s 51ms/step - loss: 0.0882 - accuracy: 0.9718 - val_loss: 0.3723 - val_accuracy: 0.8714\n",
      "Epoch 5/5\n",
      "196/196 [==============================] - 10s 53ms/step - loss: 0.0538 - accuracy: 0.9836 - val_loss: 0.5114 - val_accuracy: 0.8623\n",
      "782/782 [==============================] - 8s 11ms/step - loss: 0.5114 - accuracy: 0.8623\n",
      "• Test Loss: 0.5113770961761475\n",
      "• Test Accuracy: 0.8623200058937073\n",
      "\n",
      "782/782 [==============================] - 9s 10ms/step\n",
      "• Classification Report:\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.83      0.92      0.87     12500\n",
      "           1       0.91      0.81      0.85     12500\n",
      "\n",
      "    accuracy                           0.86     25000\n",
      "   macro avg       0.87      0.86      0.86     25000\n",
      "weighted avg       0.87      0.86      0.86     25000\n",
      "\n"
     ]
    }
   ],
   "source": [
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "model = Sequential([\n",
    "    Embedding(input_dim=10000, output_dim=128, input_length=max_length),\n",
    "    Bidirectional(LSTM(64, return_sequences=True)),\n",
    "    Bidirectional(LSTM(32)),\n",
    "    Dense(64, activation='relu'),\n",
    "    Dropout(0.3),\n",
    "    Dense(1, activation='sigmoid')\n",
    "])\n",
    "\n",
    "model.compile(optimizer=Adam(learning_rate=0.01),\n",
    "              loss='binary_crossentropy',\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "print(model.summary())\n",
    "print()\n",
    "\n",
    "history = model.fit(padded_train_data, train_labels,\n",
    "                    epochs=5,\n",
    "                    batch_size=128,\n",
    "                    validation_data=(padded_test_data, test_labels))\n",
    "\n",
    "# Evaluate the model\n",
    "test_loss, test_accuracy = model.evaluate(padded_test_data, test_labels)\n",
    "print(\"• Test Loss:\", test_loss)\n",
    "print(\"• Test Accuracy:\", test_accuracy)\n",
    "print()\n",
    "\n",
    "# Make predictions\n",
    "predictions = model.predict(padded_test_data)\n",
    "predicted_labels = (predictions > 0.5).astype(int)\n",
    "\n",
    "print(\"• Classification Report:\")\n",
    "print(classification_report(test_labels, predicted_labels))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6. Use the designed model to print the prediction on any one sample."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-02-09T15:22:17.982148Z",
     "iopub.status.busy": "2024-02-09T15:22:17.981857Z",
     "iopub.status.idle": "2024-02-09T15:22:18.059950Z",
     "shell.execute_reply": "2024-02-09T15:22:18.058846Z",
     "shell.execute_reply.started": "2024-02-09T15:22:17.982122Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1/1 [==============================] - 0s 30ms/step\n",
      "Predicted Sentiment: Negative\n"
     ]
    }
   ],
   "source": [
    "# Select a sample from the test dataset\n",
    "sample_index = 10  # Changable value to select a different sample\n",
    "sample_text = padded_test_data[sample_index]  # Get the corresponding sequence\n",
    "\n",
    "# Reshape the sample to match the input shape expected by the model\n",
    "sample_text = sample_text.reshape(1, -1)\n",
    "\n",
    "prediction = model.predict(sample_text)\n",
    "if prediction[0][0] >= 0.5:\n",
    "    print(\"Predicted Sentiment: Positive\")\n",
    "else:\n",
    "    print(\"Predicted Sentiment: Negative\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### *****************************************************************************************************************************"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <center><span style=\"font-family: Arial; font-weight:bold;font-size:1.9em;\">Part - B</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**`DOMAIN`:** Social media analytics.\n",
    "\n",
    "**`CONTEXT`:** Past studies in Sarcasm Detection mostly make use of Twitter datasets collected using hashtag based supervision\n",
    "but such datasets are noisy in terms of labels and language. Furthermore, many tweets are replies to other tweets and\n",
    "detecting sarcasm in these requires the availability of contextual tweets.In this hands-on project, the goal is to build a model\n",
    "to detect whether a sentence is sarcastic or not, using Bidirectional LSTMs.\n",
    "\n",
    "**`DATA DESCRIPTION`:** The dataset is collected from two news websites, **theonion.com and huffingtonpost.com.**\n",
    "This new dataset has the following advantages over the existing Twitter datasets: Since news headlines are written by professionals in a formal manner, there are no spelling mistakes and informal usage. This reduces the sparsity and also increases the chance of finding pre-trained embeddings. Furthermore, since the sole purpose of TheOnion is to publish sarcastic news, we get high-quality labels with much less noise as compared to Twitter datasets. Unlike tweets that reply to other tweets, the news headlines obtained are self-contained. This would help us in teasing apart the real sarcastic elements.\n",
    "Each record consists of three attributes:\n",
    "* is_sarcastic: 1 if the record is sarcastic otherwise 0.\n",
    "* headline: the headline of the news article.\n",
    "* article_link: link to the original news article. Useful in collecting supplementary data.\n",
    "\n",
    "\n",
    "    Reference: https://github.com/rishabhmisra/News-Headlines-Dataset-For-Sarcasm-Detection\n",
    "\n",
    "**`PROJECT OBJECTIVE`:** Build a sequential NLP classifier which can use input text parameters to determine the customer\n",
    "sentiments."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Read and explore the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-02-09T15:22:53.623619Z",
     "iopub.status.busy": "2024-02-09T15:22:53.623242Z",
     "iopub.status.idle": "2024-02-09T15:22:53.852381Z",
     "shell.execute_reply": "2024-02-09T15:22:53.851483Z",
     "shell.execute_reply.started": "2024-02-09T15:22:53.623591Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First few rows of the dataset:\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>is_sarcastic</th>\n",
       "      <th>headline</th>\n",
       "      <th>article_link</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>thirtysomething scientists unveil doomsday clo...</td>\n",
       "      <td>https://www.theonion.com/thirtysomething-scien...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>dem rep. totally nails why congress is falling...</td>\n",
       "      <td>https://www.huffingtonpost.com/entry/donna-edw...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>eat your veggies: 9 deliciously different recipes</td>\n",
       "      <td>https://www.huffingtonpost.com/entry/eat-your-...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>inclement weather prevents liar from getting t...</td>\n",
       "      <td>https://local.theonion.com/inclement-weather-p...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>mother comes pretty close to using word 'strea...</td>\n",
       "      <td>https://www.theonion.com/mother-comes-pretty-c...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   is_sarcastic                                           headline  \\\n",
       "0             1  thirtysomething scientists unveil doomsday clo...   \n",
       "1             0  dem rep. totally nails why congress is falling...   \n",
       "2             0  eat your veggies: 9 deliciously different recipes   \n",
       "3             1  inclement weather prevents liar from getting t...   \n",
       "4             1  mother comes pretty close to using word 'strea...   \n",
       "\n",
       "                                        article_link  \n",
       "0  https://www.theonion.com/thirtysomething-scien...  \n",
       "1  https://www.huffingtonpost.com/entry/donna-edw...  \n",
       "2  https://www.huffingtonpost.com/entry/eat-your-...  \n",
       "3  https://local.theonion.com/inclement-weather-p...  \n",
       "4  https://www.theonion.com/mother-comes-pretty-c...  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.read_json('/kaggle/input/sarcasm-headlines-dataset/Sarcasm_Headlines_Dataset.json', lines=True)\n",
    "\n",
    "print(\"First few rows of the dataset:\\n\")\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-02-09T15:22:54.776256Z",
     "iopub.status.busy": "2024-02-09T15:22:54.775531Z",
     "iopub.status.idle": "2024-02-09T15:22:54.820735Z",
     "shell.execute_reply": "2024-02-09T15:22:54.819847Z",
     "shell.execute_reply.started": "2024-02-09T15:22:54.776223Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "• Dataset information:\n",
      "\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Index: 28619 entries, 0 to 28618\n",
      "Data columns (total 3 columns):\n",
      " #   Column        Non-Null Count  Dtype \n",
      "---  ------        --------------  ----- \n",
      " 0   is_sarcastic  28619 non-null  int64 \n",
      " 1   headline      28619 non-null  object\n",
      " 2   article_link  28619 non-null  object\n",
      "dtypes: int64(1), object(2)\n",
      "memory usage: 894.3+ KB\n",
      "\n",
      "\n",
      "• Statistics Summary:\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>25%</th>\n",
       "      <th>50%</th>\n",
       "      <th>75%</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>is_sarcastic</th>\n",
       "      <td>28619.0</td>\n",
       "      <td>0.476397</td>\n",
       "      <td>0.499451</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                count      mean       std  min  25%  50%  75%  max\n",
       "is_sarcastic  28619.0  0.476397  0.499451  0.0  0.0  0.0  1.0  1.0"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(\"\\n• Dataset information:\\n\")\n",
    "data.info()\n",
    "print()\n",
    "print(\"\\n• Statistics Summary:\\n\")\n",
    "data.describe().T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Retain relevant columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To retain only the relevant columns from the dataset, we can select the columns that are necessary for our sarcasm detection task. In this case, we need to retain the 'headline' column containing the news headlines and the 'is_sarcastic' column containing the binary labels indicating whether the headline is sarcastic or not."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-02-09T15:22:56.577052Z",
     "iopub.status.busy": "2024-02-09T15:22:56.575938Z",
     "iopub.status.idle": "2024-02-09T15:22:56.592399Z",
     "shell.execute_reply": "2024-02-09T15:22:56.591353Z",
     "shell.execute_reply.started": "2024-02-09T15:22:56.577017Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First few rows of the updated dataset:\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>headline</th>\n",
       "      <th>is_sarcastic</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>thirtysomething scientists unveil doomsday clo...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>dem rep. totally nails why congress is falling...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>eat your veggies: 9 deliciously different recipes</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>inclement weather prevents liar from getting t...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>mother comes pretty close to using word 'strea...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            headline  is_sarcastic\n",
       "0  thirtysomething scientists unveil doomsday clo...             1\n",
       "1  dem rep. totally nails why congress is falling...             0\n",
       "2  eat your veggies: 9 deliciously different recipes             0\n",
       "3  inclement weather prevents liar from getting t...             1\n",
       "4  mother comes pretty close to using word 'strea...             1"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "relv_columns = ['headline', 'is_sarcastic']\n",
    "data = data[relv_columns]\n",
    "\n",
    "print(\"First few rows of the updated dataset:\\n\")\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Get length of each sentence"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-02-09T15:22:57.908097Z",
     "iopub.status.busy": "2024-02-09T15:22:57.907370Z",
     "iopub.status.idle": "2024-02-09T15:22:57.958079Z",
     "shell.execute_reply": "2024-02-09T15:22:57.957034Z",
     "shell.execute_reply.started": "2024-02-09T15:22:57.908066Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First few rows of the dataset with headline lengths:\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>headline</th>\n",
       "      <th>is_sarcastic</th>\n",
       "      <th>headline_length</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>thirtysomething scientists unveil doomsday clo...</td>\n",
       "      <td>1</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>dem rep. totally nails why congress is falling...</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>eat your veggies: 9 deliciously different recipes</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>inclement weather prevents liar from getting t...</td>\n",
       "      <td>1</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>mother comes pretty close to using word 'strea...</td>\n",
       "      <td>1</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            headline  is_sarcastic  \\\n",
       "0  thirtysomething scientists unveil doomsday clo...             1   \n",
       "1  dem rep. totally nails why congress is falling...             0   \n",
       "2  eat your veggies: 9 deliciously different recipes             0   \n",
       "3  inclement weather prevents liar from getting t...             1   \n",
       "4  mother comes pretty close to using word 'strea...             1   \n",
       "\n",
       "   headline_length  \n",
       "0                8  \n",
       "1               13  \n",
       "2                7  \n",
       "3                8  \n",
       "4                9  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Calculate the length of each sentence\n",
    "data['headline_length'] = data['headline'].apply(lambda x: len(x.split()))\n",
    "\n",
    "print(\"First few rows of the dataset with headline lengths:\\n\")\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Define parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-02-09T15:22:58.939535Z",
     "iopub.status.busy": "2024-02-09T15:22:58.938648Z",
     "iopub.status.idle": "2024-02-09T15:22:58.943466Z",
     "shell.execute_reply": "2024-02-09T15:22:58.942490Z",
     "shell.execute_reply.started": "2024-02-09T15:22:58.939506Z"
    }
   },
   "outputs": [],
   "source": [
    "max_features = 10000\n",
    "maxlen = 50 \n",
    "embedding_size = 128  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here I have defined parameters -\n",
    "\n",
    "* max_features: Number of words to take from tokenizer\n",
    "* maxlen: Maximum length of each sentence\n",
    "* embedding_size: Size of embedding vector"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. Get indices for words"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-02-09T15:23:00.981254Z",
     "iopub.status.busy": "2024-02-09T15:23:00.980924Z",
     "iopub.status.idle": "2024-02-09T15:23:02.456646Z",
     "shell.execute_reply": "2024-02-09T15:23:02.455711Z",
     "shell.execute_reply.started": "2024-02-09T15:23:00.981230Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Original Headline:\n",
      "thirtysomething scientists unveil doomsday clock of hair loss\n",
      "\n",
      "Word Indices for the Headline:\n",
      "[1, 355, 3167, 7474, 2644, 3, 661, 1119]\n",
      "\n",
      "Label (Target Variable): 1\n"
     ]
    }
   ],
   "source": [
    "tokenizer = Tokenizer(num_words=max_features, oov_token='<OOV>')\n",
    "\n",
    "# Fit the Tokenizer on the headlines\n",
    "tokenizer.fit_on_texts(data['headline'])\n",
    "\n",
    "# Get the word indices for the headlines\n",
    "headline_sequences = tokenizer.texts_to_sequences(data['headline'])\n",
    "\n",
    "# Get the word indices for the labels (target variable)\n",
    "label_sequences = data['is_sarcastic'].values\n",
    "\n",
    "example_index = 0  # Change this value to inspect different examples\n",
    "print(\"Original Headline:\")\n",
    "print(data['headline'][example_index])\n",
    "print(\"\\nWord Indices for the Headline:\")\n",
    "print(headline_sequences[example_index])\n",
    "print(\"\\nLabel (Target Variable):\", label_sequences[example_index])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the above code -\n",
    "\n",
    "* We initialized a Tokenizer with a maximum number of words (max_features) and an out-of-vocabulary token(OOV).\n",
    "* We fitted the Tokenizer on the headlines to create the vocabulary and assign unique indices to each word.\n",
    "* We converted the headlines into sequences of word indices using texts_to_sequences.\n",
    "* We obtained the word indices for the labels directly from the DataFrame.\n",
    "* Finally, we print an example headline along with its corresponding word indices and label to verify the tokenization process."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6. Create features and labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-02-09T15:23:04.548224Z",
     "iopub.status.busy": "2024-02-09T15:23:04.547872Z",
     "iopub.status.idle": "2024-02-09T15:23:04.678582Z",
     "shell.execute_reply": "2024-02-09T15:23:04.677642Z",
     "shell.execute_reply.started": "2024-02-09T15:23:04.548197Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of features (X): (28619, 50)\n",
      "Shape of labels (y): (28619,)\n"
     ]
    }
   ],
   "source": [
    "# Pad sequences to ensure uniform length\n",
    "padded_sequences = pad_sequences(headline_sequences, maxlen=maxlen, padding='post')\n",
    "\n",
    "# Convert padded sequences and labels to numpy array\n",
    "X = np.array(padded_sequences)\n",
    "y = np.array(label_sequences)\n",
    "\n",
    "print(\"Shape of features (X):\", X.shape)\n",
    "print(\"Shape of labels (y):\", y.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 7. Get vocabulary size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-02-09T15:23:07.009459Z",
     "iopub.status.busy": "2024-02-09T15:23:07.008738Z",
     "iopub.status.idle": "2024-02-09T15:23:07.014209Z",
     "shell.execute_reply": "2024-02-09T15:23:07.013009Z",
     "shell.execute_reply.started": "2024-02-09T15:23:07.009424Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Vocabulary Size: 30886\n"
     ]
    }
   ],
   "source": [
    "vocab_size = len(tokenizer.word_index) + 1\n",
    "\n",
    "print(\"Vocabulary Size:\", vocab_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 8. Create a weight matrix using GloVe embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-02-09T15:23:08.575488Z",
     "iopub.status.busy": "2024-02-09T15:23:08.574829Z",
     "iopub.status.idle": "2024-02-09T15:23:21.518647Z",
     "shell.execute_reply": "2024-02-09T15:23:21.517625Z",
     "shell.execute_reply.started": "2024-02-09T15:23:08.575456Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of embedding matrix: (30886, 100)\n"
     ]
    }
   ],
   "source": [
    "glove_file = '/kaggle/input/glove-6b/glove.6B.100d.txt'\n",
    "embedding_dim = 100  \n",
    "\n",
    "# Load GloVe embeddings into memory\n",
    "embeddings_index = {}\n",
    "with open(glove_file, encoding='utf-8') as f:\n",
    "    for line in f:\n",
    "        values = line.split()\n",
    "        word = values[0]\n",
    "        coefs = np.asarray(values[1:], dtype='float32')\n",
    "        embeddings_index[word] = coefs\n",
    "\n",
    "# Initialize embedding matrix with zeros\n",
    "embedding_matrix = np.zeros((vocab_size, embedding_dim))\n",
    "\n",
    "# Map GloVe embeddings to our vocabulary\n",
    "for word, i in tokenizer.word_index.items():\n",
    "    embedding_vector = embeddings_index.get(word)\n",
    "    if embedding_vector is not None:\n",
    "        embedding_matrix[i] = embedding_vector\n",
    "\n",
    "print(\"Shape of embedding matrix:\", embedding_matrix.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 9. Define and compile a Bidirectional LSTM model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-02-09T15:23:47.681649Z",
     "iopub.status.busy": "2024-02-09T15:23:47.681264Z",
     "iopub.status.idle": "2024-02-09T15:23:47.895028Z",
     "shell.execute_reply": "2024-02-09T15:23:47.894315Z",
     "shell.execute_reply.started": "2024-02-09T15:23:47.681619Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_2\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " embedding_2 (Embedding)     (None, 50, 100)           3088600   \n",
      "                                                                 \n",
      " lstm_4 (LSTM)               (None, 128)               117248    \n",
      "                                                                 \n",
      " dense_4 (Dense)             (None, 1)                 129       \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 3205977 (12.23 MB)\n",
      "Trainable params: 117377 (458.50 KB)\n",
      "Non-trainable params: 3088600 (11.78 MB)\n",
      "_________________________________________________________________\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.keras.initializers import Constant \n",
    "\n",
    "lstm_model = Sequential()\n",
    "\n",
    "# Add embedding layer with pre-trained GloVe embeddings\n",
    "embedding_layer = Embedding(input_dim=vocab_size,\n",
    "                            output_dim=embedding_dim,\n",
    "                            embeddings_initializer=Constant(embedding_matrix),\n",
    "                            input_length=maxlen,\n",
    "                            trainable=False)  # Set trainable to False to keep embeddings fixed\n",
    "lstm_model.add(embedding_layer)\n",
    "\n",
    "# Add LSTM layer\n",
    "lstm_model.add(LSTM(128, dropout=0.3, recurrent_dropout=0.2))\n",
    "lstm_model.add(Dense(1, activation='sigmoid')) # output layer\n",
    "\n",
    "# Compile the model\n",
    "lstm_model.compile(optimizer=Adam(learning_rate=0.001),\n",
    "                   loss='binary_crossentropy',\n",
    "                   metrics=['accuracy'])\n",
    "\n",
    "print(lstm_model.summary())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 10. Fit the model and check the validation accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-02-09T15:23:55.356598Z",
     "iopub.status.busy": "2024-02-09T15:23:55.356222Z",
     "iopub.status.idle": "2024-02-09T15:31:40.402136Z",
     "shell.execute_reply": "2024-02-09T15:31:40.401295Z",
     "shell.execute_reply.started": "2024-02-09T15:23:55.356556Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "358/358 [==============================] - 50s 129ms/step - loss: 0.5712 - accuracy: 0.7009 - val_loss: 0.4655 - val_accuracy: 0.7818\n",
      "Epoch 2/10\n",
      "358/358 [==============================] - 46s 128ms/step - loss: 0.4577 - accuracy: 0.7866 - val_loss: 0.4081 - val_accuracy: 0.8150\n",
      "Epoch 3/10\n",
      "358/358 [==============================] - 46s 129ms/step - loss: 0.4000 - accuracy: 0.8191 - val_loss: 0.3866 - val_accuracy: 0.8281\n",
      "Epoch 4/10\n",
      "358/358 [==============================] - 46s 128ms/step - loss: 0.3786 - accuracy: 0.8310 - val_loss: 0.3658 - val_accuracy: 0.8358\n",
      "Epoch 5/10\n",
      "358/358 [==============================] - 46s 127ms/step - loss: 0.3549 - accuracy: 0.8436 - val_loss: 0.3560 - val_accuracy: 0.8487\n",
      "Epoch 6/10\n",
      "358/358 [==============================] - 46s 129ms/step - loss: 0.3417 - accuracy: 0.8500 - val_loss: 0.3608 - val_accuracy: 0.8461\n",
      "Epoch 7/10\n",
      "358/358 [==============================] - 46s 128ms/step - loss: 0.3261 - accuracy: 0.8615 - val_loss: 0.3407 - val_accuracy: 0.8515\n",
      "Epoch 8/10\n",
      "358/358 [==============================] - 46s 129ms/step - loss: 0.3107 - accuracy: 0.8660 - val_loss: 0.3337 - val_accuracy: 0.8538\n",
      "Epoch 9/10\n",
      "358/358 [==============================] - 46s 128ms/step - loss: 0.2998 - accuracy: 0.8713 - val_loss: 0.3273 - val_accuracy: 0.8625\n",
      "Epoch 10/10\n",
      "358/358 [==============================] - 46s 128ms/step - loss: 0.2914 - accuracy: 0.8759 - val_loss: 0.3342 - val_accuracy: 0.8517\n",
      "179/179 [==============================] - 3s 14ms/step - loss: 0.3342 - accuracy: 0.8517\n",
      "Validation Accuracy: 0.8516771197319031\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=1)\n",
    "\n",
    "# Fit the LSTM model\n",
    "history = lstm_model.fit(X_train, y_train, batch_size=64, epochs=10, validation_data=(X_val, y_val))\n",
    "\n",
    "# Evaluate the model on the validation data\n",
    "val_loss, val_accuracy = lstm_model.evaluate(X_val, y_val)\n",
    "\n",
    "print(\"Validation Accuracy:\", val_accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### *****************************************************************************************************************************"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <center><span style=\"font-family: Arial; font-weight:bold;font-size:1.9em;\">End Of Project</center>"
   ]
  }
 ],
 "metadata": {
  "kaggle": {
   "accelerator": "nvidiaTeslaT4",
   "dataSources": [
    {
     "datasetId": 4414278,
     "sourceId": 7583076,
     "sourceType": "datasetVersion"
    },
    {
     "datasetId": 4417923,
     "sourceId": 7589763,
     "sourceType": "datasetVersion"
    }
   ],
   "dockerImageVersionId": 30648,
   "isGpuEnabled": true,
   "isInternetEnabled": true,
   "language": "python",
   "sourceType": "notebook"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}