sentiment-analysis-with-vader.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img align=\"left\" src=\"https://ithaka-labs.s3.amazonaws.com/static-files/images/tdm/tdmdocs/CC_BY.png\"><br />\n",
    "\n",
    "Created by [Nathan Kelber](http://nkelber.com) for [JSTOR Labs](https://labs.jstor.org/) under [Creative Commons CC BY License](https://creativecommons.org/licenses/by/4.0/)<br />\n",
    "For questions/comments/improvements, email nathan.kelber@ithaka.org.<br />\n",
    "___"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sentiment Analysis\n",
    "\n",
    "**Description:** This notebook describes Sentiment Analysis and demonstrates basic applications using:\n",
    "* VADER (Valence Aware Dictionary for sEntiment Reasoning), a rule-based algorithm\n",
    "* Hugging Face's transformers library\n",
    "\n",
    "**Use Case:** For Learners (Detailed explanation, not ideal for researchers)\n",
    "\n",
    "**Difficulty:** Beginner\n",
    "\n",
    "**Completion Time:** 60 minutes\n",
    "\n",
    "**Knowledge Required:** \n",
    "* Python Basics Series ([Start Python Basics I](./python-basics-1.ipynb))\n",
    "\n",
    "**Knowledge Recommended:** None\n",
    "\n",
    "**Data Format:** None\n",
    "\n",
    "**Libraries Used:** vaderSentiment, transformers\n",
    "\n",
    "**Research Pipeline:** None\n",
    "___"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Methods for Sentiment Analysis\n",
    "\n",
    "Sentiment analysis can help an analyst discover whether feedback is positive, negative, or mixed. For example, a large company like Amazon or Walmart could use sentiment analysis on user reviews to determine whether a featured product should be promoted or discontinued. Sentiment analysis generally falls into two categories:\n",
    "\n",
    "* Rule-based algorithms\n",
    "* Machine Learning models \n",
    "\n",
    "### Rule-Based Algorithms\n",
    "\n",
    "Rule-based algorithms assign sentiment scores to particular words or multi-word constructions. Simple algorithms may simply assess each word individually in a feedback document and add up an overall score. More complex algorithms may assess multi-word (or n-gram) constructions and have special rules for addressing issues such as negation, emojis, and emoticons. They can detect the difference between \"bad\", \"not bad\", and \"bad ass\". Some algorithms also support emojis and emoticons, such as \"=)\" and \"😁\".\n",
    "\n",
    "### Machine Learning Models\n",
    "\n",
    "Machine learning models rely on feedback data that has already been assessed by humans to have a particular sentiment. Each piece of feedback is **labeled** by a human reader who may place the feedback into a particular category. The categories could be as simple as positive, negative, or neutral. As long as there exists **labeled** data, a machine learning model can often identify complex concepts. For example, a car manufacturer may desire to classify the sentiment of feedback from past buyers as: \"budget-conscious\", \"eco-conscious\", \"tech-enthusiastic\", \"luxury-driven\", \"performance-driven\", etc. Assuming there is an adequately labeled **training data** for each of these categories, a machine learning model could assign a score for each category. This could help analysts understand the brand better, answering questions about what consumers do or do not like about a particular vehicle.\n",
    "\n",
    "In the humanities, sentiment analysis could be used to track emerging trends on social media. For example, we might ask: \"How are Twitter or Reddit users responding to a particular government policy or public event?\" We could look at a hashtag like \"#blm\" and get a sense of national sentiment on the Black Lives Matter movement. The project [On the Books: Jim Crow and Algorithms of Resistance](https://onthebooks.lib.unc.edu/) is using machine classification to detect racist laws based on the pioneering work of [Pauli Murray](https://en.wikipedia.org/wiki/Pauli_Murray) and [Safiya Noble](https://en.wikipedia.org/wiki/Safiya_Noble)'s concept of \"algorithmic oppression\". "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## VADER\n",
    "\n",
    "This notebook uses a rule-based algorithm named VADER (Valence Aware Dictionary and sEntiment Reasoner). VADER is a rule-based algorithm that is \"specifically attuned to sentiments expressed in social media.\" It relies on a specialized **lexicon** of words, phrases, and emojis. Each token in the lexicon is assigned a \"mean-sentiment rating\" between -4 (extremely negative) to 4 (extremely positive). Here are a few examples:\n",
    "\n",
    "|Token|Mean-Sentiment Rating|\n",
    "|---|---|\n",
    "|(:|2.2|\n",
    "|/:|-1.3|\n",
    "|):<|-1.9|\n",
    "|rotflmao|2.8|\n",
    "|aghast|-1.9|\n",
    "|awesome|3.1|\n",
    "|awful|-2.0|\n",
    "\n",
    "There are over 7500 tokens listed in VADER lexicon. (You can also add your own if you like.) VADER also considers grammatical and syntactical rules to measure intensity based on word order and sensitive relationships between terms. For example, it increases or decreases a sentiment based on degree modifers such as: \"The product is good\" versus \"the product is very good\" versus \"the product is marginally good.\" To read more about VADER, including how it works and to see its code, [visit the github page](https://github.com/cjhutto/vaderSentiment)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Applying the VADER Algorithm\n",
    "First, we need to import the SentimentIntensityAnalyzer. Here we assign the VADER lexicon object to a variable `sa`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the SentimentIntensityAnalyzer\n",
    "from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer\n",
    "\n",
    "# Creat the variable sa to hold the VADER lexicon object \n",
    "sa = SentimentIntensityAnalyzer()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can preview the contents of the lexicon by using `sa.lexicon`. This will return a dictionary, where each key is a token and each value is a sentiment rating."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Preview the lexicon contents\n",
    "# There are over 7500 tokens in the lexicon\n",
    "sa.lexicon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Check if a word is in the lexicon\n",
    "test_word = 'sweet' # The word to check for\n",
    "\n",
    "# Get the word's score or print a message for missing words\n",
    "sa.lexicon.get(test_word, 'No score for that word') "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to do our analysis, we will use a very small sample of 8 user reviews. Each review is a simple text string inside a list variable called `product_reviews`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define a list of product reviews\n",
    "\n",
    "product_reviews = [\n",
    "    'I love this product. It helps me get so much work done. I tell everyone about what a great thing it is.',\n",
    "    'This product is defective. I feel like it is broken because it does not do what it promises. Do not buy this.',\n",
    "    'Do yourself a favor and buy this product as soon as possible. I recommend it to everyone I know. It has saved me so much time!',\n",
    "    'This product is overpriced and useless. It was a waste of money and it made all my hair fall out.',\n",
    "    'Works like a dream and it is a bargain! It solves my problems with ease. I bought two!',\n",
    "    'Do not buy! This product is a ripoff. I wish it was better, but it fails constantly. What a mistake!',\n",
    "    'This thing is garbage. Do yourself a favor and save the money. Mine is a dumpster fire and fell apart.',\n",
    "    'I adore this product. =) It makes my life so much easier. And it is a deal!'\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we will analyze each product and assign it a \"normalized, weighted composite score\" based on summing the valence scores of each word in the lexicon (with some adjustments based on word order and other rules). VADER measures the proportion of text that falls into positive, negative, and neutral sentiment. The result is a sentiment score that falls between -1 (the most negative) and +1 (the most positive). (This is different from the lexicon scores that fall between -4 to +4!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# For each review in our `product_reviews` list\n",
    "# Store a polarity score in `scores`\n",
    "# Then print the score followed by the review\n",
    "for review in product_reviews:\n",
    "    scores = sa.polarity_scores(review)\n",
    "    print(scores['compound'], review)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our simple analysis does a fairly good job of assessing positive and negative sentiment. Notice that our second to last review was not very accurate though:\n",
    "> 0.5423 This thing is garbage. Do yourself a favor and save the money. Mine started on fire and fell apart.\n",
    "\n",
    "The VADER lexicon contains the following entries:\n",
    "\n",
    "|Token|Mean-Sentiment Rating|\n",
    "|---|---|\n",
    "|favor|1.7|\n",
    "|fire|-1.4|\n",
    "\n",
    "VADER assigns a value of -1.4 for \"fire\" but \"fire\" can also have a positive connotation, such as \"straight fire.\" However, words like \"garbage\" and \"dumpster,\" as in \"dumpster fire,\" are less ambiguous. If a specific token is not found in the VADER lexicon, it is considered to be neutral. Like any other statistical approach, the process benefits from having more data. In this case, the sentences are very short and several significant words do not happen to exist in our lexicon. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Adding Tokens to the VADER Lexicon\n",
    "\n",
    "The `sa.lexicon` is a simple dictionary, so we can add words that we want included. There are some guidelines for best scoring practices included in the academic paper linked on [VADER's github repository](https://github.com/cjhutto/vaderSentiment). (Remember that lexicon tokens are scored from -4 to +4.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Adding the dictionary of `new_words`\n",
    "# to sa.lexicon\n",
    "\n",
    "new_words = {\n",
    "    'garbage': -2.0,\n",
    "    'dumpster': -3.1,\n",
    "}\n",
    "\n",
    "sa.lexicon.update(new_words)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's try our analysis again with the new lexicon."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# For each review in our `product_reviews` list\n",
    "# Store a polarity score in `scores`\n",
    "# Then print the score followed by the review\n",
    "\n",
    "for review in product_reviews:\n",
    "    scores = sa.polarity_scores(review)\n",
    "    print(scores['compound'], review)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sentiment analysis with machine learning\n",
    "\n",
    "The primary advantage of using a machine learning classifier for sentiment analysis is there is no need to maintain a lexicon, assign sentiment scores to particular words, develop linguistic rules based on grammatical structures (negation, intensifiers), or keep track of novel expressions (slang, emoticons, etc.). \n",
    "\n",
    "The very best models for sentiment analysis will be trained or tuned on the type of data you are analyzing. We always recommend trying existing models first though, since training a model from scratch takes significant resources, both on the computational side and on the labor side for data quality assurance. If you are interested in training your own models, then you may want to invest in high-end hardware, especially a computer with a powerful graphics processing unit (GPU). If your data or model requires significant resources, consider purchasing cloud-computing resources.\n",
    "\n",
    "There are many existing models that are a great place to start with sentiment analysis. Let's try using an existing model with the popular [transfomers](https://github.com/huggingface/transformers) library from [HuggingFace](https://huggingface.co/). We will use a [dataset of Amazon game reviews](https://huggingface.co/datasets/LoganKells/amazon_product_reviews_video_games/) created by Logan Kells."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install the transformers package\n",
    "!pip install -q transformers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install TensorFlow, a popular library for machine learning\n",
    "!pip install TensorFlow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Download the dataset\n",
    "from pathlib import Path\n",
    "import urllib.request\n",
    "\n",
    "# The file URL\n",
    "url = 'https://huggingface.co/datasets/LoganKells/amazon_product_reviews_video_games/resolve/main/data.csv'\n",
    "\n",
    "# Check if a data folder exists. If not, create it.\n",
    "data_folder = Path('./data/')\n",
    "data_folder.mkdir(exist_ok=True)\n",
    "\n",
    "# Download the file\n",
    "path_url = Path(url)\n",
    "urllib.request.urlretrieve(url, f'{data_folder.as_posix()}/{path_url.name}')\n",
    "    \n",
    "## Success message\n",
    "print('Data downloaded.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import Pandas and Read Data CSV file\n",
    "import pandas as pd\n",
    "df = pd.read_csv('data/data.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Check the size of the dataframe\n",
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Preview the dataframe\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a list of the first 100 review texts\n",
    "review_texts = df['reviewText'].tolist()[:100]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Using pipeline class to make predictions from models available in the Hub in an easy way \n",
    "from transformers import pipeline\n",
    "sentiment_pipeline = pipeline(\"sentiment-analysis\")\n",
    "\n",
    "sentiment_scores = sentiment_pipeline(review_texts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Examine each review to see how the model did\n",
    "review = 0\n",
    "\n",
    "# Print Sentiment\n",
    "print(sentiment_scores[review])\n",
    "\n",
    "# Print Review\n",
    "print(review_texts[review])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Show reviews with lower certainty scores\n",
    "for review_number, sentiment_score in enumerate(sentiment_scores):\n",
    "    if sentiment_score['score'] < .9:\n",
    "        print(sentiment_score, review_texts[review_number], '\\n')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.4"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": true,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "199px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}