Movie_recommendation_sys.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Movie_recommendation_sys.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.5"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/unguyen14/recommendation_system/blob/main/Movie_recommendation_sys.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m7htG-Awn2Ie"
      },
      "source": [
        "# Movie Recommendation System\n",
        "## Authors: Fabio Turazzi and Winnie Nguyen\n",
        "\n",
        "This notebook demonstrates a recommendation system combining the ratings dataset [MovieLens 25M dataset](https://grouplens.org/datasets/movielens/25m/) and movie metadata from [The Movie Dataset](https://www.kaggle.com/rounakbanik/the-movies-dataset) on Kaggle. The two datasets were joined using movie Ids from IMDb.\n",
        "In order to provide different perspectives on recommendation systems, we examine multiple approaches to generate a lists of movies based on their genres, contents, as well as ratings by users.\n",
        "\n",
        "## Datasets Used\n",
        "We use 4 datasets:\n",
        "- links.csv: containing movieId, imdbId, tmdbId\n",
        "- movies.csv: containing movieId, title, genres\n",
        "- ratings.csv: containing userId, movieId, rating, timestamp\n",
        "- movies_metadata.csv: containing imdb_id, overview\n",
        "\n",
        "\n",
        "## Notebook Structure:\n",
        "In this notebook, we will follow the following process in order to build our robusted recommendation engine:\n",
        "- 1. **Dataset Loading and Preparation**: loading necessary libraries, importing datasets, joining datasets\n",
        "\n",
        "- 2. **Simple Recommendation System**: generally analyzing most popular and highest rated movies across genres.\n",
        "\n",
        "- 3. **Content-Based Recommendataion**: using contents as the basis to suggest similar movies.\n",
        "    - 3.1. *Genre similarity*: recommendations based on the similarities between movie genres.\n",
        "    - 3.2. *Description similarity*: based on text similarities in movie descriptions.\n",
        "    \n",
        "- 4. **Collaborative Filtering**: using ratings as the basis to suggest similar movies.\n",
        "    - 4.1. *Item-based filtering*: provides recommendations given a movie title by analyzing other items similarly rated by users.\n",
        "    - 4.2. *User-based filtering*: focuses on a target user to provide suggestions, by analyzing favorite movies by other similar users. Includes a model for rating prediction for a given user and movie.\n",
        "    \n",
        "## Methodologies:\n",
        "We apply multiple statistical, Machine Learning, and Natural Language Processing methods to solve the different prioritization problems. The used methodologies used include:\n",
        "* Jaccard similarity coefficient\n",
        "* Cosine similarity\n",
        "* TF-IDF\n",
        "* KNNRegressor\n",
        "\n",
        "\n",
        "## References:\n",
        "- Datacamp Recommendation System materials\n",
        "- [Scikit-learn documentation](https://scikit-learn.org/0.21/documentation.html)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IlFLsaDVn2Ij"
      },
      "source": [
        "# 1. Dataset Loading and Preparation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9GIHBY0nn2Ij"
      },
      "source": [
        "### Loading necessary libraries"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2-qfmDfvtTjX"
      },
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "import warnings\n",
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns\n",
        "from sklearn.neighbors import KNeighborsRegressor\n",
        "from sklearn.metrics import jaccard_score\n",
        "from scipy.spatial.distance import pdist, squareform\n",
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "from sklearn.metrics.pairwise import cosine_similarity"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ADAAnDjTn2Il"
      },
      "source": [
        "### Import datasets"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cBuFR4YEs93g"
      },
      "source": [
        "#read files\n",
        "df_links = pd.read_csv('data/links.csv', low_memory=False)\n",
        "df_movies = pd.read_csv('data/movies.csv', low_memory=False)\n",
        "df_ratings = pd.read_csv('data/ratings.csv', low_memory=False)\n",
        "df_metadata = pd.read_csv('data/movies_metadata.csv', low_memory=False)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XZ0ar5STn2In"
      },
      "source": [
        "### Dataset Overview"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 406
        },
        "id": "SSJMEtV4xLti",
        "outputId": "4b39bfe2-4454-4cfe-8242-95e965072ad1"
      },
      "source": [
        "df_links.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>movieId</th>\n",
              "      <th>imdbId</th>\n",
              "      <th>tmdbId</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>114709</td>\n",
              "      <td>862.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>113497</td>\n",
              "      <td>8844.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>113228</td>\n",
              "      <td>15602.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>114885</td>\n",
              "      <td>31357.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>113041</td>\n",
              "      <td>11862.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   movieId  imdbId   tmdbId\n",
              "0        1  114709    862.0\n",
              "1        2  113497   8844.0\n",
              "2        3  113228  15602.0\n",
              "3        4  114885  31357.0\n",
              "4        5  113041  11862.0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 197
        },
        "id": "LUS_EapvxMag",
        "outputId": "ed92ac94-28f7-4ff7-9dac-4f459f5e465b"
      },
      "source": [
        "df_movies.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>movieId</th>\n",
              "      <th>title</th>\n",
              "      <th>genres</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>Jumanji (1995)</td>\n",
              "      <td>Adventure|Children|Fantasy</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>Grumpier Old Men (1995)</td>\n",
              "      <td>Comedy|Romance</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>Waiting to Exhale (1995)</td>\n",
              "      <td>Comedy|Drama|Romance</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>Father of the Bride Part II (1995)</td>\n",
              "      <td>Comedy</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   movieId                               title  \\\n",
              "0        1                    Toy Story (1995)   \n",
              "1        2                      Jumanji (1995)   \n",
              "2        3             Grumpier Old Men (1995)   \n",
              "3        4            Waiting to Exhale (1995)   \n",
              "4        5  Father of the Bride Part II (1995)   \n",
              "\n",
              "                                        genres  \n",
              "0  Adventure|Animation|Children|Comedy|Fantasy  \n",
              "1                   Adventure|Children|Fantasy  \n",
              "2                               Comedy|Romance  \n",
              "3                         Comedy|Drama|Romance  \n",
              "4                                       Comedy  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 406
        },
        "id": "5jaeZLlTKZyZ",
        "outputId": "a5ebbab4-c1f7-42e8-f53d-bddbfb39940a"
      },
      "source": [
        "# Keeping just the description column for our content based filtering\n",
        "df_descriptions = df_metadata[['imdb_id', 'overview']].copy()\n",
        "df_descriptions.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>imdb_id</th>\n",
              "      <th>overview</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>tt0114709</td>\n",
              "      <td>Led by Woody, Andy's toys live happily in his ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>tt0113497</td>\n",
              "      <td>When siblings Judy and Peter discover an encha...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>tt0113228</td>\n",
              "      <td>A family wedding reignites the ancient feud be...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>tt0114885</td>\n",
              "      <td>Cheated on, mistreated and stepped on, the wom...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>tt0113041</td>\n",
              "      <td>Just when George Banks has recovered from his ...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "     imdb_id                                           overview\n",
              "0  tt0114709  Led by Woody, Andy's toys live happily in his ...\n",
              "1  tt0113497  When siblings Judy and Peter discover an encha...\n",
              "2  tt0113228  A family wedding reignites the ancient feud be...\n",
              "3  tt0114885  Cheated on, mistreated and stepped on, the wom...\n",
              "4  tt0113041  Just when George Banks has recovered from his ..."
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 197
        },
        "id": "eKEvpJMrxSEP",
        "outputId": "505cba2c-97ff-4134-9d62-7840ae466782"
      },
      "source": [
        "df_ratings.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>userId</th>\n",
              "      <th>movieId</th>\n",
              "      <th>rating</th>\n",
              "      <th>timestamp</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>296</td>\n",
              "      <td>5.0</td>\n",
              "      <td>1147880044</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>306</td>\n",
              "      <td>3.5</td>\n",
              "      <td>1147868817</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>307</td>\n",
              "      <td>5.0</td>\n",
              "      <td>1147868828</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1</td>\n",
              "      <td>665</td>\n",
              "      <td>5.0</td>\n",
              "      <td>1147878820</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>1</td>\n",
              "      <td>899</td>\n",
              "      <td>3.5</td>\n",
              "      <td>1147868510</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   userId  movieId  rating   timestamp\n",
              "0       1      296     5.0  1147880044\n",
              "1       1      306     3.5  1147868817\n",
              "2       1      307     5.0  1147868828\n",
              "3       1      665     5.0  1147878820\n",
              "4       1      899     3.5  1147868510"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K4OGlcSFMRy0"
      },
      "source": [
        "### Joining Movie Descriptions"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GYngqzPzMhVP",
        "outputId": "9ace8cf1-96e6-4dac-87a4-8d21bb43a028"
      },
      "source": [
        "#Formatting imdb_id to join dataframes\n",
        "df_descriptions.imdb_id = df_descriptions.imdb_id.apply(lambda x: str(x).replace(\"tt\", \"\"))\n",
        "df_descriptions.imdb_id = pd.to_numeric(df_descriptions.imdb_id, errors=\"coerce\")\n",
        "df_descriptions.dropna(inplace=True)\n",
        "\n",
        "#Merging dataframes using the \"links\" dataset\n",
        "df_descriptions = pd.merge(df_links, df_descriptions, left_on=['imdbId'], right_on=['imdb_id'], how='inner')\n",
        "df_descriptions = df_descriptions.drop(columns=['imdb_id', 'tmdbId'], axis=1)\n",
        "df_descriptions = pd.merge(df_descriptions, df_movies, how='inner')\n",
        "df_descriptions = df_descriptions.drop(columns=['imdbId'], axis=1)\n",
        "df_descriptions\n",
        "# #Replace NaN with an empty string\n",
        "df_descriptions['overview'] = df_descriptions['overview'].fillna('')\n",
        "\n",
        "df_descriptions.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>movieId</th>\n",
              "      <th>overview</th>\n",
              "      <th>title</th>\n",
              "      <th>genres</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>Led by Woody, Andy's toys live happily in his ...</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>When siblings Judy and Peter discover an encha...</td>\n",
              "      <td>Jumanji (1995)</td>\n",
              "      <td>Adventure|Children|Fantasy</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>A family wedding reignites the ancient feud be...</td>\n",
              "      <td>Grumpier Old Men (1995)</td>\n",
              "      <td>Comedy|Romance</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>Cheated on, mistreated and stepped on, the wom...</td>\n",
              "      <td>Waiting to Exhale (1995)</td>\n",
              "      <td>Comedy|Drama|Romance</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>Just when George Banks has recovered from his ...</td>\n",
              "      <td>Father of the Bride Part II (1995)</td>\n",
              "      <td>Comedy</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   movieId                                           overview  \\\n",
              "0        1  Led by Woody, Andy's toys live happily in his ...   \n",
              "1        2  When siblings Judy and Peter discover an encha...   \n",
              "2        3  A family wedding reignites the ancient feud be...   \n",
              "3        4  Cheated on, mistreated and stepped on, the wom...   \n",
              "4        5  Just when George Banks has recovered from his ...   \n",
              "\n",
              "                                title  \\\n",
              "0                    Toy Story (1995)   \n",
              "1                      Jumanji (1995)   \n",
              "2             Grumpier Old Men (1995)   \n",
              "3            Waiting to Exhale (1995)   \n",
              "4  Father of the Bride Part II (1995)   \n",
              "\n",
              "                                        genres  \n",
              "0  Adventure|Animation|Children|Comedy|Fantasy  \n",
              "1                   Adventure|Children|Fantasy  \n",
              "2                               Comedy|Romance  \n",
              "3                         Comedy|Drama|Romance  \n",
              "4                                       Comedy  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5-hEWJUp1OQn"
      },
      "source": [
        "# 2. Simple Recommendation System\n",
        "- Our simple recommendation system represents the general suggestion on main pages, considering overall most popular and well rated items to be displayed. \n",
        "- For this purpose, we will use the IMDB's top movie calculation formula to calculate weighted rating (WR) for each movie:\n",
        "\n",
        "<div align=\"center\">$\\Large WR = (v ÷ (v+m)) \\times R + (m ÷ (v+m)) \\times C$</div>\n",
        "    \n",
        "Where:\n",
        "\n",
        "    R = average for the movie (mean) = (Rating)\n",
        "    v = number of votes for the movie = (votes)\n",
        "    m = minimum votes required to be listed in the Top Movies list\n",
        "    C = the mean vote across the whole report"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fkiLxMZLn2It"
      },
      "source": [
        "### Grouping ratings by movie"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 406
        },
        "id": "YJG54k_n0T4Q",
        "outputId": "1cfd8569-3f48-4e5a-b6f0-0321d2e2ef0a"
      },
      "source": [
        "#Grouping rating by movie\n",
        "movie_avg_rating = df_ratings.groupby([\"movieId\"], as_index=False).agg(avg_rating = (\"rating\", \"mean\"),\n",
        "                                                 rating_count = (\"movieId\", 'count')).reset_index(drop=True)\n",
        "movie_avg_rating.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>movieId</th>\n",
              "      <th>avg_rating</th>\n",
              "      <th>rating_count</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>3.893708</td>\n",
              "      <td>57309</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>3.251527</td>\n",
              "      <td>24228</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>3.142028</td>\n",
              "      <td>11804</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>2.853547</td>\n",
              "      <td>2523</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>3.058434</td>\n",
              "      <td>11714</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   movieId  avg_rating  rating_count\n",
              "0        1    3.893708         57309\n",
              "1        2    3.251527         24228\n",
              "2        3    3.142028         11804\n",
              "3        4    2.853547          2523\n",
              "4        5    3.058434         11714"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IHRXJ0Ik3O4B",
        "outputId": "aed7a33b-65d7-4a77-96fd-a9bde99512df"
      },
      "source": [
        "#merging rating with movies' names\n",
        "movie_avg_rating = pd.merge(movie_avg_rating,\n",
        "                 df_movies,\n",
        "                 on='movieId')\n",
        "\n",
        "movie_avg_rating.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>movieId</th>\n",
              "      <th>avg_rating</th>\n",
              "      <th>rating_count</th>\n",
              "      <th>title</th>\n",
              "      <th>genres</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>3.893708</td>\n",
              "      <td>57309</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>3.251527</td>\n",
              "      <td>24228</td>\n",
              "      <td>Jumanji (1995)</td>\n",
              "      <td>Adventure|Children|Fantasy</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>3.142028</td>\n",
              "      <td>11804</td>\n",
              "      <td>Grumpier Old Men (1995)</td>\n",
              "      <td>Comedy|Romance</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>2.853547</td>\n",
              "      <td>2523</td>\n",
              "      <td>Waiting to Exhale (1995)</td>\n",
              "      <td>Comedy|Drama|Romance</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>3.058434</td>\n",
              "      <td>11714</td>\n",
              "      <td>Father of the Bride Part II (1995)</td>\n",
              "      <td>Comedy</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   movieId  avg_rating  rating_count                               title  \\\n",
              "0        1    3.893708         57309                    Toy Story (1995)   \n",
              "1        2    3.251527         24228                      Jumanji (1995)   \n",
              "2        3    3.142028         11804             Grumpier Old Men (1995)   \n",
              "3        4    2.853547          2523            Waiting to Exhale (1995)   \n",
              "4        5    3.058434         11714  Father of the Bride Part II (1995)   \n",
              "\n",
              "                                        genres  \n",
              "0  Adventure|Animation|Children|Comedy|Fantasy  \n",
              "1                   Adventure|Children|Fantasy  \n",
              "2                               Comedy|Romance  \n",
              "3                         Comedy|Drama|Romance  \n",
              "4                                       Comedy  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6fjfRRJc24qy"
      },
      "source": [
        "### Checking the 95th percentile to decide rating threshold\n",
        "- To determine a threshold for popular movies, we choose the ones with top 5% amount of rating from users, by calculating the 95 quantile. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 264
        },
        "id": "nVTFhqE149X-",
        "outputId": "3f977daa-cfab-43b9-913d-cc46ce401a79"
      },
      "source": [
        "#Generate histogram\n",
        "plt.hist(x=movie_avg_rating.rating_count, bins=30)\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAD4CAYAAADsKpHdAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAS1UlEQVR4nO3cf6zd9X3f8eerdkLIDwg/DPJsNBNhVQXUkmAxZ0xVF3fFSaqYP0C6kTI8yZUlRqVkm1TZq7Qpf1iCaSod2mBDIcPQNuDSZFhUrEGmUbUJ4VwSUjDgcVMoeLjYDZTQSaCavvfHed9wfDn2Pfdi+1zk50P66vs97/P9fM/7ezjmdb8/zklVIUnSz026AUnS0mAgSJIAA0GS1AwESRJgIEiS2vJJN7BY559/fq1Zs2bSbUjSB8oTTzzx11W1YtRzH9hAWLNmDdPT05NuQ5I+UJL85bGe85SRJAkwECRJzUCQJAEGgiSpGQiSJMBAkCQ1A0GSBBgIkqRmIEiSgA/wN5XfjzXb/nis9V68+YsnuRNJWjo8QpAkAQaCJKkZCJIkwECQJDUDQZIEGAiSpGYgSJIAA0GS1AwESRJgIEiSmoEgSQIMBElSMxAkSYCBIElqBoIkCTAQJEnNQJAkAQaCJKkZCJIkwECQJDUDQZIEjBkISV5M8lSSJ5NMd+3cJI8keb7n5wytvz3JTJL9Sa4Zql/Z25lJcluSdP2MJPd3/fEka07wfkqS5rGQI4R/WlVXVNW6frwN2FNVa4E9/ZgklwJTwGXARuD2JMt6zB3AVmBtTxu7vgV4vaouAW4Fbln8LkmSFuP9nDLaBOzs5Z3AtUP1+6rq7ap6AZgBrkqyEjirqh6rqgLumTNmdlsPABtmjx4kSafGuIFQwHeTPJFka9curKqDAD2/oOurgJeHxh7o2qpenls/akxVHQHeAM6b20SSrUmmk0wfPnx4zNYlSeNYPuZ6V1fVK0kuAB5J8txx1h31l30dp368MUcXqu4E7gRYt27de56XJC3eWEcIVfVKzw8B3wGuAl7t00D0/FCvfgC4aGj4auCVrq8eUT9qTJLlwNnAawvfHUnSYs0bCEk+luQTs8vArwFPA7uBzb3aZuDBXt4NTPWdQxczuHi8t08rvZlkfV8fuGHOmNltXQc82tcZJEmnyDinjC4EvtPXeJcDf1BV/zPJ94FdSbYALwHXA1TVviS7gGeAI8BNVfVOb+tG4G7gTODhngDuAu5NMsPgyGDqBOybJGkB5g2EqvoL4JdG1H8CbDjGmB3AjhH1aeDyEfW36ECRJE2G31SWJAEGgiSpGQiSJMBAkCQ1A0GSBBgIkqRmIEiSAANBktQMBEkSYCBIkpqBIEkCDARJUjMQJEmAgSBJagaCJAkwECRJzUCQJAEGgiSpGQiSJMBAkCQ1A0GSBBgIkqRmIEiSAANBktQMBEkSYCBIkpqBIEkCDARJUhs7EJIsS/LDJA/143OTPJLk+Z6fM7Tu9iQzSfYnuWaofmWSp/q525Kk62ckub/rjydZcwL3UZI0hoUcIXwVeHbo8TZgT1WtBfb0Y5JcCkwBlwEbgduTLOsxdwBbgbU9bez6FuD1qroEuBW4ZVF7I0latLECIclq4IvAN4bKm4CdvbwTuHaofl9VvV1VLwAzwFVJVgJnVdVjVVXAPXPGzG7rAWDD7NGDJOnUGPcI4XeB3wL+fqh2YVUdBOj5BV1fBbw8tN6Brq3q5bn1o8ZU1RHgDeC8uU0k2ZpkOsn04cOHx2xdkjSOeQMhya8Dh6rqiTG3Oeov+zpO/Xhjji5U3VlV66pq3YoVK8ZsR5I0juVjrHM18KUkXwA+ApyV5PeAV5OsrKqDfTroUK9/ALhoaPxq4JWurx5RHx5zIMly4GzgtUXukyRpEeY9Qqiq7VW1uqrWMLhY/GhVfQXYDWzu1TYDD/bybmCq7xy6mMHF4719WunNJOv7+sANc8bMbuu6fo33HCFIkk6ecY4QjuVmYFeSLcBLwPUAVbUvyS7gGeAIcFNVvdNjbgTuBs4EHu4J4C7g3iQzDI4Mpt5HX5KkRVhQIFTV94Dv9fJPgA3HWG8HsGNEfRq4fET9LTpQJEmT4TeVJUmAgSBJagaCJAkwECRJzUCQJAEGgiSpGQiSJMBAkCQ1A0GSBBgIkqRmIEiSAANBktQMBEkSYCBIkpqBIEkCDARJUjMQJEmAgSBJagaCJAkwECRJzUCQJAEGgiSpGQiSJMBAkCQ1A0GSBBgIkqRmIEiSgDECIclHkuxN8qMk+5J8vevnJnkkyfM9P2dozPYkM0n2J7lmqH5lkqf6uduSpOtnJLm/648nWXMS9lWSdBzjHCG8DXyuqn4JuALYmGQ9sA3YU1VrgT39mCSXAlPAZcBG4PYky3pbdwBbgbU9bez6FuD1qroEuBW45f3vmiRpIeYNhBr42374oZ4K2ATs7PpO4Npe3gTcV1VvV9ULwAxwVZKVwFlV9VhVFXDPnDGz23oA2DB79CBJOjXGuoaQZFmSJ4FDwCNV9ThwYVUdBOj5Bb36KuDloeEHuraql+fWjxpTVUeAN4DzRvSxNcl0kunDhw+PtYOSpPGMFQhV9U5VXQGsZvDX/uXHWX3UX/Z1nPrxxszt486qWldV61asWDFP15KkhVjQXUZV9TfA9xic+3+1TwPR80O92gHgoqFhq4FXur56RP2oMUmWA2cDry2kN0nS+zPOXUYrknyyl88EfhV4DtgNbO7VNgMP9vJuYKrvHLqYwcXjvX1a6c0k6/v6wA1zxsxu6zrg0b7OIEk6RZaPsc5KYGffKfRzwK6qeijJY8CuJFuAl4DrAapqX5JdwDPAEeCmqnqnt3UjcDdwJvBwTwB3AfcmmWFwZDB1InZOkjS+eQOhqv4c+PSI+k+ADccYswPYMaI+Dbzn+kNVvUUHiiRpMvymsiQJMBAkSc1AkCQBBoIkqRkIkiTAQJAkNQNBkgQYCJKkZiBIkgADQZLUDARJEmAgSJKagSBJAgwESVIzECRJgIEgSWoGgiQJMBAkSc1AkCQBBoIkqRkIkiTAQJAkNQNBkgQYCJKkZiBIkgADQZLUDARJEjBGICS5KMmfJnk2yb4kX+36uUkeSfJ8z88ZGrM9yUyS/UmuGapfmeSpfu62JOn6GUnu7/rjSdachH2VJB3HOEcIR4B/U1W/AKwHbkpyKbAN2FNVa4E9/Zh+bgq4DNgI3J5kWW/rDmArsLanjV3fArxeVZcAtwK3nIB9kyQtwLyBUFUHq+oHvfwm8CywCtgE7OzVdgLX9vIm4L6qeruqXgBmgKuSrATOqqrHqqqAe+aMmd3WA8CG2aMHSdKpsaBrCH0q59PA48CFVXUQBqEBXNCrrQJeHhp2oGurenlu/agxVXUEeAM4b8Trb00ynWT68OHDC2ldkjSPsQMhyceBPwK+VlU/Pd6qI2p1nPrxxhxdqLqzqtZV1boVK1bM17IkaQHGCoQkH2IQBr9fVd/u8qt9GoieH+r6AeCioeGrgVe6vnpE/agxSZYDZwOvLXRnJEmLN85dRgHuAp6tqt8Zemo3sLmXNwMPDtWn+s6hixlcPN7bp5XeTLK+t3nDnDGz27oOeLSvM0iSTpHlY6xzNfDPgaeSPNm1fwvcDOxKsgV4CbgeoKr2JdkFPMPgDqWbquqdHncjcDdwJvBwTzAInHuTzDA4Mph6f7slSVqoeQOhqv4Xo8/xA2w4xpgdwI4R9Wng8hH1t+hAkSRNht9UliQBBoIkqRkIkiTAQJAkNQNBkgQYCJKkZiBIkgADQZLUDARJEmAgSJKagSBJAgwESVIzECRJgIEgSWoGgiQJMBAkSc1AkCQBBoIkqRkIkiTAQJAkNQNBkgQYCJKkZiBIkgADQZLUDARJEmAgSJKagSBJAsYIhCTfTHIoydNDtXOTPJLk+Z6fM/Tc9iQzSfYnuWaofmWSp/q525Kk62ckub/rjydZc4L3UZI0hnGOEO4GNs6pbQP2VNVaYE8/JsmlwBRwWY+5PcmyHnMHsBVY29PsNrcAr1fVJcCtwC2L3RlJ0uLNGwhV9WfAa3PKm4CdvbwTuHaofl9VvV1VLwAzwFVJVgJnVdVjVVXAPXPGzG7rAWDD7NGDJOnUWew1hAur6iBAzy/o+irg5aH1DnRtVS/PrR81pqqOAG8A5y2yL0nSIp3oi8qj/rKv49SPN+a9G0+2JplOMn348OFFtihJGmWxgfBqnwai54e6fgC4aGi91cArXV89on7UmCTLgbN57ykqAKrqzqpaV1XrVqxYscjWJUmjLDYQdgObe3kz8OBQfarvHLqYwcXjvX1a6c0k6/v6wA1zxsxu6zrg0b7OIEk6hZbPt0KSbwG/Apyf5ADw74GbgV1JtgAvAdcDVNW+JLuAZ4AjwE1V9U5v6kYGdyydCTzcE8BdwL1JZhgcGUydkD2TJC3IvIFQVV8+xlMbjrH+DmDHiPo0cPmI+lt0oEiSJsdvKkuSAANBktQMBEkSYCBIkpqBIEkCDARJUjMQJEmAgSBJagaCJAkwECRJzUCQJAEGgiSpGQiSJMBAkCQ1A0GSBBgIkqRmIEiSAANBktQMBEkSYCBIkpqBIEkCDARJUjMQJEmAgSBJagaCJAkwECRJzUCQJAEGgiSpLZ90A7OSbAT+E7AM+EZV3Tzhlliz7Y/HXvfFm794EjuRpJNvSQRCkmXAfwH+GXAA+H6S3VX1zGQ7G9+44WFwSFqqlkQgAFcBM1X1FwBJ7gM2AR+YQBjXQo46TiSDSNJ8lkogrAJeHnp8APhHc1dKshXY2g//Nsn+Rb7e+cBfL3LsyXTS+sotix66FN+rpdgTLM2+7Gl8S7Gvk9HTPzzWE0slEDKiVu8pVN0J3Pm+XyyZrqp173c7J9pS7MuexrcU+7Kn8S3Fvk51T0vlLqMDwEVDj1cDr0yoF0k6LS2VQPg+sDbJxUk+DEwBuyfckySdVpbEKaOqOpLkN4E/YXDb6Terat9JfMn3fdrpJFmKfdnT+JZiX/Y0vqXY1yntKVXvOVUvSToNLZVTRpKkCTMQJEnAaRgISTYm2Z9kJsm2k7D9byY5lOTpodq5SR5J8nzPzxl6bnv3sj/JNUP1K5M81c/dliRdPyPJ/V1/PMmaMXq6KMmfJnk2yb4kX510X0k+kmRvkh91T1+fdE9D21uW5IdJHlpCPb3Y23syyfRS6CvJJ5M8kOS5/mx9dgn09PP9Hs1OP03ytSXQ17/qz/nTSb6Vwed/4p+r96iq02ZicMH6x8CngA8DPwIuPcGv8cvAZ4Cnh2r/AdjWy9uAW3r50u7hDODi7m1ZP7cX+CyD72g8DHy+6/8S+K+9PAXcP0ZPK4HP9PIngP/Trz2xvnr8x3v5Q8DjwPpJv1e97r8G/gB4aCn89+t1XwTOn1Ob9OdqJ/Abvfxh4JOT7mnEv/e/YvBFrEl+1lcBLwBn9uNdwL9YSu/Vz3pdzKAP6tRv5J8MPd4ObD8Jr7OGowNhP7Cyl1cC+0e9PoO7rD7b6zw3VP8y8N+G1+nl5Qy+xZgF9vcgg9+NWhJ9AR8FfsDg2+kT7YnBd2D2AJ/j3UCY+PvE6ECYWF/AWQz+J5el0tOIHn8N+N+T7ot3f4nh3F7/oe5tybxXs9Ppdspo1E9krDoFr3thVR0E6PkF8/SzqpdH9fmzMVV1BHgDOG/cRvpQ8tMM/iKfaF99auZJ4BDwSFVNvCfgd4HfAv5+qDbpnmDwzf3vJnkig59wmXRfnwIOA/+9T699I8nHJtzTXFPAt3p5Yn1V1f8F/iPwEnAQeKOqvjvJno7ldAuEsX4i4xQ6Vj/H63PR+5Dk48AfAV+rqp9Ouq+qeqeqrmDwV/lVSS6fZE9Jfh04VFVPHKePU9rTkKur6jPA54GbkvzyhPtazuDU6B1V9Wng/zE47THJnt59scEXXL8E/OF8q57svvrawCYGp3/+AfCxJF+ZZE/HcroFwqR+IuPVJCsBen5onn4O9PKoPn82Jsly4GzgtfkaSPIhBmHw+1X17aXSF0BV/Q3wPWDjhHu6GvhSkheB+4DPJfm9CfcEQFW90vNDwHcY/ELwJPs6ABzoozqABxgExMTfq/Z54AdV9Wo/nmRfvwq8UFWHq+rvgG8D/3jCPY10ugXCpH4iYzewuZc3MziHP1uf6jsELgbWAnv78PHNJOv7LoIb5oyZ3dZ1wKPVJw6PpbdxF/BsVf3OUugryYokn+zlMxn8o3lukj1V1faqWl1Vaxh8Nh6tqq9Msqd+fz6W5BOzywzOPz894ffqr4CXk/x8lzYw+Ln6ib5XQ77Mu6eL5m7rVPf1ErA+yUd7WxuAZyfc02gLvejwQZ+ALzC4y+bHwG+fhO1/i8F5wr9jkNpbGJzL2wM83/Nzh9b/7e5lP33HQNfXMfhH/2PgP/Put8o/wuAweIbBHQefGqOnf8Lg8PHPgSd7+sIk+wJ+Efhh9/Q08O+6PtH3amibv8K7F5Un/d/vUwzuOvkRsG/2c7sE+roCmO7/hv8DOGfSPfW4jwI/Ac4eqk36vfo6gz94ngbuZXAH0cTfq7mTP10hSQJOv1NGkqRjMBAkSYCBIElqBoIkCTAQJEnNQJAkAQaCJKn9f99vjAlHreAAAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Kn9iSgy82mNz",
        "outputId": "6b3a6a72-83a7-4de4-c88d-51833e8d3b24"
      },
      "source": [
        "#Calculate rating count threshold\n",
        "count_thresh = movie_avg_rating.rating_count.quantile(0.95).astype(int)\n",
        "print(count_thresh)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "1503\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o-CrwU6T6pl7"
      },
      "source": [
        "### Comments:\n",
        "- For this first simple recommendation system, we will consider the movies with more than 1503 ratings."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 406
        },
        "id": "FAt4GXZP3Lue",
        "outputId": "55abb1e0-e670-4c95-ee9d-0c6cc869acc3"
      },
      "source": [
        "#Filtering the movies below the threshold\n",
        "movie_avg_rating = movie_avg_rating[movie_avg_rating.rating_count > count_thresh]\n",
        "movie_avg_rating.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>movieId</th>\n",
              "      <th>avg_rating</th>\n",
              "      <th>rating_count</th>\n",
              "      <th>title</th>\n",
              "      <th>genres</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>3.893708</td>\n",
              "      <td>57309</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>3.251527</td>\n",
              "      <td>24228</td>\n",
              "      <td>Jumanji (1995)</td>\n",
              "      <td>Adventure|Children|Fantasy</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>3.142028</td>\n",
              "      <td>11804</td>\n",
              "      <td>Grumpier Old Men (1995)</td>\n",
              "      <td>Comedy|Romance</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>2.853547</td>\n",
              "      <td>2523</td>\n",
              "      <td>Waiting to Exhale (1995)</td>\n",
              "      <td>Comedy|Drama|Romance</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>3.058434</td>\n",
              "      <td>11714</td>\n",
              "      <td>Father of the Bride Part II (1995)</td>\n",
              "      <td>Comedy</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   movieId  avg_rating  rating_count                               title  \\\n",
              "0        1    3.893708         57309                    Toy Story (1995)   \n",
              "1        2    3.251527         24228                      Jumanji (1995)   \n",
              "2        3    3.142028         11804             Grumpier Old Men (1995)   \n",
              "3        4    2.853547          2523            Waiting to Exhale (1995)   \n",
              "4        5    3.058434         11714  Father of the Bride Part II (1995)   \n",
              "\n",
              "                                        genres  \n",
              "0  Adventure|Animation|Children|Comedy|Fantasy  \n",
              "1                   Adventure|Children|Fantasy  \n",
              "2                               Comedy|Romance  \n",
              "3                         Comedy|Drama|Romance  \n",
              "4                                       Comedy  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZI9AW7gL9ScB",
        "outputId": "baacc876-7dab-4c65-a049-661eceecc59b"
      },
      "source": [
        "#Storing the total rating average for calculation\n",
        "total_rating_avg = df_ratings.rating.mean()\n",
        "print(\"Total rating average:\", total_rating_avg)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Total rating average: 3.533854451353085\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H1XZideV70gj"
      },
      "source": [
        "### Creating Recommendations\n",
        "- Using the formula from IMDB, we will calculate and add to the dataframe the weighted rating for each movie. \n",
        "- The recommendations will then be generated by highlighting the items with highest weighted rating value, with the option to use genres as a filter."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 197
        },
        "id": "3fg7a0QlxVM9",
        "outputId": "3f9bef08-7cbb-4e1b-d745-efa4d69f0314"
      },
      "source": [
        "#Calculating the weighted average of ratings\n",
        "movie_avg_rating['weighted_avg'] = (movie_avg_rating.rating_count / (movie_avg_rating.rating_count + count_thresh)) * movie_avg_rating.avg_rating \\\n",
        "                                        + (count_thresh / (count_thresh + movie_avg_rating.rating_count)) * total_rating_avg\n",
        "#Sorting the dataframe\n",
        "movie_avg_rating = movie_avg_rating.sort_values(by=['weighted_avg'], ascending=False).reset_index(drop=True)\n",
        "#Deleting unnecessary columns\n",
        "movie_avg_rating = movie_avg_rating.drop(columns=['rating_count', 'movieId', 'avg_rating'], axis=1)\n",
        "#Renaming the rating column\n",
        "movie_avg_rating.rename(columns={\"weighted_avg\": \"rating\"}, inplace=True)\n",
        "\n",
        "movie_avg_rating.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>title</th>\n",
              "      <th>genres</th>\n",
              "      <th>rating</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Shawshank Redemption, The (1994)</td>\n",
              "      <td>Crime|Drama</td>\n",
              "      <td>4.397643</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Godfather, The (1972)</td>\n",
              "      <td>Crime|Drama</td>\n",
              "      <td>4.302335</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Usual Suspects, The (1995)</td>\n",
              "      <td>Crime|Mystery|Thriller</td>\n",
              "      <td>4.264518</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Godfather: Part II, The (1974)</td>\n",
              "      <td>Crime|Drama</td>\n",
              "      <td>4.231105</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Schindler's List (1993)</td>\n",
              "      <td>Drama|War</td>\n",
              "      <td>4.230253</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                              title                  genres    rating\n",
              "0  Shawshank Redemption, The (1994)             Crime|Drama  4.397643\n",
              "1             Godfather, The (1972)             Crime|Drama  4.302335\n",
              "2        Usual Suspects, The (1995)  Crime|Mystery|Thriller  4.264518\n",
              "3    Godfather: Part II, The (1974)             Crime|Drama  4.231105\n",
              "4           Schindler's List (1993)               Drama|War  4.230253"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JqBzAffyBDdM"
      },
      "source": [
        "#Declaring the function that will be used to generate a list of popular movies, accepting genre and number of movies as arguments\n",
        "def get_recommendation(genre = \"All\", num_recommendations = 10):\n",
        "    if genre == \"All\":\n",
        "        results = movie_avg_rating.head(num_recommendations)\n",
        "    else:\n",
        "        results = movie_avg_rating[movie_avg_rating['genres'].str.contains(genre)].head(num_recommendations)\n",
        "        results = results.reset_index(drop=True)\n",
        "    return results"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a_V-K41CB6h9"
      },
      "source": [
        "### Getting Top Movies Overall\n",
        "- Using the function written above, we will generate a list of top 20 most popular and highest rated movies in the list"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 647
        },
        "id": "WBypFoAAB3pE",
        "outputId": "e0a3e7c4-e8d6-4259-fc5f-2e25b4d5781e"
      },
      "source": [
        "recommendations = get_recommendation(num_recommendations=20)\n",
        "recommendations"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>title</th>\n",
              "      <th>genres</th>\n",
              "      <th>rating</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Shawshank Redemption, The (1994)</td>\n",
              "      <td>Crime|Drama</td>\n",
              "      <td>4.397643</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Godfather, The (1972)</td>\n",
              "      <td>Crime|Drama</td>\n",
              "      <td>4.302335</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Usual Suspects, The (1995)</td>\n",
              "      <td>Crime|Mystery|Thriller</td>\n",
              "      <td>4.264518</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Godfather: Part II, The (1974)</td>\n",
              "      <td>Crime|Drama</td>\n",
              "      <td>4.231105</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Schindler's List (1993)</td>\n",
              "      <td>Drama|War</td>\n",
              "      <td>4.230253</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>Fight Club (1999)</td>\n",
              "      <td>Action|Crime|Drama|Thriller</td>\n",
              "      <td>4.210994</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>One Flew Over the Cuckoo's Nest (1975)</td>\n",
              "      <td>Drama</td>\n",
              "      <td>4.191259</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>Rear Window (1954)</td>\n",
              "      <td>Mystery|Thriller</td>\n",
              "      <td>4.189101</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>12 Angry Men (1957)</td>\n",
              "      <td>Drama</td>\n",
              "      <td>4.184035</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>Seven Samurai (Shichinin no samurai) (1954)</td>\n",
              "      <td>Action|Adventure|Drama</td>\n",
              "      <td>4.181902</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>Dr. Strangelove or: How I Learned to Stop Worr...</td>\n",
              "      <td>Comedy|War</td>\n",
              "      <td>4.179480</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>Pulp Fiction (1994)</td>\n",
              "      <td>Comedy|Crime|Drama|Thriller</td>\n",
              "      <td>4.176783</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>Casablanca (1942)</td>\n",
              "      <td>Drama|Romance</td>\n",
              "      <td>4.170954</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>Spirited Away (Sen to Chihiro no kamikakushi) ...</td>\n",
              "      <td>Adventure|Animation|Fantasy</td>\n",
              "      <td>4.170171</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>Goodfellas (1990)</td>\n",
              "      <td>Crime|Drama</td>\n",
              "      <td>4.152078</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>Dark Knight, The (2008)</td>\n",
              "      <td>Action|Crime|Drama|IMAX</td>\n",
              "      <td>4.144435</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>North by Northwest (1959)</td>\n",
              "      <td>Action|Adventure|Mystery|Romance|Thriller</td>\n",
              "      <td>4.142341</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17</th>\n",
              "      <td>Matrix, The (1999)</td>\n",
              "      <td>Action|Sci-Fi|Thriller</td>\n",
              "      <td>4.141532</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>18</th>\n",
              "      <td>Silence of the Lambs, The (1991)</td>\n",
              "      <td>Crime|Horror|Thriller</td>\n",
              "      <td>4.139070</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19</th>\n",
              "      <td>City of God (Cidade de Deus) (2002)</td>\n",
              "      <td>Action|Adventure|Crime|Drama|Thriller</td>\n",
              "      <td>4.136088</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                                title  \\\n",
              "0                    Shawshank Redemption, The (1994)   \n",
              "1                               Godfather, The (1972)   \n",
              "2                          Usual Suspects, The (1995)   \n",
              "3                      Godfather: Part II, The (1974)   \n",
              "4                             Schindler's List (1993)   \n",
              "5                                   Fight Club (1999)   \n",
              "6              One Flew Over the Cuckoo's Nest (1975)   \n",
              "7                                  Rear Window (1954)   \n",
              "8                                 12 Angry Men (1957)   \n",
              "9         Seven Samurai (Shichinin no samurai) (1954)   \n",
              "10  Dr. Strangelove or: How I Learned to Stop Worr...   \n",
              "11                                Pulp Fiction (1994)   \n",
              "12                                  Casablanca (1942)   \n",
              "13  Spirited Away (Sen to Chihiro no kamikakushi) ...   \n",
              "14                                  Goodfellas (1990)   \n",
              "15                            Dark Knight, The (2008)   \n",
              "16                          North by Northwest (1959)   \n",
              "17                                 Matrix, The (1999)   \n",
              "18                   Silence of the Lambs, The (1991)   \n",
              "19                City of God (Cidade de Deus) (2002)   \n",
              "\n",
              "                                       genres    rating  \n",
              "0                                 Crime|Drama  4.397643  \n",
              "1                                 Crime|Drama  4.302335  \n",
              "2                      Crime|Mystery|Thriller  4.264518  \n",
              "3                                 Crime|Drama  4.231105  \n",
              "4                                   Drama|War  4.230253  \n",
              "5                 Action|Crime|Drama|Thriller  4.210994  \n",
              "6                                       Drama  4.191259  \n",
              "7                            Mystery|Thriller  4.189101  \n",
              "8                                       Drama  4.184035  \n",
              "9                      Action|Adventure|Drama  4.181902  \n",
              "10                                 Comedy|War  4.179480  \n",
              "11                Comedy|Crime|Drama|Thriller  4.176783  \n",
              "12                              Drama|Romance  4.170954  \n",
              "13                Adventure|Animation|Fantasy  4.170171  \n",
              "14                                Crime|Drama  4.152078  \n",
              "15                    Action|Crime|Drama|IMAX  4.144435  \n",
              "16  Action|Adventure|Mystery|Romance|Thriller  4.142341  \n",
              "17                     Action|Sci-Fi|Thriller  4.141532  \n",
              "18                      Crime|Horror|Thriller  4.139070  \n",
              "19      Action|Adventure|Crime|Drama|Thriller  4.136088  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1HWrWpphCGks"
      },
      "source": [
        "### Getting Top 10 Movies by Genre\n",
        "- Using the same function, we can filter out movies specifying a genre. \n",
        "- The 2 examples below will get the top Comedy and Action movies respectively."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 347
        },
        "id": "yRi9M1EQCFI7",
        "outputId": "bdb26312-33eb-4552-e275-20d3f67e10fe"
      },
      "source": [
        "recommendations = get_recommendation(\"Comedy\")\n",
        "recommendations"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>title</th>\n",
              "      <th>genres</th>\n",
              "      <th>rating</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Dr. Strangelove or: How I Learned to Stop Worr...</td>\n",
              "      <td>Comedy|War</td>\n",
              "      <td>4.179480</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Pulp Fiction (1994)</td>\n",
              "      <td>Comedy|Crime|Drama|Thriller</td>\n",
              "      <td>4.176783</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Monty Python and the Holy Grail (1975)</td>\n",
              "      <td>Adventure|Comedy|Fantasy</td>\n",
              "      <td>4.124137</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Life Is Beautiful (La Vita è bella) (1997)</td>\n",
              "      <td>Comedy|Drama|Romance|War</td>\n",
              "      <td>4.117739</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Princess Bride, The (1987)</td>\n",
              "      <td>Action|Adventure|Comedy|Fantasy|Romance</td>\n",
              "      <td>4.106371</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>Fargo (1996)</td>\n",
              "      <td>Comedy|Crime|Drama|Thriller</td>\n",
              "      <td>4.093822</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>Sting, The (1973)</td>\n",
              "      <td>Comedy|Crime</td>\n",
              "      <td>4.089119</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>Amelie (Fabuleux destin d'Amélie Poulain, Le) ...</td>\n",
              "      <td>Comedy|Romance</td>\n",
              "      <td>4.077475</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>Wallace &amp; Gromit: The Wrong Trousers (1993)</td>\n",
              "      <td>Animation|Children|Comedy|Crime</td>\n",
              "      <td>4.071835</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>Intouchables (2011)</td>\n",
              "      <td>Comedy|Drama</td>\n",
              "      <td>4.062422</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                               title  \\\n",
              "0  Dr. Strangelove or: How I Learned to Stop Worr...   \n",
              "1                                Pulp Fiction (1994)   \n",
              "2             Monty Python and the Holy Grail (1975)   \n",
              "3         Life Is Beautiful (La Vita è bella) (1997)   \n",
              "4                         Princess Bride, The (1987)   \n",
              "5                                       Fargo (1996)   \n",
              "6                                  Sting, The (1973)   \n",
              "7  Amelie (Fabuleux destin d'Amélie Poulain, Le) ...   \n",
              "8        Wallace & Gromit: The Wrong Trousers (1993)   \n",
              "9                                Intouchables (2011)   \n",
              "\n",
              "                                    genres    rating  \n",
              "0                               Comedy|War  4.179480  \n",
              "1              Comedy|Crime|Drama|Thriller  4.176783  \n",
              "2                 Adventure|Comedy|Fantasy  4.124137  \n",
              "3                 Comedy|Drama|Romance|War  4.117739  \n",
              "4  Action|Adventure|Comedy|Fantasy|Romance  4.106371  \n",
              "5              Comedy|Crime|Drama|Thriller  4.093822  \n",
              "6                             Comedy|Crime  4.089119  \n",
              "7                           Comedy|Romance  4.077475  \n",
              "8          Animation|Children|Comedy|Crime  4.071835  \n",
              "9                             Comedy|Drama  4.062422  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 347
        },
        "id": "BZdvTafNCfYL",
        "outputId": "1f2f7d13-ff03-4e23-aa25-1ed1c4495c7c"
      },
      "source": [
        "recommendations = get_recommendation(\"Action\")\n",
        "recommendations"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>title</th>\n",
              "      <th>genres</th>\n",
              "      <th>rating</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Fight Club (1999)</td>\n",
              "      <td>Action|Crime|Drama|Thriller</td>\n",
              "      <td>4.210994</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Seven Samurai (Shichinin no samurai) (1954)</td>\n",
              "      <td>Action|Adventure|Drama</td>\n",
              "      <td>4.181902</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Dark Knight, The (2008)</td>\n",
              "      <td>Action|Crime|Drama|IMAX</td>\n",
              "      <td>4.144435</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>North by Northwest (1959)</td>\n",
              "      <td>Action|Adventure|Mystery|Romance|Thriller</td>\n",
              "      <td>4.142341</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Matrix, The (1999)</td>\n",
              "      <td>Action|Sci-Fi|Thriller</td>\n",
              "      <td>4.141532</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>City of God (Cidade de Deus) (2002)</td>\n",
              "      <td>Action|Adventure|Crime|Drama|Thriller</td>\n",
              "      <td>4.136088</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>Inception (2010)</td>\n",
              "      <td>Action|Crime|Drama|Mystery|Sci-Fi|Thriller|IMAX</td>\n",
              "      <td>4.132380</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>Star Wars: Episode V - The Empire Strikes Back...</td>\n",
              "      <td>Action|Adventure|Sci-Fi</td>\n",
              "      <td>4.128540</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>Raiders of the Lost Ark (Indiana Jones and the...</td>\n",
              "      <td>Action|Adventure</td>\n",
              "      <td>4.108136</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>Star Wars: Episode IV - A New Hope (1977)</td>\n",
              "      <td>Action|Adventure|Sci-Fi</td>\n",
              "      <td>4.107639</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                               title  \\\n",
              "0                                  Fight Club (1999)   \n",
              "1        Seven Samurai (Shichinin no samurai) (1954)   \n",
              "2                            Dark Knight, The (2008)   \n",
              "3                          North by Northwest (1959)   \n",
              "4                                 Matrix, The (1999)   \n",
              "5                City of God (Cidade de Deus) (2002)   \n",
              "6                                   Inception (2010)   \n",
              "7  Star Wars: Episode V - The Empire Strikes Back...   \n",
              "8  Raiders of the Lost Ark (Indiana Jones and the...   \n",
              "9          Star Wars: Episode IV - A New Hope (1977)   \n",
              "\n",
              "                                            genres    rating  \n",
              "0                      Action|Crime|Drama|Thriller  4.210994  \n",
              "1                           Action|Adventure|Drama  4.181902  \n",
              "2                          Action|Crime|Drama|IMAX  4.144435  \n",
              "3        Action|Adventure|Mystery|Romance|Thriller  4.142341  \n",
              "4                           Action|Sci-Fi|Thriller  4.141532  \n",
              "5            Action|Adventure|Crime|Drama|Thriller  4.136088  \n",
              "6  Action|Crime|Drama|Mystery|Sci-Fi|Thriller|IMAX  4.132380  \n",
              "7                          Action|Adventure|Sci-Fi  4.128540  \n",
              "8                                 Action|Adventure  4.108136  \n",
              "9                          Action|Adventure|Sci-Fi  4.107639  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cgADxbAPMOgY"
      },
      "source": [
        "# 3. Content-Based Filtering\n",
        "- In content-based filtering, we will consider movies' *genres and short overviews* to generate lists of similar movies. \n",
        "- Due to computational expense and runtime, we applied this model considering a sample of 5000 most popular movies.\n",
        "- In Section 3.1, we use jaccard similarity to compare genre overlapping between movies and determine most relevant movies.\n",
        "- In Section 3.2, we will analyze movie descriptions using NLP to identify items with similar contents."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m2RpqoLOs1id"
      },
      "source": [
        "## 3.1 Content-Based - Genres Similarity"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ywYF7ISln2I3"
      },
      "source": [
        "- Jaccard index, or Jaccard similarity coefficient, measures the similarity between sample sets A and B, using the formula:\n",
        "\n",
        "<div align=\"center\">$\\Large J(A,B) = \\frac{|A \\bigcap B|}{|A \\bigcup B|}$  with  $\\Large 0 \\leq J(A,B)  \\leq 1$</div>\n",
        "\n",
        "- Using this Jaccard coefficient, we will determine movies with a high correlation of overlapping genres with any given target movie, outputting a list of recommendations. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "O3sZCYlzxnG1"
      },
      "source": [
        "# Sample top 5000 movies to be analyzed\n",
        "df_movies_genres = get_recommendation(\"All\",5000).copy()\n",
        "df_movies_genres.drop(columns=['rating'], inplace=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 197
        },
        "id": "E-W4hq8Baxzr",
        "outputId": "d6cc0aca-844c-448b-9f8a-9925a130bb26"
      },
      "source": [
        "# Split genre strings into lists\n",
        "df_movies_genres.genres = df_movies_genres.genres.apply(lambda x: x.split('|'))\n",
        "df_movies_genres.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>title</th>\n",
              "      <th>genres</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Shawshank Redemption, The (1994)</td>\n",
              "      <td>[Crime, Drama]</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Godfather, The (1972)</td>\n",
              "      <td>[Crime, Drama]</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Usual Suspects, The (1995)</td>\n",
              "      <td>[Crime, Mystery, Thriller]</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Godfather: Part II, The (1974)</td>\n",
              "      <td>[Crime, Drama]</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Schindler's List (1993)</td>\n",
              "      <td>[Drama, War]</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                              title                      genres\n",
              "0  Shawshank Redemption, The (1994)              [Crime, Drama]\n",
              "1             Godfather, The (1972)              [Crime, Drama]\n",
              "2        Usual Suspects, The (1995)  [Crime, Mystery, Thriller]\n",
              "3    Godfather: Part II, The (1974)              [Crime, Drama]\n",
              "4           Schindler's List (1993)                [Drama, War]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 347
        },
        "id": "LVtMApJ5bGQw",
        "outputId": "958c558d-d0da-41ed-c21a-07f1a8a77339"
      },
      "source": [
        "#Exploding genres to individual rows\n",
        "df_movies_genres = df_movies_genres.explode('genres').reset_index(drop=True)\n",
        "# Delete row indexes with no genres\n",
        "df_movies_genres.drop(df_movies_genres[ df_movies_genres['genres'] == \"(no genres listed)\" ].index , inplace=True)\n",
        "\n",
        "df_movies_genres.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>title</th>\n",
              "      <th>genres</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Shawshank Redemption, The (1994)</td>\n",
              "      <td>Crime</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Shawshank Redemption, The (1994)</td>\n",
              "      <td>Drama</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Godfather, The (1972)</td>\n",
              "      <td>Crime</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Godfather, The (1972)</td>\n",
              "      <td>Drama</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Usual Suspects, The (1995)</td>\n",
              "      <td>Crime</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                              title genres\n",
              "0  Shawshank Redemption, The (1994)  Crime\n",
              "1  Shawshank Redemption, The (1994)  Drama\n",
              "2             Godfather, The (1972)  Crime\n",
              "3             Godfather, The (1972)  Drama\n",
              "4        Usual Suspects, The (1995)  Crime"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 227
        },
        "id": "qjHG6GaYndFS",
        "outputId": "d02f33ec-16bb-4124-a476-31a1eda6b0a7"
      },
      "source": [
        "# Create cross-tabulated DataFrame from name and genre_list columns\n",
        "movie_cross_table = pd.crosstab(df_movies_genres['title'], df_movies_genres['genres'])\n",
        "movie_cross_table.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th>genres</th>\n",
              "      <th>Action</th>\n",
              "      <th>Adventure</th>\n",
              "      <th>Animation</th>\n",
              "      <th>Children</th>\n",
              "      <th>Comedy</th>\n",
              "      <th>Crime</th>\n",
              "      <th>Documentary</th>\n",
              "      <th>Drama</th>\n",
              "      <th>Fantasy</th>\n",
              "      <th>Film-Noir</th>\n",
              "      <th>Horror</th>\n",
              "      <th>IMAX</th>\n",
              "      <th>Musical</th>\n",
              "      <th>Mystery</th>\n",
              "      <th>Romance</th>\n",
              "      <th>Sci-Fi</th>\n",
              "      <th>Thriller</th>\n",
              "      <th>War</th>\n",
              "      <th>Western</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>title</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>'burbs, The (1989)</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>(500) Days of Summer (2009)</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>*batteries not included (1987)</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10 Cloverfield Lane (2016)</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10 Things I Hate About You (1999)</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "genres                             Action  Adventure  Animation  Children  \\\n",
              "title                                                                       \n",
              "'burbs, The (1989)                      0          0          0         0   \n",
              "(500) Days of Summer (2009)             0          0          0         0   \n",
              "*batteries not included (1987)          0          0          0         1   \n",
              "10 Cloverfield Lane (2016)              0          0          0         0   \n",
              "10 Things I Hate About You (1999)       0          0          0         0   \n",
              "\n",
              "genres                             Comedy  Crime  Documentary  Drama  Fantasy  \\\n",
              "title                                                                           \n",
              "'burbs, The (1989)                      1      0            0      0        0   \n",
              "(500) Days of Summer (2009)             1      0            0      1        0   \n",
              "*batteries not included (1987)          1      0            0      0        1   \n",
              "10 Cloverfield Lane (2016)              0      0            0      0        0   \n",
              "10 Things I Hate About You (1999)       1      0            0      0        0   \n",
              "\n",
              "genres                             Film-Noir  Horror  IMAX  Musical  Mystery  \\\n",
              "title                                                                          \n",
              "'burbs, The (1989)                         0       0     0        0        0   \n",
              "(500) Days of Summer (2009)                0       0     0        0        0   \n",
              "*batteries not included (1987)             0       0     0        0        0   \n",
              "10 Cloverfield Lane (2016)                 0       0     0        0        0   \n",
              "10 Things I Hate About You (1999)          0       0     0        0        0   \n",
              "\n",
              "genres                             Romance  Sci-Fi  Thriller  War  Western  \n",
              "title                                                                       \n",
              "'burbs, The (1989)                       0       0         0    0        0  \n",
              "(500) Days of Summer (2009)              1       0         0    0        0  \n",
              "*batteries not included (1987)           0       1         0    0        0  \n",
              "10 Cloverfield Lane (2016)               0       0         1    0        0  \n",
              "10 Things I Hate About You (1999)        1       0         0    0        0  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Kmhpar1ynQau",
        "outputId": "9756290f-1c6a-4129-835b-bc4ce24e2248"
      },
      "source": [
        "# Calculate all pairwise distances\n",
        "jaccard_distances = pdist(movie_cross_table.values, metric='jaccard')\n",
        "\n",
        "# Convert the distances to a square matrix\n",
        "jaccard_similarity_array = 1 - squareform(jaccard_distances)\n",
        "\n",
        "# Wrap the array in a pandas DataFrame\n",
        "jaccard_similarity_df = pd.DataFrame(jaccard_similarity_array, index=movie_cross_table.index, columns=movie_cross_table.index)\n",
        "\n",
        "jaccard_similarity_df.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th>title</th>\n",
              "      <th>'burbs, The (1989)</th>\n",
              "      <th>(500) Days of Summer (2009)</th>\n",
              "      <th>*batteries not included (1987)</th>\n",
              "      <th>10 Cloverfield Lane (2016)</th>\n",
              "      <th>10 Things I Hate About You (1999)</th>\n",
              "      <th>10,000 BC (2008)</th>\n",
              "      <th>101 Dalmatians (1996)</th>\n",
              "      <th>101 Dalmatians (One Hundred and One Dalmatians) (1961)</th>\n",
              "      <th>102 Dalmatians (2000)</th>\n",
              "      <th>12 Angry Men (1957)</th>\n",
              "      <th>...</th>\n",
              "      <th>Zero Dark Thirty (2012)</th>\n",
              "      <th>Zero Effect (1998)</th>\n",
              "      <th>Zodiac (2007)</th>\n",
              "      <th>Zombieland (2009)</th>\n",
              "      <th>Zoolander (2001)</th>\n",
              "      <th>Zootopia (2016)</th>\n",
              "      <th>[REC] (2007)</th>\n",
              "      <th>eXistenZ (1999)</th>\n",
              "      <th>xXx (2002)</th>\n",
              "      <th>¡Three Amigos! (1986)</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>title</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>'burbs, The (1989)</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.333333</td>\n",
              "      <td>0.250000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.333333</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.333333</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.333333</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.200000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.500000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>(500) Days of Summer (2009)</th>\n",
              "      <td>0.333333</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.166667</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.666667</td>\n",
              "      <td>0.200000</td>\n",
              "      <td>0.200000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.250000</td>\n",
              "      <td>0.333333</td>\n",
              "      <td>...</td>\n",
              "      <td>0.200000</td>\n",
              "      <td>0.200000</td>\n",
              "      <td>0.200000</td>\n",
              "      <td>0.200000</td>\n",
              "      <td>0.333333</td>\n",
              "      <td>0.142857</td>\n",
              "      <td>0.200000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.250000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>*batteries not included (1987)</th>\n",
              "      <td>0.250000</td>\n",
              "      <td>0.166667</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.200000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.400000</td>\n",
              "      <td>0.166667</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.166667</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.166667</td>\n",
              "      <td>0.250000</td>\n",
              "      <td>0.285714</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.166667</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.200000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10 Cloverfield Lane (2016)</th>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.333333</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.333333</td>\n",
              "      <td>0.333333</td>\n",
              "      <td>0.333333</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.333333</td>\n",
              "      <td>0.333333</td>\n",
              "      <td>0.333333</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10 Things I Hate About You (1999)</th>\n",
              "      <td>0.500000</td>\n",
              "      <td>0.666667</td>\n",
              "      <td>0.200000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.250000</td>\n",
              "      <td>0.250000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.333333</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.250000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.250000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>0.166667</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.333333</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 2952 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "title                              'burbs, The (1989)  \\\n",
              "title                                                   \n",
              "'burbs, The (1989)                           1.000000   \n",
              "(500) Days of Summer (2009)                  0.333333   \n",
              "*batteries not included (1987)               0.250000   \n",
              "10 Cloverfield Lane (2016)                   0.000000   \n",
              "10 Things I Hate About You (1999)            0.500000   \n",
              "\n",
              "title                              (500) Days of Summer (2009)  \\\n",
              "title                                                            \n",
              "'burbs, The (1989)                                    0.333333   \n",
              "(500) Days of Summer (2009)                           1.000000   \n",
              "*batteries not included (1987)                        0.166667   \n",
              "10 Cloverfield Lane (2016)                            0.000000   \n",
              "10 Things I Hate About You (1999)                     0.666667   \n",
              "\n",
              "title                              *batteries not included (1987)  \\\n",
              "title                                                               \n",
              "'burbs, The (1989)                                       0.250000   \n",
              "(500) Days of Summer (2009)                              0.166667   \n",
              "*batteries not included (1987)                           1.000000   \n",
              "10 Cloverfield Lane (2016)                               0.000000   \n",
              "10 Things I Hate About You (1999)                        0.200000   \n",
              "\n",
              "title                              10 Cloverfield Lane (2016)  \\\n",
              "title                                                           \n",
              "'burbs, The (1989)                                        0.0   \n",
              "(500) Days of Summer (2009)                               0.0   \n",
              "*batteries not included (1987)                            0.0   \n",
              "10 Cloverfield Lane (2016)                                1.0   \n",
              "10 Things I Hate About You (1999)                         0.0   \n",
              "\n",
              "title                              10 Things I Hate About You (1999)  \\\n",
              "title                                                                  \n",
              "'burbs, The (1989)                                          0.500000   \n",
              "(500) Days of Summer (2009)                                 0.666667   \n",
              "*batteries not included (1987)                              0.200000   \n",
              "10 Cloverfield Lane (2016)                                  0.000000   \n",
              "10 Things I Hate About You (1999)                           1.000000   \n",
              "\n",
              "title                              10,000 BC (2008)  101 Dalmatians (1996)  \\\n",
              "title                                                                        \n",
              "'burbs, The (1989)                         0.000000               0.333333   \n",
              "(500) Days of Summer (2009)                0.200000               0.200000   \n",
              "*batteries not included (1987)             0.000000               0.400000   \n",
              "10 Cloverfield Lane (2016)                 0.333333               0.000000   \n",
              "10 Things I Hate About You (1999)          0.250000               0.250000   \n",
              "\n",
              "title                              101 Dalmatians (One Hundred and One Dalmatians) (1961)  \\\n",
              "title                                                                                       \n",
              "'burbs, The (1989)                                                          0.000000        \n",
              "(500) Days of Summer (2009)                                                 0.000000        \n",
              "*batteries not included (1987)                                              0.166667        \n",
              "10 Cloverfield Lane (2016)                                                  0.000000        \n",
              "10 Things I Hate About You (1999)                                           0.000000        \n",
              "\n",
              "title                              102 Dalmatians (2000)  12 Angry Men (1957)  \\\n",
              "title                                                                           \n",
              "'burbs, The (1989)                              0.500000             0.000000   \n",
              "(500) Days of Summer (2009)                     0.250000             0.333333   \n",
              "*batteries not included (1987)                  0.500000             0.000000   \n",
              "10 Cloverfield Lane (2016)                      0.000000             0.000000   \n",
              "10 Things I Hate About You (1999)               0.333333             0.000000   \n",
              "\n",
              "title                              ...  Zero Dark Thirty (2012)  \\\n",
              "title                              ...                            \n",
              "'burbs, The (1989)                 ...                 0.000000   \n",
              "(500) Days of Summer (2009)        ...                 0.200000   \n",
              "*batteries not included (1987)     ...                 0.000000   \n",
              "10 Cloverfield Lane (2016)         ...                 0.333333   \n",
              "10 Things I Hate About You (1999)  ...                 0.000000   \n",
              "\n",
              "title                              Zero Effect (1998)  Zodiac (2007)  \\\n",
              "title                                                                  \n",
              "'burbs, The (1989)                           0.333333       0.000000   \n",
              "(500) Days of Summer (2009)                  0.200000       0.200000   \n",
              "*batteries not included (1987)               0.166667       0.000000   \n",
              "10 Cloverfield Lane (2016)                   0.333333       0.333333   \n",
              "10 Things I Hate About You (1999)            0.250000       0.000000   \n",
              "\n",
              "title                              Zombieland (2009)  Zoolander (2001)  \\\n",
              "title                                                                    \n",
              "'burbs, The (1989)                          0.333333          1.000000   \n",
              "(500) Days of Summer (2009)                 0.200000          0.333333   \n",
              "*batteries not included (1987)              0.166667          0.250000   \n",
              "10 Cloverfield Lane (2016)                  0.000000          0.000000   \n",
              "10 Things I Hate About You (1999)           0.250000          0.500000   \n",
              "\n",
              "title                              Zootopia (2016)  [REC] (2007)  \\\n",
              "title                                                              \n",
              "'burbs, The (1989)                        0.200000      0.000000   \n",
              "(500) Days of Summer (2009)               0.142857      0.200000   \n",
              "*batteries not included (1987)            0.285714      0.000000   \n",
              "10 Cloverfield Lane (2016)                0.000000      0.333333   \n",
              "10 Things I Hate About You (1999)         0.166667      0.000000   \n",
              "\n",
              "title                              eXistenZ (1999)  xXx (2002)  \\\n",
              "title                                                            \n",
              "'burbs, The (1989)                        0.000000    0.000000   \n",
              "(500) Days of Summer (2009)               0.000000    0.000000   \n",
              "*batteries not included (1987)            0.166667    0.000000   \n",
              "10 Cloverfield Lane (2016)                0.333333    0.333333   \n",
              "10 Things I Hate About You (1999)         0.000000    0.000000   \n",
              "\n",
              "title                              ¡Three Amigos! (1986)  \n",
              "title                                                     \n",
              "'burbs, The (1989)                              0.500000  \n",
              "(500) Days of Summer (2009)                     0.250000  \n",
              "*batteries not included (1987)                  0.200000  \n",
              "10 Cloverfield Lane (2016)                      0.000000  \n",
              "10 Things I Hate About You (1999)               0.333333  \n",
              "\n",
              "[5 rows x 2952 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DMrbk9t8n2I5"
      },
      "source": [
        "### Creating Recommendations"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wf8ftE5esw0P"
      },
      "source": [
        "#Funtion to find similar movies based on genres, accepting a target movie and the desired size of returned list\n",
        "def find_similar_genres(movie_name, num_recommendations = 10):\n",
        "    #locate the row with specified movie and the similarity coefficient\n",
        "    jaccard_similarity_series = jaccard_similarity_df.loc[movie_name]\n",
        "    #sort value\n",
        "    ordered_similarities = jaccard_similarity_series.drop([movie_name]).sort_values(ascending=False)\n",
        "    # Returns the results\n",
        "    return ordered_similarities.head(num_recommendations)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NrkxWJnItIw6",
        "outputId": "567ba3ff-56a2-4608-87b1-94a13e49639c"
      },
      "source": [
        "find_similar_genres(\"Star Wars: Episode V - The Empire Strikes Back (1980)\")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "title\n",
              "Sky Captain and the World of Tomorrow (2004)           1.0\n",
              "Demolition Man (1993)                                  1.0\n",
              "Black Panther (2017)                                   1.0\n",
              "Star Trek Beyond (2016)                                1.0\n",
              "Star Trek III: The Search for Spock (1984)             1.0\n",
              "Star Wars: Episode I - The Phantom Menace (1999)       1.0\n",
              "Star Wars: Episode III - Revenge of the Sith (2005)    1.0\n",
              "Star Wars: Episode IV - A New Hope (1977)              1.0\n",
              "Star Wars: Episode VI - Return of the Jedi (1983)      1.0\n",
              "Stargate (1994)                                        1.0\n",
              "Name: Star Wars: Episode V - The Empire Strikes Back (1980), dtype: float64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-64G8GRjuNtY"
      },
      "source": [
        "## 3.2 Content-based: Description similarities\n",
        "- In order to compare the movies' descriptions, we will use TF-IDF to determine keywords' importance in all desciptions, and then compare the result between movies using Cosine similarity. \n",
        "\n",
        "### TF-IDF\n",
        "-  **TF-IDF (Term frequency inverse document frequency)** can determine keywords' importance based on their frequencies in the corpus, calculated using the following formula:\n",
        "\n",
        "<div align=\"center\">$\\Large tfidf(t,D) = tf(t,d) \\times idf(t,D)$</div>\n",
        "\n",
        "    with:\n",
        "\n",
        "$\\Large tf(t,D) = 0.5 + 0.5 \\times \\frac{f_{t,d}}{max\\{f_{t',d} : t'\\in d\\}}$\n",
        "\n",
        "$\\Large idf(t,D) = log \\frac{N}{|\\{d \\in D : t \\in d\\}|}$\n",
        "\n",
        "- For more information on tf-idf, please visit [TF-IDF Website](http://www.tfidf.com/)\n",
        "\n",
        "### Cosine similarity\n",
        "- Cosine similarity measures the distance between 2 non-zero vectors, equating to the cosine of angle between them. Using this calculation, we can identify similar movie descriptions by cross-referencing word occurances."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KO0l9UOANMXD",
        "outputId": "a921185a-f52e-4039-fd26-d4403f42cb48"
      },
      "source": [
        "# Sample top 5000 movies to be analyzed\n",
        "df_desc_recomm = get_recommendation(\"All\",5000).copy()\n",
        "df_desc_recomm = pd.merge(df_desc_recomm, df_descriptions, on='title')\n",
        "df_desc_recomm.drop(columns=['genres_x', 'genres_y', 'rating', 'movieId'], inplace=True)\n",
        "\n",
        "df_desc_recomm.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>title</th>\n",
              "      <th>overview</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Shawshank Redemption, The (1994)</td>\n",
              "      <td>Framed in the 1940s for the double murder of h...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Godfather, The (1972)</td>\n",
              "      <td>Spanning the years 1945 to 1955, a chronicle o...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Usual Suspects, The (1995)</td>\n",
              "      <td>Held in an L.A. interrogation room, Verbal Kin...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Godfather: Part II, The (1974)</td>\n",
              "      <td>In the continuing saga of the Corleone crime f...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Schindler's List (1993)</td>\n",
              "      <td>The true story of how businessman Oskar Schind...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                              title  \\\n",
              "0  Shawshank Redemption, The (1994)   \n",
              "1             Godfather, The (1972)   \n",
              "2        Usual Suspects, The (1995)   \n",
              "3    Godfather: Part II, The (1974)   \n",
              "4           Schindler's List (1993)   \n",
              "\n",
              "                                            overview  \n",
              "0  Framed in the 1940s for the double murder of h...  \n",
              "1  Spanning the years 1945 to 1955, a chronicle o...  \n",
              "2  Held in an L.A. interrogation room, Verbal Kin...  \n",
              "3  In the continuing saga of the Corleone crime f...  \n",
              "4  The true story of how businessman Oskar Schind...  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GC5bRQvizLfG"
      },
      "source": [
        "# get words with at least 2 mentions or which appear in less than 75% of movies, then filter stop words\n",
        "tfidf = TfidfVectorizer(min_df=2, max_df=0.75, stop_words='english')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uwXauhbuMrtE"
      },
      "source": [
        "# fit the description data to tfidf\n",
        "vertorized_data = tfidf.fit_transform(df_desc_recomm.overview)\n",
        "# print(tfidf.get_feature_names())"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Qp9L_xZkR7-Y",
        "outputId": "5137d4e2-60c9-4a7a-9f92-9a95c6c05364"
      },
      "source": [
        "# Add words to a dataframe containing the movie title\n",
        "df_tfidf = pd.DataFrame(vertorized_data.toarray(),\n",
        "                        columns=tfidf.get_feature_names())\n",
        "\n",
        "df_tfidf.index = df_desc_recomm['title']\n",
        "df_tfidf.head(3)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>00</th>\n",
              "      <th>000</th>\n",
              "      <th>007</th>\n",
              "      <th>10</th>\n",
              "      <th>100</th>\n",
              "      <th>10th</th>\n",
              "      <th>11</th>\n",
              "      <th>12</th>\n",
              "      <th>13</th>\n",
              "      <th>14</th>\n",
              "      <th>...</th>\n",
              "      <th>zeus</th>\n",
              "      <th>zion</th>\n",
              "      <th>zoe</th>\n",
              "      <th>zombie</th>\n",
              "      <th>zombies</th>\n",
              "      <th>zone</th>\n",
              "      <th>zones</th>\n",
              "      <th>zoo</th>\n",
              "      <th>zooey</th>\n",
              "      <th>zorro</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>title</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>Shawshank Redemption, The (1994)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Godfather, The (1972)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Usual Suspects, The (1995)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>3 rows × 8072 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "                                   00  000  007   10  100  10th   11   12  \\\n",
              "title                                                                       \n",
              "Shawshank Redemption, The (1994)  0.0  0.0  0.0  0.0  0.0   0.0  0.0  0.0   \n",
              "Godfather, The (1972)             0.0  0.0  0.0  0.0  0.0   0.0  0.0  0.0   \n",
              "Usual Suspects, The (1995)        0.0  0.0  0.0  0.0  0.0   0.0  0.0  0.0   \n",
              "\n",
              "                                   13   14  ...  zeus  zion  zoe  zombie  \\\n",
              "title                                       ...                            \n",
              "Shawshank Redemption, The (1994)  0.0  0.0  ...   0.0   0.0  0.0     0.0   \n",
              "Godfather, The (1972)             0.0  0.0  ...   0.0   0.0  0.0     0.0   \n",
              "Usual Suspects, The (1995)        0.0  0.0  ...   0.0   0.0  0.0     0.0   \n",
              "\n",
              "                                  zombies  zone  zones  zoo  zooey  zorro  \n",
              "title                                                                      \n",
              "Shawshank Redemption, The (1994)      0.0   0.0    0.0  0.0    0.0    0.0  \n",
              "Godfather, The (1972)                 0.0   0.0    0.0  0.0    0.0    0.0  \n",
              "Usual Suspects, The (1995)            0.0   0.0    0.0  0.0    0.0    0.0  \n",
              "\n",
              "[3 rows x 8072 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZkQK4hmWn2I8"
      },
      "source": [
        "### Filter numeric columns not corresponding to year"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 923
        },
        "id": "G-ZlGqZpWkn7",
        "outputId": "0ace6109-2393-40be-f250-302b82c1a91b"
      },
      "source": [
        "#Find numeric columns\n",
        "int_columns = [s for s in df_tfidf.columns if s.isdigit()]\n",
        "#Filter columns that start with 1 or 2 and have 4 characters (likely years)\n",
        "int_columns = list(filter(lambda x: (not x.startswith('1') and not x.startswith('2')) or len(x) != 4, int_columns))\n",
        "\n",
        "#Dropping the int columns\n",
        "df_tfidf.drop(columns=int_columns, inplace=True)\n",
        "df_tfidf.head(3)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>10th</th>\n",
              "      <th>15th</th>\n",
              "      <th>16th</th>\n",
              "      <th>17th</th>\n",
              "      <th>1800</th>\n",
              "      <th>1863</th>\n",
              "      <th>1890s</th>\n",
              "      <th>18th</th>\n",
              "      <th>1900s</th>\n",
              "      <th>1910</th>\n",
              "      <th>...</th>\n",
              "      <th>zeus</th>\n",
              "      <th>zion</th>\n",
              "      <th>zoe</th>\n",
              "      <th>zombie</th>\n",
              "      <th>zombies</th>\n",
              "      <th>zone</th>\n",
              "      <th>zones</th>\n",
              "      <th>zoo</th>\n",
              "      <th>zooey</th>\n",
              "      <th>zorro</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>title</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>Shawshank Redemption, The (1994)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Godfather, The (1972)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Usual Suspects, The (1995)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>3 rows × 8029 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "                                  10th  15th  16th  17th  1800  1863  1890s  \\\n",
              "title                                                                         \n",
              "Shawshank Redemption, The (1994)   0.0   0.0   0.0   0.0   0.0   0.0    0.0   \n",
              "Godfather, The (1972)              0.0   0.0   0.0   0.0   0.0   0.0    0.0   \n",
              "Usual Suspects, The (1995)         0.0   0.0   0.0   0.0   0.0   0.0    0.0   \n",
              "\n",
              "                                  18th  1900s  1910  ...  zeus  zion  zoe  \\\n",
              "title                                                ...                    \n",
              "Shawshank Redemption, The (1994)   0.0    0.0   0.0  ...   0.0   0.0  0.0   \n",
              "Godfather, The (1972)              0.0    0.0   0.0  ...   0.0   0.0  0.0   \n",
              "Usual Suspects, The (1995)         0.0    0.0   0.0  ...   0.0   0.0  0.0   \n",
              "\n",
              "                                  zombie  zombies  zone  zones  zoo  zooey  \\\n",
              "title                                                                        \n",
              "Shawshank Redemption, The (1994)     0.0      0.0   0.0    0.0  0.0    0.0   \n",
              "Godfather, The (1972)                0.0      0.0   0.0    0.0  0.0    0.0   \n",
              "Usual Suspects, The (1995)           0.0      0.0   0.0    0.0  0.0    0.0   \n",
              "\n",
              "                                  zorro  \n",
              "title                                    \n",
              "Shawshank Redemption, The (1994)    0.0  \n",
              "Godfather, The (1972)               0.0  \n",
              "Usual Suspects, The (1995)          0.0  \n",
              "\n",
              "[3 rows x 8029 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 30
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oQvlDFGWT-6j"
      },
      "source": [
        "### Identify similarity between data with Cosine Similarity"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eKcCBWxvUkU7"
      },
      "source": [
        "# calculate similarity between all items\n",
        "cosine_sim = cosine_similarity(df_tfidf)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ko-Z9n_dWgS-",
        "outputId": "e3efa11b-7608-4ab5-f80c-85a340bc67b1"
      },
      "source": [
        "# pur results in a dataframe\n",
        "df_similarity = pd.DataFrame(cosine_sim, columns=df_tfidf.index, index=df_tfidf.index)\n",
        "df_similarity.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th>title</th>\n",
              "      <th>Shawshank Redemption, The (1994)</th>\n",
              "      <th>Godfather, The (1972)</th>\n",
              "      <th>Usual Suspects, The (1995)</th>\n",
              "      <th>Godfather: Part II, The (1974)</th>\n",
              "      <th>Schindler's List (1993)</th>\n",
              "      <th>Fight Club (1999)</th>\n",
              "      <th>One Flew Over the Cuckoo's Nest (1975)</th>\n",
              "      <th>Rear Window (1954)</th>\n",
              "      <th>12 Angry Men (1957)</th>\n",
              "      <th>Seven Samurai (Shichinin no samurai) (1954)</th>\n",
              "      <th>...</th>\n",
              "      <th>Showgirls (1995)</th>\n",
              "      <th>Wild Wild West (1999)</th>\n",
              "      <th>Grease 2 (1982)</th>\n",
              "      <th>Spice World (1997)</th>\n",
              "      <th>Home Alone 3 (1997)</th>\n",
              "      <th>Super Mario Bros. (1993)</th>\n",
              "      <th>Anaconda (1997)</th>\n",
              "      <th>Batman &amp; Robin (1997)</th>\n",
              "      <th>Speed 2: Cruise Control (1997)</th>\n",
              "      <th>Battlefield Earth (2000)</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>title</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>Shawshank Redemption, The (1994)</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.006940</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.008126</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.008379</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.034679</td>\n",
              "      <td>0.017646</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.026994</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.021276</td>\n",
              "      <td>0.008312</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.033757</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.007657</td>\n",
              "      <td>0.011338</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Godfather, The (1972)</th>\n",
              "      <td>0.006940</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.069111</td>\n",
              "      <td>0.424837</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.014397</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.030850</td>\n",
              "      <td>0.046529</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.034888</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Usual Suspects, The (1995)</th>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.069111</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.059527</td>\n",
              "      <td>0.015213</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.030280</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.035370</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Godfather: Part II, The (1974)</th>\n",
              "      <td>0.008126</td>\n",
              "      <td>0.424837</td>\n",
              "      <td>0.059527</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.008079</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.021443</td>\n",
              "      <td>0.009771</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.132776</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.043473</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.026333</td>\n",
              "      <td>0.023739</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Schindler's List (1993)</th>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.015213</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.018385</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.019657</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.075014</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 2927 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "title                             Shawshank Redemption, The (1994)  \\\n",
              "title                                                                \n",
              "Shawshank Redemption, The (1994)                          1.000000   \n",
              "Godfather, The (1972)                                     0.006940   \n",
              "Usual Suspects, The (1995)                                0.000000   \n",
              "Godfather: Part II, The (1974)                            0.008126   \n",
              "Schindler's List (1993)                                   0.000000   \n",
              "\n",
              "title                             Godfather, The (1972)  \\\n",
              "title                                                     \n",
              "Shawshank Redemption, The (1994)               0.006940   \n",
              "Godfather, The (1972)                          1.000000   \n",
              "Usual Suspects, The (1995)                     0.069111   \n",
              "Godfather: Part II, The (1974)                 0.424837   \n",
              "Schindler's List (1993)                        0.000000   \n",
              "\n",
              "title                             Usual Suspects, The (1995)  \\\n",
              "title                                                          \n",
              "Shawshank Redemption, The (1994)                    0.000000   \n",
              "Godfather, The (1972)                               0.069111   \n",
              "Usual Suspects, The (1995)                          1.000000   \n",
              "Godfather: Part II, The (1974)                      0.059527   \n",
              "Schindler's List (1993)                             0.015213   \n",
              "\n",
              "title                             Godfather: Part II, The (1974)  \\\n",
              "title                                                              \n",
              "Shawshank Redemption, The (1994)                        0.008126   \n",
              "Godfather, The (1972)                                   0.424837   \n",
              "Usual Suspects, The (1995)                              0.059527   \n",
              "Godfather: Part II, The (1974)                          1.000000   \n",
              "Schindler's List (1993)                                 0.000000   \n",
              "\n",
              "title                             Schindler's List (1993)  Fight Club (1999)  \\\n",
              "title                                                                          \n",
              "Shawshank Redemption, The (1994)                 0.000000           0.008379   \n",
              "Godfather, The (1972)                            0.000000           0.000000   \n",
              "Usual Suspects, The (1995)                       0.015213           0.000000   \n",
              "Godfather: Part II, The (1974)                   0.000000           0.008079   \n",
              "Schindler's List (1993)                          1.000000           0.000000   \n",
              "\n",
              "title                             One Flew Over the Cuckoo's Nest (1975)  \\\n",
              "title                                                                      \n",
              "Shawshank Redemption, The (1994)                                     0.0   \n",
              "Godfather, The (1972)                                                0.0   \n",
              "Usual Suspects, The (1995)                                           0.0   \n",
              "Godfather: Part II, The (1974)                                       0.0   \n",
              "Schindler's List (1993)                                              0.0   \n",
              "\n",
              "title                             Rear Window (1954)  12 Angry Men (1957)  \\\n",
              "title                                                                       \n",
              "Shawshank Redemption, The (1994)            0.034679             0.017646   \n",
              "Godfather, The (1972)                       0.000000             0.014397   \n",
              "Usual Suspects, The (1995)                  0.000000             0.030280   \n",
              "Godfather: Part II, The (1974)              0.021443             0.009771   \n",
              "Schindler's List (1993)                     0.000000             0.000000   \n",
              "\n",
              "title                             Seven Samurai (Shichinin no samurai) (1954)  \\\n",
              "title                                                                           \n",
              "Shawshank Redemption, The (1994)                                          0.0   \n",
              "Godfather, The (1972)                                                     0.0   \n",
              "Usual Suspects, The (1995)                                                0.0   \n",
              "Godfather: Part II, The (1974)                                            0.0   \n",
              "Schindler's List (1993)                                                   0.0   \n",
              "\n",
              "title                             ...  Showgirls (1995)  \\\n",
              "title                             ...                     \n",
              "Shawshank Redemption, The (1994)  ...          0.026994   \n",
              "Godfather, The (1972)             ...          0.000000   \n",
              "Usual Suspects, The (1995)        ...          0.000000   \n",
              "Godfather: Part II, The (1974)    ...          0.132776   \n",
              "Schindler's List (1993)           ...          0.000000   \n",
              "\n",
              "title                             Wild Wild West (1999)  Grease 2 (1982)  \\\n",
              "title                                                                      \n",
              "Shawshank Redemption, The (1994)               0.000000         0.021276   \n",
              "Godfather, The (1972)                          0.000000         0.030850   \n",
              "Usual Suspects, The (1995)                     0.000000         0.000000   \n",
              "Godfather: Part II, The (1974)                 0.000000         0.043473   \n",
              "Schindler's List (1993)                        0.018385         0.000000   \n",
              "\n",
              "title                             Spice World (1997)  Home Alone 3 (1997)  \\\n",
              "title                                                                       \n",
              "Shawshank Redemption, The (1994)            0.008312                  0.0   \n",
              "Godfather, The (1972)                       0.046529                  0.0   \n",
              "Usual Suspects, The (1995)                  0.000000                  0.0   \n",
              "Godfather: Part II, The (1974)              0.000000                  0.0   \n",
              "Schindler's List (1993)                     0.000000                  0.0   \n",
              "\n",
              "title                             Super Mario Bros. (1993)  Anaconda (1997)  \\\n",
              "title                                                                         \n",
              "Shawshank Redemption, The (1994)                  0.033757         0.000000   \n",
              "Godfather, The (1972)                             0.000000         0.000000   \n",
              "Usual Suspects, The (1995)                        0.000000         0.000000   \n",
              "Godfather: Part II, The (1974)                    0.000000         0.000000   \n",
              "Schindler's List (1993)                           0.000000         0.019657   \n",
              "\n",
              "title                             Batman & Robin (1997)  \\\n",
              "title                                                     \n",
              "Shawshank Redemption, The (1994)               0.007657   \n",
              "Godfather, The (1972)                          0.034888   \n",
              "Usual Suspects, The (1995)                     0.035370   \n",
              "Godfather: Part II, The (1974)                 0.026333   \n",
              "Schindler's List (1993)                        0.000000   \n",
              "\n",
              "title                             Speed 2: Cruise Control (1997)  \\\n",
              "title                                                              \n",
              "Shawshank Redemption, The (1994)                        0.011338   \n",
              "Godfather, The (1972)                                   0.000000   \n",
              "Usual Suspects, The (1995)                              0.000000   \n",
              "Godfather: Part II, The (1974)                          0.023739   \n",
              "Schindler's List (1993)                                 0.000000   \n",
              "\n",
              "title                             Battlefield Earth (2000)  \n",
              "title                                                       \n",
              "Shawshank Redemption, The (1994)                  0.000000  \n",
              "Godfather, The (1972)                             0.000000  \n",
              "Usual Suspects, The (1995)                        0.000000  \n",
              "Godfather: Part II, The (1974)                    0.000000  \n",
              "Schindler's List (1993)                           0.075014  \n",
              "\n",
              "[5 rows x 2927 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XvAMpBGvn2I-"
      },
      "source": [
        "### Creating Recommendations"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7UGtcx2HXdnr"
      },
      "source": [
        "#Function to find the most similar movide based on the description\n",
        "def find_similar_des(movie_name, num_recommendations=10):\n",
        "    cosine_sim_series = df_similarity.loc[movie_name]\n",
        "    #sort the dataframe\n",
        "    sorted_result = cosine_sim_series.drop([movie_name]).sort_values(ascending=False)\n",
        "    \n",
        "    return sorted_result.head(num_recommendations)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gAT3HqQMZGrB",
        "outputId": "8fd47ec7-f217-4292-82f2-62a376678c8f"
      },
      "source": [
        "find_similar_des(\"Star Wars: Episode V - The Empire Strikes Back (1980)\")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "title\n",
              "Star Wars: Episode IV - A New Hope (1977)              0.451199\n",
              "Star Wars: Episode VI - Return of the Jedi (1983)      0.349217\n",
              "Star Wars: Episode VII - The Force Awakens (2015)      0.266170\n",
              "Star Wars: Episode III - Revenge of the Sith (2005)    0.226523\n",
              "Cool Hand Luke (1967)                                  0.124833\n",
              "Star Wars: Episode I - The Phantom Menace (1999)       0.122387\n",
              "Star Wars: Episode II - Attack of the Clones (2002)    0.100162\n",
              "Aladdin (1992)                                         0.097780\n",
              "Sleeping Beauty (1959)                                 0.092597\n",
              "Baraka (1992)                                          0.089945\n",
              "Name: Star Wars: Episode V - The Empire Strikes Back (1980), dtype: float64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 34
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mJyb7W1KbSji"
      },
      "source": [
        "# 4. Collaborative filtering\n",
        "- For the collaborative filtering system, we analyzed correlations between user ratings in order to identify similarities. We consider the following 2 approaches:\n",
        "    - Item-based: generate recommendations for a target movie by identifying other similarly rated titles, done by applying cosine similarity.\n",
        "        - Includes an improved item-based model, combining an input from our description-based recommender to generate more cohesive suggestions.  \n",
        "    - User-based: generate recommendations for a target user by identifying well-rated movies by similar users and filtering out the recommendations already seen by the target user.\n",
        "        - Includes a predictive model to estimate how a given user would rate a movie. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hgGopmW1jv5_"
      },
      "source": [
        "## 4.1. Item-Based Collaborative Filtering\n",
        "- Similarly to our content-based approach, we will focus on a sample of 5000 most popular movies across all genres. We performed this filter using our function written in the Simple Recommendation System section.\n",
        "- Since the titles rated by each user varies, we have centered the ratings around 0 in order to provide better scaling for estimations. \n",
        "- We then applied cosine similarity analysis to obtain a table of movie relationships based on user ratings, which was then used to generate our recommendations."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PwMs7aFSbgid",
        "outputId": "22ae8338-fecd-4d1f-b064-f8caf396812c"
      },
      "source": [
        "# Merge dataframes to bring movie names\n",
        "df_ratings = pd.merge(df_ratings, df_movies, on='movieId')\n",
        "#Drop unnecessary columns\n",
        "df_ratings = df_ratings.drop(columns=['timestamp','movieId','genres'], axis=1)\n",
        "#Drop duplicates to keep last rating given by user\n",
        "df_ratings = df_ratings.drop_duplicates(subset=['userId', 'title'], keep='last')\n",
        "\n",
        "# Filter most polular movies\n",
        "df_pop_ratings = pd.merge(df_ratings, get_recommendation(\"All\",5000).copy().drop(columns=['rating', 'genres']), on='title')\n",
        "df_pop_ratings = df_pop_ratings.reset_index(drop=True)\n",
        "\n",
        "df_pop_ratings.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>userId</th>\n",
              "      <th>rating</th>\n",
              "      <th>title</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>5.0</td>\n",
              "      <td>Pulp Fiction (1994)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>3</td>\n",
              "      <td>5.0</td>\n",
              "      <td>Pulp Fiction (1994)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>4</td>\n",
              "      <td>4.0</td>\n",
              "      <td>Pulp Fiction (1994)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>5</td>\n",
              "      <td>4.0</td>\n",
              "      <td>Pulp Fiction (1994)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>7</td>\n",
              "      <td>4.0</td>\n",
              "      <td>Pulp Fiction (1994)</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   userId  rating                title\n",
              "0       1     5.0  Pulp Fiction (1994)\n",
              "1       3     5.0  Pulp Fiction (1994)\n",
              "2       4     4.0  Pulp Fiction (1994)\n",
              "3       5     4.0  Pulp Fiction (1994)\n",
              "4       7     4.0  Pulp Fiction (1994)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 35
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KMxkxH_qclCd",
        "outputId": "8ac80543-7713-4b74-c164-f2d90a5ec0a2"
      },
      "source": [
        "#Create pivot table of ratings\n",
        "title_ratings_pivot = df_pop_ratings.pivot(index='userId', columns='title', values='rating')\n",
        "title_ratings_pivot.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th>title</th>\n",
              "      <th>'burbs, The (1989)</th>\n",
              "      <th>(500) Days of Summer (2009)</th>\n",
              "      <th>*batteries not included (1987)</th>\n",
              "      <th>10 Cloverfield Lane (2016)</th>\n",
              "      <th>10 Things I Hate About You (1999)</th>\n",
              "      <th>10,000 BC (2008)</th>\n",
              "      <th>101 Dalmatians (1996)</th>\n",
              "      <th>101 Dalmatians (One Hundred and One Dalmatians) (1961)</th>\n",
              "      <th>102 Dalmatians (2000)</th>\n",
              "      <th>12 Angry Men (1957)</th>\n",
              "      <th>...</th>\n",
              "      <th>Zero Dark Thirty (2012)</th>\n",
              "      <th>Zero Effect (1998)</th>\n",
              "      <th>Zodiac (2007)</th>\n",
              "      <th>Zombieland (2009)</th>\n",
              "      <th>Zoolander (2001)</th>\n",
              "      <th>Zootopia (2016)</th>\n",
              "      <th>[REC] (2007)</th>\n",
              "      <th>eXistenZ (1999)</th>\n",
              "      <th>xXx (2002)</th>\n",
              "      <th>¡Three Amigos! (1986)</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>userId</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>4.0</td>\n",
              "      <td>3.5</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>3.5</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>4.0</td>\n",
              "      <td>4.5</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 2953 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "title   'burbs, The (1989)  (500) Days of Summer (2009)  \\\n",
              "userId                                                    \n",
              "1                      NaN                          NaN   \n",
              "2                      NaN                          NaN   \n",
              "3                      NaN                          NaN   \n",
              "4                      NaN                          NaN   \n",
              "5                      NaN                          NaN   \n",
              "\n",
              "title   *batteries not included (1987)  10 Cloverfield Lane (2016)  \\\n",
              "userId                                                               \n",
              "1                                  NaN                         NaN   \n",
              "2                                  NaN                         NaN   \n",
              "3                                  NaN                         NaN   \n",
              "4                                  NaN                         NaN   \n",
              "5                                  NaN                         NaN   \n",
              "\n",
              "title   10 Things I Hate About You (1999)  10,000 BC (2008)  \\\n",
              "userId                                                        \n",
              "1                                     NaN               NaN   \n",
              "2                                     NaN               NaN   \n",
              "3                                     NaN               NaN   \n",
              "4                                     NaN               NaN   \n",
              "5                                     NaN               NaN   \n",
              "\n",
              "title   101 Dalmatians (1996)  \\\n",
              "userId                          \n",
              "1                         NaN   \n",
              "2                         NaN   \n",
              "3                         NaN   \n",
              "4                         NaN   \n",
              "5                         NaN   \n",
              "\n",
              "title   101 Dalmatians (One Hundred and One Dalmatians) (1961)  \\\n",
              "userId                                                           \n",
              "1                                                     NaN        \n",
              "2                                                     NaN        \n",
              "3                                                     NaN        \n",
              "4                                                     NaN        \n",
              "5                                                     NaN        \n",
              "\n",
              "title   102 Dalmatians (2000)  12 Angry Men (1957)  ...  \\\n",
              "userId                                              ...   \n",
              "1                         NaN                  NaN  ...   \n",
              "2                         NaN                  NaN  ...   \n",
              "3                         2.0                  NaN  ...   \n",
              "4                         NaN                  NaN  ...   \n",
              "5                         NaN                  NaN  ...   \n",
              "\n",
              "title   Zero Dark Thirty (2012)  Zero Effect (1998)  Zodiac (2007)  \\\n",
              "userId                                                               \n",
              "1                           NaN                 NaN            NaN   \n",
              "2                           NaN                 NaN            NaN   \n",
              "3                           NaN                 NaN            NaN   \n",
              "4                           NaN                 NaN            NaN   \n",
              "5                           NaN                 NaN            NaN   \n",
              "\n",
              "title   Zombieland (2009)  Zoolander (2001)  Zootopia (2016)  [REC] (2007)  \\\n",
              "userId                                                                       \n",
              "1                     NaN               NaN              NaN           NaN   \n",
              "2                     NaN               NaN              NaN           NaN   \n",
              "3                     4.0               3.5              NaN           NaN   \n",
              "4                     NaN               4.0              4.5           NaN   \n",
              "5                     NaN               NaN              NaN           NaN   \n",
              "\n",
              "title   eXistenZ (1999)  xXx (2002)  ¡Three Amigos! (1986)  \n",
              "userId                                                      \n",
              "1                   NaN         NaN                    NaN  \n",
              "2                   NaN         NaN                    NaN  \n",
              "3                   NaN         3.5                    NaN  \n",
              "4                   NaN         NaN                    NaN  \n",
              "5                   NaN         NaN                    NaN  \n",
              "\n",
              "[5 rows x 2953 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 36
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "f68L_DLndWOv"
      },
      "source": [
        "# recenter data around 0 and fill Nan  \n",
        "# Get the average rating for each user \n",
        "avg_ratings = title_ratings_pivot.mean(axis=1)\n",
        "\n",
        "# Center each users ratings around 0\n",
        "title_ratings_table_centered = title_ratings_pivot.sub(avg_ratings, axis=0)\n",
        "\n",
        "# Fill in the missing data with 0s\n",
        "title_ratings_table_normed = title_ratings_table_centered.fillna(0)\n",
        "title_ratings_table_normed.head()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 350
        },
        "id": "_KQI7wPMg5XQ",
        "outputId": "decbc5fc-dc47-476f-b48c-34543905b7ed"
      },
      "source": [
        "# transpose movie to index for item based\n",
        "movie_based = title_ratings_table_normed.T\n",
        "movie_based.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th>userId</th>\n",
              "      <th>1</th>\n",
              "      <th>2</th>\n",
              "      <th>3</th>\n",
              "      <th>4</th>\n",
              "      <th>5</th>\n",
              "      <th>6</th>\n",
              "      <th>7</th>\n",
              "      <th>8</th>\n",
              "      <th>9</th>\n",
              "      <th>10</th>\n",
              "      <th>...</th>\n",
              "      <th>162532</th>\n",
              "      <th>162533</th>\n",
              "      <th>162534</th>\n",
              "      <th>162535</th>\n",
              "      <th>162536</th>\n",
              "      <th>162537</th>\n",
              "      <th>162538</th>\n",
              "      <th>162539</th>\n",
              "      <th>162540</th>\n",
              "      <th>162541</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>title</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>'burbs, The (1989)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>-1.917178</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>(500) Days of Summer (2009)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>*batteries not included (1987)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10 Cloverfield Lane (2016)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10 Things I Hate About You (1999)</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>-0.421569</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>-1.088983</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 162538 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "userId                             1       2       3       4       5       \\\n",
              "title                                                                       \n",
              "'burbs, The (1989)                    0.0     0.0     0.0     0.0     0.0   \n",
              "(500) Days of Summer (2009)           0.0     0.0     0.0     0.0     0.0   \n",
              "*batteries not included (1987)        0.0     0.0     0.0     0.0     0.0   \n",
              "10 Cloverfield Lane (2016)            0.0     0.0     0.0     0.0     0.0   \n",
              "10 Things I Hate About You (1999)     0.0     0.0     0.0     0.0     0.0   \n",
              "\n",
              "userId                             6       7       8       9         10      \\\n",
              "title                                                                         \n",
              "'burbs, The (1989)                    0.0     0.0     0.0     0.0  0.000000   \n",
              "(500) Days of Summer (2009)           0.0     0.0     0.0     0.0  0.000000   \n",
              "*batteries not included (1987)        0.0     0.0     0.0     0.0  0.000000   \n",
              "10 Cloverfield Lane (2016)            0.0     0.0     0.0     0.0  0.000000   \n",
              "10 Things I Hate About You (1999)     0.0     0.0     0.0     0.0 -0.421569   \n",
              "\n",
              "userId                             ...  162532    162533  162534  162535  \\\n",
              "title                              ...                                     \n",
              "'burbs, The (1989)                 ...     0.0  0.000000     0.0     0.0   \n",
              "(500) Days of Summer (2009)        ...     0.0  0.000000     0.0     0.0   \n",
              "*batteries not included (1987)     ...     0.0  0.000000     0.0     0.0   \n",
              "10 Cloverfield Lane (2016)         ...     0.0  0.000000     0.0     0.0   \n",
              "10 Things I Hate About You (1999)  ...     0.0 -1.088983     0.0     0.0   \n",
              "\n",
              "userId                             162536  162537  162538  162539  162540  \\\n",
              "title                                                                       \n",
              "'burbs, The (1989)                    0.0     0.0     0.0     0.0     0.0   \n",
              "(500) Days of Summer (2009)           0.0     0.0     1.0     0.0     0.0   \n",
              "*batteries not included (1987)        0.0     0.0     0.0     0.0     0.0   \n",
              "10 Cloverfield Lane (2016)            0.0     0.0     0.0     0.0     0.0   \n",
              "10 Things I Hate About You (1999)     0.0     0.0     1.0     0.0     0.0   \n",
              "\n",
              "userId                               162541  \n",
              "title                                        \n",
              "'burbs, The (1989)                -1.917178  \n",
              "(500) Days of Summer (2009)        0.000000  \n",
              "*batteries not included (1987)     0.000000  \n",
              "10 Cloverfield Lane (2016)         0.000000  \n",
              "10 Things I Hate About You (1999)  0.000000  \n",
              "\n",
              "[5 rows x 162538 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 38
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 167
        },
        "id": "9Pe2nwSRjt82",
        "outputId": "bee3f15b-c255-4b0f-b200-72a6d3764283"
      },
      "source": [
        "# Calculate cosine similarities for our item-based dataframe\n",
        "similarities = cosine_similarity(movie_based)\n",
        "# Wrap the similarities in a DataFrame\n",
        "df_movie_similarity = pd.DataFrame(similarities, index=movie_based.index, columns=movie_based.index)\n",
        "df_movie_similarity.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th>title</th>\n",
              "      <th>'burbs, The (1989)</th>\n",
              "      <th>(500) Days of Summer (2009)</th>\n",
              "      <th>*batteries not included (1987)</th>\n",
              "      <th>10 Cloverfield Lane (2016)</th>\n",
              "      <th>10 Things I Hate About You (1999)</th>\n",
              "      <th>10,000 BC (2008)</th>\n",
              "      <th>101 Dalmatians (1996)</th>\n",
              "      <th>101 Dalmatians (One Hundred and One Dalmatians) (1961)</th>\n",
              "      <th>102 Dalmatians (2000)</th>\n",
              "      <th>12 Angry Men (1957)</th>\n",
              "      <th>...</th>\n",
              "      <th>Zero Dark Thirty (2012)</th>\n",
              "      <th>Zero Effect (1998)</th>\n",
              "      <th>Zodiac (2007)</th>\n",
              "      <th>Zombieland (2009)</th>\n",
              "      <th>Zoolander (2001)</th>\n",
              "      <th>Zootopia (2016)</th>\n",
              "      <th>[REC] (2007)</th>\n",
              "      <th>eXistenZ (1999)</th>\n",
              "      <th>xXx (2002)</th>\n",
              "      <th>¡Three Amigos! (1986)</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>title</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>'burbs, The (1989)</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.003306</td>\n",
              "      <td>0.040960</td>\n",
              "      <td>0.003214</td>\n",
              "      <td>0.007690</td>\n",
              "      <td>0.005386</td>\n",
              "      <td>0.027102</td>\n",
              "      <td>-0.020474</td>\n",
              "      <td>0.015117</td>\n",
              "      <td>-0.043731</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.001074</td>\n",
              "      <td>-0.020870</td>\n",
              "      <td>-0.004836</td>\n",
              "      <td>-0.008369</td>\n",
              "      <td>0.014527</td>\n",
              "      <td>-0.004555</td>\n",
              "      <td>-0.003419</td>\n",
              "      <td>0.017637</td>\n",
              "      <td>0.012628</td>\n",
              "      <td>0.084297</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>(500) Days of Summer (2009)</th>\n",
              "      <td>-0.003306</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.006602</td>\n",
              "      <td>0.018147</td>\n",
              "      <td>0.023853</td>\n",
              "      <td>-0.029539</td>\n",
              "      <td>-0.023602</td>\n",
              "      <td>-0.011066</td>\n",
              "      <td>-0.032766</td>\n",
              "      <td>0.017625</td>\n",
              "      <td>...</td>\n",
              "      <td>0.019949</td>\n",
              "      <td>0.004549</td>\n",
              "      <td>0.023080</td>\n",
              "      <td>0.041274</td>\n",
              "      <td>-0.010333</td>\n",
              "      <td>0.012507</td>\n",
              "      <td>0.015779</td>\n",
              "      <td>-0.005618</td>\n",
              "      <td>-0.030105</td>\n",
              "      <td>-0.010460</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>*batteries not included (1987)</th>\n",
              "      <td>0.040960</td>\n",
              "      <td>-0.006602</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.001200</td>\n",
              "      <td>0.016142</td>\n",
              "      <td>0.022138</td>\n",
              "      <td>0.024278</td>\n",
              "      <td>0.009570</td>\n",
              "      <td>0.016061</td>\n",
              "      <td>-0.035216</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.004734</td>\n",
              "      <td>-0.009043</td>\n",
              "      <td>-0.014289</td>\n",
              "      <td>-0.009894</td>\n",
              "      <td>0.001684</td>\n",
              "      <td>-0.003461</td>\n",
              "      <td>0.007467</td>\n",
              "      <td>0.001050</td>\n",
              "      <td>0.024019</td>\n",
              "      <td>0.006309</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10 Cloverfield Lane (2016)</th>\n",
              "      <td>0.003214</td>\n",
              "      <td>0.018147</td>\n",
              "      <td>-0.001200</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.008594</td>\n",
              "      <td>-0.028963</td>\n",
              "      <td>-0.022987</td>\n",
              "      <td>-0.012007</td>\n",
              "      <td>-0.032139</td>\n",
              "      <td>0.022239</td>\n",
              "      <td>...</td>\n",
              "      <td>0.026332</td>\n",
              "      <td>-0.003410</td>\n",
              "      <td>0.049127</td>\n",
              "      <td>0.042108</td>\n",
              "      <td>-0.011681</td>\n",
              "      <td>0.034286</td>\n",
              "      <td>0.049506</td>\n",
              "      <td>-0.005217</td>\n",
              "      <td>-0.050854</td>\n",
              "      <td>-0.002578</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10 Things I Hate About You (1999)</th>\n",
              "      <td>0.007690</td>\n",
              "      <td>0.023853</td>\n",
              "      <td>0.016142</td>\n",
              "      <td>-0.008594</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.014515</td>\n",
              "      <td>0.020798</td>\n",
              "      <td>0.016509</td>\n",
              "      <td>-0.000475</td>\n",
              "      <td>-0.032232</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.011749</td>\n",
              "      <td>0.006919</td>\n",
              "      <td>-0.015923</td>\n",
              "      <td>0.001618</td>\n",
              "      <td>0.004124</td>\n",
              "      <td>0.011097</td>\n",
              "      <td>-0.000867</td>\n",
              "      <td>-0.000620</td>\n",
              "      <td>0.032241</td>\n",
              "      <td>0.011114</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 2953 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "title                              'burbs, The (1989)  \\\n",
              "title                                                   \n",
              "'burbs, The (1989)                           1.000000   \n",
              "(500) Days of Summer (2009)                 -0.003306   \n",
              "*batteries not included (1987)               0.040960   \n",
              "10 Cloverfield Lane (2016)                   0.003214   \n",
              "10 Things I Hate About You (1999)            0.007690   \n",
              "\n",
              "title                              (500) Days of Summer (2009)  \\\n",
              "title                                                            \n",
              "'burbs, The (1989)                                   -0.003306   \n",
              "(500) Days of Summer (2009)                           1.000000   \n",
              "*batteries not included (1987)                       -0.006602   \n",
              "10 Cloverfield Lane (2016)                            0.018147   \n",
              "10 Things I Hate About You (1999)                     0.023853   \n",
              "\n",
              "title                              *batteries not included (1987)  \\\n",
              "title                                                               \n",
              "'burbs, The (1989)                                       0.040960   \n",
              "(500) Days of Summer (2009)                             -0.006602   \n",
              "*batteries not included (1987)                           1.000000   \n",
              "10 Cloverfield Lane (2016)                              -0.001200   \n",
              "10 Things I Hate About You (1999)                        0.016142   \n",
              "\n",
              "title                              10 Cloverfield Lane (2016)  \\\n",
              "title                                                           \n",
              "'burbs, The (1989)                                   0.003214   \n",
              "(500) Days of Summer (2009)                          0.018147   \n",
              "*batteries not included (1987)                      -0.001200   \n",
              "10 Cloverfield Lane (2016)                           1.000000   \n",
              "10 Things I Hate About You (1999)                   -0.008594   \n",
              "\n",
              "title                              10 Things I Hate About You (1999)  \\\n",
              "title                                                                  \n",
              "'burbs, The (1989)                                          0.007690   \n",
              "(500) Days of Summer (2009)                                 0.023853   \n",
              "*batteries not included (1987)                              0.016142   \n",
              "10 Cloverfield Lane (2016)                                 -0.008594   \n",
              "10 Things I Hate About You (1999)                           1.000000   \n",
              "\n",
              "title                              10,000 BC (2008)  101 Dalmatians (1996)  \\\n",
              "title                                                                        \n",
              "'burbs, The (1989)                         0.005386               0.027102   \n",
              "(500) Days of Summer (2009)               -0.029539              -0.023602   \n",
              "*batteries not included (1987)             0.022138               0.024278   \n",
              "10 Cloverfield Lane (2016)                -0.028963              -0.022987   \n",
              "10 Things I Hate About You (1999)          0.014515               0.020798   \n",
              "\n",
              "title                              101 Dalmatians (One Hundred and One Dalmatians) (1961)  \\\n",
              "title                                                                                       \n",
              "'burbs, The (1989)                                                         -0.020474        \n",
              "(500) Days of Summer (2009)                                                -0.011066        \n",
              "*batteries not included (1987)                                              0.009570        \n",
              "10 Cloverfield Lane (2016)                                                 -0.012007        \n",
              "10 Things I Hate About You (1999)                                           0.016509        \n",
              "\n",
              "title                              102 Dalmatians (2000)  12 Angry Men (1957)  \\\n",
              "title                                                                           \n",
              "'burbs, The (1989)                              0.015117            -0.043731   \n",
              "(500) Days of Summer (2009)                    -0.032766             0.017625   \n",
              "*batteries not included (1987)                  0.016061            -0.035216   \n",
              "10 Cloverfield Lane (2016)                     -0.032139             0.022239   \n",
              "10 Things I Hate About You (1999)              -0.000475            -0.032232   \n",
              "\n",
              "title                              ...  Zero Dark Thirty (2012)  \\\n",
              "title                              ...                            \n",
              "'burbs, The (1989)                 ...                -0.001074   \n",
              "(500) Days of Summer (2009)        ...                 0.019949   \n",
              "*batteries not included (1987)     ...                -0.004734   \n",
              "10 Cloverfield Lane (2016)         ...                 0.026332   \n",
              "10 Things I Hate About You (1999)  ...                -0.011749   \n",
              "\n",
              "title                              Zero Effect (1998)  Zodiac (2007)  \\\n",
              "title                                                                  \n",
              "'burbs, The (1989)                          -0.020870      -0.004836   \n",
              "(500) Days of Summer (2009)                  0.004549       0.023080   \n",
              "*batteries not included (1987)              -0.009043      -0.014289   \n",
              "10 Cloverfield Lane (2016)                  -0.003410       0.049127   \n",
              "10 Things I Hate About You (1999)            0.006919      -0.015923   \n",
              "\n",
              "title                              Zombieland (2009)  Zoolander (2001)  \\\n",
              "title                                                                    \n",
              "'burbs, The (1989)                         -0.008369          0.014527   \n",
              "(500) Days of Summer (2009)                 0.041274         -0.010333   \n",
              "*batteries not included (1987)             -0.009894          0.001684   \n",
              "10 Cloverfield Lane (2016)                  0.042108         -0.011681   \n",
              "10 Things I Hate About You (1999)           0.001618          0.004124   \n",
              "\n",
              "title                              Zootopia (2016)  [REC] (2007)  \\\n",
              "title                                                              \n",
              "'burbs, The (1989)                       -0.004555     -0.003419   \n",
              "(500) Days of Summer (2009)               0.012507      0.015779   \n",
              "*batteries not included (1987)           -0.003461      0.007467   \n",
              "10 Cloverfield Lane (2016)                0.034286      0.049506   \n",
              "10 Things I Hate About You (1999)         0.011097     -0.000867   \n",
              "\n",
              "title                              eXistenZ (1999)  xXx (2002)  \\\n",
              "title                                                            \n",
              "'burbs, The (1989)                        0.017637    0.012628   \n",
              "(500) Days of Summer (2009)              -0.005618   -0.030105   \n",
              "*batteries not included (1987)            0.001050    0.024019   \n",
              "10 Cloverfield Lane (2016)               -0.005217   -0.050854   \n",
              "10 Things I Hate About You (1999)        -0.000620    0.032241   \n",
              "\n",
              "title                              ¡Three Amigos! (1986)  \n",
              "title                                                     \n",
              "'burbs, The (1989)                              0.084297  \n",
              "(500) Days of Summer (2009)                    -0.010460  \n",
              "*batteries not included (1987)                  0.006309  \n",
              "10 Cloverfield Lane (2016)                     -0.002578  \n",
              "10 Things I Hate About You (1999)               0.011114  \n",
              "\n",
              "[5 rows x 2953 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 39
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "smCCwXFvn2JB"
      },
      "source": [
        "### Creating Recommendations"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yK7uijBxkx4u"
      },
      "source": [
        "#function to find similar movies based on users' rating\n",
        "def find_similar_rating(movie_name, num_recommendations = 10):\n",
        "    cosine_similarity_series = df_movie_similarity.loc[movie_name]\n",
        "    # Sort these values highest to lowest\n",
        "    ordered_similarities = cosine_similarity_series.drop([movie_name]).sort_values(ascending=False)\n",
        "    \n",
        "    return ordered_similarities.head(num_recommendations)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GGJklJw_lNit",
        "outputId": "13efb2b8-1667-44af-f952-227af3530729"
      },
      "source": [
        "find_similar_rating('Star Wars: Episode V - The Empire Strikes Back (1980)')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "title\n",
              "Star Wars: Episode IV - A New Hope (1977)                                         0.658184\n",
              "Star Wars: Episode VI - Return of the Jedi (1983)                                 0.592826\n",
              "Raiders of the Lost Ark (Indiana Jones and the Raiders of the Lost Ark) (1981)    0.447282\n",
              "Indiana Jones and the Last Crusade (1989)                                         0.335323\n",
              "Matrix, The (1999)                                                                0.283854\n",
              "Lord of the Rings: The Fellowship of the Ring, The (2001)                         0.269870\n",
              "Aliens (1986)                                                                     0.263332\n",
              "Alien (1979)                                                                      0.261108\n",
              "Back to the Future (1985)                                                         0.254422\n",
              "Blade Runner (1982)                                                               0.251197\n",
              "Name: Star Wars: Episode V - The Empire Strikes Back (1980), dtype: float64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 41
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BGXkyLarn2JB"
      },
      "source": [
        "### Improved Item-Based Collaborative Filtering\n",
        "- Since this analysis only takes into consideration the user ratings to generate recommendations, we found beneficial to improve it by combining the output from our description-based model.\n",
        "- To combine the results, we averaged the scores provided by each of the two models to prioritize similarly rated movies that also contain similar plots/descriptions."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fuzSxtBrn2JC"
      },
      "source": [
        "def improved_item_based_recommend(movie_name, num_recommendations = 10):\n",
        "    \n",
        "    #Combine rating results from both models, take average and sort list\n",
        "    combined_results = pd.merge(find_similar_des(movie_name, 1000), find_similar_rating(movie_name, 1000), \n",
        "                                   on='title').mean(axis=1).sort_values(ascending=False)\n",
        "    return combined_results.head(num_recommendations)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sTr6_Cfin2JC",
        "outputId": "db81a228-43eb-499f-b4b5-5231f44aa599"
      },
      "source": [
        "improved_item_based_recommend('Star Wars: Episode V - The Empire Strikes Back (1980)')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "title\n",
              "Star Wars: Episode IV - A New Hope (1977)                    0.554692\n",
              "Star Wars: Episode VI - Return of the Jedi (1983)            0.471022\n",
              "Star Wars: Episode VII - The Force Awakens (2015)            0.199830\n",
              "Lord of the Rings: The Fellowship of the Ring, The (2001)    0.141694\n",
              "Princess Bride, The (1987)                                   0.136866\n",
              "Star Wars: Episode III - Revenge of the Sith (2005)          0.132364\n",
              "Die Hard (1988)                                              0.125950\n",
              "Terminator 2: Judgment Day (1991)                            0.120448\n",
              "Godfather: Part II, The (1974)                               0.118786\n",
              "Cool Hand Luke (1967)                                        0.116750\n",
              "dtype: float64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 43
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zECwIP09sHtP"
      },
      "source": [
        "## 4.2. User-Based Collaborative Filtering\n",
        "- In order to build a personalized system for individual users, we analyzed correlations in user ratings' data to identify similar users and infer potential movies that a target user would like.\n",
        "- To provide a more complete implementation, we also applied the Nearest Neighbors concept to provide a predictor of estimated ratings for the given user using a KNNRegressor. \n",
        "- Due to the limited processing resources, we will consider a sample of 1000 users which provided the highest number of ratings in the dataset."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "D4I-zho9n2JC",
        "outputId": "3c94545f-9953-4e11-dd2c-f5a4c9af7af4"
      },
      "source": [
        "#Group ratings dataframe by user\n",
        "df_top_users = df_ratings.groupby([\"userId\"], as_index=False).agg(num_ratings = (\"rating\", \"count\")).reset_index(drop=True)\n",
        "#Define most active users\n",
        "df_top_users = df_top_users.sort_values(by=\"num_ratings\", ascending=False).head(1000)\n",
        "# Filter most active users \n",
        "df_user_ratings = pd.merge(df_ratings, df_top_users.copy().drop(columns=['num_ratings']), on='userId')\n",
        "df_user_ratings"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>userId</th>\n",
              "      <th>rating</th>\n",
              "      <th>title</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>548</td>\n",
              "      <td>4.5</td>\n",
              "      <td>Pulp Fiction (1994)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>548</td>\n",
              "      <td>4.0</td>\n",
              "      <td>Dirty Dancing (1987)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>548</td>\n",
              "      <td>4.0</td>\n",
              "      <td>Gattaca (1997)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>548</td>\n",
              "      <td>4.0</td>\n",
              "      <td>Back to the Future Part II (1989)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>548</td>\n",
              "      <td>3.0</td>\n",
              "      <td>Back to the Future Part III (1990)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2205010</th>\n",
              "      <td>75309</td>\n",
              "      <td>5.0</td>\n",
              "      <td>Throwback (2013)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2205011</th>\n",
              "      <td>75309</td>\n",
              "      <td>5.0</td>\n",
              "      <td>Warlock Moon (1973)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2205012</th>\n",
              "      <td>75309</td>\n",
              "      <td>5.0</td>\n",
              "      <td>Spike (2008)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2205013</th>\n",
              "      <td>75309</td>\n",
              "      <td>5.0</td>\n",
              "      <td>Dying God (2008)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2205014</th>\n",
              "      <td>75309</td>\n",
              "      <td>5.0</td>\n",
              "      <td>Ex-Patriot (2017)</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>2205015 rows × 3 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "         userId  rating                               title\n",
              "0           548     4.5                 Pulp Fiction (1994)\n",
              "1           548     4.0                Dirty Dancing (1987)\n",
              "2           548     4.0                      Gattaca (1997)\n",
              "3           548     4.0   Back to the Future Part II (1989)\n",
              "4           548     3.0  Back to the Future Part III (1990)\n",
              "...         ...     ...                                 ...\n",
              "2205010   75309     5.0                    Throwback (2013)\n",
              "2205011   75309     5.0                 Warlock Moon (1973)\n",
              "2205012   75309     5.0                        Spike (2008)\n",
              "2205013   75309     5.0                    Dying God (2008)\n",
              "2205014   75309     5.0                   Ex-Patriot (2017)\n",
              "\n",
              "[2205015 rows x 3 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 44
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6LOxkb5mn2JD",
        "outputId": "2b06d00d-9790-4706-9bc3-3be57f146668"
      },
      "source": [
        "#Create pivot table of ratings\n",
        "user_ratings_pivot = df_user_ratings.pivot(index='userId', columns='title', values='rating')\n",
        "user_ratings_pivot.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th>title</th>\n",
              "      <th>\"Great Performances\" Cats (1998)</th>\n",
              "      <th>#1 Cheerleader Camp (2010)</th>\n",
              "      <th>#Captured (2017)</th>\n",
              "      <th>#Female Pleasure (2018)</th>\n",
              "      <th>#FollowMe (2019)</th>\n",
              "      <th>#Horror (2015)</th>\n",
              "      <th>#Lucky Number (2015)</th>\n",
              "      <th>#SCREAMERS (2016)</th>\n",
              "      <th>#SquadGoals (2018)</th>\n",
              "      <th>#Stuck (2014)</th>\n",
              "      <th>...</th>\n",
              "      <th>Я худею (2018)</th>\n",
              "      <th>ארבינקא (1967)</th>\n",
              "      <th>أهواك (2015)</th>\n",
              "      <th>به نام پدر</th>\n",
              "      <th>‘Rameau’s Nephew’ by Diderot (Thanx to Dennis Young) by Wilma Schoen (1974)</th>\n",
              "      <th>…And the Fifth Horseman Is Fear (1965)</th>\n",
              "      <th>キサラギ (2007)</th>\n",
              "      <th>チェブラーシカ (2010)</th>\n",
              "      <th>貞子3D (2012)</th>\n",
              "      <th>줄탁동시 (2012)</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>userId</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>548</th>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>626</th>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>847</th>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>997</th>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1401</th>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 53437 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "title   \"Great Performances\" Cats (1998)  #1 Cheerleader Camp (2010)  \\\n",
              "userId                                                                 \n",
              "548                                  NaN                         NaN   \n",
              "626                                  NaN                         NaN   \n",
              "847                                  NaN                         NaN   \n",
              "997                                  NaN                         NaN   \n",
              "1401                                 NaN                         NaN   \n",
              "\n",
              "title   #Captured (2017)  #Female Pleasure (2018)  #FollowMe (2019)  \\\n",
              "userId                                                                \n",
              "548                  NaN                      NaN               NaN   \n",
              "626                  NaN                      NaN               NaN   \n",
              "847                  NaN                      NaN               NaN   \n",
              "997                  NaN                      NaN               NaN   \n",
              "1401                 NaN                      NaN               NaN   \n",
              "\n",
              "title   #Horror (2015)  #Lucky Number (2015)  #SCREAMERS (2016)  \\\n",
              "userId                                                            \n",
              "548                NaN                   NaN                NaN   \n",
              "626                NaN                   NaN                NaN   \n",
              "847                NaN                   NaN                NaN   \n",
              "997                NaN                   NaN                NaN   \n",
              "1401               NaN                   NaN                NaN   \n",
              "\n",
              "title   #SquadGoals (2018)  #Stuck (2014)  ...  Я худею (2018)  \\\n",
              "userId                                     ...                   \n",
              "548                    NaN            NaN  ...             NaN   \n",
              "626                    NaN            NaN  ...             NaN   \n",
              "847                    NaN            NaN  ...             NaN   \n",
              "997                    NaN            NaN  ...             NaN   \n",
              "1401                   NaN            NaN  ...             NaN   \n",
              "\n",
              "title   ארבינקא (1967)  أهواك (2015)  به نام پدر  \\\n",
              "userId                                             \n",
              "548                NaN           NaN         NaN   \n",
              "626                NaN           NaN         NaN   \n",
              "847                NaN           NaN         NaN   \n",
              "997                NaN           NaN         NaN   \n",
              "1401               NaN           NaN         NaN   \n",
              "\n",
              "title   ‘Rameau’s Nephew’ by Diderot (Thanx to Dennis Young) by Wilma Schoen (1974)  \\\n",
              "userId                                                                                \n",
              "548                                                   NaN                             \n",
              "626                                                   NaN                             \n",
              "847                                                   NaN                             \n",
              "997                                                   NaN                             \n",
              "1401                                                  NaN                             \n",
              "\n",
              "title   …And the Fifth Horseman Is Fear (1965)  キサラギ (2007)  チェブラーシカ (2010)  \\\n",
              "userId                                                                        \n",
              "548                                        NaN          NaN             NaN   \n",
              "626                                        NaN          NaN             NaN   \n",
              "847                                        NaN          NaN             NaN   \n",
              "997                                        NaN          NaN             NaN   \n",
              "1401                                       NaN          NaN             NaN   \n",
              "\n",
              "title   貞子3D (2012)  줄탁동시 (2012)  \n",
              "userId                            \n",
              "548             NaN          NaN  \n",
              "626             NaN          NaN  \n",
              "847             NaN          NaN  \n",
              "997             NaN          NaN  \n",
              "1401            NaN          NaN  \n",
              "\n",
              "[5 rows x 53437 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 45
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xSYoBxFBn2JD",
        "outputId": "20d66c4c-f380-4203-8d87-bc17f2611824"
      },
      "source": [
        "# recenter data around 0 and fill Nan  \n",
        "# Get the average rating for each user \n",
        "avg_ratings = user_ratings_pivot.mean(axis=1)\n",
        "\n",
        "# Center each users ratings around 0\n",
        "user_ratings_table_centered = user_ratings_pivot.sub(avg_ratings, axis=0)\n",
        "\n",
        "# Fill in the missing data with 0s\n",
        "user_based = user_ratings_table_centered.fillna(0)\n",
        "user_based"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th>title</th>\n",
              "      <th>\"Great Performances\" Cats (1998)</th>\n",
              "      <th>#1 Cheerleader Camp (2010)</th>\n",
              "      <th>#Captured (2017)</th>\n",
              "      <th>#Female Pleasure (2018)</th>\n",
              "      <th>#FollowMe (2019)</th>\n",
              "      <th>#Horror (2015)</th>\n",
              "      <th>#Lucky Number (2015)</th>\n",
              "      <th>#SCREAMERS (2016)</th>\n",
              "      <th>#SquadGoals (2018)</th>\n",
              "      <th>#Stuck (2014)</th>\n",
              "      <th>...</th>\n",
              "      <th>Я худею (2018)</th>\n",
              "      <th>ארבינקא (1967)</th>\n",
              "      <th>أهواك (2015)</th>\n",
              "      <th>به نام پدر</th>\n",
              "      <th>‘Rameau’s Nephew’ by Diderot (Thanx to Dennis Young) by Wilma Schoen (1974)</th>\n",
              "      <th>…And the Fifth Horseman Is Fear (1965)</th>\n",
              "      <th>キサラギ (2007)</th>\n",
              "      <th>チェブラーシカ (2010)</th>\n",
              "      <th>貞子3D (2012)</th>\n",
              "      <th>줄탁동시 (2012)</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>userId</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>548</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>626</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>847</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>997</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1401</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>161586</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>161928</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>162047</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>162271</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>162516</th>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>1000 rows × 53437 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "title   \"Great Performances\" Cats (1998)  #1 Cheerleader Camp (2010)  \\\n",
              "userId                                                                 \n",
              "548                                  0.0                         0.0   \n",
              "626                                  0.0                         0.0   \n",
              "847                                  0.0                         0.0   \n",
              "997                                  0.0                         0.0   \n",
              "1401                                 0.0                         0.0   \n",
              "...                                  ...                         ...   \n",
              "161586                               0.0                         0.0   \n",
              "161928                               0.0                         0.0   \n",
              "162047                               0.0                         0.0   \n",
              "162271                               0.0                         0.0   \n",
              "162516                               0.0                         0.0   \n",
              "\n",
              "title   #Captured (2017)  #Female Pleasure (2018)  #FollowMe (2019)  \\\n",
              "userId                                                                \n",
              "548                  0.0                      0.0               0.0   \n",
              "626                  0.0                      0.0               0.0   \n",
              "847                  0.0                      0.0               0.0   \n",
              "997                  0.0                      0.0               0.0   \n",
              "1401                 0.0                      0.0               0.0   \n",
              "...                  ...                      ...               ...   \n",
              "161586               0.0                      0.0               0.0   \n",
              "161928               0.0                      0.0               0.0   \n",
              "162047               0.0                      0.0               0.0   \n",
              "162271               0.0                      0.0               0.0   \n",
              "162516               0.0                      0.0               0.0   \n",
              "\n",
              "title   #Horror (2015)  #Lucky Number (2015)  #SCREAMERS (2016)  \\\n",
              "userId                                                            \n",
              "548                0.0                   0.0                0.0   \n",
              "626                0.0                   0.0                0.0   \n",
              "847                0.0                   0.0                0.0   \n",
              "997                0.0                   0.0                0.0   \n",
              "1401               0.0                   0.0                0.0   \n",
              "...                ...                   ...                ...   \n",
              "161586             0.0                   0.0                0.0   \n",
              "161928             0.0                   0.0                0.0   \n",
              "162047             0.0                   0.0                0.0   \n",
              "162271             0.0                   0.0                0.0   \n",
              "162516             0.0                   0.0                0.0   \n",
              "\n",
              "title   #SquadGoals (2018)  #Stuck (2014)  ...  Я худею (2018)  \\\n",
              "userId                                     ...                   \n",
              "548                    0.0            0.0  ...             0.0   \n",
              "626                    0.0            0.0  ...             0.0   \n",
              "847                    0.0            0.0  ...             0.0   \n",
              "997                    0.0            0.0  ...             0.0   \n",
              "1401                   0.0            0.0  ...             0.0   \n",
              "...                    ...            ...  ...             ...   \n",
              "161586                 0.0            0.0  ...             0.0   \n",
              "161928                 0.0            0.0  ...             0.0   \n",
              "162047                 0.0            0.0  ...             0.0   \n",
              "162271                 0.0            0.0  ...             0.0   \n",
              "162516                 0.0            0.0  ...             0.0   \n",
              "\n",
              "title   ארבינקא (1967)  أهواك (2015)  به نام پدر  \\\n",
              "userId                                             \n",
              "548                0.0           0.0         0.0   \n",
              "626                0.0           0.0         0.0   \n",
              "847                0.0           0.0         0.0   \n",
              "997                0.0           0.0         0.0   \n",
              "1401               0.0           0.0         0.0   \n",
              "...                ...           ...         ...   \n",
              "161586             0.0           0.0         0.0   \n",
              "161928             0.0           0.0         0.0   \n",
              "162047             0.0           0.0         0.0   \n",
              "162271             0.0           0.0         0.0   \n",
              "162516             0.0           0.0         0.0   \n",
              "\n",
              "title   ‘Rameau’s Nephew’ by Diderot (Thanx to Dennis Young) by Wilma Schoen (1974)  \\\n",
              "userId                                                                                \n",
              "548                                                   0.0                             \n",
              "626                                                   0.0                             \n",
              "847                                                   0.0                             \n",
              "997                                                   0.0                             \n",
              "1401                                                  0.0                             \n",
              "...                                                   ...                             \n",
              "161586                                                0.0                             \n",
              "161928                                                0.0                             \n",
              "162047                                                0.0                             \n",
              "162271                                                0.0                             \n",
              "162516                                                0.0                             \n",
              "\n",
              "title   …And the Fifth Horseman Is Fear (1965)  キサラギ (2007)  チェブラーシカ (2010)  \\\n",
              "userId                                                                        \n",
              "548                                        0.0          0.0             0.0   \n",
              "626                                        0.0          0.0             0.0   \n",
              "847                                        0.0          0.0             0.0   \n",
              "997                                        0.0          0.0             0.0   \n",
              "1401                                       0.0          0.0             0.0   \n",
              "...                                        ...          ...             ...   \n",
              "161586                                     0.0          0.0             0.0   \n",
              "161928                                     0.0          0.0             0.0   \n",
              "162047                                     0.0          0.0             0.0   \n",
              "162271                                     0.0          0.0             0.0   \n",
              "162516                                     0.0          0.0             0.0   \n",
              "\n",
              "title   貞子3D (2012)  줄탁동시 (2012)  \n",
              "userId                            \n",
              "548             0.0          0.0  \n",
              "626             0.0          0.0  \n",
              "847             0.0          0.0  \n",
              "997             0.0          0.0  \n",
              "1401            0.0          0.0  \n",
              "...             ...          ...  \n",
              "161586          0.0          0.0  \n",
              "161928          0.0          0.0  \n",
              "162047          0.0          0.0  \n",
              "162271          0.0          0.0  \n",
              "162516          0.0          0.0  \n",
              "\n",
              "[1000 rows x 53437 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 46
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4o1--Phfn2JD"
      },
      "source": [
        "### Predicting user ratings using KNN"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PVr6Tq_B4Wzc"
      },
      "source": [
        "def predict_score(user, movie):\n",
        "    # Drop the column you are trying to predict\n",
        "    user_scores = user_based.drop(movie, axis=1)\n",
        "\n",
        "    # Get the data for the user you are predicting for\n",
        "    user_x = user_scores.loc[[user]]\n",
        "\n",
        "    # Get the target data from user_ratings_table\n",
        "    others_y = user_ratings_pivot[movie]\n",
        "\n",
        "    # Get the data for only those that have seen the movie\n",
        "    others_x = user_scores[others_y.notnull()]\n",
        "\n",
        "    # Remove those that have not seen the movie from the target\n",
        "    others_y.dropna(inplace=True)\n",
        "\n",
        "    # Instantiate the user KNN model\n",
        "    user_knn = KNeighborsRegressor(metric='cosine', n_neighbors=10)\n",
        "\n",
        "    # Fit the model and predict the target user\n",
        "    user_knn.fit(others_x, others_y)\n",
        "    pred_score = user_knn.predict(user_x)[0]\n",
        "\n",
        "    return pred_score"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VKe1EiAWn2JE",
        "outputId": "223a6273-d39f-45ff-f605-1c02310352ee"
      },
      "source": [
        "print(\"Predicted score for a given user and movie:\", predict_score(548, \"Apollo 13 (1995)\"))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Predicted score for a given user and movie: 3.5\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yjieAOZRn2JE"
      },
      "source": [
        "### Using Cosine Similarity to recommend a list of movies\n",
        "- using top 5% most similar users\n",
        "- mean of scores for each movie"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 167
        },
        "id": "1GsfZDOHn2JE",
        "outputId": "bee3f15b-c255-4b0f-b200-72a6d3764283"
      },
      "source": [
        "similarities = cosine_similarity(user_based)\n",
        "# Wrap the similarities in a DataFrame\n",
        "df_user_similarity = pd.DataFrame(similarities, index=user_based.index, columns=user_based.index)\n",
        "df_user_similarity.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th>userId</th>\n",
              "      <th>548</th>\n",
              "      <th>626</th>\n",
              "      <th>847</th>\n",
              "      <th>997</th>\n",
              "      <th>1401</th>\n",
              "      <th>1652</th>\n",
              "      <th>1748</th>\n",
              "      <th>1920</th>\n",
              "      <th>1977</th>\n",
              "      <th>2003</th>\n",
              "      <th>...</th>\n",
              "      <th>160951</th>\n",
              "      <th>161184</th>\n",
              "      <th>161342</th>\n",
              "      <th>161383</th>\n",
              "      <th>161544</th>\n",
              "      <th>161586</th>\n",
              "      <th>161928</th>\n",
              "      <th>162047</th>\n",
              "      <th>162271</th>\n",
              "      <th>162516</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>userId</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>548</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.098919</td>\n",
              "      <td>0.014155</td>\n",
              "      <td>0.215703</td>\n",
              "      <td>0.139115</td>\n",
              "      <td>0.142881</td>\n",
              "      <td>0.120889</td>\n",
              "      <td>0.274242</td>\n",
              "      <td>0.129787</td>\n",
              "      <td>0.062130</td>\n",
              "      <td>...</td>\n",
              "      <td>0.259374</td>\n",
              "      <td>0.195574</td>\n",
              "      <td>0.180144</td>\n",
              "      <td>0.054047</td>\n",
              "      <td>0.082037</td>\n",
              "      <td>0.146003</td>\n",
              "      <td>0.098940</td>\n",
              "      <td>0.136444</td>\n",
              "      <td>0.053942</td>\n",
              "      <td>0.183845</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>626</th>\n",
              "      <td>0.098919</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.022193</td>\n",
              "      <td>0.068807</td>\n",
              "      <td>0.039690</td>\n",
              "      <td>0.074311</td>\n",
              "      <td>0.122133</td>\n",
              "      <td>0.116372</td>\n",
              "      <td>0.154064</td>\n",
              "      <td>0.040243</td>\n",
              "      <td>...</td>\n",
              "      <td>0.095846</td>\n",
              "      <td>0.106153</td>\n",
              "      <td>0.045150</td>\n",
              "      <td>0.141541</td>\n",
              "      <td>-0.012065</td>\n",
              "      <td>0.067175</td>\n",
              "      <td>0.075466</td>\n",
              "      <td>0.051487</td>\n",
              "      <td>0.022263</td>\n",
              "      <td>0.098727</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>847</th>\n",
              "      <td>0.014155</td>\n",
              "      <td>0.022193</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.038995</td>\n",
              "      <td>0.013853</td>\n",
              "      <td>0.014937</td>\n",
              "      <td>-0.027510</td>\n",
              "      <td>0.027439</td>\n",
              "      <td>0.065770</td>\n",
              "      <td>0.145654</td>\n",
              "      <td>...</td>\n",
              "      <td>0.087720</td>\n",
              "      <td>0.071641</td>\n",
              "      <td>0.052734</td>\n",
              "      <td>-0.022233</td>\n",
              "      <td>0.119895</td>\n",
              "      <td>0.121661</td>\n",
              "      <td>0.009932</td>\n",
              "      <td>0.050526</td>\n",
              "      <td>0.077448</td>\n",
              "      <td>-0.005177</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>997</th>\n",
              "      <td>0.215703</td>\n",
              "      <td>0.068807</td>\n",
              "      <td>0.038995</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.180115</td>\n",
              "      <td>0.172095</td>\n",
              "      <td>0.136436</td>\n",
              "      <td>0.284800</td>\n",
              "      <td>0.130451</td>\n",
              "      <td>0.112039</td>\n",
              "      <td>...</td>\n",
              "      <td>0.203909</td>\n",
              "      <td>0.169936</td>\n",
              "      <td>0.185976</td>\n",
              "      <td>0.025221</td>\n",
              "      <td>0.121291</td>\n",
              "      <td>0.189271</td>\n",
              "      <td>0.142812</td>\n",
              "      <td>0.144200</td>\n",
              "      <td>0.098583</td>\n",
              "      <td>0.187010</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1401</th>\n",
              "      <td>0.139115</td>\n",
              "      <td>0.039690</td>\n",
              "      <td>0.013853</td>\n",
              "      <td>0.180115</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.080465</td>\n",
              "      <td>0.039456</td>\n",
              "      <td>0.153744</td>\n",
              "      <td>0.087234</td>\n",
              "      <td>0.053545</td>\n",
              "      <td>...</td>\n",
              "      <td>0.142850</td>\n",
              "      <td>0.127917</td>\n",
              "      <td>0.059083</td>\n",
              "      <td>0.000316</td>\n",
              "      <td>0.048369</td>\n",
              "      <td>0.118844</td>\n",
              "      <td>0.064470</td>\n",
              "      <td>0.110306</td>\n",
              "      <td>0.075344</td>\n",
              "      <td>0.130986</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 1000 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "userId    548       626       847       997       1401      1652      1748    \\\n",
              "userId                                                                         \n",
              "548     1.000000  0.098919  0.014155  0.215703  0.139115  0.142881  0.120889   \n",
              "626     0.098919  1.000000  0.022193  0.068807  0.039690  0.074311  0.122133   \n",
              "847     0.014155  0.022193  1.000000  0.038995  0.013853  0.014937 -0.027510   \n",
              "997     0.215703  0.068807  0.038995  1.000000  0.180115  0.172095  0.136436   \n",
              "1401    0.139115  0.039690  0.013853  0.180115  1.000000  0.080465  0.039456   \n",
              "\n",
              "userId    1920      1977      2003    ...    160951    161184    161342  \\\n",
              "userId                                ...                                 \n",
              "548     0.274242  0.129787  0.062130  ...  0.259374  0.195574  0.180144   \n",
              "626     0.116372  0.154064  0.040243  ...  0.095846  0.106153  0.045150   \n",
              "847     0.027439  0.065770  0.145654  ...  0.087720  0.071641  0.052734   \n",
              "997     0.284800  0.130451  0.112039  ...  0.203909  0.169936  0.185976   \n",
              "1401    0.153744  0.087234  0.053545  ...  0.142850  0.127917  0.059083   \n",
              "\n",
              "userId    161383    161544    161586    161928    162047    162271    162516  \n",
              "userId                                                                        \n",
              "548     0.054047  0.082037  0.146003  0.098940  0.136444  0.053942  0.183845  \n",
              "626     0.141541 -0.012065  0.067175  0.075466  0.051487  0.022263  0.098727  \n",
              "847    -0.022233  0.119895  0.121661  0.009932  0.050526  0.077448 -0.005177  \n",
              "997     0.025221  0.121291  0.189271  0.142812  0.144200  0.098583  0.187010  \n",
              "1401    0.000316  0.048369  0.118844  0.064470  0.110306  0.075344  0.130986  \n",
              "\n",
              "[5 rows x 1000 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 49
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fKIXp1wvn2JF"
      },
      "source": [
        "### Creating Recommendations"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7rnTedoOtAwt"
      },
      "source": [
        "#Function to find a list of movie recommendations based on similar users\n",
        "def recommend_movies_by_neighbors(user, num_recommendations=10):\n",
        "    # Isolate the similarity scores for a given user and sort\n",
        "    single_user_sim = df_user_similarity.loc[user]\n",
        "    ordered_similarities = single_user_sim.sort_values(ascending=False).drop([user])\n",
        "\n",
        "    num_neighbors =int(len(ordered_similarities)*(0.05))\n",
        "    # Find the top 10 most similar users\n",
        "    similar_users = ordered_similarities.head(num_neighbors).index\n",
        "\n",
        "    # Removing movies that the user has already watched\n",
        "    movies_watched = list(user_ratings_pivot.loc[[user]].dropna(axis='columns').columns)\n",
        "    \n",
        "    # Extract the ratings of the neighbors\n",
        "    similar_user_ratings = user_ratings_pivot.drop(columns=movies_watched).reindex(similar_users).T\n",
        "\n",
        "    #Drop movies that more than half didn't watch\n",
        "    similar_user_ratings = similar_user_ratings.dropna(thresh=num_neighbors/2)\n",
        "\n",
        "    similar_user_ratings_mean = similar_user_ratings.mean(axis=1).sort_values(ascending=False).head(num_recommendations)\n",
        "\n",
        "    return similar_user_ratings_mean"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "McSKoLVIn2JF",
        "outputId": "73499baf-288f-417f-f346-c95b180bf84b"
      },
      "source": [
        "recommend_movies_by_neighbors(548)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "title\n",
              "Godfather: Part II, The (1974)                                               4.539474\n",
              "Godfather, The (1972)                                                        4.500000\n",
              "Good, the Bad and the Ugly, The (Buono, il brutto, il cattivo, Il) (1966)    4.296875\n",
              "One Flew Over the Cuckoo's Nest (1975)                                       4.196970\n",
              "Jaws (1975)                                                                  4.166667\n",
              "Deadpool (2016)                                                              4.145161\n",
              "Blade Runner (1982)                                                          4.097561\n",
              "Rear Window (1954)                                                           4.068966\n",
              "City of God (Cidade de Deus) (2002)                                          4.062500\n",
              "Pianist, The (2002)                                                          4.053571\n",
              "dtype: float64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 51
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Wf1ope0qn2JG",
        "outputId": "1a89b3a1-124e-40aa-ec7e-5f2e83540afc"
      },
      "source": [
        "recommend_movies_by_neighbors(1401)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "title\n",
              "City of God (Cidade de Deus) (2002)                              4.351351\n",
              "Once Upon a Time in the West (C'era una volta il West) (1968)    4.258065\n",
              "Let the Right One In (Låt den rätte komma in) (2008)             4.250000\n",
              "Old Boy (2003)                                                   4.226190\n",
              "Citizen Kane (1941)                                              4.214286\n",
              "Drive (2011)                                                     4.144737\n",
              "Casablanca (1942)                                                4.125000\n",
              "Amelie (Fabuleux destin d'Amélie Poulain, Le) (2001)             4.125000\n",
              "Pan's Labyrinth (Laberinto del fauno, El) (2006)                 4.088889\n",
              "Blood Simple (1984)                                              4.083333\n",
              "dtype: float64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 52
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xVSANFh3n2JG"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}