Description: This project, YouTube Data Harvesting & Analysis, is all about gathering, storing, and studying information from different YouTube channels. It uses the YouTube Data API to get the data, then stores it in MongoDB and PostgreSQL databases. You can interact with and analyze this data through an easy-to-use interface made with Streamlit.
Key Features:
Data Collection: Grabs channel details, video information, playlist information and comments using the YouTube Data API. Data Storage: Saves all this information in MongoDB for flexibility and PostgreSQL for organized storage. Streamlit Interface: Offers a user-friendly interface with Streamlit, making it simple to query and analyze the data. Query Capabilities: Lets you run pre-set queries to find things like top videos, channel stats, and analyze comments. Customization: Allows users to add new channels for data collection and move existing data to SQL for more in-depth analysis.
Technologies Used:
Python Google API Client Library MongoDB PostgreSQL Streamlit
How to Use:
Navigate to the "Collect and Store Data" section to add new YouTube channels for data collection. Explore "Choose Channel" to transfer existing data to SQL. Under the "Select The Table For View" section, tables were displayed categorized for channels, playlists, videos, and comments. Under the "Select your question" area, 10 questions will be listed for studying the collected data. This can be done either using built-in queries or by directly checking the raw data tables.