previous episode

Machine Learning based GDP Nowcasting Workshop

next episode

Download Google Trends Data

Overview

Teaching: 130 min
Exercises: 130 min
Questions

  • How can Google Trends data be leveraged to gain insights into public economic sentiment?

  • How can Google Trends data be downloaded using different methods?

  • What are the key advantages and potential limitations of using Google Trends data in economic forecasting?

  • Is there a list of Google Trends keywords used by different research institutions?

  • What considerations should be taken into account when integrating Google Trends data with traditional economic indicators?

Objectives

  • Understand the significance of Google Trends data in economic analysis.

  • Learn how to utilize Google Trends for nowcasting GDP by analyzing public search behavior.

  • Recognize the advantages and limitations of using Google Trends data in economic research.

  • Download Google Trends data using different methods.

Introduction to Google Trends Data

Google Trends is a powerful tool that provides insights into the relative popularity of search queries over time across various regions and languages. It is widely used to analyze search trends, compare keywords, and explore interest in topics. The data is normalized on a scale of 0 to 100, with 100 representing the peak popularity of a term within the selected parameters.

Google Trends data is valuable for various applications, including market research, forecasting, sentiment analysis, and academic research. By analyzing trends, businesses and researchers can identify patterns, understand user behavior, and make informed decisions based on public interest.

In this project, Google Trends data is used as real-time survey of public sentiment that people’s search behavior on Google reflects their current economic situation and concerns. Google Trends help us to know how people are feeling about the economy. For example, during economic booms, consumer confidence is high, leading to increased searches for terms like “loans,” “investment opportunities,” or “luxury goods.” On the other hand, economic downturns (economic crisis) trigger a shift in search behavior towards terms like “unemployment benefits,” “debt consolidation,” “budgeting tips,” or “frugal living.” By analyzing these trends, economists can get a sense of how people are feeling about the economy. Essentially, Google Trends provides a real-time window into public sentiment and economic concerns, potentially providing early warnings of economic shifts. When combined with other economic data, it becomes a valuable tool for economists and policymakers.

Advantages:

Limitations:

Key features of Google Trends data include:

With both manual exploration and automated methods using tools like the pytrends Python library, Google Trends data is accessible and versatile for a wide range of analyses.

Google Trends Keywords:

Various research centers and economic institutes have utilized different sets of keywords for their studies. Identifying appropriate keywords for a specific country or geographic region often requires in-depth research. However, here are examples of keywords used by three notable institutions:

1) University of British Columbia (UBC), Canada

2) Organisation for Economic Co-operation and Development (OECD)

3) Institute of Business Administration (IBA), Karachi, Pakistan

Download Google Trends Data

Download Google Tresnds Data Directly from Google Trends Account.

Step1 : Logging in with a Google account.

Step 2: Open the Google Trends website in your web browser.

Step 3: Click the Explore button to begin your search.

Step 4: Search for Keywords. Enter a key term or word in the search bar.

Step 5: Country/Region: Select a specific country.

Step 6: Time Period: Choose a predefined range (e.g., past 7 days, 12 months) or specify a custom range.

Step 7: Category: Set to “All Categories” unless a specific category is relevant to your search.

Step 8: Search Type: Select “Web Search” for general interest trends or adjust to other types like “Image Search” or “YouTube Search” as needed.

Step 9: Review Data Visualization: Analyze the data presented in various formats, such as interest over time, geographic distribution, and related topics or queries.

Step 10: Download Data: Click the Download icon in the top-right corner of the chart. The data will be saved in CSV format on your computer.

Step 11: Open and Analyze the CSV File: Open the file using a spreadsheet application like Google Sheets, Microsoft Excel, or another data analysis tool.

Automated Google Tresnds Data Download using web appliction devloped at UNECA

The GDP Nowcasting app is a web-based application currently being developed at the African Centre for Statistics (ACS) under the United Nations Economic Commission for Africa (UNECA). It is designed to nowcast GDP by utilizing Google Trends data and applying various machine learning algorithms. The app aims to fully automate the GDP nowcasting process, encompassing workflows, deployments, and the integration of diverse machine learning methodologies.

You can use the web application to download Google Trends data by following these steps:

Step 1: Open the Web application link

Step 2: Go the Navigation Bar and select Google Trend Data Downloder module

Step 3: Select a country from the dropdown menu

Step 4: Choose the desired time range.

Step 5: Specify keywords either by manually entering them or selecting from the provided dictionary.

Step 6: Finally, download the data in CSV format.

Occasionally, the Google Trends API may block downloads due to multiple requests being made in a short period. To address this issue, you can try using a VPN, clearing your browser cache, or reopening the application after some time.

To overcome this limitation, we pre-downloaded monthly Google Trends data using the Institute of Business Administration (IBA) methodology, covering the period from 2004 to December 20, 2024, for the following countries:

1) Nigeria

2) Kenya

3) Ivory Coast

4) Cameroon

5) Egypt

6) Mauritius

7) Ghana

If needed, we can also provide additional keywords from other sources upon request.

Automated Google Tresnds Data Download using the pytrends Python library

What is Pytrends?

Pytrends is an unofficial Google Trends API for Python. It allows us to extract data of all kinds related to the interest of users in a specific topic or query.

As in Google’s own tool, we can extract information at a global or geolocated level, in specific time periods and even based on our own categorization of queries.

Link for Pytrends

Step 1: Install Pytrends

First, install the pytrends library, which allows you to interact with the Google Trends API.

pip install pytrends
Step 2: Import Necessary Libraries

Import the required libraries for working with Google Trends data and for visualizing it.

from pytrends.request import TrendReq
import matplotlib.pyplot as plt

Establish a connection to the Google Trends API.

# Connect to Google Trends
d pytrends = TrendReq(hl='en-US', tz=360)

Define the list of keywords you want to analyze using Google Trends.

# Keywords to search for
pytrends.build_payload(kw_list=['data science'])

Step 5: Retrieve Data

Fetch the interest over time data for the specified keyword(s).

# Retrieve data
time_df = pytrends.interest_over_time()

Step 6: Preview the Data

View the first few rows of the retrieved data.

time_df.head()

Step 7: Visualize the Data

Create a graph to visualize the trends over time.

# Create a graph
fig, ax = plt.subplots(figsize=(12, 6))
time_df['data science'].plot(color='purple')

Step 8: Add Titles and Labels

Enhance the graph with a title and axis labels.

# Add title and labels
plt.title('Total Google Searches for "Data Science"', fontweight='bold')
plt.xlabel('Year')
plt.ylabel('Total Count')

Final Output

The output will be a graph displaying the total Google searches for “Data Science” over time.

Parameters for Build Payload in Pytrends

The build_payload function in Pytrends has the following parameters:

1. kw_list

Example:

kw_list = ['Pizza', 'Italian', 'Spaghetti', 'Breadsticks', 'Sausage']

2. cat

Example: Choose a category ID for targeted results, such as food, travel, or technology.

3. geo

Example:

geo = 'US'  # United States
geo = 'IN'  # India

4. timeframe

Predefined Formats:

Custom Date Range:

Example:

'2016-12-14 2017-01-25'

All Time Data:

5. gprop

Available Options:

Example:

gprop = 'youtube'  # Fetch trends for YouTube searches.

Example Usage

pytrends.build_payload(
    kw_list=['Pizza', 'Italian'], 
    cat=0, 
    geo='US', 
    timeframe='today 5-y', 
    gprop='web'
)

Step 1: Define the Keyword List

Create a list of keywords to analyze in Google Trends.

kw_list = ["crypto", "bitcoin", "ethereum", "AI", "Tesla"]

Step 2: Build Payload

Pass the keyword list and additional parameters to the build_payload function to fetch data.

pytrends.build_payload(kw_list, cat=0, timeframe='today 5-y', geo='US', gprop='')

Step 3: Retrieve Interest Over Time Data

Fetch the interest over time for the specified keywords.

iot = pytrends.interest_over_time()

Step 4: Plot the Data

Visualize the interest over time data.

iot.plot()

Final Output

The output will be a line graph showing the relative search interest of the keywords “crypto,” “bitcoin,” “ethereum,” “AI,” and “Tesla” in the United States over the last 5 years.

Key Points

  • Introduction to Google Trends data.

  • Applications of Google Trends data in Economic Analysis.

  • Advantages and Limitations of Google Trends data.

  • Ways to download Google Trends data.

  • Keyword Examples from Research Institutions.

previous episode

next episode

Copyright © 2024 UNECA-ACS

Contact