Site icon Easy2Digital

Python Tutorial for Digital Marketers 9: Big Picture Matters, How to Pull Data from Google Trends API via Pytrends

google trends api

In the previous Python Tutorial for digital marketers, I talked about leveraging Shopify APIs to scrape the competitors’ product feed. Also, you can monitor up-to-trend products and pricing from there. It’s for the purpose to adjust tactics and keep your business cutting-edge from the same selling marketplaces.

In a way, the product feed is not sufficient to understand the market demand. It’s because you might not want to anchor a dot, where they’re selling low-demand products, or the product trend has been going down. Then, unfortunately, you test along with these sellers and lose money at the end. It’s because you just follow without further step analyzing the big picture.

The big picture is critical in the business battleground. And the end consumer search trend is a key implication to tell you how’s the demand going out there. Also, you can learn what topics they are looking for.

In this piece of Python Tutorial, I would walk through how to pull search data from Google Trends API via Pytrends. Then people can learn the data that is integrated with your in-house database, and identify the opportunities. By the end of this Python Tutorial, you can master how to install Pytrends and necessary modules. And you can understand what available API methods and parameters you can leverage. It’s for scraping available data, and custom the data sheet based on actual needs.

Python Tutorial: Install Pytrends & Import Modules

As usual, before writing the codes, we start with identifying the modules needed and import those in the Python script. Installing Pytrends is pretty simple, that is just by typing this line of code in the macOS terminal and it can be done. As I have installed it before, so the screen attached below shows it has existed already. If you wanna check whether it exists on your laptop, please check by using the pip3 list.

 pip3 install pytrends

After installing the pytrends, we install the pytrends request module that can let us call data based on the parameters used in the codings.

from pytrends.request import TrendReq

Trend data visualization is important well as we type in Google Trends onsite and the result comes out a graph of the trend, this is the key value of Google trend allowing people to immediately identify the opportunity. Therefore, we would need to install the Matplot library in the macOS terminal and import the module in the script, which can allow us to generate the data along with the keyword trend photo.

 pip3 install matplotlib

import matplotlib

Then, we import pandas library, which pytrends has included in the package. Pandas can offer data structures and operations for manipulating numerical tables and time series

Import pandas as pd

Last but not least, CSV is necessary to store all the data in the file. Having said that, we could upload it to Google sheet as well. If you like to learn more about the information, please check out the other article – Google Sheet API.

import csv

Pytrends API Methods & Parameters

Pytrends has 8 API methods with common parameters shared with each other, and also each method includes parameters for developers to custom the data pull dimension. For example, if we like to pull the keyword search data trend, interest_over_time() can fulfill the function. It returns historical, indexed data for when the keyword was searched most as shown on Google Trends’ Interest Over Time section.

In the method, there are some shared parameters, such as kw_list, geo, etc. Developers can fill in the value and target to pull related data for business opportunity analytic. For more details, please visit the official pip.org 

Target Keyword and Related Dimension

Data generation roots from what keywords you aim to research, so whatever data type you like to pull, such as data segmented by search trend, country, topics, related queries, or keyword suggestions, etc, we have to tell Python what keywords are and custom the scope of keyword data for Python scraping for you.

Basically, we need to use the build_payload method and it will have 3 lines of coding before we select which API method. 

First of all, defining pytrends with language and timezone, which impact the visualization and data time endpoint

pytrends = TrendReq(hl='en-US', tz=360)

Secondly, defining what keywords you aim to fetch data, and pytrends allows developers to use up to 5 keywords in each script. 

keyword = ['iphone case','laptop stand','osmo pocket','smart home','xiaomi']

Then, using the build_payload method to custom the time frame, geo. Basically, if we leave it as an empty value, it will be set as a default value. For example, geo=’’ means it’s set to scrape worldwide data. On the other hand, if we set the US, the data set only includes US market sources. 

pytrends.build_payload(kw_list = keyword, cat=0, timeframe='today 5-y', geo='', gprop='')

Interest_over_time() method

Defining a variable with interest over time method, and here doesn’t need any parameter inside, because we define in build_payload already. If we try to print this out, as you can see all data are pulled out in the result.

Python Tutorial: Search Trend Image Generation

The trend graph is what we know from the 1st day using Google Trends, and definitely, in terms of being direct and straightforward, rather than leverage the trend raw data, we need to generate a graph photo. 

First of all, we would remove the isPartial columns, because it’s nothing dealing with the trend. Using the drop() method with labels and axis parameters, we can custom what information to remove.

data2 = data.drop(labels=['isPartial'],axis='columns')

Then, define the image using the plot() method to name the image title.

image = data2.plot(title = 'Last 5 years on Google Trends')

For converting the fetched data into a graph, we use the get_figure() method.

fig = image.get_figure()

Last but not least, use save the photo as a jpg format by using the savefig() method

fig.savefig('figure.jpg')

Python Tutorial: Pull out and Save Data in CSV

In the previous Python Tutorial, I talked about how to use the CSV module and save data into a CSV. file. So I’m not gonna deep dive and elaborate. If you’re interested, please check out the other article.

data2.to_csv('googletrends2.csv', encoding='utf_8_sig')

Why Search Trend Data Matter

There is a typical scenario to explain the importance of search trend for identifying business opportunities is just like the perspective and flow we check out the math question as list below

{(9809032 * 43)/(5380 + 3435 - 98090/889343)}*0 = 0

Sometimes we are really just deep-diving into the specialist area and just start from there to resolve a problem, like calculating the formulate from 1st place. In fact, standing an upper and bigger picture to analyze is much more important than all.

If you are running a business and selling products, take the iPhone case for example here, you might be aware the demand for iPhone cases is going down, which implies if you use the same amount of investment that includes time, money, and creativity, the outcome would be less and less year by year. Competition is definitely fierce, but the key challenge is fewer customers purchase.

Thus you would want to leverage python and integrate the trend data into your product research and P&L table, for the purpose to identify the best proper business and product marketing strategies in the right format.

Full Python Script of Google Trends API via Pytrends Scraper

If you would like to have the full version of the Python Script of Google Trends API via Pytrends Scraper, please subscribe to our newsletter by adding the message Python Tutorial 9. We would send you the script immediately to your mailbox.

Contact us

So easy, right? I hope you enjoy reading Python Tutorial for Digital Marketers 9: Big Picture Matters, How to Pull Data from Google Trends API via Pytrends. If you did, please support us by doing one of the things listed below, because it always helps out our channel.

If you are interested in scraping all web pagination and Shopify product pages, please check out the article below

Python Tutorial for Digital Marketers 10 – Python Web Scraping – Pagination and Shopify Product Pages

Exit mobile version