Chapter 9: Big Picture Matters, Get Data from Google Trends Using Easy2Digital API

In the previous Python Tutorial for digital marketers, I talked about leveraging Shopify APIs to scrape the competitors’ product feeds. Also, you can monitor up-to-trend products and pricing from there. It’s for the purpose to adjust tactics and keep your business cutting-edge from the same selling marketplaces.

In a way, the product feed is not sufficient to understand the market demand. It’s because you might not want to anchor a dot, where they’re selling low-demand products, or the product trend has been going down. Then, unfortunately, you test along with these sellers and lose money in the end. It’s because you just follow without further steps to analyze the big picture.

The big picture is critical in the business battleground. And the end consumer search trend is a key implication to tell you how’s the demand going out there. Also, you can learn what topics they are looking for.

In this piece of Python Tutorial, I would walk through how to pull search data from Google Trends API via Pytrends. Then people can learn the data that is integrated with your in-house database, and identify the opportunities. By the end of this Python Tutorial, you can master how to install Pytrends and the necessary modules. And you can understand what available API methods and parameters you can leverage. It’s for scraping available data and customizing the data sheet based on actual needs.

Table of Contents: Big Picture Matters, Get Data from Google Trends Using Easy2Digital API

• Install Pytrends & Import Modules
• API methods and Parameters
• Keywords and Related Dimensions
• Interest over time method
• Search Trend Image Generation
• Pull out the data and save it in CSV
• Why search trend data matter
• Easy2Digital API – Google Trend Data Scraper
• Free Easy2Digital Token and Full Python Script of Google Trends Data Scraper

Install Pytrends & Import Modules

As usual, before writing the codes, we start with identifying the modules needed and importing those in the Python script. Installing Pytrends is pretty simple, that is just by typing this line of code in the macOS terminal and it can be done. As I have installed it before, the screen attached below shows it has existed already. If you wanna check whether it exists on your laptop, please check by using the pip3 list.

 pip3 install pytrends

After installing the pytrends, we install the pytrends request module that can let us call data based on the parameters used in the codings.

from pytrends.request import TrendReq

Trend data visualization is important well as we type in Google Trends on the site and the result comes out as a graph of the trend, this is the key value of Google Trends allowing people to immediately identify the opportunity. Therefore, we would need to install the Matplot library in the macOS terminal and import the module in the script, which can allow us to generate the data along with the keyword trend photo.

 pip3 install matplotlib

import matplotlib

Then, we import the pandas’ library, which pytrends has included in the package. Pandas can offer data structures and operations for manipulating numerical tables and time series

Import pandas as pd

Last but not least, CSV is necessary to store all the data in the file. Having said that, we could upload it to Google Sheets as well. If you like to learn more about the information, please check out the other article – Google Sheet API.

import csv

Methods & Parameters

Pytrends has 8 API methods with common parameters shared with each other, and also each method includes parameters for developers to custom the data pull dimension. For example, if we like to pull the keyword search data trend, interest_over_time() can fulfill the function. It returns historical, indexed data for when the keyword was searched most as shown in Google Trends’ Interest Over Time section.

In the method, there are some shared parameters, such as kw_list, geo, etc. Developers can fill in the value and target to pull related data for business opportunity analytics. For more details, please visit the official pip.org 

Target Keyword and Related Dimension

Data generation roots from what keywords you aim to research, so whatever data type you like to pull, such as data segmented by search trend, country, topics, related queries, or keyword suggestions, etc, we have to tell Python what keywords are and custom the scope of keyword data for Python scraping for you.

Basically, we need to use the build_payload method and it will have 3 lines of coding before we select which API method. 

First of all, defining pytrends with language and timezone, impact the visualization and data time endpoint

pytrends = TrendReq(hl='en-US', tz=360)

Secondly, defining what keywords you aim to fetch data, and pytrends allows developers to use up to 5 keywords in each script. 

keyword = ['iphone case','laptop stand','osmo pocket','smart home','xiaomi']

Then, using the build_payload method to custom the time frame, geo. Basically, if we leave it as an empty value, it will be set as a default value. For example, geo=’’ means it’s set to scrape worldwide data. On the other hand, if we set the US, the data set only includes US market sources. 

pytrends.build_payload(kw_list = keyword, cat=0, timeframe='today 5-y', geo='', gprop='')

Interest_over_time() method

Defining a variable with interest over time method, and here doesn’t need any parameter inside, because we define in build_payload already. If we try to print this out, as you can see all data are pulled out in the result.

Search Trend Image Generation

The trend graph is what we know from the 1st day using Google Trends, and definitely, in terms of being direct and straightforward, rather than leveraging the trend raw data, we need to generate a graph photo. 

First of all, we would remove the isPartial columns, because it’s nothing dealing with the trend. Using the drop() method with labels and axis parameters, we can customize what information to remove.

data2 = data.drop(labels=['isPartial'],axis='columns')

Then, define the image using the plot() method to name the image title.

image = data2.plot(title = 'Last 5 years on Google Trends')

For converting the fetched data into a graph, we use the get_figure() method.

fig = image.get_figure()

Last but not least, use save the photo in a jpg format by using the savefig() method

fig.savefig('figure.jpg')

Pull out and Save Data in CSV

In the previous Python Tutorial, I talked about how to use the CSV module and save data into a CSV. file. So I’m not gonna deep dive and elaborate. If you’re interested, please check out the other article.

data2.to_csv('googletrends2.csv', encoding='utf_8_sig')

Why Search Trend Data Matter

There is a typical scenario to explain the importance of search trends for identifying business opportunities is just like the perspective and flow we check out the math question as listed below

{(9809032 * 43)/(5380 + 3435 - 98090/889343)}*0 = 0

Sometimes we are really just deep-diving into the specialist area and just start from there to resolve a problem, like calculating the formulate from 1st place. In fact, standing an upper and bigger picture to analyze is much more important than all.

If you are running a business and selling products, take the iPhone case for example here, you might be aware the demand for iPhone cases is going down, which implies if you use the same amount of investment that includes time, money, and creativity, the outcome would be less and less year by year. Competition is definitely fierce, but the key challenge is fewer customers purchase.

Thus you would want to leverage python and integrate the trend data into your product research and P&L table, for the purpose to identify the best proper business and product marketing strategies in the right format.

Easy2Digital APIs – Google Trend Data Scraper

If you find the script might be complicated and also requires you to update scripts and fix bugs on and off, you can leverage Easy2Digital Google Trend Data Scraper API. Here is the token endpoint as follows:

https://www.buyfromlo.com/api/1/googleTrend?token=&keyword=

https://www.buyfromlo.com/api/1/topqueries?token=&keyword=

https://www.buyfromlo.com/api/1/trendGEO?token=&keyword=

By using these API endpoints, you just need to select the specific endpoint, add the Easy2Digital token, and related keywords, you aim to scrape. The scraped result is the same as the one shown above.

For more details regarding Marketing APIs, please check out this page.

Free Easy2Digital Token and Full Python Script of Google Trends Data Scraper

If you would like to have a free Easy2Digital token and the full version of the Python Script of Google Trends data scraper, please subscribe to our newsletter by adding the message Chapter 9. We would send you asap to your mailbox.

Contact us

So easy, right? I hope you enjoy reading Chapter 9: Big Picture Matters, Get Data from Google Trends Using Easy2Digital API. If you did, please support us by doing one of the things listed below, because it always helps out our channel.

• Support and Donate to our channel through PayPal (paypal.me/Easy2digital)
• Subscribe to my channel and turn on the notification bell Easy2Digital Youtube channel.
• Follow and like my page Easy2Digital Facebook page
• Share the article on your social network with the hashtag #easy2digital
• You sign up for our weekly newsletter to receive Easy2Digital latest articles, videos, and discount codes
• Subscribe to our monthly membership through Patreon to enjoy exclusive benefits (www.patreon.com/louisludigital)

If you are interested in scraping all web pagination and Shopify product pages, please check out the article below

Chapter 10 – Build a Shopify Bot to Scrape Store Product Data at Scale Using Easy2Digital APIs