In the previous Python Tutorial for digital marketers, I talked about leveraging Shopify APIs to scrape the competitors’ product feeds. Also, you can monitor up-to-trend products and pricing from there. It’s for the purpose to adjust tactics and keep your business cutting-edge from the same selling marketplaces.
In a way, the product feed is not sufficient to understand the market demand. It’s because you might not want to anchor a dot, where they’re selling low-demand products, or the product trend has been going down. Then, unfortunately, you test along with these sellers and lose money in the end. It’s because you just follow without further steps to analyze the big picture.
The big picture is critical in the business battleground. And the end consumer search trend is a key implication to tell you how’s the demand going out there. Also, you can learn what topics they are looking for.
In this piece of Python Tutorial, I would walk through how to pull search data from Google Trends API via Pytrends. Then people can learn the data that is integrated with your in-house database, and identify the opportunities. By the end of this Python Tutorial, you can master how to install Pytrends and the necessary modules. And you can understand what available API methods and parameters you can leverage. It’s for scraping available data and customizing the data sheet based on actual needs.
Table of Contents: Big Picture Matters, Get Data from Google Trends Using Easy2Digital API
- Install Pytrends & Import Modules
- API methods and Parameters
- Keywords and Related Dimensions
- Interest over time method
- Search Trend Image Generation
- Pull out the data and save it in CSV
- Why search trend data matter
- Easy2Digital API – Google Trend Data Scraper
- Free Easy2Digital Token and Full Python Script of Google Trends Data Scraper
- FAQ
Install Pytrends & Import Modules
As usual, before writing the codes, we start with identifying the modules needed and importing those in the Python script. Installing Pytrends is pretty simple, that is just by typing this line of code in the macOS terminal and it can be done. As I have installed it before, the screen attached below shows it has existed already. If you wanna check whether it exists on your laptop, please check by using the pip3 list.
pip3 install pytrends
After installing the pytrends, we install the pytrends request module that can let us call data based on the parameters used in the codings.
from pytrends.request import TrendReq
Trend data visualization is important well as we type in Google Trends on the site and the result comes out as a graph of the trend, this is the key value of Google Trends allowing people to immediately identify the opportunity. Therefore, we would need to install the Matplot library in the macOS terminal and import the module in the script, which can allow us to generate the data along with the keyword trend photo.
pip3 install matplotlib
import matplotlib
Then, we import the pandas’ library, which pytrends has included in the package. Pandas can offer data structures and operations for manipulating numerical tables and time series
Import pandas as pd
Last but not least, CSV is necessary to store all the data in the file. Having said that, we could upload it to Google Sheets as well. If you like to learn more about the information, please check out the other article – Google Sheet API.
import csv
Methods & Parameters
Pytrends has 8 API methods with common parameters shared with each other, and also each method includes parameters for developers to custom the data pull dimension. For example, if we like to pull the keyword search data trend, interest_over_time() can fulfill the function. It returns historical, indexed data for when the keyword was searched most as shown in Google Trends’ Interest Over Time section.
In the method, there are some shared parameters, such as kw_list, geo, etc. Developers can fill in the value and target to pull related data for business opportunity analytics. For more details, please visit the official pip.org
Target Keyword and Related Dimension
Data generation roots from what keywords you aim to research, so whatever data type you like to pull, such as data segmented by search trend, country, topics, related queries, or keyword suggestions, etc, we have to tell Python what keywords are and custom the scope of keyword data for Python scraping for you.
Basically, we need to use the build_payload method and it will have 3 lines of coding before we select which API method.
First of all, defining pytrends with language and timezone, impact the visualization and data time endpoint
pytrends = TrendReq(hl='en-US', tz=360)
Secondly, defining what keywords you aim to fetch data, and pytrends allows developers to use up to 5 keywords in each script.
keyword = ['iphone case','laptop stand','osmo pocket','smart home','xiaomi']
Then, using the build_payload method to custom the time frame, geo. Basically, if we leave it as an empty value, it will be set as a default value. For example, geo=’’ means it’s set to scrape worldwide data. On the other hand, if we set the US, the data set only includes US market sources.
pytrends.build_payload(kw_list = keyword, cat=0, timeframe='today 5-y', geo='', gprop='')
Interest_over_time() method
Defining a variable with interest over time method, and here doesn’t need any parameter inside, because we define in build_payload already. If we try to print this out, as you can see all data are pulled out in the result.
Search Trend Image Generation
The trend graph is what we know from the 1st day using Google Trends, and definitely, in terms of being direct and straightforward, rather than leveraging the trend raw data, we need to generate a graph photo.
First of all, we would remove the isPartial columns, because it’s nothing dealing with the trend. Using the drop() method with labels and axis parameters, we can customize what information to remove.
data2 = data.drop(labels=['isPartial'],axis='columns')
Then, define the image using the plot() method to name the image title.
image = data2.plot(title = 'Last 5 years on Google Trends')
For converting the fetched data into a graph, we use the get_figure() method.
fig = image.get_figure()
Last but not least, use save the photo in a jpg format by using the savefig() method
fig.savefig('figure.jpg')
Pull out and Save Data in CSV
In the previous Python Tutorial, I talked about how to use the CSV module and save data into a CSV. file. So I’m not gonna deep dive and elaborate. If you’re interested, please check out the other article.
data2.to_csv('googletrends2.csv', encoding='utf_8_sig')
Why Search Trend Data Matter
There is a typical scenario to explain the importance of search trends for identifying business opportunities is just like the perspective and flow we check out the math question as listed below
{(9809032 * 43)/(5380 + 3435 - 98090/889343)}*0 = 0
Sometimes we are really just deep-diving into the specialist area and just start from there to resolve a problem, like calculating the formulate from 1st place. In fact, standing an upper and bigger picture to analyze is much more important than all.
If you are running a business and selling products, take the iPhone case for example here, you might be aware the demand for iPhone cases is going down, which implies if you use the same amount of investment that includes time, money, and creativity, the outcome would be less and less year by year. Competition is definitely fierce, but the key challenge is fewer customers purchase.
Thus you would want to leverage python and integrate the trend data into your product research and P&L table, for the purpose to identify the best proper business and product marketing strategies in the right format.
Easy2Digital APIs – Google Trend Data Scraper
If you find the script might be complicated and also requires you to update scripts and fix bugs on and off, you can leverage Easy2Digital Google Trend Data Scraper API. Here is the token endpoint as follows:
https://www.buyfromlo.com/api/1/googleTrend?token=&keyword=
https://www.buyfromlo.com/api/1/topqueries?token=&keyword=
https://www.buyfromlo.com/api/1/trendGEO?token=&keyword=
By using these API endpoints, you just need to select the specific endpoint, add the Easy2Digital token, and related keywords, you aim to scrape. The scraped result is the same as the one shown above.
For more details regarding Marketing APIs, please check out this page.
Free Easy2Digital Token and Full Python Script of Google Trends Data Scraper
If you would like to have a free Easy2Digital token and the full version of the Python Script of Google Trends data scraper, please subscribe to our newsletter by adding the message Chapter 9. We would send you asap to your mailbox.
So easy, right? I hope you enjoy reading Chapter 9: Big Picture Matters, Get Data from Google Trends Using Easy2Digital API. If you did, please support us by doing one of the things listed below, because it always helps out our channel.
- Support and Donate to our channel through PayPal (paypal.me/Easy2digital)
- Subscribe to my channel and turn on the notification bell Easy2Digital Youtube channel.
- Follow and like my page Easy2Digital Facebook page
- Share the article on your social network with the hashtag #easy2digital
- You sign up for our weekly newsletter to receive Easy2Digital latest articles, videos, and discount codes
- Subscribe to our monthly membership through Patreon to enjoy exclusive benefits (www.patreon.com/louisludigital)
If you are interested in scraping all web pagination and Shopify product pages, please check out the article below
Chapter 10 – Build a Shopify Bot to Scrape Store Product Data at Scale Using Easy2Digital APIs
FAQ:
Q1: What is Google Trends API?
A: Google Trends API is an application programming interface that allows developers to access and retrieve data from Google Trends. It provides real-time and historical search data, allowing businesses to analyze and understand search trends and patterns.
Q2: How can Google Trends API help eCommerce businesses?
A: Google Trends API can help eCommerce businesses by providing valuable insights into consumer search behavior. It allows businesses to identify popular search terms and topics related to their products, enabling them to optimize their SEO strategies and target relevant keywords to drive more traffic and conversions.
Q3: What type of data can be obtained from Google Trends API?
A: Google Trends API provides data such as search interest over time, related queries, regional interest, and category-specific trends. This data can be used to track the popularity of specific keywords, compare search interest across different regions, and discover related topics and queries.
Q4: How can I integrate Google Trends API into my eCommerce website?
A: To integrate Google Trends API into your eCommerce website, you will need to use the API documentation and follow the provided instructions. This usually involves obtaining an API key, making API requests using specified endpoints, and handling the returned data in your website’s code.
Q5: Can Google Trends API be used for competitor analysis?
A: Yes, Google Trends API can be used for competitor analysis. By comparing search interest and related queries for your brand and your competitors, you can identify areas of strength and weakness in your SEO strategy. This can help you refine your marketing efforts and stay ahead of the competition.
Q6: Is Google Trends API free to use?
A: No, Google Trends API is not free to use. It is a paid service offered by Google. The pricing and available plans can be found on the Google Cloud Platform website. However, there may be limitations on the amount of data that can be accessed and the number of requests that can be made within a certain timeframe.
Q7: What are the benefits of using Google Trends API for SEO?
A: Using Google Trends API for SEO can provide several benefits. It allows you to stay updated with the latest search trends, discover new keywords and topics to target, and monitor the performance of your SEO efforts. It can also help you identify emerging trends and capitalize on them to gain a competitive advantage.
Q8: Can Google Trends API be used for content marketing?
A: Yes, Google Trends API can be a valuable tool for content marketing. By analyzing search interest and related queries, you can identify popular topics and create content that is relevant and engaging to your target audience. This can help drive more traffic to your website and increase brand visibility.
Q9: Are there any limitations to using Google Trends API?
A: Yes, there are some limitations to using Google Trends API. These may include restrictions on the amount of data that can be accessed, the frequency of API requests, and the availability of certain features and functionalities. It is important to review the API documentation and terms of service to understand these limitations.
Q10: Where can I find more information about Google Trends API?
A: For more information about Google Trends API, you can visit the Google Cloud Platform website or refer to the API documentation. These resources provide detailed information about the API’s features, pricing, usage guidelines, and integration instructions.