May 8, 2021

Python Tutorial for Digital Marketers 6: Scrape View, Comment, Like Data of More than 50 Videos from the Youtube Search List

In the previous Python Tutorial, I walked you through how to create a youtube API key and scrape videos by search queries via python programming. Basically, the beauty of youtube video scraping is you don’t need to pay any fees to research potential YouTubers for your brand and eCommerce store product review, which is a super-efficient way.

But having said that for filtering the better YouTubers, it’s not sufficient for us to select just by those fetched videos, on the other hand, we also need to look into the video data, such as view, comments, likes, dislikes, etc, which can help us make a better candidate list. Also, by default Youtube data API v3 allows developers to scrape the top 50 pieces of videos, I would say it might be not enough because you might miss some secondary tier YouTubers whose videos might rank lower positions, but the content is pretty good and engaging.

As I haven’t talked about how to fetch those data above, so in this piece, I’ll continue to use the previous Youtube video scraping python script, and walk you through Youtube data API video data, how to write the codes to fetch the data, and scrape more than 50 pieces of videos from the search query result.

In the previous Python Tutorial, I walked you through how to create a youtube API key and scrape videos by search queries via python programming. Basically, the beauty of youtube video scraping is you don’t need to pay any fees and make the research done in minutes and focus more on content creation for your brand and eCommerce store product review, which is a super-efficient way.

But having said that for filtering the better YouTubers, it’s not sufficient for us to select just by those fetched videos, on the other hand, we also need to look into the video data, such as view, comments, likes, dislikes, etc, which can help us make a better candidate list. Also, by default Youtube data API v3 allows developers to scrape the top 50 pieces of videos, I would say it might be not enough because you might miss some secondary tier YouTubers whose videos might rank lower positions, but the content is pretty good and engaging.

As I haven’t talked about how to fetch those data above, so in this Python Tutorial 6, I’ll continue to use the previous Youtube video scraping python script, and walk you through Youtube data API video data, how to write the codes to fetch the data, and scrape more than 50 pieces of videos from the search query result.

Youtube Data API – Video Documentation

First thing first, we always start with this platform API documentation reference, whichever platforms you are going to leverage the APIs, such as WeChat auto-reply, Facebook messenger, etc. Last time, we checked out the search list method, and here I try to check out the video sections.

python tutorial

As we can see from the following JSON structure shows the format of a video resource, the video has the list() to list all videos we are going to fetch data, and the data we need such as view, comment, like, etc is an understandable statistic.

      “statistics”: {

    “viewCount”: unsigned long,

    “likeCount”: unsigned long,

    “dislikeCount”: unsigned long,

    “favoriteCount”: unsigned long,

    “commentCount”: unsigned long

  },…….

We create a variable vid_response, which we pass in the youtube video list method and request the statistic data division. Vid_ids are from the previous search.list (), which variable represents the videos from the search query result.

vid_request = youtube.videos().list(

part=’statistics’,

id=vid_ids

)

If we try to execute the codings, we find it’s working because the data we need are calibrated. The video data are following up the video URL yt_link.

Video Data Sample:

HomeAutomationX

Is the Ring Spotlight Camera Worth Buying?

https://youtu.be/U-06WEwtaSk

{‘kind’: ‘youtube#videoListResponse’, ‘etag’: ‘DUSIAmpvqb5UpMMas5J6IiT_RnM’, ‘items’: [{‘kind’: ‘youtube#video’, ‘etag’: ‘Rak9ja0G6clMeVEuQlcqhe0IvMc’, ‘id’: ‘U-06WEwtaSk’, ‘statistics’: {‘viewCount’: ‘136516’, ‘likeCount’: ‘1214’, ‘dislikeCount’: ‘232’, ‘favoriteCount’: ‘0’, ‘commentCount’: ‘234’}}], ‘pageInfo’: {‘totalResults’: 1, ‘resultsPerPage’: 1}}

How to Fetch Video Performance from Youtube Search Query Result

We can see all data are under items, and the data we are going to fetch are under statistics as we mentioned above. Also, we need to fetch each video id-data, so we create a loop as well. I’m not going to go into details because it’s similar to my previous article. If you’d like to learn how to identify the data location, please check out this article.

Python Tutorial for Digital Marketers 5: Create Youtube API & Scrape Youtube Videos

for item in vid_response[‘items’]:

vid_view = (item[‘statistics’][‘viewCount’])

                       print(vid_view)

And some videos might not have comments, likes, so we can use try/except to pass the missing data value, in order to avoid errors in the scraping process

Then, we try to command B and can see the top 50 video performances all show up in the result.

How to Scrape as many videos as possible from the search result

As we know, Youtube data API by default can respond to the max. 50 videos from a search query, however, in fact, the search result video amounts are far more than 50. So first thing first, we need to check if the Youtube search list provides related parameters for us to fetch data by page.

As the screencap shows below, we can find a parameter called pageToken, which can be used to fetch videos on the pre or next page.

Also, as we need to loop and fetch the whole block and body we create to scrape one page, we need to use the other loop called while in the script of master level.

Syntax and argument:

while <expr>:

    <statement(s)>

<statement(s)> represents the block to be repeatedly executed, often referred to as the body of the loop. This is denoted with indentation, just as in an if statement.

First, we create a variable named nextPageToken, which value is none, because we start from the first page.

Then, we pass in the while function, which is defined as it’s true. Just remember to indent the current ready body and block to create a single page, because it tells Python While is used for looping the whole block. Also, we set the max result = 50 and use parameter pageToken, and set the variable nextPageToken as its value.

After a page fetching is done, we need to add a line of code that tells Python to scrape the next page and stop to fetch if the next page doesn’t exit, which can be:

nextPageToken = search_response.get(‘nextPageToken’)

           if not nextPageToken:

                break

Just remember to keep the CSV write demand within the block because it tells Python to store the new page video data into the file in each loop.

If things are done as instructed, you can generate and pass all video data into the CSV.

python tutorial

For example, I search ‘ring spotlight camera” and grab 271 videos from this search query. So you can imagine, if you set this file in a place where it has connected with your influencer marketing research and recruitment dashboard, the dashboard can automatically show you what new YouTubers appear, what new better-engaging videos about ring spotlight cameras released.  I’ll walk you through in the coming Python Tutorial chapters, how to create automatic refresh with external raw data updated.

So easy, right? I hope you enjoy reading Python Tutorial for Digital Marketers 6: Scrape View, Comment, Like Data of More than 50 Videos from the Youtube Search List. If you did, please support us by doing one of the things listed below, because it always helps out our channel.

  • Support my channel through PayPal (paypal.me/Easy2digital)
  • Subscribe to my channel and turn on the notification bell Easy2Digital Youtube channel.
  • Follow and like my page Easy2Digital Facebook page
  • Share the article to your social network with the hashtag #easy2digital
  • Buy products with Easy2Digital 10% OFF Discount code (Easy2DigitalNewBuyers2021)
  • You sign up for our weekly newsletter to receive Easy2Digital latest articles, videos, and discount code on Buyfromlo products and digital software
  • Subscribe to our monthly membership through Patreon to enjoy exclusive benefits (www.patreon.com/louisludigital)

If you are interested in learning how to save the data in a Google Sheet via API, please check out this article:

Python Tutorial for Digital Marketers 7: Save Web Scraping Data in Google Sheet via API

Youtuber Research and Recruitment Service

If you are looking for a freelancer to help you with Youtube influencer research and recruitment, I’m able to help you with this, and below has 3 service and quotation ($US) plans you can select. For details, please visit here and contact me.

python tutorial

Leave a Reply

Your email address will not be published. Required fields are marked *