Chapter 40 – Utilize Youtube Bots to Scrape Videos, Profiles, and Contacts Using Easy2Digital APIs and Youtube API
Automatic chain effect supplier is a nice phrase to describe the youtube data source. Most popular video content in SERP, active and famous Youtube channels, and the contact points such as email, website and social channels are in place as long as the youtubers add on. Basically people are able to grab content ideas, youtuber profile and automate outreach from every time of youtube bot scraping command.
In this chapter, I’m going to show you how to build a Youtube bot and leverage it to finish a series of commercial tasks. You can apply these actions to complete lots of business tasks, such as demand generation, partnership recruitment, etc.
Table of Contents: Youtube Bot
- Youtube API Key and URL Endpoint
- Scrape the polarity video content and channel URL using keywords
- Easy2Digital API – Youtube Video Content Scraper
- Fetch specific channel data at Scale
- Grab the contact points and automate the messaging
- Full Python script of Youtube Bot
Youtube API Key and URL Endpoint
Before starting coding, there are two ingredients we need. The 1st one is a key to connecting with the Youtube data center. We need to create an account in the Google API console and enable the youtube data v3 app. For more details, please check out this article I released earlier
Chapter 5 – Build a Youtube Bot to Scrape Trending Videos Using Youtube and Easy2Digital APIs
Once the key is in place, we would use the youtube API endpoint this time to scrape the data. It’s different from the method I used to scrape top videos in SEPR and the video performance data.
There are two URL endpoints this time. One is the search API endpoint, and the other is for channel data fetch. Each endpoint has many parameters for you to filter and extract the specific data you like to grab. I’ll introduce and explain how to use the parameters to complete our task in a moment.
Search Data:
https://www.googleapis.com/youtube/v3/search?key=&maxResult&......
Channel Data:
https://www.googleapis.com/youtube/v3/channels?key=&maxResult&......
If you like to explore the other method I use to scrape the polarity videos in SERP using Google APIs and outh client modules in python, please check out this article
Scrape the polarity video content and channel URL using keywords
There are 6 parameters you will use in the URL endpoint to scrape the popular video content and get the creator channel page URL.
Api parameter– Insert the Youtube API key you generate from the Google APIs console credentialmaxResult– Each SERP has max.50 results to scrape. Normally you would input 50nextPageToken– The next page value from the Youtube JSONQuery– the keywordOrder– you can filter the SERP result by viewCount or video count. From my point of view, I will use the viewCount because it shows you the polarity and it’s easier to consolidate popular channel profilesregionCode– It allows you to generate video content from a specific country, such as Singapore, Japan, etc
When the parameters’ values are ready, below is a sample using requests
You might find that it just comes up with response 200 if you just print out the url2 variable above. It’s because you need to format the data from youtube data in a JSON format.
Easy2Digital API – Youtube Video Content Scraper
If you find the script might be complicated and also requires you to update scripts and fix bugs on and off, you can leverage Easy2Digital Youtube Bot API. Here is the token endpoint as follows:
https://www.buyfromlo.com?token=&keyword=&totalSEPR=
By using this API endpoint, you just need to add the Easy2Digital token, the target shop domain brand name, and the type of top-level domain or subdomain (www, us, HK, etc) you aim to scrape. The scraped result is the same as the one shown above.
For more details regarding Marketing APIs, please check out this page.
Grab the next page token Value and Leverage Nested Loop for more than 50 SERPs
It might miss out on some valuable content and channels if just referring to the top 50 SERPs. So we need to leverage the next page token parameter mentioned above. For the value of the next page token, Youtube API would include a dynamic value in each response. So first thing first, we need to grab the value from each SERP JSON data like the script below
I would recommend using 2 or 3 SERPs from each keyword because the quality of channel profiles is getting lower after the 4th SERP.
Once the next page token value is in place, we could create a nested loop, which would loop the page token first and then loop scraping the search snippet value, such as channel ID
Fetch specific channel data at Scale
Scraping the top and most polarity content is the first step where you can grab the great content as inspiration. However, if you like to network with more top-tier YouTubers and content creators for commercial purposes, specific channel data is the next step.
The Youtube channel URL structure is like this below.
https://www.youtube.com/channel/ + Channel ID (UCF2oW5-MO8dB6ul9WH9xi0A
)
In the SERP scraping section, you can grab a list of channel IDs from each top-ranking video. So you can set up a timer to wait a while and automatically visit this channel one after one by combining the youtube channel folder and the channel ID.
For the channel API URL endpoint, there are also different parameters you can check and leverage based on your needs.
For finding the qualified profiles, usually I would use these parameters
Statistics– we can grab the follower amountbrandingSettings– Fetch the channel about page description, sometimes the owner leaves the email and phone number contact there.topicDetails– Here tells what is the main content topics from this channel page.
Once the choices are ready, below is the sample of the URL with optimal parameters
url1234 = requests.get('https://www.googleapis.com/youtube/v3/channels?key=' + api + '&part=statistics' + '&id=' + chID).json()
Grab the contact points and automate the messaging
Now you have a list of qualified channel profiles you find opportunities to collaborate. So the next step is to scrape the contact points and automate the messaging.
Except for the description part, generally, Youtubers add their contact in the channel about page. The URL structure of the about page adds the slash tag “about” after the channel ID page.
https://www.youtube.com/channel/UCF2oW5-MO8dB6ul9WH9xi0A/about
As you can see, there is one section called for business inquiries. Youtube sets up a very strict human verification for constraining the bot scraping action. Normally I would use Python to detect if the YouTuber leaves an email on the about page by using this code. Then I can manually get them once I find this channel is worthy of investing time.
The turnaround solution is the Youtuber’s social channels and website. We can integrate the scraped social channel address and website URL with the specific social bots I shared previously. So we can contact them via social messenger, like Instagram messages, TikTok, and through email, such as Gmass.
For more details, please check out this list of articles to build up an RPA
Social Bots
Email Automation
Full Python script of Youtube Bot
If you would like to have a free Easy2Digital API token and the full version of the Youtube Bot for scraping content, profiles, and contacts, please subscribe to our newsletter by adding the message “Chapter 40. We would send you the script immediately to your mailbox.
I hope you enjoy reading Chapter 40 – Utilize Youtube Bots to Scrape Videos, Profiles, and Contacts Using Easy2Digital APIs. If you did, please support us by doing one of the things listed below, because it always helps out our channel.
- Support and donate to our channel through PayPal (paypal.me/Easy2digital)
- Subscribe to my channel and turn on the notification bell Easy2Digital Youtube channel.
- Follow and like my page Easy2Digital Facebook page
- Share the article on your social network with the hashtag #easy2digital
- Buy products with Easy2Digital 10% OFF Discount code (Easy2DigitalNewBuyers2021)
- You sign up for our weekly newsletter to receive Easy2Digital latest articles, videos, and discount codes
- Subscribe to our monthly membership through Patreon to enjoy exclusive benefits (www.patreon.com/louisludigital)
Useful info. Fortunate me I found your site by accident, and I am stunned why this accident didn’t came about
earlier! I bookmarked it.
When some one searches for his necessary thing, therefore he/she needs to be available that in detail, thus that thing is maintained over here.
Your mode of telling the whole thing in this article is in fact fastidious, all be able to easily be aware of it, Thanks a lot.
awesome, thanks for sharing