Python Tutorial 30: TikTok Bot Scrapes Top Ranking Content, Profiles & Automates Messagings
It was unbelievable how one app under two names has become the fastest growing social media application. They are respectively TikTok for global audiences, and Douyinfor China. In fact, TikTok amassed over 3 billion downloads and penetrated one-third of all social media users on this planet in less than four short years. This number took Facebook and Instagram almost a decade. If we look into these two platforms, it makes sense why it is growing so fast. Content is the king and this piece talks how to fast scrape the top profiles, best performing videos, meanwhile implement the automatic communication
It was unbelievable how one app under two names has become the fastest growing social media application. They are respectively TikTok for global audiences, and Douyin for China. In fact, TikTok amassed over 3 billion downloads and penetrated one-third of all social media users on this planet in less than four short years. This number took Facebook and Instagram almost a decade. If we look into these two platforms, it makes sense why it is growing so fast. Content is the king and this piece talks how to fast scrape the top profiles, best performing videos, meanwhile implement the automatic communication
Python Tutorial 23: Douyin Bot & Content Scraper – Top Ranking Videos & KOL Profiles
In this piece, I’ll take the TikTok SERP by keyword search for an example. And it will finish a task, which is from fetching the top ranking posts to automating the messaging to the influencers. The Python Tutorial will walk you through the codings.
If you are interested in the full script of TikTok bot creating customized messagings to different connections, please subscribe to our newsletter by adding the message “Python tutorial 30”. We would send you the script immediately to your mailbox.
Python Modules to use: BeautifulSoup, Selenium, Pandas, Time, CSV, gspread
- What TikTok search mechanicals are available?
- TikTok Login Function
- Load More Button
- Scrape SERP by keywords, country
- Full Script of TikTok Scraper
What TikTok search mechanicals are available?
TikTok is a mobile-driven app that provides users a platform to upload diversified video content though, it also has a web application version. Basically users can have three methods to search their favorite video content.
- Keyword (multi keywords in the search box are doable, it’s different with Instagram)
- User account
- Hashtags
Thus, we can leverage these search mechanicals and write a scraper to fetch the top ranking content. Of course, mostly they are from the active, popular and well-known influencers. And here are the three search URL with related parameters:
https://www.tiktok.com/search?lang=en&q=nfthttps://www.tiktok.com/search/user?lang=en&q=nfthttps://www.tiktok.com/tag/budgeting?lang=en
TikTok Login
Compared to Instagram, TikTok login has more options and also more security checkers. Thus, I’ll suggest making it simple and manually to login to our TikTok account by setting up a 60 seconds timer.
## ****** Tiktok Logins *********
def tiktokLogin():
tmall = driver.get('https://www.tiktok.com/')
time.sleep(60)
return
For the login method, I’ll suggest using an email address to login. It’s because it has fewer factors affecting the login process. For example, you might have latency in receiving the verification code by using phone number login. Or you are rejected from using Google when using selenium.
Last but not least, using a real TikTok account with your profile pic, videos and bio is vital. As you might use it to automate messaging, such as influencer outreach. However, using a phone to create an account on the web application doesn’t allow you to update profile information. Furthermore, Tiktok might not be available in some cities or countries, such as Hong Kong and India. I’ll release another article regarding creating a TikTok account by using a mobile phone.
Load More Button
TikTok SERP has a load more button. In general, users need to click the button to load more video posts. Thus, below is the code that resolves this block and you can load as many video posts as you aim to fetch from one query.
## Tiktok ********* TopSERP ********
def TiktokbyTop(query, filename):
tiktokSERP = [ ]
tiktokTop = driver.get('https://www.tiktok.com/search?lang=en&q=' + query)
try:
loadMore = driver.find_element_by_xpath('//*[@id="app"]/div[2]/div[2]/div[2]/div[2]/button')
loadMore.click()
except Exception as e:
loadMore = None
time.sleep(3)
In fact, I might only use a timer that is set to 60 seconds. So I can manually click the button, it seems faster when you are in front of the computer. But at least you are in front of the laptop.
Scrape SERP by Keywords
In the SERP, the scrapable elements are the video link, video views, user profile page, description. For making it more simply, I use the beautifulsoup to lock the path of those elements as listed below:
TikTok Profile and Following
We can either use CSV to read the profile links just now that are scraped above, or access to Google Sheets using gspread. It’s totally up to your preference.
In the profile page, I would say follower amount and the website URL data are primary to scrape. It’s because it can tell us if the influencer quality can fit our criteria, and we can learn more about the influencer activities out of TikTok.
TikTok only opens the private message access when two profiles are connected. So apart from getting the data, please don’t forget to follow the page automatically.
Automate the Messagings
Compared to Instagram, TikTok gives more spaces for the bots to connect and engage with users. So you can tailor the messaging and send messages to your connections by using this.
Full Script of TikTok Scraper
If you are interested in the full script of TikTok bot creating customized messagings to different connections, please subscribe to our newsletter by adding the message “Python tutorial 30”. We would send you the script immediately to your mailbox.
Social Commerce Douyin vs TikTok vs Instagram – Social Consumable and Shoppable Content
I hope you enjoy reading Python Robotic Process Automation – Def Functions, Import Custom Modules to Create a Multi-functional Bot. If you did, please support us by doing one of the things listed below, because it always helps out our channel.
- Support my channel through PayPal (paypal.me/Easy2digital)
- Subscribe to my channel and turn on the notification bell Easy2Digital Youtube channel.
- Follow and like my page Easy2Digital Facebook page
- Share the article to your social network with the hashtag #easy2digital
- Buy products with Easy2Digital 10% OFF Discount code (Easy2DigitalNewBuyers2021)
- You sign up for our weekly newsletter to receive Easy2Digital latest articles, videos, and discount code on Buyfromlo products and digital software
- Subscribe to our monthly membership through Patreon to enjoy exclusive benefits (www.patreon.com/louisludigital)
