Skip to content
February 3, 2023
  • easy2digital usaEN
  • easy2digital japanJP
  • Facebook
  • Linkedin
  • Youtube
  • Instagram
  • Pinterest
  • Twitter

Easy2Digital

Find Ways to Save Time For Your Life

Primary Menu

Easy2Digital

  • About
    • What They Say
    • Successful Cases
  • News
    • Technology
  • Lifestyle
    • Smart Home
    • Smart Device
  • Automation
    • Python
    • Web & Mobile Application
    • Email Scraper
    • Digital Advertising Automation
    • Google Bots
    • China Social Bot
    • Global Social Bot
    • Global eCommerce Bot
  • Investment
    • Smart Finance
  • Marketing
    • eCommerce
    • SaaS
    • Strategy
  • Web3.0
  • Data Science
    • Pandas
  • APIs Hub
    • Easy2Digital Marketing APIs Documentation
    • Easy2Digital Finance APIs Documentation
  • Contact us
  • Buyfromlo Store
  • Home
  • Automation
  • Chapter 12 – Build an Instagram Bot and Use Hashtags to Scrape Top Instagram Posts and Instagram Users
  • Automation
  • data

Chapter 12 – Build an Instagram Bot and Use Hashtags to Scrape Top Instagram Posts and Instagram Users

1 week ago

Instagram influencers are those users with up to hundreds, thousands, and millions of followers who are trusted by other users for their lifestyle, aesthetic, and expertise. Take fashion for example. There are so many fashion influencers, who create Livestream videos on IGTV or video posts. If you are a clothing brand marketer, scraping the profiles of famous influencers gives you candidate lists who can access a large audience of people all interested in fashion and lifestyle. For either the complimentary product review or content collaboration, it’s a great way to accumulate the influencer databases.

Also, scraping these Instagram influencers and their posts also gives you content marketing insights about what content might have better engagement with audiences. If you’re looking to strengthen your brand identity, learning from the profiles of influencers is a good place to start.

python tutorial

Scraping Instagram influencers and their posts can not only help collect influencer candidate lists to work with in the future but also give you content marketing insights about what content might have better engagement with audiences. If you’re looking to strengthen your brand identity, learning from the profiles of influencers is a good place to start.

However, scraping social data is not the same as website scraping, because almost all social platforms require you to log in first before using any features on the platform. So in this Python Tutorial, I would walk you through how to use Selenium to stimulate you to log into the platform, browse Instagram, and search the hashtags for downloading the top posts’ links. By the end of the Python Tutorial, you can start downloading all top Instagram posts by changing the hashtags as you like.

Table of Contents: Build an Instagram Bot and Use Hashtags to Scrape Top Instagram Posts and Instagram Users

  • Install Selenium and ChromeDriver
  • Log in Instagram account
  • Simulate you to select the options
  • Search posts using hashtags and scroll for more posts
  • Find the elements you like to scrape and save them in a CSV file
  • Full Python Script of Instagram Top Ranking Post and Influencer Profile Scraper By Using Hashtag

Instagram Bot – Install Selenium and ChromeDriver

Selenium is a free open-source automated testing framework used to validate web applications across different browsers and platforms. You can use multiple programming languages like Java, C#, Python, etc to create Selenium Test Scripts. Testing done using the Selenium testing tool is usually referred to as Selenium Testing.

If you have read my Python Tutorial article regarding setting up the pip3 previously, installing Selenium is very easy. You just need to type in this code in your Mac terminal

Then, you need a virtual driver to act on your behalf of you in the process. I would recommend ChromeDrive in this tutorial. First thing first, please go to Google and search ChromeDriver and click through to their website. You can see two versions basically – the beta and the latest standard. Just click the standard now!

python tutorial

You can select the version that is configurable with your device, here we would select mac64.zip. After downloading, you need to extract the zip and install the ChromeDriver. Quick notes for you to copy the ChromeDriver location path to the clipboard. It will be used in a moment.

In the python script, first of all, we need to import modules as well as other Python scripts we created before.

Then, we create a variable called the driver and add on the path copied just now using executable_path. Also, we type into a code about the requests to browse instagram.com. It’s similar to the request.get, but we need to use a driver in a selenium environment.

driver = webdriver.Chrome(executable_path='/Users/louislu/Desktop/Python/chromedriver')
driver.get("https://www.instagram.com")

Instagram Bot – Log in Instagram account

Basically, Selenium testing would simulate my normal browsing on Instagram. So first thing first must be the account login.

First, we go to the login page and right-click to select inspect, for finding out what elements are used to function in the username and password type-in box. As we can see, basically it is using the< input name=”username”> element representing this box, as well as the password is using input either. So we can use By.CSS_SELECTOR it to specifically point out this section.

instagram bot

In the selenium expected_conditions, there is one argument we can use which means the element is clickable is element_to_be_clickable. And as we might need to consider the loading speed, we can create the lines of coding also by using WebDriverWait.

Here are the codes:

username = WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.CSS_SELECTOR,"input[name='username']")))
password = WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.CSS_SELECTOR,"input[name='password']")))

Secondly, we would send the account username and password value to the box. Before that, I would recommend clearing up the box first, to ensure that the box is empty. Then, we use a method from selenium API – send_keys, for sending the value to the box.

Last but not least, we also need to inspect what elements the login button is, as well as check the username and password box. Then it continues to use element_to_be_clickable and By.CSS_SELECTOR. As we need to click the button, at the end, a method, click(), needs adding.

instagram bot

log_in = WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.CSS_SELECTOR,"button[type='submit']"))).click()

Simulate you to select the options

Some platforms would have some pop-up windows after you logged in. In this case, you also need to clarify what pop-up windows that might have. Instagram generally has two windows and to smoothly browse our target content, we can click not now.

Here we also can use XPATH to click the not now button. Here are the codes:

not_now = WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//button[contains(text(),'Not Now')]"))).click()
not_now2 = WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//button[contains(text(),'Not Now')]"))).click()

Search posts using hashtags and scroll down for more posts

For searching posts using hashtags, Instagram has a fixed path which is https://www.instagram.com/explore/tags/ + keyword. So we need to create a query variable first, here I presume to search “moussy”. Then, we also create a variable page to visit the page.

When you scroll down for more posts, you might find out it will load a while for more posts. So we need to write codes to scroll and also add codes to avoid the scraping being stopped due to the loading time. We would use window.scrollBy () and time.sleep() methods. The x and y number in the scroll method represents the max. The height you like to scroll down. But as the scrolling would be stopped due to loading time, I would recommend you set a bigger number first and add more lines if you aim to scrape more posts.

driver.execute_script("window.scrollBy(0,1000000)")
time.sleep(5)

Find the elements you like to scrape and save them in a CSV file

Now basically all posts are ready, and what we need to do is to fetch the post links. Again, we can inspect and find the elements. In the Selenium argument, there are two methods, elements_by_tag_name and get_attribute()

links = driver.find_elements_by_tag_name('a')
links = [link.get_attribute('href') for link in links]

If you try to print these codes and the result comes up in the links, it means it’s working

instagram bot

So you can use Pandas to append the column and save it as a CSV file. Because of this, I shared previously and I am not going to elaborate on the details here.

Full Python Script of Instagram Top Ranking Post and Influencer Profile Scraper By Using Hashtag

If you would like to have the full version of the Python Script of Instagram Post and Influencer Scraper By Using Hashtag, please subscribe to our newsletter by adding the message “Chapter 12”. We would send you the script asap to your mailbox.

Contact us

So easy, right? I hope you enjoy reading Chapter 12 – Build an Instagram Bot and Use Hashtags to Scrape Top Instagram Posts and Instagram Users. If you did, please support us by doing one of the things listed below, because it always helps out our channel.

  • Support and donate any amount to our channel through PayPal (paypal.me/Easy2digital)
  • Subscribe to my channel and turn on the notification bell Easy2Digital Youtube channel.
  • Follow and like my page Easy2Digital Facebook page
  • Share the article on your social network with the hashtag #easy2digital
  • Buy products with Easy2Digital 10% OFF Discount code (Easy2DigitalNewBuyers2021)
  • You sign up for our weekly newsletter to receive Easy2Digital latest articles, videos, and discount code
Tags: Global Social Bot Collection, instagram, Python for Digital Marketers

Continue Reading

Previous Chapter 11: Google SERP Bot to Scrape SERP Data Using Google Search and Easy2Digital APIs
Next Chapter 13 – Build an Instagram Profile Scraper to Scrape Instagram Email, Followers, Posts, and More

More Stories

Featured Video Play Icon
  • Automation
  • data

Chapter 72 – Build a Blog Content Generator Using OpenAI GPT3 and Easy2Digital API

6 days ago
product detail page
  • Automation
  • data

Chapter 71 – Build Online Shop Product Detail Pages or PDP Using Flask, Javascript, Bootstrap

6 days ago
discord bot
  • Automation
  • data

Chapter 70 – Build a Discord Bot Using Python, Hikari, Lightbulb, MongoDB

6 days ago
  1. kucreawbet on Meta Ads API – Break Through Platform Ceiling and Implement Practical Dynamic Content and Targeting Ads StrategiesJanuary 30, 2023

    Does your blog have a contact page? I'm having trouble locating it but, I'd like to send you an email.…

  2. rcronin on Chapter 56 – Use Flask Limiter to Customise Rate Limits on Web Application, API & SaaS UsageJanuary 19, 2023

    Greetings! Very useful advice within this post! It is the little changes which will make the biggest changes. Thanks a…

  3. Mypartner on Meta Ads API – Break Through Platform Ceiling and Implement Practical Dynamic Content and Targeting Ads StrategiesJanuary 18, 2023

    My partner and I absolutely love your blog and find nearly all of your post's to be exactly I'm looking…

  4. Natjeta on Chapter 42 – Pinterest Bot for Scraping Web URLs, Emails, and Automating MessagesJanuary 13, 2023

    Thanks for the information you shared I appreciate your efforts and taking the time and sharing this content.

  5. Yuru Tian on Chapter 29 – Build an Indiegogo Bot for Scraping Most Crowdfunded ProjectsJanuary 13, 2023

    Pls send me Python Tutorial 29 – Create an Indiegogo Bot for Scraping Most Crowdfunded Projects

Tags

amazon Apple Baidu CBD CDP China Social Bot Collections CRM Data Science Digital Advertising Automation DJI DMP Douyin e-Commerce Email Scraper Facebook fitbit Global eCommerce Bot Collection Global Social Bot Collection Google Google adsense Google Bots google sheets google shopping Google vs Amazon Collection instagram Investment kickstarter lazada linkedin Marketing non-us citizen Pandas Python for Digital Marketers ring doorbell SEO Shopify Subscription-Business taobao TikTok tmall Twitter Web & Mobile Application WeChat youtube zhihu

Follow Us

  • Facebook
  • Linkedin
  • Youtube
  • Instagram
  • Pinterest
  • Twitter

Product & Services

  • APIs Hub
  • RPA Applications
  • Help center

About

  • About Us
  • Influencer Program
  • Library
  • Privacy & Terms
  • Contact Us
  • About Us
  • Influencer Program
  • Library
  • Privacy & Terms
  • Contact Us
  • Facebook
  • Linkedin
  • Youtube
  • Instagram
  • Pinterest
  • Twitter
Copyright © All rights reserved | Easy2Digital
Go to mobile version