In this Python Tutorial, I’ll walk you through how to write an Alibaba bot to help you scrape the potential suppliers and product price with MOQ. And you can use this bot to automate collecting supplier information and updating the landed cost and COGS data. It can visualize a profit margin at the souricing stage.
Python Module and Libraries: Selenium, BeautifulSoup, Pandas
- Scrape suppliers, landed cost and COGS
- Alibaba Search URL Parameters
- SERP Product Data
- Product Page Data
- Full Python Script of Alibaba Bot
Research Suppliers, Landed Cost and COGS
Alibaba, which is a familiar name in the B2B trading sector, lists thousands of supplier information from diversified industries. Basically, it’s very difficult if you can’t find some product supplies from this platform.
As an eCommerce seller, we often walk around among different suppliers to find the most optimal product sourcing cost, or landed cost. What’s more, we like to find verified suppliers with more added value, such as free samples, logistic service, and lower min. MOQ requirement.
It’s because this information and data are relevant to the product R&D, sustainable supply side, and profit margin. When talking about the profit margin, landed cost or cost of good sale is highly related to the supply you partner with. And this is one of the primary reasons why an Alibaba bot is necessary. The bot can automatically scrape the specific data using some filtering parameters, and consolidate and update intelligent data in your dashboard.
For more details regarding product sourcing, please check out this article
If you like to learn the cost-wise in the eCommerce, please check out these two articles
Alibaba Search URL Parameters
I’ll use Beautifulsoup and selenium to scrape the data. First thing first, we would use the search function on the platform. Below is the search URL and there are some parameters I think it’s always very useful.
URL = driver.get(
'https://www.alibaba.com/trade/search?IndexArea=product_en&SearchText=' + query + '&page=' + str(page) + '&Country=CN&assessment_company=ASS¶m_order=CNTRY-CN&f0=y'
IndexArea = product_en: It’s the international side of product and supplier information in English
- SearchText: Here adds the keyword
- Page: The pagination. Usually I would set 3 pages
- Country: Select the country of the suppliers
assessment_company=ASS: It represents only verified suppliers
SERP Product Data
There are mainly two types of products in the SERP. One type is the business as usual product information, and the other type is the promotion data. For the element locators, the pricing elements have different class values because of this.
For the SERP product data, normally we like to collect the product price and MOQ data that can automatically plugin into your dashboard. So you can immediately learn how many suppliers are selling the SKU and a rough profit margin at the early stage. My experience can tell it’s very useful.
Last but not least, some data is also scrapable such as photos, product name, product page URL, etc.
Product Page Data
BeautifulSoup doesn’t work on the Alibaba product page. So software like selenium is only to scrape the product page data.For me, I would like to fetch the free sample, product variant, company profile link data. As it’s helpful for the team to follow up the conversation with the potential ones.
By using selenium to convert the fetched session ID into the textual content, there are one method and three values I recommend you to use.
- Scrape the html elements:
- Fetch the string content:
- Grab the href link:
From the SERP scraping, we have got the product page URL and you can continue to scrape the specific product page data using those URLs you scrape from the SERP.
Full Python Script of Alibaba Bot
If you are interested in the full script of Alibaba Bot for Scraping Product Price, MOQ and Supplier Profiles, please subscribe to our newsletter by adding the message “Python tutorial 39”. We would send you the script immediately to your mailbox.
I hope you enjoy reading Python Tutorial 39 – Alibaba Bot for Scraping Product Price, MOQ and Supplier Profiles. If you did, please support us by doing one of the things listed below, because it always helps out our channel.
- Support my channel through PayPal (paypal.me/Easy2digital)
- Subscribe to my channel and turn on the notification bell Easy2Digital Youtube channel.
- Follow and like my page Easy2Digital Facebook page
- Share the article to your social network with the hashtag #easy2digital
- Buy products with Easy2Digital 10% OFF Discount code (Easy2DigitalNewBuyers2021)
- You sign up for our weekly newsletter to receive Easy2Digital latest articles, videos, and discount code on Buyfromlo products and digital software
- Subscribe to our monthly membership through Patreon to enjoy exclusive benefits (www.patreon.com/louisludigital)