Chapter 39 – Alibaba Bot for Scraping Product Price, MOQ, and Supplier Profiles
Stable and sustainable contribution from supply sides to any eCommerce business is a vital thing to succeed. It’s a tough task of course, so on and off identifying the potential suppliers is a must-have to-do-item. It’s not just for diversifying the suppliers, but also it’s a tip to optimize the landed cost and COGS. The question is how to save time to complete this task. Alibaba bot is the answer for you.
In this Chapter, I’ll walk you through how to write an Alibaba bot to help you scrape the potential suppliers and product prices with MOQ. And you can use this bot to automate collecting supplier information and updating the landed cost and COGS data. It can visualize a profit margin at the sourcing stage.
Table of Contents: Alibaba Bot
- Scrape suppliers landed cost and COGS
- Alibaba Search URL Parameters
- SERP Product Data
- Product Page Data
- Full Python Script of Alibaba Bot
Research Suppliers, Landed Costs, and COGS
Alibaba, which is a familiar name in the B2B trading sector, lists thousands of supplier information from diversified industries. Basically, it’s very difficult if you can’t find some product supplies from this platform.
As eCommerce sellers, we often walk around among different suppliers to find the most optimal product sourcing cost or landed cost. What’s more, we like to find verified suppliers with more added value, such as free samples, logistic service, and lower min. MOQ requirement.
It’s because this information and data are relevant to product R&D, sustainable supply side, and profit margin. When talking about the profit margin, the landed cost or cost of good sale is highly related to the supply you partner with. And this is one of the primary reasons why an Alibaba bot is necessary. The bot can automatically scrape specific data using some filtering parameters, and consolidate and update intelligent data in your dashboard.
For more details regarding product sourcing, please check out this article
OEM, Private Label, White Label, or Dropshipping – Tips for Product Sourcing for the eCommerce Store
If you like to learn the cost-wise in the eCommerce, please check out these two articles
Selling Well But Be Losing Money? 7 Variable Costs Are Impacting Your Online Store Profit Margin
Chapter 33: Create an eCommerce Profit & Loss Calculator Using OOP
Alibaba Search URL Parameters
I’ll use Beautifulsoup and selenium to scrape the data. First thing first, we would use the search function on the platform. Below is the search URL and there are some parameters I think it’s always very useful.
URL = driver.get(
'https://www.alibaba.com/trade/search?IndexArea=product_en&SearchText=' + query + '&page=' + str(page) + '&Country=CN&assessment_company=ASS¶m_order=CNTRY-CN&f0=y'
)
IndexArea = product_en: It’s the international side of product and supplier information in English- SearchText: Here add the keyword
- Page: The pagination. Usually, I would set 3 pages
- Country: Select the country of the suppliers
assessment_company=ASS: It represents only verified suppliers
SERP Product Data
There are mainly two types of products in the SERP. One type is the business-as-usual product information, and the other type is the promotion data. For the element locators, the pricing elements have different class values because of this.
For the SERP product data, normally we like to collect the product price and MOQ data that can automatically plugin into your dashboard. So you can immediately learn how many suppliers are selling the SKU and a rough profit margin at the early stage. My experience can tell it’s very useful.
Last but not least, some data is also scrapable such as photos, product names, product page URLs, etc.
Product Page Data
BeautifulSoup doesn’t work on the Alibaba product page. So software like selenium is only to scrape the product page data. For me, I would like to fetch the free sample, product variant, and company profile link data. As it’s helpful for the team to follow up on the conversation with the potential ones.
By using selenium to convert the fetched session ID into the textual content, there are one method and three values I recommend you use.
- Method:
get_attribute - Scrape the HTML elements:
get_attribute(“innerHTML) - Fetch the string content:
get_attribute(“textContent”) - Grab the Href link:
get_attribute(“href”)
From the SERP scraping, we have got the product page URL and you can continue to scrape the specific product page data using those URLs you scrape from the SERP.
Full Python Script of Alibaba Bot
If you are interested in the full script of Alibaba Bot for Scraping Product Price, MOQ, and Supplier Profiles, please subscribe to our newsletter by adding the message “Chapter 39”. We would send you the script immediately to your mailbox.
I hope you enjoy reading Chapter 39 – Alibaba Bot for Scraping Product Price, MOQ, and Supplier Profiles. If you did, please support us by doing one of the things listed below, because it always helps out our channel.
- Support and donate to our channel through PayPal (paypal.me/Easy2digital)
- Subscribe to my channel and turn on the notification bell Easy2Digital Youtube channel.
- Follow and like my page Easy2Digital Facebook page
- Share the article on your social network with the hashtag #easy2digital
- Buy products with Easy2Digital 10% OFF Discount code (Easy2DigitalNewBuyers2021)
- You sign up for our weekly newsletter to receive Easy2Digital latest articles, videos, and discount codes
- Subscribe to our monthly membership through Patreon to enjoy exclusive benefits (www.patreon.com/louisludigital)
It’s awesome folk, thank you for sharing!