Site icon EASY2DIGITAL

Chapter 52 – Utilise Google Cloud Platform to Implement Python Script Automation with Cloud Function and Cloud Scheduler

Louis Lu

This chapter is relevant to coaching on how to automate your Python applications with the Google cloud platform. Compared to local device crontab and cron jobs, Cloud platforms like Google can allow your application 24 hours and 7 days stand-by and run by triggering preset events. You need to either keep the laptop running for 24 hours or sit beside the device to monitor. I will take the trending topic bot for example and walk you through all components and script codings.

Table of Contents: Use Google Cloud Platform to Automate Web Applications

Google Cloud Platform – How does it work and why do we need it?

Basically, the Google Cloud Platform allows us to deploy a Python script to run automatically based on the schedule and workflow settings. Depending on the work volume, it can either be free or not. And even if you need to pay for some excess volume, it’s the pay-as-you-go model.

Python automation has great business and marketing value. It’s because it makes it possible to real-time optimize the advertising campaign, update the dashboard to grab intelligent data, and refresh the latest trending content automatically. Previously I tried to write some articles regarding APIs, price trackers, and trending topics. If you are interested in some application scenarios, please feel free to check them out

Meta Ads API – Break Through Platform Ceiling and Implement Practical Dynamic Content and Targeting Ads Strategies

Chapter 47: Product Price Trackers of Google Shopping, Walmart, HomeDepot Product Using Easy2Digital APIs

Chapter 51 – Trending News Bot with Twitter and Easy2Digital News APIs

Sign up for a Google Cloud Account and Create a Billing Account

Compared to the Google APIs console account, we need to add a credit card to sign up for a Google cloud account, although both are in the same interface. It’s because Google needs us to enable the billing account for the Cloud function, App engine, or computer engine. If your usage exceeds the free tier, GCP would automatically charge you based on the actual volume. And as well as activating Google APIs, we need to create a project as well using the account with a billing account.

Cloud Functions provides a perpetual free tier for compute-time resources, which includes an allocation of both GB-seconds and GHz-seconds. In addition to the 2 million invocations, the free tier provides 400,000 GB seconds, 200,000 GHz seconds of computing time, and 5GB of Internet egress traffic per month.

The free tier is measured as a dollar equivalent to the Tier 1 pricing. The equivalent dollar amount is provided whether you run your functions in a region with Tier 1 pricing or Tier 2 pricing or both. However, the deduction of usage of the free tier will be based on whether the region in which the function is running is in Tier 1 or Tier 2

Google Cloud Platform – Configure, Deploy and Set up Google Cloud Function

The most complex step is the Python script you are going to upload to GCP, and in terms of the function, deployment is not difficult. There are basically 4 steps.

Create a Function

Here we need to select the server we’re going to store our application and scripts and the methods of triggers. There are several options that we could select based on the project’s needs. In terms of operational automation and marketing automation, Cloud Pub/Sub can be the best one. It’s because it’s much easier to deploy with all user authentication done internally, so the credentials are not necessary.

Set up Runtime Environment Variables in the Function Section

In a Python script, it might include and involve some sensitive data and numbers, such as the password, etc. This section called runtime environment variables gives people to create some variables that have sensitive data value instead of putting them onto the script directly.

Upload the Python Script and Requirement TXT

In the Cloud function script interface, Google sets to start with a def with two arguments. It’s mandatory whether you use them or not. If you don’t have any arguments, just leave them both there as what it is.

The other thing you need to be aware of is the entry point name must be the same as your function name. For example, here I name it the trendingNews2022, and the def function name is also the same.

Then, we need to tell Google what Python modules the script requires. So we can add the module and version bylines in the requirement.txt.

Deploy and Test the Script

When things are ready, we can click the deploy button, and Google would take some time to process. When you see a green light turning on on the left-hand side of your function. That means the script doesn’t have outstanding issues at the moment.

However, the green light doesn’t mean that your automation is working properly and correctly. We need to test the function and check the logs and see if there are some bugs or something else before launching.

Google Cloud Platform – Set up Cloud Scheduler Connecting with the Specific Cloud Function Topics

We have finished the most difficult part, but it seems it’s not that difficult in fact. Here we need to create a cloud schedule and connect the function topic we just created.

The thing we need to do here is to set up the frequency using the cron methodology. I’m not going to go into details in this section as I have another article relevant to this topic. If you are interested, please check out this.

Chapter 18: Utilize Macbook Crontab to Automate Running Amazon Competitor Price Tracker and Updating P&L Calculator and Product Market Value

Full Python Application Scripts that can be enhanced with the automation capability

If you are interested in the full script of Chapter 52 – Utilise Google Cloud Platform to Implement Python Script Automation with Cloud Function and Cloud Scheduler, please subscribe to our newsletter by adding the message “Chapter 53”. We would send you the script immediately to your mailbox.

I hope you enjoy reading Chapter 52 – Utilise Google Cloud Platform to Implement Python Script Automation with Cloud Function and Cloud Scheduler. If you did, please support us by doing one of the things listed below, because it always helps out our channel.

FAQ:

Q1: What is Google Cloud Function?

A: Google Cloud Function is a serverless execution environment for building and connecting cloud services. It allows developers to write and deploy event-driven functions that automatically respond to events triggered by cloud infrastructure and services.

Q2: Why should I use Google Cloud Function?

A: Google Cloud Function offers several benefits including scalability, automatic scaling, cost-efficiency, and ease of use. It allows you to focus on writing code without worrying about infrastructure management.

Q3: How can I deploy a Google Cloud Function?

A: You can deploy a Google Cloud Function using the Cloud Console, the gcloud command-line tool, or the Cloud Functions API. These deployment methods provide flexibility and ease of use.

Q4: What programming languages are supported by Google Cloud Function?

A: Google Cloud Function supports several programming languages including Node.js, Python, Go, Java, and .NET.

Q5: Can I automate the execution of Google Cloud Functions?

A: Yes, Google Cloud Functions can be triggered by various events such as changes in Google Cloud Storage, Firebase Realtime Database updates, Pub/Sub messages, HTTP requests, and more. You can configure your functions to automatically execute in response to these events.

Q6: What is the pricing model for Google Cloud Function?

A: Google Cloud Function offers a pay-as-you-go pricing model, where you are only charged for the resources used during function execution. The pricing is based on factors such as the number of function invocations, execution time, and memory usage.

Q7: Can I monitor the performance of my Google Cloud Functions?

A: Yes, Google Cloud Function provides monitoring and logging capabilities. You can view logs, set up alerts, and monitor the performance of your functions using the Cloud Console or Stackdriver Monitoring.

Q8: Is there any limit on the execution time of Google Cloud Functions?

A: Yes, currently the maximum execution time for a single Google Cloud Function is 9 minutes.

Q9: Can I use third-party libraries or dependencies in my Google Cloud Functions?

A: Yes, you can include third-party libraries or dependencies in your functions by specifying them in your function’s deployment configuration.

Q10: Is there a limit on the number of Google Cloud Functions I can deploy?

A: There is no hard limit on the number of Google Cloud Functions you can deploy. However, there are certain limits on concurrent invocations, deployment size, and resource allocation, which you should consider while designing your application.

Google API Endpoint Recommendation

Google Shopping Product Scraper API

Price: US$18

Google Shopping SERP scraper crawls the product information from Google Shopping channel. API allows to filter by platform country domain, user location, language. Users can scrape the product information using a keyword query. The scraped dataset include product name, pricing, shipping fees, brand name, product page URL etc.

More API options from the Google collection. 

SAVE UP TO 50% and EXPLORE MORE!
Exit mobile version