Editor’s Note: Because our bloggers have lots of useful tips, every now and then we update and bring forward a popular post from the past. Today’s post was originally published on August 15, 2019.
In this post, I’ll describe a few takeaways for deploying or submitting machine learning (ML) tasks on Google Cloud Platform (GCP). If you have less experience as a ML engineer, or if you’re a solution architect, you might be in the right place to learn some tips.
What exactly is an ML task? Before building an ML model, you first need to specify what you’re planning to accomplish with the data. Having this in mind helps you to identify what the exact ML tasks are for your use case. Some broad ML tasks could be: data ingestion, feature transformation, supervised learning, semi-supervised learning, unsupervised learning, dimensionality reduction, active learning, reinforcement learning and model prediction. This is too much material to cover in one blog post, so expect another post to cover all those tasks. In the meantime, you can find a general overview on Wikipedia.
8. Orchestrated tasks manually triggered that take any amount of time:
Cloud Build - through Triggers, Cloud Composer(Airflow), Kubeflow
Here, you are implementing an orchestrated workflow that will perform tasks in sequence depending on conditions. Each task may take a different amount of time to finalize. A modern ML workflow / pipeline usually involves running containers in a third container orchestrator environment. On GCP it can be Kubernetes or AI Platform.
Note that we now have ML Engine Custom Containers for training and serving, which is the only option where you can be careless about scalable cluster management and environment (e.g. Java, Python, Go, etc.).