Skip to content

3.3

Task 3.3: Use automated orchestration tools to set up continuous integration and continuous delivery (CI/CD) pipelines

Knowledge of:

  • Capabilities and quotas for AWS CodePipeline, AWS CodeBuild, and AWS CodeDeploy
  • Automation and integration of data ingestion with orchestration services
  • Version control systems and basic usage (for example, Git)
  • CI/CD principles and how they fit into ML workflows
  • Deployment strategies and rollback actions (for example, blue/green, canary, linear)
  • How code repositories and pipelines work together

Skills in:

  • Configuring and troubleshooting CodeBuild, CodeDeploy, and CodePipeline, including stages
  • Applying continuous deployment flow structures to invoke pipelines (for example, Gitflow, GitHub Flow)
  • Using AWS services to automate orchestration (for example, to deploy ML models, automate model building)
  • Configuring training and inference jobs (for example, by using Amazon EventBridge rules, SageMaker Pipelines, CodePipeline)
  • Creating automated tests in CI/CD pipelines (for example, integration tests, unit tests, end-to-end tests)
  • Building and integrating mechanisms to retrain models

Let's get started with the third task statement from Domain 3, which is to use automated orchestration tools to set up continuous integration and continuous delivery pipelines. This task statement is split into three lessons. The first two steps in the machine learning engineer lifecycle are collecting data and processing the data. After that, you use the data to train and validate the machine learning model. You can then deploy the trained and validated model as a prediction service that other applications can access through APIs. Exploratory data analysis usually requires you to experiment with different models until the best model version is ready for deployment. And this leads to frequent model version deployments and data versioning. Experiment tracking and machine learning training pipeline management are important before your applications can integrate or consume the model. In your code, MLOps is a practice that joins machine learning application development with machine learning system deployment and operations. You can use MLOps to automate and standardize processes across the machine learning lifecycle such as model development, testing, integration, release, and infrastructure management. You deploy machine learning models with the applications and services they use as part of the whole process. Ensure you understand the principles of MLOps, which are version control, automation, continuous testing, and model governance. AWS has MLOps Workload Orchestrator solution, and I added a link for this solution under additional resources. But I wanted to mention that the template gives you the ability to train models, upload your trained models, configure the pipeline orchestration, and monitor the pipeline's operations to increase your agility and efficiency and to help repeat successful processes at scale for both single account deployment and multi-account deployment. In our last lesson, we mentioned version control. After you have your CloudFormation template definition file, you can commit that file to a source code repository such as AWS CodeCommit or GitHub. Then your CI/CD pipeline can be automated to run when a new version of a file is committed to the source code repository. For the exam, dive deeper into using code repositories in deployment environments. Could you use AWS CodePipeline to deploy code maintained in AWS CodeCommit repository to a single Amazon EC2 instance? Yes, your pipeline is initiated when you push a change to the CodeCommit repository, and then the pipeline deploys your changes to an Amazon EC2 instance using AWS CodeDeploy as a deployment service. You can also use CodePipeline to orchestrate each step in your release process. By defining stages in a pipeline, you can retrieve code from a source code repository, build that source code into a releaseable artifact, test the artifact and deploy it to production. AWS Step Functions can also be used to orchestrate workflows and can handle nested workflow logic errors and retries. Ensure you know how to configure and integrate other AWS services into AWS CodePipeline to complete your machine learning pipeline and also understand the set of AWS CI/CD developer tools. I'm gonna stop this lesson here, and in the next lesson, we'll continue talking about task statement 3.3.

Let's continue with task statement 3.3, which is to use automated orchestration tools to set up continuous integration and continuous delivery pipelines. For this exam, also ensure you know how, when, and why to use different deployment methods such as blue/green, canary, immutable, rolling, rolling with additional batch and so on. Here is a question. What are some deployment strategies for serverless? AWS CloudFormation, AWS Serverless Application Model, or AWS SAM, but also all-at-once, blue/green, and canary or linear. One basic deployment strategy in serverless is to update your function code in production, publish a new version, invoke the version directly to run your integration test, and after validating, you can switch the alias that your production uses to point to the new version. AWS CodeDeploy has different deployment strategies such as linear and canary, and you can monitor the deployment, create alarms, and initiate the rollback if needed. It also can define pre-traffic and post-traffic hooks to verify that the newly deployed code is configured correctly and your application works as expected. As you scale your machine learning operations, you can use SageMaker manage workflow services to implement continuous integration and deployment practices for your machine learning lifecycle. With the SageMaker Model Building Pipelines and the SageMaker Pipelines SDK, you can choose and integrate pipeline steps into a solution that automates the building process from data preparation to model deployment. For Kubernetes orchestration and Kubernetes-based architectures, you can install SageMaker Operators on your Kubernetes cluster to create SageMaker jobs using the Kubernetes API and command line Kubernetes tool, such as kubectl. For SageMaker notebook jobs, and to schedule non-interactive batch runs of your Jupyter notebook. You can use the notebook-based workflow service to initiate standalone or regular runs on a defined schedule. SageMaker Notebooks runs Jupyter notebooks and models that can be trained, saved, loaded, and there are also environments or kernels inside the notebooks. You can also use other services such as Airflow workflows and AWS Step Functions to integrate with SageMaker to build your workflows. For Airflow workflows, the SageMaker API exports configurations for creating and managing airflow workflows. And for Step Functions, you can use multi-step machine learning workflows in Python that orchestrate SageMaker infrastructure without provisioning your resources separately. Here's a question. Let's say you have existing infrastructure. How do you select the correct deployment orchestrator in AWS? One option is to use Amazon Managed Workflows for Apache Airflow, which is a managed orchestration service for Airflow, and you can use it to set up and operate data pipelines in AWS and create workflows without having to manage the infrastructure for scalability, availability, and security. You can also use SageMaker Model Building Pipelines and set up SageMaker projects for orchestration using a tool that handles most of the step creation and management for you. What is your solution to optimize functions that act as orchestrators that include idle time in the function and increased costs? One solution is to move the orchestration flow to Step Functions. You can also automate data ingestion with AWS Data Exchange and send the data to Amazon S3. AWS Data Exchange helps to securely exchange file-based datasets in AWS, then you can use the ingested data with the AWS analytics and machine learning services. Here is a question. You are granted access to a SQL server hosted on Amazon RDS to train a model in SageMaker. You need to design an automated workflow. What is your solution? One solution is to use AWS Data Pipeline to copy the data from the SQL server to an Amazon S3 bucket, and then provide the Amazon S3 endpoint within the SageMaker notebook. Here's another question. How can you train a model in SageMaker? One solution is to create a training job that includes the URL for the S3 bucket where the data is stored, the compute resources you want SageMaker to use for the model training, the URL of the S3 bucket where you want to store the output of the job and the Amazon Elastic Container Registry, or Amazon ECR, path where the training code is stored. I'm gonna stop this lesson here, and in the next lesson we'll continue talking about task statement 3.3.

Let's continue with task statement 3.3, which is to use automated orchestration tools to set up continuous integration and continuous delivery pipelines. For the exam, ensure you understand how GitFlow works and how to implement CI/CD with the AWS developer tools. Compiling and testing the code is a continuous integration part. Producing an artifact from your code that is ready to be deployed is a continuous delivery, and deploying the application automatically is the continuous deployment. And when you add automation to the test, it can speed up the whole pipeline and deployment and reduce risks. You can use CloudFormation templates to set up the required role and environment for the main and developed branches, including the CodeCommit repository, your Amazon VPC, Amazon EC2 instance, AWS CodeBuild, AWS CodeDeploy, and AWS CodePipeline. I added a link to a blog that talks about how to use triggers in environments in multiple workflows to implement GitFlow with Amazon CodeCatalyst. For this task statement, also ensure you understand writing and running unit test in your development environment. What services could we use for testing? Dive deeper into AWS SAM, CodePipeline, CloudFormation, and how to automate your test environments. Code quality testing begins when the code is checked out from the source code repository. If you compile your code, your code will be tested to see if it can create a fresh build. Then you run unit tests to configure your CI/CD tool to execute the tests that are in your codebase. How can you test proposed code changes and pull requests before merging? My first thoughts are using AWS CodeCommit, AWS CodeBuild, and AWS Lambda. We can also use services such as AWS CodePipeline for visualizing and creating our pipeline and automatically building and deploying merged code that has met the validation bar for the pull requests. It is important to understand your user data flows and the user interface too. How can you test your serverless applications end-to-end? One solution is to configure tests using AWS Lambda. Another solution is AWS Glue that provides both visual and code-based interfaces to help with data integration, can be integrated with unit test Python scripts for Glue jobs, and can deploy AWS Glue jobs using AWS CodePipeline. Here's a question. You recently migrated your application to AWS and you're using AWS CodePipeline. Your application has a test environment and a production environment and are both configured in the pipeline. You want to ensure that any issues are identified before the deployment for new updates, revisions, or versions. What steps can you take? Well, this is a bit vague, but what I hope you're thinking is to add a test action to the pipeline for both unit and functional testing using AWS CodeBuild to verify the test results passed before moving on to the deployment. You could also use AWS CodeDeploy action to deploy the latest version to a pre-production test environment and set up manual approval actions in the pipeline to have the needed test performed. Then have another CodeDeploy action that deploys the verified code to the production environment. For running load or stress tests, performance benchmarking, and application testing at scale, check out the AWS Prescriptive Guidance for load testing applications. I added a link under additional resources. Let's wrap up this task statement and talk about how to configure training and inference jobs in your CI/CD pipelines. There are additional considerations for the pipeline when you implement continuous integration and continuous deployment processes such as automated re-training, scheduled inference, and similar extensions of the pipeline. Here is one solution. You can develop the initial machine learning model and evaluate its inference performance with Jupyter notebooks. You can integrate the model with runtime Python scripts so that it can be managed and provisioned by a machine learning framework using SageMaker. These Python scripts include predefined SageMaker callback functions and environment variables. Remember, they run inside a SageMaker container that is hosted on an Amazon EC2 instance. For processing jobs, SageMaker provides two options for performing batch-mode model inference. You can use SageMaker processing job or batch transform job. A processing job consists of a Python file that runs inside a SageMaker container. Batch transfer jobs are an extension of the concept of a SageMaker endpoint. At runtime, the jobs import callback functions, which then handle the input/output for reading the data, loading the model, and making the predictions. Then you define your pipeline. With SageMaker, you can use AWS Step Functions or ML Max Framework, which also support other pipeline or state machine systems for the development and deployment of machine learning pipelines. You can define the input and output placeholders for each step of the pipeline with a focus on training, inference, cross-validation, and back-testing. You can create the underlying infrastructure using AWS CloudFormation and include the AWS Step Function state machine instance or Airflow. You can also prepare metadata and data or data locations to fill in concrete values for the input/output placeholders that you define, including the runtime scripts and model hyperparameters. Then you run your pipeline and implement continuous integration and continuous deployment processes, automated re-training, scheduled inference, and similar extensions of the pipeline, and monitor. What can you configure to monitor for any SageMaker model state changes? You can use Amazon EventBridge to monitor status change events in SageMaker. Let's get started with our fifth walkthrough question.