Skip to content

Domain 3: Deployment and Orchestration of ML and AI Workflows

Question: Orchestrating CI/CD for ML Workloads

Category: MLA – Deployment and Orchestration of ML and AI Workflows

Review Bucket:

  • [x] Bucket A (Term Mapping)
  • [ ] Bucket B (Missed Constraint)
  • [ ] Bucket C (Knowledge Gap)

A manufacturing company wants to develop a machine learning (ML) model for predicting equipment failures and optimizing maintenance schedules. The company has a team of data scientists who have developed custom Python scripts for data preprocessing, feature engineering, and model training. The company wants to leverage AWS services to build, train, and deploy its ML models in a scalable and automated manner.

Which AWS service should the company use to orchestrate continuous integration and continuous deployment (CI/CD) pipeline?

  • A) AWS CodePipeline
  • B) Amazon CodeGuru Reviewer
  • C) AWS CodeDeploy
  • D) AWS CodeBuild
Click to reveal Answer & Explanation

Correct Answer: A

Correct Option Analysis

  • A is Correct: AWS CodePipeline is a fully managed continuous integration and continuous delivery (CI/CD) service. It is designed to automate the build, test, and deploy phases of your release process. This makes it the correct "orchestration" tool, as it can string together other services (like CodeBuild and CodeDeploy) into a single automated workflow for the ML models.

Distractor Analysis

  • B is Incorrect: Amazon CodeGuru Reviewer is an automated code review service that uses machine learning to identify critical issues, security vulnerabilities, and hard-to-find bugs in your code. It is not used to orchestrate pipelines but rather to analyze code quality within a build stage.
  • C is Incorrect: AWS CodeDeploy is a deployment service that automates code deployments to any instance (EC2, Lambda, on-premises). While it is often triggered by a pipeline, it is just one component of the CI/CD process, not the overall orchestration engine that coordinates the different stages.
  • D is Incorrect: AWS CodeBuild is a fully managed build service that compiles source code, runs tests, and produces software packages. Like CodeDeploy, it is a critical part of a pipeline but serves as a specific build stage rather than the orchestrator that ties stages (build -> deploy) together.

Key Takeaway: Understand the distinct roles of AWS Developer/Tooling services. CodePipeline is the orchestrator that integrates services like CodeBuild (Build) and CodeDeploy (Deploy). If the question asks for a service to "orchestrate" or "automate steps," CodePipeline is usually the right choice.

self-notes

Code pipeline = CI/CD full pipeline Code Deploy = Deploy to compute resources (EC2)

Question: SageMaker Pipelines Decorators – @step vs @remote

Category: MLA – Deployment and Orchestration of ML and AI Workflows
Review Bucket: [ ] Bucket A (Term Mapping) | [ ] Bucket B (Missed Constraint) | [x] Bucket C (Knowledge Gap)

A Machine Learning Engineer is working on an Amazon SageMaker Pipelines pipeline to build and deploy a model. The pipeline was provisioned using SageMaker::CreatePipeline and is tracked through SageMaker Experiments for reproducibility. The pipeline includes data processing, model training, and model deployment steps. The ML Engineer needs to incorporate a custom data validation step to ensure the quality of the input data before proceeding with model training.

Which approach should the ML Engineer take to add the custom data validation step to the existing SageMaker Pipelines pipeline?

  • A) Create a custom Python function for data validation and use the @step decorator to integrate it into the pipeline. Add the decorated function as a step in the pipeline definition.
  • B) Create a custom data validation script and use the TransformStep definition to integrate it into the pipeline.
  • C) Create a custom AWS Glue job that performs the data validation step, and use the GlueJobStep definition to add the Glue job as a step in the pipeline.
  • D) Create a custom Python function for data validation and use the @remote decorator to run it remotely. Add the decorated function as a step in the pipeline definition.
Click to reveal Answer & Explanation

Correct Answer: A

Correct Option Analysis

  • A is Correct: The @step decorator in the SageMaker Python SDK converts a standard Python function directly into a step within a SageMaker Pipeline DAG. This provides a lightweight, pythonic way to incorporate custom data validation or processing steps without explicitly managing container specifications or external orchestrators.

Distractor Analysis

  • B is Incorrect: TransformStep is used to run batch transform jobs on datasets to generate offline predictions, not for running custom data validation code.
  • C is Incorrect: While AWS Glue can perform data validation, bringing in external Glue resources adds unnecessary operational complexity compared to native Python function decoration via @step.
  • D is Incorrect: The @remote decorator executes a Python function as a standalone remote SageMaker Training Job. It is used for running code remotely from local notebooks, not for defining steps inside a SageMaker Pipelines DAG.

Key SDK Distinction: Use @step to turn a Python function into a SageMaker Pipeline Step. Use @remote to execute a Python function as a standalone Remote SageMaker Training Job.

self-notes
Step Primitive / Class How It Is Defined Best Used For
@step Decorator Native Python function with @step annotation. Quick, pythonic custom data validation, light transformation, or evaluation logic directly in a pipeline.
ProcessingStep Uses a ScriptProcessor or SKLearnProcessor with an S3 input/output config. Heavy-duty, containerized ETL data preprocessing and feature engineering (e.g., Spark, Pandas, Scikit-Learn).
LambdaStep References an AWS Lambda ARN or inline function code. Lightweight, serverless operational triggers (e.g., sending a Slack notification, updating a database record).
ConditionStep Evaluates a logical expression (e.g., F1 > 0.85). Branching pipeline execution (e.g., if model metrics exceed a threshold, register the model; otherwise, fail).
RegisterModel References a ModelPackageGroup. Automatically publishing a trained model artifact to the SageMaker Model Registry for approval.

Question: SageMaker Production Variants for Multi-Model A/B Testing

Category: MLA – Deployment and Orchestration of ML and AI Workflows
Review Bucket: [ ] Bucket A (Term Mapping) | [x] Bucket B (Missed Constraint) | [ ] Bucket C (Knowledge Gap)

A logistics company uses Amazon SageMaker AI to predict delivery delays in real time, while Amazon DynamoDB maintains current shipment information. Recent performance evaluations revealed a decline in prediction accuracy, prompting the development of three updated models. The company wants to perform A/B testing to the updated models and comparing accuracy before selecting a replacement. The solution must minimize infrastructure management.

Which approach will satisfy these requirements while minimizing operational effort?

  • A) Create three separate SageMaker AI endpoints for the updated models. Use Amazon API Gateway deployment stages and canary settings to control the percentage of incoming requests.
  • B) Configure a single SageMaker AI endpoint with three production variants, one for each updated model. Specify the traffic distribution for each variant and adjust the weights based on the evaluation results.
  • C) Configure SageMaker AI shadow tests to evaluate each updated model against the existing production model. Mirror selected inference requests and compare the recorded predictions to determine model accuracy.
  • D) Update the existing SageMaker AI endpoint with each revised model using deployment guardrails. Configure canary traffic shifting to evaluate prediction accuracy during each deployment.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

  • B is Correct: SageMaker natively supports A/B testing by configuring multiple Production Variants on a single endpoint. You specify traffic weights (InitialVariantWeight) to split live incoming requests across all three updated models without deploying separate endpoints or managing external routing logic, minimizing operational management.

Distractor Analysis

  • A is Incorrect: Managing three separate endpoints and configuring custom routing via API Gateway introduces heavy operational overhead and violates the constraint to minimize infrastructure management.
  • C is Incorrect: Shadow testing duplicates requests asynchronously so candidate responses are dropped rather than returned to callers. The prompt explicitly requests A/B testing across the updated models.
  • D is Incorrect: Deployment guardrails with canary traffic shifting is a gradual rollout strategy for replacing an old model version with a new one, not a multi-candidate A/B test.

Key Takeaway: For A/B testing multiple candidate models with minimal infrastructure overhead, use multiple Production Variants on a single SageMaker Endpoint.

self-notes

Auto distribute traffic for multiple model version in single endpoint

Shadow testing is wrong coz question require: allocate portions of incoming production traffic across the three

Question: Selecting a cost-effective, fully managed hosting for a 2 GB custom ML model with variable traffic

Category: MLA – Deployment and Orchestration of ML and AI Workflows

Review Bucket:

  • [ ] Bucket A (Term Mapping)
  • [x] Bucket B (Missed Constraint)
  • [ ] Bucket C (Knowledge Gap)

A logistics company relies on a 2 GB custom machine learning model hosted in an on-premises environment to forecast delivery times. The company needs to migrate this model to AWS to lower operational overhead and use a fully managed service. The model experiences unpredictable traffic with peak loads during business hours and low activity at night and on weekends. The solution must scale automatically based on demand and be cost-effective during idle times.

Which AWS service is the most suitable for these requirements?

  • A) Use AWS Lambda to host the model, loading it from Amazon S3 on each invocation.
  • B) Deploy the model as a serverless SageMaker AI endpoint with Provisioned Concurrency.
  • C) Deploy the model as a serverless SageMaker AI endpoint without Provisioned Concurrency.
  • D) Use Amazon SageMaker AI real-time endpoints with auto-scaling enabled.
Click to reveal Answer & Explanation

Correct Answer: C

Correct Option Analysis

  • C is Correct: A SageMaker AI serverless endpoint without Provisioned Concurrency is fully managed, automatically scales with demand, and can scale to zero when there is no traffic. This makes it highly cost-effective for workloads with peak business-hour traffic and low or no usage at night and on weekends. It also supports hosting larger models like the 2 GB model described.

Distractor Analysis

  • A is Incorrect: AWS Lambda is not suitable for hosting a 2 GB model. Lambda has deployment package size limits that are far smaller, and loading a large model from Amazon S3 on every invocation introduces high latency and cost.
  • B is Incorrect: Provisioned Concurrency on a serverless endpoint keeps capacity warm to reduce cold starts, but it also incurs costs even when there is no traffic. This conflicts with the requirement to be cost-effective during idle times.
  • D is Incorrect: SageMaker AI real-time endpoints with auto-scaling can scale based on demand, but they do not scale to zero. An endpoint typically requires at least one instance running, which continues to incur charges during periods of low or no activity.

Key Takeaway: For intermittent ML inference workloads requiring automatic scaling and cost-effectiveness during idle periods, choose SageMaker AI Serverless Inference without Provisioned Concurrency. Provisioned concurrency is useful for low-latency, predictable workloads but is not cost-effective for idle periods.

self-notes

real-time endpoint with scaling => doesn't scale down during downtime vs serverless

Practice Question: Event-Driven Preprocessing Pipeline Architecture

Category: MLA – Deployment and Orchestration of ML and AI Workflows
Review Bucket:

  • [X] Bucket A (Term Mapping)
  • [ ] Bucket B (Missed Constraint)
  • [ ] Bucket C (Knowledge Gap)

A company is developing a sentiment analysis model that processes text data from customer reviews. The company collects and stores these reviews in Amazon S3 buckets. The data preprocessing pipeline involves text cleaning, tokenization, and embedding generation using machine learning frameworks such as TensorFlow and Hugging Face’s Transformers. The processed data is then used for training and fine-tuning the model.

Which set of actions will accomplish these tasks with the MINIMUM management effort? (Select THREE.)

  • A) Create an AWS Lambda function to trigger a SageMaker Processing Job when new data is uploaded to the S3 bucket.
  • B) Utilize AWS Batch to manage and execute the SageMaker Processing Jobs for data preprocessing.
  • C) Use AWS Glue to preprocess and clean the text data and store the output in Amazon S3.
  • D) Create a SageMaker Processing Job to preprocess the text data, generate embeddings, and store the output in Amazon S3.
  • E) Deploy a custom Amazon ECS cluster on AWS Fargate to run the preprocessing scripts.
  • F) Use Amazon S3 Event Notifications to trigger an AWS Step Functions workflow for data preprocessing.
Click to reveal Answer & Explanation

Correct Answers: A, D, and F

Correct Option Analysis

  • A is Correct: An AWS Lambda function serves as an event-driven serverless worker that executes light orchestration code (calling the CreateProcessingJob API) without requiring dedicated server infrastructure.
  • D is Correct: SageMaker Processing Jobs provide fully managed, ephemeral compute pre-configured to run heavy deep learning frameworks (TensorFlow, PyTorch, Hugging Face Transformers) for tokenization and embedding generation, terminating automatically upon completion.
  • F is Correct: Amazon S3 Event Notifications paired with AWS Step Functions provides a serverless, decoupled orchestration workflow that automatically triggers the data preprocessing pipeline in real time whenever new review files are uploaded to S3.

Distractor Analysis

  • B is Incorrect: SageMaker Processing inherently manages its own underlying ephemeral compute instances and scheduling. Wrapping SageMaker Processing inside AWS Batch adds redundant operational overhead and architectural complexity.
  • C is Incorrect: AWS Glue is optimized for tabular ETL and PySpark data transformations; heavy deep learning framework workloads (TensorFlow/Transformers) for dense embedding generation belong in SageMaker Processing Jobs.
  • E is Incorrect: Deploying a custom Amazon ECS cluster on Fargate requires building, configuring, patching, and maintaining custom container task definitions, violating the requirement for MINIMUM management effort.

Key Decision Rules: * Heavy ML Preprocessing (TensorFlow / Hugging Face Embeddings): Use SageMaker Processing Jobs. * Event-Driven Serverless Pipeline Triggering: Use S3 Event Notifications \(\rightarrow\) Step Functions \(\rightarrow\) Lambda. * Avoid Redundant Containers/Orchestrators: Reject custom ECS clusters or AWS Batch wrappers around native SageMaker jobs.

self-notes

AWS batch -> only for AWS resources job like EC2/ECS

Practice Question: Deploying R Models on SageMaker

Category: MLA – Deployment and Orchestration of ML and AI Workflows
Review Bucket: [ ] Bucket A (Term Mapping) | [x] Bucket B (Missed Constraint) | [ ] Bucket C (Knowledge Gap)

A data scientist at a healthcare company has developed multiple ML models using R and wants to deploy on Amazon SageMaker AI with minimal operational overhead. The team has already eliminated the ImportModel API operation because it does not natively support R-based model artifacts without an appropriate container runtime. Amazon EKS was also ruled out due to its high cluster and node management demands, which exceed the team’s overhead limits, even though it supports containerized R workloads. The solution must operate within the SageMaker Processing job execution role, which controls access to container images and deployment resources.

Which solution should the data scientist choose?

  • A) Manually install R and all necessary dependencies on an Amazon EC2 instance, then use SageMaker’s local mode to deploy the models.
  • B) Use the SageMaker Python SDK to convert the R models into Python models and deploy them using SageMaker’s built-in Python containers.
  • C) Create a custom R Dockerfile, build the Docker image locally, push it to Amazon ECR, and use it to create a SageMaker endpoint.
  • D) Use SageMaker’s built-in R kernel to develop and deploy the models directly from a Jupyter notebook.
Click to reveal Answer & Explanation

Correct Answer: C

Why Option D is INCORRECT

  • Notebook Kernel \(\neq\) Endpoint Runtime: A SageMaker Jupyter notebook running an R kernel provides an interactive compute environment for exploratory data analysis and model training only. It does not provide an execution container or web server for production model serving.
  • No Built-in R Inference Containers: AWS SageMaker provides pre-built Deep Learning Containers (DLCs) for Python-based frameworks (PyTorch, TensorFlow, Scikit-learn, Hugging Face, vLLM). AWS does not offer pre-built SageMaker managed inference containers for R. Calling .deploy() from an R notebook cannot create a SageMaker endpoint without pointing to a container image that implements the SageMaker HTTP contract.
  • SageMaker Endpoint Contract: To serve predictions on a SageMaker endpoint, the underlying container must host a web server listening on Port 8080 that handles POST /invocations and GET /ping requests (typically using the R plumber package). An R Jupyter notebook kernel cannot fulfill this server contract.

Analysis of Other Options

  • C is Correct: Because SageMaker lacks native built-in inference containers for R, deploying an R model onto a SageMaker Real-Time Endpoint requires a Bring Your Own Container (BYOC) approach. You package the R model artifacts and an HTTP web server (e.g., using the R plumber library) into a custom Docker image, push it to Amazon ECR, and register the endpoint.
  • A is Incorrect: Installing R manually on an EC2 instance and using SageMaker "local mode" is meant for local testing and debugging, not a managed, scalable production endpoint solution.
  • B is Incorrect: The SageMaker Python SDK cannot automatically convert native R model objects (.rds / .RData files) into Python models.

Key Decision Rule: * SageMaker Notebook R Kernel: For training & EDA only. * SageMaker Endpoint Deployment for R: Requires Bring Your Own Container (BYOC) via Amazon ECR (R + plumber package on Port 8080)

self-notes

R Kernel is not enough to host model

Practice Question: Serverless Preprocessing & Orchestration Pipeline

Category: MLA – Deployment and Orchestration of ML and AI Workflows
Review Bucket: [ ] Bucket A (Term Mapping) | [ ] Bucket B (Missed Constraint) | [ ] Bucket C (Knowledge Gap)

An ML Engineer is developing a multi-step pipeline that initially utilizes a computer vision ML model to extract text from scanned images of customer feedback forms. The resulting text data is stored in Amazon S3 and must undergo preparation steps, such as tokenization, stop-word removal, and vectorization, before being used for downstream model training using TensorFlow and PyTorch.

The engineer requires a scalable and serverless solution to manage these preprocessing and training tasks. Furthermore, the engineer plans to implement the text processing logic using the Natural Language Toolkit (NLTK) library and needs an environment that can seamlessly execute these Python-based scripts without managing or maintaining underlying infrastructure.

What set of actions should the engineer configure to accomplish this with the absolute minimum administrative burden? (Select THREE.)

  • A) Use Amazon SageMaker Processing jobs to handle the data preprocessing steps like tokenization and vectorization.
  • B) Use AWS Lambda to preprocess the text data and store the results back into S3.
  • C) Enable S3 Event Notifications to trigger the preprocessing workflows upon new data uploads.
  • D) Deploy an AWS Lambda function to monitor and detect newly uploaded files in Amazon S3.
  • E) Use Amazon Managed Workflows for Apache Airflow (Amazon MWAA) to manage and orchestrate the preprocessing and model training workflows.
  • F) Use AWS Step Functions to orchestrate the preprocessing and model training workflows in a serverless manner.
Click to reveal Answer & Explanation

Correct Answers: A, C, and F

Correct Option Analysis

  • A is Correct: SageMaker Processing jobs provide fully managed, ephemeral compute instances pre-configured for Python-based ML tasks. It natively supports running custom NLTK, TensorFlow, and PyTorch scripts in custom or pre-built containers without execution time limits or infrastructure maintenance overhead.
  • C is Correct: Amazon S3 Event Notifications provide a native, event-driven trigger mechanism that immediately fires when a new file lands in S3 without requiring polling loops or custom monitoring code.
  • F is Correct: AWS Step Functions is a 100% serverless, visual workflow orchestrator that manages multi-step execution flows, retries, and state transitions between S3, SageMaker Processing, and SageMaker Training with zero infrastructure administration.

Distractor Analysis

  • B is Incorrect: Running NLTK tokenization, corpus downloads, and vectorization across large text datasets inside AWS Lambda risks hitting Lambda's strict 15-minute execution timeout, memory limits, and deployment package/layer size limits.
  • D is Incorrect: Deploying a custom Lambda function to "monitor and detect" S3 file uploads is an anti-pattern that introduces unnecessary polling code and maintenance compared to native S3 Event Notifications.
  • E is Incorrect: Amazon MWAA (Managed Apache Airflow) requires configuring and managing Airflow environments, worker instance sizing, plugin management, and DAG directories, violating the absolute minimum administrative burden requirement when compared to fully serverless AWS Step Functions.

Key Decision Rules: * Serverless Pipeline Orchestration (Zero Admin): AWS Step Functions (not MWAA/Airflow). * ML Preprocessing with Heavy Libraries (NLTK/Vectorization/PyTorch): SageMaker Processing Jobs (prevents Lambda 15-min timeouts). * Event Detection on Storage: S3 Event Notifications (native push trigger, not Lambda polling).

Practice Question: Automated End-to-End ML Pipeline Architecture

Category: MLA – Deployment and Orchestration of ML and AI Workflows
Review Bucket: [ ] Bucket A (Term Mapping) | [x] Bucket B (Missed Constraint) | [ ] Bucket C (Knowledge Gap)

An ML engineer is developing a natural language processing (NLP) model. The engineer must preprocess a large text dataset from multiple sources and create a scalable and automated pipeline that handles the machine learning workflow, from data ingestion to model deployment, while minimizing manual intervention.

Which combination of AWS services would be most suitable for this requirement? (Select TWO.)

  • A) Set up Amazon Kinesis Data Streams to ingest the text data from various sources.
  • B) Use AWS Glue to extract, transform, and load the text data into Amazon S3.
  • C) Use Amazon Managed Workflows for Apache Airflow (MWAA) to orchestrate data workflows and pipeline management.
  • D) Use Amazon SageMaker Pipelines to automate the entire ML workflow, including data preprocessing, model training, and deployment.
  • E) Use Amazon SageMaker Data Wrangler to clean and preprocess the data.
Click to reveal Answer & Explanation

Correct Answers: B and D

Correct Option Analysis

  • B is Correct: AWS Glue is a serverless, fully managed ETL service ideal for extracting large unstructured/text datasets from heterogeneous data sources, transforming/cleaning the data at scale using PySpark, and loading the output into Amazon S3 as an ingestion target.
  • D is Correct: Amazon SageMaker Pipelines is the native, purpose-built CI/CD and MLOps orchestration service designed specifically to automate the entire end-to-end ML lifecycle (data preprocessing, training, model evaluation, model registration, and endpoint deployment) with zero manual intervention.

Distractor Analysis

  • A is Incorrect: Amazon Kinesis Data Streams is designed for real-time streaming ingestion of high-velocity time-series data (e.g., IoT telemetry, clickstreams). It is not suited for batch ETL and text preprocessing of large static datasets from diverse database sources.
  • C is Incorrect: While Amazon MWAA can orchestrate general data workflows, using it instead of native SageMaker Pipelines introduces unnecessary operational management (managing Airflow environments, worker scaling, and custom DAG code) when SageMaker Pipelines natively automates ML pipelines.
  • E is Incorrect: SageMaker Data Wrangler is primarily an interactive, visual GUI tool optimized for quick exploratory data preparation on tabular data. It is not designed to serve as an automated, programmatic pipeline for large-scale text ETL across multi-source datasets.

Key Decision Rules: * Batch ETL / Extraction from Multiple Sources \(\rightarrow\) Amazon S3: Use AWS Glue. * End-to-End Native ML Pipeline Automation: Use Amazon SageMaker Pipelines. * Avoid Extra Cluster/DAG Overhead: Choose SageMaker Pipelines over MWAA (Airflow) when orchestrating ML-centric workflows natively in SageMaker.

self-notes

Text processing, question didn't mention real-time stream -> Kinesis is wrong