Skip to content

Skill 3.3.3: Configure training and inference jobs

Question Answer & Explanation
### Question 1 - SageMaker Pipeline Training Job Configuration



A machine learning team is building an automated CI/CD pipeline using AWS CodePipeline and Amazon SageMaker Pipelines to train a custom XGBoost model. The training script is stored in an Amazon S3 bucket, and the training data is also located in S3. The pipeline must configure the training job with the least operational overhead and ensure that compute resources are released immediately after the job completes.



Which ONE approach should the engineer use to configure the training job within the pipeline?



- A) Provision an Amazon EC2 instance using a custom CloudFormation script to run the training script and terminate upon completion.


- B) Configure an AWS Lambda function to invoke the CreateTrainingJob API with a custom ECR image and instance configuration.


- C) Use a SageMaker Estimator step in SageMaker Pipelines, specifying the S3 data inputs, instance type, and a built-in or custom training container URI.


- D) Deploy an Amazon ECS Fargate task to run the training script and save the model artifacts back to S3. | Correct Answer: C



Correct Answer Analysis:


- C is Correct: Using a SageMaker Estimator within a SageMaker Pipeline is the native, AWS-recommended approach for configuring training jobs. It automatically provisions the compute instances, mounts the S3 data channels, executes the container, and releases the resources immediately after the job finishes, minimizing operational overhead.



Distractor Analysis:


- A is Incorrect: Provisioning and managing raw EC2 instances introduces significant operational overhead compared to SageMaker's managed training infrastructure.


- B is Incorrect: While invoking CreateTrainingJob via Lambda is possible, it requires writing and maintaining custom orchestration code, which violates the "least operational overhead" constraint when SageMaker Pipelines exists natively.


- D is Incorrect: Using ECS Fargate requires building a custom container orchestration setup for training, data downloading, and artifact uploading, rather than relying on SageMaker's built-in capabilities.



> Key Takeaway: SageMaker Pipelines and Estimators provide a managed, serverless way to configure and run training jobs without managing underlying compute instances. | | ### Question 2 - Asynchronous Inference for Large Payloads



An AI engineering team is automating the deployment of an inference endpoint for a computer vision model. The model generates predictions from large image payloads (up to 100 MB), and processing can take up to 5 minutes. The CI/CD pipeline must configure the inference job to handle these payloads without blocking the client application or dropping requests due to timeout limits.



Which ONE configuration satisfies these requirements?



- A) Configure a SageMaker Real-Time Inference endpoint with multiple instances and Application Auto Scaling.


- B) Configure a SageMaker Asynchronous Inference endpoint with an Amazon S3 bucket for input and output payloads.


- C) Configure a SageMaker Batch Transform job to run on an hourly schedule to process queued images.


- D) Deploy the model as an AWS Lambda function with extended timeout limits and provisioned concurrency.



| Correct Answer: B



Correct Answer Analysis:


- B is Correct: SageMaker Asynchronous Inference endpoints are designed specifically for large payload sizes (up to 1GB) and long processing times (up to 15 minutes). Clients place requests in an S3 queue, and the endpoint processes them asynchronously, returning results to an output S3 bucket without blocking or timing out.



Distractor Analysis:


- A is Incorrect: Real-Time Inference endpoints have a strict 60-second timeout limit per request and a 6MB payload limit for synchronous invocations, which would fail under the 5-minute processing time and 100MB payload constraints.


- C is Incorrect: Batch Transform is for offline, bulk processing of datasets, not for handling individual, on-demand client requests in near real-time.


- D is Incorrect: AWS Lambda has a maximum payload size of 10MB for synchronous invocations and a maximum execution timeout of 15 minutes, but it is not suitable for hosting heavy ML inference workloads natively without SageMaker.



> Key Takeaway: Use SageMaker Asynchronous Inference for payloads up to 1GB or inference requests that take up to 15 minutes to process. | | ### Question 3 - IAM Role for Encrypted Training Jobs



An MLOps engineer is defining a training job in a SageMaker Pipeline. The job reads data from a private Amazon S3 bucket and writes model artifacts to another S3 bucket. Both buckets are encrypted using customer-managed keys (CMKs) in AWS KMS. The training job must assume an IAM role with exactly the required permissions to access the data and the KMS keys, adhering to the principle of least privilege.



Which TWO configurations are required to allow the training job to read the encrypted S3 data and write the encrypted artifacts?



- A) Grant s3:GetObject and s3:PutObject permissions to the IAM role assumed by the SageMaker training job.


- B) Grant kms:Decrypt and kms:GenerateDataKey permissions to the IAM role assumed by the SageMaker training job.


- C) Attach a policy to the IAM role allowing the sagemaker:CreateModel permission for the specific KMS key.


- D) Configure the S3 bucket policy to allow public read access so the SageMaker compute instances can download the data without IAM constraints.



| Correct Answers: A and B



Correct Answer Analysis:


- A is Correct: The execution role assumed by SageMaker must have explicit S3 permissions (s3:GetObject, s3:PutObject) on the specific buckets to read the training data and write the output artifacts.


- B is Correct: Because the S3 buckets are encrypted with a KMS CMK, the SageMaker execution role must have kms:Decrypt (to read the input data) and kms:GenerateDataKey (to encrypt the output artifacts) permissions on the specific KMS key ARN.



Distractor Analysis:


- C is Incorrect: sagemaker:CreateModel is an IAM permission for creating a model package, not for KMS interactions or data access during training.


- D is Incorrect: Making an S3 bucket public violates security best practices and least privilege principles. IAM roles should be used to grant private, scoped access.



> Key Takeaway: When reading from or writing to KMS-encrypted S3 buckets, the SageMaker execution role requires both S3 bucket permissions and explicit KMS key permissions. | | ### Question 4 - Event-Driven Retraining Triggers



An AI startup is setting up a CI/CD pipeline for a predictive maintenance model. They want to trigger a retraining pipeline automatically whenever the F1 score of the deployed model drops below a certain threshold, as detected by Amazon SageMaker Model Monitor. The trigger must require the least amount of custom code and operational overhead.



Which ONE AWS service combination provides the event-driven trigger to start the SageMaker Pipeline?



- A) Use AWS Lambda to poll SageMaker Model Monitor metrics and trigger the pipeline using the Boto3 SDK.


- B) Use Amazon EventBridge to listen for SageMaker Model Monitor alerts and trigger the SageMaker Pipeline execution.


- C) Use AWS CloudTrail to log model monitor alerts and an S3 Event Notification to trigger the pipeline.


- D) Use Amazon CloudWatch Alarms to detect metric drops and send an SNS message to a custom microservice that starts the pipeline.



| Correct Answer: B



Correct Answer Analysis:


- B is Correct: Amazon SageMaker Model Monitor emits baseline drift or model quality violation events directly to Amazon EventBridge. EventBridge can be configured with a rule to match these violation events and automatically invoke the SageMaker StartPipelineExecution API, creating a native, event-driven retraining trigger with zero custom polling code.



Distractor Analysis:


- A is Incorrect: Polling metrics with a Lambda function introduces unnecessary operational overhead and custom code compared to native EventBridge integration.


- C is Incorrect: CloudTrail logs API calls but does not provide real-time metric evaluation or native pipeline triggering capabilities for Model Monitor baseline violations.


- D is Incorrect: Routing through CloudWatch Alarms and a custom microservice adds architectural complexity and points of failure; EventBridge can trigger the pipeline directly.



> Key Takeaway: SageMaker Model Monitor integrates natively with Amazon EventBridge, which can trigger automated retraining pipelines when model quality degrades. | | ### Question 5 - Manual Approval in MLOps Pipelines



A company is deploying an automated MLOps pipeline using SageMaker Pipelines. The pipeline successfully trains a new model version and evaluates its accuracy. The team mandates that a human must review the model's metrics before it is deployed to the production endpoint, and the pipeline must not execute the deployment step automatically.



Which ONE configuration fulfills this requirement?



- A) Add a Manual Approval step in AWS CodePipeline after the model registration step in SageMaker Model Registry.


- B) Configure the SageMaker Pipeline to pause indefinitely after the training step and wait for an API call to resume.


- C) Use an AWS Lambda function to send an email via Amazon SES and halt the pipeline execution until a link is clicked.


- D) Register the model in the SageMaker Model Registry with a "PendingManualApproval" status and separate the deployment into a CodePipeline stage gated by an approval action.



| Correct Answer: D



Correct Answer Analysis:


- D is Correct: The best practice for human-in-the-loop MLOps is to have the SageMaker Pipeline register the newly trained model in the SageMaker Model Registry with a PendingManualApproval status. A separate CI/CD tool (like AWS CodePipeline) can then be configured to trigger a deployment stage only when a human manually updates the model status to Approved in the registry.



Distractor Analysis:


- A is Incorrect: While AWS CodePipeline has a manual approval action, placing it after a SageMaker registry step without checking the registry's approval status doesn't enforce the registry's governance model natively. Option D correctly describes linking the approval status to the deployment gate.


- B is Incorrect: Pausing a SageMaker Pipeline indefinitely is not a supported, robust native feature for human-in-the-loop approvals; pipelines are meant to execute to completion (e.g., up to registration).


- C is Incorrect: Building a custom email and link-clicking mechanism via SES and Lambda introduces unnecessary operational overhead and security complexities compared to using Model Registry's built-in approval statuses.



> Key Takeaway: SageMaker Model Registry's PendingManualApproval status combined with AWS CodePipeline provides a native, auditable human-in-the-loop gating mechanism. | | ### Question 6 - Warm Pools for Frequent Training Jobs



A data science team needs to configure a SageMaker training job for a large NLP model. The team wants to reduce the cold start time of the training jobs, as they run frequently in their CI/CD pipeline. The instances must retain the conda environments, downloaded packages, and pre-built Docker images between runs without requiring the instances to run continuously.



Which ONE configuration achieves this with the least operational overhead?



- A) Configure a SageMaker Processing job with KeepAlive set to True.


- B) Configure the SageMaker Estimator in the pipeline with KeepAliveConfig enabled.


- C) Deploy a persistent Amazon EMR cluster and submit training jobs via Spark.


- D) Use a custom Amazon ECR image with all dependencies pre-installed and skip dependency installation in the script.



| Correct Answer: B



Correct Answer Analysis:


- B is Correct: SageMaker Managed Warm Pools (KeepAliveConfig) allow a provisioned training cluster to be retained in a warm state after a training job completes. This drastically reduces cold start times for subsequent training jobs because the cluster retains the EBS volume contents, conda environments, and downloaded data, without incurring continuous compute costs while idle.



Distractor Analysis:


- A is Incorrect: KeepAlive is not a parameter for SageMaker Processing jobs; it is specifically a configuration for Training jobs via KeepAliveConfig.


- C is Incorrect: Deploying an EMR cluster introduces significant operational overhead and continuous cost, violating the "least operational overhead" and "not running continuously" constraints.


- D is Incorrect: While a custom ECR image reduces dependency installation time, it does not solve the cold start issue of provisioning the compute instances and downloading the massive training datasets each time.



> Key Takeaway: SageMaker Managed Warm Pools (KeepAliveConfig) persist EBS volumes and environments between training jobs to minimize pipeline cold start times. | | ### Question 7 - Automated Bedrock Model Evaluation



An AI engineering team is building a pipeline to evaluate a fine-tuned foundation model on Amazon Bedrock. The pipeline needs to run a model evaluation job automatically whenever a new fine-tuned model is created. The evaluation must assess the model's toxicity and accuracy against a custom dataset stored in S3. The solution must minimize operational overhead.



Which ONE service should the team use to configure this automated evaluation job natively?



- A) Amazon SageMaker Clarify


- B) Amazon Bedrock Model Evaluation


- C) AWS CodeBuild with custom Python evaluation scripts


- D) Amazon Comprehend



| Correct Answer: B



Correct Answer Analysis:


- B is Correct: Amazon Bedrock Model Evaluation provides a native, managed capability to evaluate foundation models or fine-tuned models. It supports automatic evaluation of metrics like toxicity, accuracy, and robustness using both built-in and custom datasets stored in S3, without requiring custom evaluation code.



Distractor Analysis:


- A is Incorrect: SageMaker Clarify is used for bias detection and explainability of traditional SageMaker ML models, not for native evaluation of Bedrock foundation models.


- C is Incorrect: Using AWS CodeBuild with custom scripts introduces significant development and operational overhead to maintain, package, and run evaluation logic manually.


- D is Incorrect: Amazon Comprehend is a natural language processing (NLP) service for entity recognition, sentiment analysis, etc., not a model evaluation framework for generative AI.



> Key Takeaway: Amazon Bedrock Model Evaluation is the native service for automatically assessing foundation model quality, toxicity, and accuracy. | | ### Question 8 - Batch Transform Input Configuration



A machine learning team is configuring a SageMaker Batch Transform job to generate offline predictions for a daily reporting pipeline. The input data consists of a large JSON file where each line is a separate JSON object. The team needs to ensure the Batch Transform job correctly parses each JSON object as an individual invocation and outputs the predictions to an S3 bucket without requiring custom pre-processing code.



Which ONE configuration is required to handle the JSON input effectively?



- A) Set the MaxPayloadInMB parameter to the maximum allowed value to prevent timeouts.


- B) Configure the AssembleWith parameter to "Line" to concatenate the results properly.


- C) Specify the ContentType parameter as "application/json" and the SplitType parameter as "Line".


- D) Set the ContentType parameter to "application/x-jsonlines" and the SplitType parameter to "None".



| Correct Answer: C



Correct Answer Analysis:


- C is Correct: By setting SplitType to "Line", SageMaker Batch Transform will automatically split the input JSON file by newline characters. Each line (JSON object) is sent as an individual invocation to the model container. Setting ContentType to "application/json" tells the container how to parse the individual payload.



Distractor Analysis:


- A is Incorrect: MaxPayloadInMB governs the maximum size of a single payload sent to the container, but does not dictate how the input file is split into individual records.


- B is Incorrect: AssembleWith controls how the output files are assembled after inference, not how the input file is parsed or split.


- D is Incorrect: Setting SplitType to "None" would send the entire massive JSON file as a single payload to the container, likely causing memory issues or failing to process records individually.



> Key Takeaway: In SageMaker Batch Transform, SplitType="Line" is used to break up newline-delimited JSON files into individual invocations without custom preprocessing. | | ### Question 9 - Custom Inference Container Requirements



An enterprise is deploying a custom inference container for a PyTorch model. The container must run on a SageMaker Real-Time Inference endpoint. The CI/CD pipeline builds the container image and pushes it to Amazon ECR. The team needs to ensure the container adheres to the SageMaker inference container requirements to receive traffic correctly.



Which ONE interface must the container implement to handle invocation requests correctly?



- A) The container must expose port 8080 and implement the /ping and /invocations endpoints.


- B) The container must expose port 443 and implement the /health and /predict endpoints.


- C) The container must implement a Flask application listening on port 80.


- D) The container must run an nginx web server that proxies requests to the model using gRPC.



| Correct Answer: A



Correct Answer Analysis:


- A is Correct: SageMaker Real-Time Inference containers must expose port 8080. They are required to respond to HTTP GET requests on /ping (for health checks) and HTTP POST requests on /invocations (to receive inference payloads).



Distractor Analysis:


- B is Incorrect: SageMaker does not use port 443 or the /health and /predict endpoints for standard container inference interfaces.


- C is Incorrect: While Flask is commonly used to build the web server inside the container, SageMaker strictly requires port 8080, not port 80, and it must implement the specific endpoints /ping and /invocations.


- D is Incorrect: Nginx is not a requirement for SageMaker containers, and gRPC is not the default required interface for basic real-time endpoints.



> Key Takeaway: SageMaker requires custom inference containers to serve HTTP traffic on port 8080 and implement the /ping and /invocations endpoints. | | ### Question 10 - Automating Instance Selection for Inference



A company is preparing to deploy a highly optimized custom ML model to a SageMaker Real-Time Inference endpoint. They want to minimize inference costs while meeting strict latency and throughput requirements. They need to automate the process of finding the optimal instance type and configuration without manual load testing in their CI/CD pipeline.



Which ONE SageMaker feature should they use to find the best instance configuration?



- A) SageMaker Hyperparameter Tuning


- B) SageMaker Inference Recommender


- C) SageMaker Autopilot


- D) SageMaker Debugger



| Correct Answer: B



Correct Answer Analysis:


- B is Correct: SageMaker Inference Recommender is a feature that automatically tests and evaluates different instance types and configurations for a given model container. It runs load tests, measures latency, throughput, and cost, and provides a recommendation for the optimal endpoint configuration, fulfilling the CI/CD automation requirement.



Distractor Analysis:


- A is Incorrect: SageMaker Hyperparameter Tuning is used to optimize model weights and algorithm hyperparameters during the training phase, not for finding the optimal inference compute infrastructure.


- C is Incorrect: SageMaker Autopilot is an automated machine learning (AutoML) tool for training models from a dataset, not for load-testing or recommending endpoint instances.


- D is Incorrect: SageMaker Debugger is used to monitor training metrics, detect anomalies, and debug training scripts, not for deployment infrastructure optimization.



> Key Takeaway: Use SageMaker Inference Recommender to automate instance type selection and endpoint configuration load testing to minimize costs and meet latency targets. |