Skip to content

Skill 3.2.3: Build and maintain containers for ML and AI workloads

Question 1 - Custom Inference Container Requirements

A financial services company is deploying a fraud detection model that requires a custom feature engineering library written in Rust, which must be compiled and executed during model inference. The machine learning team has decided to use Amazon SageMaker AI for hosting.

The container must be built from scratch because no pre-built framework container includes the Rust runtime. The team requires that the container image be versioned immutably so that a redeployment always pulls the exact same image, and the build process must comply with the corporate policy of scanning images for vulnerabilities before deployment.

Which TWO actions should the team take to meet these requirements? (Select TWO)

  • A) Build the container locally, push it to Amazon ECR with a mutable tag such as latest, and configure the SageMaker endpoint to reference that tag.
  • B) Create the container to implement the SageMaker serving stack requirements, including handling /invocations and /ping HTTP endpoints.
  • C) Deploy the container to Amazon SageMaker using the latest tag after running a security scan with Amazon Inspector.
  • D) Push the container to Amazon ECR with an immutable tag (e.g., the Git commit SHA), enable Amazon Inspector on the repository, and configure the SageMaker endpoint to reference that SHA-based tag.
  • E) Use AWS Lambda to execute the Rust library during inference, calling it from a SageMaker-managed container using the generic PyTorch image.
Click to reveal Answer & Explanation

Correct Answers: B and D

Correct Options Analysis

  • B is Correct: When building a custom container from scratch for SageMaker hosting, the container must implement the required HTTP endpoints—/ping for health checks and /invocations for inference requests—along with the proper directory structure for model artifacts.
  • D is Correct: Immutable image tags (like Git commit SHAs) ensure that redeployments always pull the exact same container version. Amazon Inspector integration with ECR provides automated vulnerability scanning.

Distractor Analysis

  • A is Incorrect: Using a mutable tag like latest violates the immutability requirement. A redeployment may pull a different image than expected, causing configuration drift and reproducibility issues.
  • C is Incorrect: While scanning is good practice, using the latest tag after scanning does not guarantee immutability—a new latest could be pushed without scanning, and the endpoint might pull an unintended version.
  • E is Incorrect: Lambda is not appropriate for executing custom compiled libraries during low-latency inference calls; it introduces network overhead, cold starts, and an entirely different execution model than what a custom container is designed for.

Key Takeaway: Custom SageMaker containers must implement /ping and /invocations endpoints, and immutable image tags (not latest) are essential for reproducible ML deployments.

Question 2 - Prebuilt vs. Extended vs. Custom Container Selection

A healthcare analytics team is deploying a PyTorch-based medical image segmentation model to Amazon SageMaker AI. The model uses standard PyTorch operations but requires a few additional Python packages (opencv-python-headless, pydicom) for image preprocessing.

The team wants to minimize operational overhead while maintaining the ability to customize the inference environment. They do not want to manage the entire container build pipeline from scratch.

What is the MOST appropriate container strategy? (Select ONE)

  • A) Build a container completely from scratch, installing the PyTorch framework, the inference server, and all dependencies manually.
  • B) Use a prebuilt SageMaker Deep Learning Container (DLC) for PyTorch, and install additional dependencies by extending the DLC's Dockerfile using FROM and pip install.
  • C) Use the prebuilt SageMaker Deep Learning Container (DLC) for PyTorch as-is, and include the additional packages in the requirements.txt file inside the model artifact source_dir.
  • D) Deploy the model using Amazon Bedrock instead, which natively supports custom PyTorch models.
  • E) Package the model as an AWS Lambda layer and invoke it through API Gateway.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

  • B is Correct: Extending a SageMaker Deep Learning Container (DLC) is the recommended approach when you need pre-installed framework support plus additional dependencies. You start with the DLC base image and add your packages, keeping most of the maintenance burden on AWS while giving the flexibility to customize the environment.

Distractor Analysis

  • A is Incorrect: Building from scratch introduces unnecessary operational complexity when a pre-built DLC already provides the PyTorch framework and inference server. This violates the "minimize operational overhead" constraint.
  • C is Incorrect: SageMaker DLCs do not automatically install packages from requirements.txt in source_dir at runtime for real-time endpoints—the container image itself must include the packages. This option incorrectly suggests that model artifacts can modify the container's installed packages.
  • D is Incorrect: Amazon Bedrock does not support deploying custom PyTorch models. Bedrock is for managed foundation models and knowledge bases, not for bringing your own framework-specific custom models.
  • E is Incorrect: Lambda functions have deployment package size limits (250 MB unzipped) that are unsuitable for PyTorch models, and Lambda is not designed for GPU-accelerated inference workloads.

Key Takeaway: Extending a SageMaker DLC with a custom Dockerfile is the sweet spot between using a prebuilt container and building from scratch, offering customization with minimal operational overhead.

Question 3 - SageMaker Endpoint in VPC with Private S3 Access

A pharmaceutical company is deploying a drug-protein interaction model to Amazon SageMaker AI. Compliance policies mandate that the model endpoint must not have access to the public internet, and all data transfers—including model artifact retrieval from Amazon S3 and calls to other AWS services—must occur exclusively over the AWS private network.

The model is deployed in private subnets across two Availability Zones, and the endpoint's execution role has the necessary permissions. However, the endpoint fails during initialization because it cannot download the model artifacts from S3.

Which configuration change addresses this failure? (Select ONE)

  • A) Add a NAT Gateway to the private subnet's route table so the endpoint can reach S3 through the internet.
  • B) Create an Amazon S3 Gateway Endpoint in the VPC, add it to the route table of the private subnets, and configure the endpoint policy to allow access to the model artifact bucket.
  • C) Move the model artifacts from S3 to Amazon EBS volumes attached directly to the endpoint instances.
  • D) Configure the endpoint to use a public subnet with an Internet Gateway so it can reach S3.
  • E) Increase the IAM role permissions to include s3:* on all buckets.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

  • B is Correct: An Amazon S3 Gateway Endpoint (VPC Endpoint) provides a private connection from VPC subnets to Amazon S3 without traversing the public internet. Adding it to the route table of the private subnets ensures that traffic to S3 flows through the AWS private backbone, satisfying both the connectivity and compliance requirements without public internet exposure.

Distractor Analysis

  • A is Incorrect: A NAT Gateway allows instances in private subnets to reach the public internet, which directly violates the compliance policy that prohibits internet access. It also introduces additional cost and a managed resource.
  • C is Incorrect: SageMaker endpoints pull model artifacts from S3 during deployment initialization; they do not support attaching persistent EBS volumes for model artifact storage as a primary deployment mechanism. This approach would also complicate model updates.
  • D is Incorrect: Placing the endpoint in a public subnet with an Internet Gateway exposes the endpoint to the public internet, violating the requirement that all traffic stays on the private network.
  • E is Incorrect: While SageMaker execution roles need s3:GetObject permission on the artifact bucket, the failure described is a network connectivity issue, not an IAM permission issue. The question states the execution role already has necessary permissions.

Key Takeaway: For private-only SageMaker deployments, use VPC Endpoints (like S3 Gateway Endpoints) to provide private network paths to AWS services—NAT Gateways or public subnets violate the "no internet" constraint.

Question 4 - Programmatic Deployment with SageMaker Python SDK

A retail company's MLOps team needs to automate the deployment of a recommendation model to Amazon SageMaker AI. The deployment pipeline is built using AWS CDK and must:

  1. Create the SageMaker Model from an ECR image and S3 artifact.
  2. Create an Endpoint Configuration with the appropriate instance type and initial instance count.
  3. Deploy the Endpoint.
  4. Allow the endpoint ARN to be referenced by a downstream CloudFormation stack that configures API Gateway.

The team prefers the least verbose approach using the high-level SageMaker Python SDK for deployment, and they need the endpoint ARN exported for cross-stack use.

Which TWO components should the team implement? (Select TWO)

  • A) Use the high-level SageMaker Python SDK (sagemaker.Model.deploy()), which automatically creates the Model, Endpoint Configuration, and Endpoint in a single call.
  • B) Use the low-level boto3 client and manually create the Model, Endpoint Config, and Endpoint with separate API calls.
  • C) Use an AWS CloudFormation template that creates all three SageMaker resources and exports the Endpoint ARN using the Export field in the Outputs block.
  • D) Store the endpoint ARN as an SSM Parameter Store entry, and have the downstream stack read that parameter directly.
  • E) Use the SageMaker Python SDK's predict method to deploy the model directly to API Gateway.
Click to reveal Answer & Explanation

Correct Answers: A and C

Correct Options Analysis

  • A is Correct: The high-level SageMaker Python SDK's deploy() method is the correct approach for programmatic deployment when minimizing code verbosity and operational overhead—it creates the Model, Endpoint Configuration, and Endpoint in a single call.
  • C is Correct: For cross-stack reference, CloudFormation's Export/ImportValue mechanism is the standard approach. The hosting stack can export the Endpoint ARN, and the API Gateway stack can import it, maintaining proper infrastructure as code practices.

Distractor Analysis

  • B is Incorrect: Using low-level boto3 with separate API calls for each step is more verbose and error-prone compared to the high-level SDK's deploy() method, which meets the "least verbose" requirement.
  • D is Incorrect: While SSM Parameter Store can hold values, CloudFormation's native cross-stack export/import is more appropriate for stack-to-stack linkage in a CDK-based IaC pipeline. SSM adds another dependency and doesn't integrate as cleanly with CloudFormation's dependency ordering.
  • E is Incorrect: The SageMaker SDK's predict method is for generating predictions from a deployed endpoint, not for deploying models or configuring API Gateway. This is a nonexistent workflow.

Key Takeaway: Use sagemaker.Model.deploy() for concise programmatic deployment, and CloudFormation Exports/Imports for cross-stack resource references—don't mix concerns between SDK deployment and infrastructure orchestration.

Question 5 - High-Resolution Scaling for Streaming Inference

A generative AI startup is deploying an LLM to Amazon SageMaker AI for real-time chat inference. Each request can take 15–30 seconds to complete, and traffic can spike dramatically within seconds during product launches.

The team has configured Application Auto Scaling with a target-tracking policy on SageMakerVariantInvocationsPerInstance, but they observe that the scaling response is too slow—instances become overwhelmed before new capacity is added.

Which TWO changes should the team make to improve responsiveness? (Select TWO)

  • A) Change the scaling metric to a high-resolution concurrency metric like ConcurrentRequestsPerModel with target tracking.
  • B) Increase the scale-out cooldown period to prevent rapid fluctuations.
  • C) Decrease the scale-in cooldown period to maintain more instances available during traffic spikes.
  • D) Keep the current metric but increase the target value to allow more invocations per instance before scaling.
  • E) Configure a scheduled scaling action that adds capacity during peak product launch windows.
Click to reveal Answer & Explanation

Correct Answer: A

Correct Options Analysis

  • A is Correct: High-resolution concurrency metrics (like ConcurrentRequestsPerModel or ConcurrentRequestsPerCopy) are published at sub-minute intervals (e.g., 10 seconds), allowing Application Auto Scaling to detect in-flight request spikes much faster than standard 1-minute aggregation metrics. This aligns with the need for rapid scale-out during burst traffic.

Distractor Analysis

  • B is Incorrect: Increasing the scale-out cooldown delays subsequent scale-out actions, making the endpoint response even slower to traffic spikes—this makes the problem worse.
  • C is Incorrect: Scale-in cooldown affects how quickly instances are removed when traffic decreases, not how quickly they are added during traffic spikes. It doesn't address scale-out responsiveness.
  • D is Incorrect: Increasing the target value means each instance handles more traffic before triggering a scale-out, which delays scaling further and can cause instances to become overloaded and drop requests.
  • E is Incorrect: Scheduled scaling is useful for predictable events but cannot handle unexpected or rapid traffic spikes that occur within seconds or minutes—only a high-resolution metric can react quickly enough.

Key Takeaway: For bursty or streaming workloads, high-resolution concurrency metrics are essential for responsive auto scaling—standard invocation metrics react too slowly, and cooldown/target adjustments often make the problem worse.

Question 6 - Bedrock Knowledge Base Vector Store Selection

A legal tech company is building a Amazon Bedrock Knowledge Base to enable semantic search over its document repository. The company has two primary requirements:

  1. Support for hybrid search (combining semantic vector similarity with keyword/BM25 matching) to find both related content and exact legal case citations.
  2. The ability to filter results by metadata fields such as court, jurisdiction, and document_type.

The system must be fully managed with no infrastructure provisioning overhead, and the company wants to minimize cost.

Which vector store configuration should the company use? (Select ONE)

  • A) Use an Amazon OpenSearch Serverless collection configured with a hybrid search field, and define metadata field mappings during index creation.
  • B) Use Amazon RDS for PostgreSQL with the pgvector extension, connecting it to Bedrock Knowledge Bases.
  • C) Use Amazon S3 with vector embeddings stored as Parquet files to minimize cost.
  • D) Use Amazon Neptune as a graph store for retrieval following relationships between legal entities.
  • E) Use Amazon DynamoDB with a vector search index.
Click to reveal Answer & Explanation

Correct Answer: A

Correct Options Analysis

  • A is Correct: Amazon OpenSearch Serverless is a fully managed search service that supports hybrid search (combining dense vector k-NN with sparse lexical BM25), metadata filtering, and is natively supported as a vector store for Bedrock Knowledge Bases. Serverless eliminates infrastructure provisioning, and OPU (OpenSearch Processing Unit) pricing is cost-effective for many workloads.

Distractor Analysis

  • B is Incorrect: While Amazon RDS for PostgreSQL with pgvector supports vector search and metadata filtering, it does not natively support hybrid search (BM25 + vector) in the same way OpenSearch does. Additionally, RDS requires more infrastructure configuration and is less optimized for search workloads.
  • C is Incorrect: Amazon S3 is a low-cost object storage option for vectors, but it does not support real-time querying, hybrid search, or metadata filtering as a retrieval engine. Bedrock Knowledge Bases supports S3 for document ingestion, not as a vector store for retrieval.
  • D is Incorrect: Neptune is a graph database for entity-relationship traversal, which is not aligned with the requirements of hybrid search over documents and metadata filtering. Bedrock Knowledge Bases does not support Neptune as a primary vector store.
  • E is Incorrect: Amazon DynamoDB does not have native vector search capabilities. While it can store metadata and references to embeddings, it requires integration with another service (like OpenSearch or Kendra) for similarity search.

Key Takeaway: OpenSearch Serverless is the go-to vector store for Bedrock Knowledge Bases when hybrid search and rich metadata filtering are requirements, offering full management while minimizing infrastructure overhead.

Question 7 - Bedrock Knowledge Base Dimension Mismatch

A technology company is setting up a new Amazon Bedrock Knowledge Base for customer support search. The team selects Amazon Titan Text Embeddings v2 as the embedding model, which produces 1024-dimensional vectors.

They create an OpenSearch Serverless collection and begin the ingestion job. However, the ingestion fails with an indexing error. Upon investigation, the team discovers that the vector field in the OpenSearch index has been mapped to 768 dimensions.

What is the correct action to resolve this issue? (Select ONE)

  • A) Modify the existing vector field mapping in OpenSearch to change the dimension from 768 to 1024, then restart the ingestion job.
  • B) Create a new OpenSearch Serverless collection (or index) with the correct vector dimension mapping (1024), and reconfigure the Bedrock Knowledge Base to use that new vector store before restarting ingestion.
  • C) Change the embedding model in the Bedrock Knowledge Base to one that outputs 768-dimensional vectors, such as Titan Embeddings v1.
  • D) Use dimensionality reduction (e.g., PCA) to reduce the 1024-dimensional vectors to 768 before indexing.
  • E) Ignore the dimension mismatch and allow the OpenSearch index to automatically pad or truncate the vectors to fit.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

  • B is Correct: Vector field mappings in OpenSearch cannot be changed after index creation. When a dimension mismatch occurs between the embedding model and the vector index, the correct approach is to create a new index/collection with the correct vector dimension mapping and point the Knowledge Base to that new vector store. This may require re-running ingestion to re-index all documents.

Distractor Analysis

  • A is Incorrect: OpenSearch field mappings are immutable once defined—you cannot modify a vector field's dimension after index creation. This option suggests a non-existent capability.
  • C is Incorrect: Changing the embedding model may resolve the dimension mismatch, but the team already selected Titan Text Embeddings v2 for valid reasons (quality, performance). This is a workaround, not a proper fix, and requires evaluating whether the alternative model meets requirements.
  • D is Incorrect: Applying PCA to reduce dimensions would degrade the quality of embeddings, lose information, and requires additional infrastructure. This is not a best practice when the index can simply be recreated with the correct dimension.
  • E is Incorrect: OpenSearch does not automatically pad or truncate vectors to fit mismatched dimensions. This option describes invalid behavior.

Key Takeaway: Vector index dimensions must match the embedding model's output dimensions at index creation—this is immutable and requires recreating the index if mismatched.

Question 8 - Agent Long-Term Memory with Bedrock AgentCore

A customer service company is deploying an AI agent using Amazon Bedrock AgentCore. The agent successfully handles multi-turn conversations within a single session using session-scoped memory.

However, the company wants to enable returning users to have persistent preferences (e.g., "always show prices in USD", "prefer email notifications"). These preferences should be automatically remembered across different sessions without requiring the user to re-state them.

Which configuration should the team implement? (Select ONE)

  • A) Store user preferences in a DynamoDB table managed by the agent's Lambda functions, with manual lookup logic in each session.
  • B) Enable Long-Term Memory (LTM) in Bedrock AgentCore, which automatically extracts and persists cross-session user facts and preferences scoped to the user identity.
  • C) Configure a more aggressive session timeout to keep short-term memory alive longer, reducing the chance of context loss.
  • D) Use a retrieval pipeline from a knowledge base to inject user preferences as searchable documents.
  • E) Implement session isolation with stricter boundaries so each session cannot access another's working memory.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

  • B is Correct: Bedrock AgentCore's Long-Term Memory (LTM) is a managed capability that automatically extracts persistent, cross-session facts and preferences from conversations and stores them in a user-specific namespace. When a returning user starts a new session, the agent automatically retrieves this persisted context, eliminating the need for users to repeat themselves.

Distractor Analysis

  • A is Incorrect: While DynamoDB can store user preferences, this approach requires the team to manually implement the logic for extracting, storing, and retrieving preferences—adding significant operational overhead compared to using AgentCore's native LTM.
  • C is Incorrect: Extending session timeout does not persist context across sessions; it only keeps short-term memory alive longer within the same session. Once the session terminates, the context is lost.
  • D is Incorrect: Retrieving user preferences from a knowledge base treats preferences as searchable documents, which is not the correct pattern—preferences are structured user-specific state, not semantic knowledge to be retrieved via similarity search.
  • E is Incorrect: Session isolation prevents security boundary violations between concurrent sessions; it does not persist state across sessions. In fact, stronger isolation might work against cross-session continuity.

Key Takeaway: Use Bedrock AgentCore's Long-Term Memory for persistent, cross-session user context—it is managed, automatic, and user-scoped, unlike manual DynamoDB storage or retrieval-based approaches.

Question 9 - SageMaker HyperPod for Resilient Training

A research lab is training a large language model (LLM) on a cluster of GPUs using Amazon SageMaker HyperPod. The training job is expected to run for 3 weeks, and the team needs the infrastructure to be resilient to hardware failures—if a GPU instance fails mid-training, the job should continue without manual intervention.

The team is evaluating configuration options for HyperPod. Which TWO statements accurately describe SageMaker HyperPod's capabilities? (Select TWO)

  • A) HyperPod automatically detects faulty GPU instances and replaces them without disrupting the training job, assuming the training framework supports checkpointing and recovery.
  • B) HyperPod requires the team to manually monitor instance health and manually replace failed nodes via the AWS Management Console.
  • C) HyperPod can be orchestrated with either a workload manager (e.g., Slurm) or Kubernetes (e.g., Amazon EKS), providing flexibility for different ML engineering workflows.
  • D) HyperPod only supports single-node training; multi-node distributed training requires a separate managed cluster service.
  • E) HyperPod automatically configures checkpointing and fault tolerance in the training script without requiring any code changes.
Click to reveal Answer & Explanation

Correct Answer: A and C

Correct Options Analysis

  • A is Correct: SageMaker HyperPod is designed for resilient large-scale training. It includes automated health monitoring and node replacement—if a GPU instance becomes faulty, HyperPod can detect and replace it automatically, allowing training to resume from the last checkpoint without manual intervention.
  • C is Correct: HyperPod supports two orchestration models: a workload manager (like Slurm) for HPC-style workflows, or Kubernetes (like Amazon EKS) for container-native workflows. This flexibility is a key feature for diverse ML engineering teams.

Distractor Analysis

  • B is Incorrect: This statement contradicts HyperPod's core value proposition—automated failure detection and recovery. Manual node replacement would be the approach for a self-managed EC2 cluster, not HyperPod.
  • D is Incorrect: HyperPod is designed explicitly for large-scale, multi-node distributed training workloads. It manages clusters of interconnected GPU instances, not single-node jobs.
  • E is Incorrect: HyperPod provides infrastructure resilience (node replacement), but this statement incorrectly implies it manages fault tolerance in the training script itself—checkpointing and recovery logic still needs to be implemented in the training code (e.g., using PyTorch Lightning, Ray, or custom logic).

Key Takeaway: SageMaker HyperPod offers automated hardware failure detection and replacement for long-running distributed training jobs, with orchestration flexibility between Slurm and Kubernetes.

Question 10 - Bedrock Agent Gateway and Observability

A bank is deploying a production AI agent using Amazon Bedrock AgentCore. The agent must call three existing internal microservices (a credit score API, a customer profile API, and a fraud detection API) to make loan decisions.

The bank's requirements are:

  1. The agent must access internal services through a secure, managed gateway without exposing them to the public internet.
  2. The bank must be able to trace each step the agent took (including tool calls, inputs, outputs, and intermediate reasoning) for audit and debugging.

Which TWO configurations should the bank implement? (Select TWO)

  • A) Implement a custom API Gateway in the agent's Lambda functions to expose the internal services, with the Lambda functions handling authentication and routing.
  • B) Use Bedrock AgentCore's Gateway to expose the internal APIs as tools, with built-in identity and access management.
  • C) Enable observability and tracing in AgentCore (via CloudWatch logs and tracing), which records each step the agent took, including tool calls and intermediate outputs.
  • D) Manually add logging statements to each internal API to record when it is called by the agent.
  • E) Store all agent conversation logs in Amazon S3 without any structured tracing.
Click to reveal Answer & Explanation

Correct Answer: B and C

Correct Options Analysis

  • B is Correct: Bedrock AgentCore provides a managed Gateway that exposes internal APIs and functions as tools for the agent, handling the protocol translation (e.g., OpenAPI/MCP), identity, and secure access—eliminating the need to rebuild services as custom agent tools.
  • C is Correct: AgentCore includes built-in observability and tracing capabilities that record each step of the agent's execution, including tool calls, function inputs/outputs, and intermediate reasoning. This provides the audit trail and debugging capability required by the bank.

Distractor Analysis

  • A is Incorrect: While custom Lambda + API Gateway can expose internal services, this requires the team to build and maintain authentication, routing, and protocol translation logic—increasing operational overhead compared to using AgentCore's managed Gateway.
  • D is Incorrect: Manual logging only records that a specific API was called, not the full agent workflow (reasoning steps, orchestration decisions, or which agent step triggered the call). It doesn't satisfy the requirement for tracing each step the agent took.
  • E is Incorrect: Storing raw conversation logs in S3 without structured tracing does not provide the step-by-step observability needed for debugging agent failures or auditing tool calls. It's an unstructured data dump rather than a trace.

Key Takeaway: Bedrock AgentCore's Gateway and built-in observability are production-grade features for secure tool access and step-level tracing of agent execution—avoid building custom infrastructure when AgentCore already provides these capabilities.