Skip to content

Skill 3.1.4: Evaluate and select appropriate foundation model (FM) deployment options

Question 1 - Bedrock On-Demand vs SageMaker Endpoint for FM Inference

A financial services company is building a generative AI application that summarizes customer support tickets using a foundation model. The application is used by support agents during business hours only (9 AM–5 PM, Monday–Friday), with traffic dropping to nearly zero overnight and on weekends. The company wants to minimize infrastructure management and only pay for actual model usage. However, the application is not latency-sensitive, so occasional cold starts are acceptable.

Which deployment option should the ML engineer select to meet these requirements?

A) Deploy the foundation model on a SageMaker real-time endpoint with auto scaling B) Deploy the foundation model on SageMaker Serverless Inference C) Invoke the foundation model through Amazon Bedrock with on-demand pricing D) Deploy the foundation model on SageMaker Asynchronous Inference endpoints E) Use Amazon Bedrock Provisioned Throughput to reserve dedicated capacity

Click to reveal Answer & Explanation

Correct Answer: C

Correct Answer Analysis

  • C is Correct: Amazon Bedrock on-demand pricing has no minimum fees and no upfront commitments. You pay only for what you use, and there are no instances to manage. Bedrock handles idle periods automatically without requiring you to configure scaling policies or manage cold-start mitigation. For a generative AI application with intermittent traffic and no strict latency requirements, Bedrock on-demand is the most operationally efficient choice.

Distractor Analysis

  • A is Incorrect: A SageMaker real-time endpoint requires provisioning and managing instances, even during idle periods. While auto scaling can reduce costs during low-traffic hours, it cannot scale to zero without additional configuration (inference components plus scale-to-zero policies), and maintaining warm capacity overnight incurs unnecessary costs.
  • B is Incorrect: While SageMaker Serverless Inference does scale to zero and charges per execution, it is designed for hosting models you supply rather than served foundation models. For a foundation model, Bedrock is more operationally efficient as it requires no model packaging, container configuration, or endpoint management.
  • D is Incorrect: Asynchronous Inference is for workloads with large payloads or processing times exceeding 60 seconds. A summarization task triggered interactively by support agents does not fit this pattern—agents are actively waiting for the response.
  • E is Incorrect: Provisioned Throughput reserves dedicated model processing capacity for a fixed term. This is the opposite of "pay only for actual usage" and is designed for workloads that need guaranteed throughput and low latency, not intermittent business-hours traffic.

Key Takeaway: For serving foundation models with intermittent traffic, no latency SLA, and minimal operational overhead, Amazon Bedrock on-demand pricing is the default choice—it removes both instance management and capacity planning.

Question 2 - Bedrock Custom Model Import vs SageMaker Hosting

A retail organization has fine-tuned an open-weight Llama 3 model in their own training environment for product recommendation generation. The model artifacts are stored in Amazon S3. The team wants to serve this custom model through a managed API—they do not want to manage inference containers, configure health checks, or maintain scaling policies. The model must be invocable via the same Converse API as other foundation models.

Which deployment approach should the ML engineer use?

A) Package the model into a SageMaker real-time endpoint using a custom inference container B) Use Amazon Bedrock Custom Model Import to import the fine-tuned model from S3 C) Deploy the model on a SageMaker multi-model endpoint D) Deploy the model using Amazon ECS with GPU-accelerated instances E) Use Amazon SageMaker JumpStart to deploy the fine-tuned model

Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

  • B is Correct: Amazon Bedrock Custom Model Import allows you to import supported open-weight foundation models (like Llama 3, Mistral, etc.) that have been fine-tuned outside of Bedrock. The model is uploaded from Amazon S3, and Bedrock serves it through the same InvokeModel/Converse APIs as native Bedrock foundation models. No container management, health checks, or scaling policy configuration is required.

Distractor Analysis

  • A is Incorrect: While a SageMaker real-time endpoint with a custom inference container is technically valid, it requires the team to build, maintain, and manage the inference container, configure health checks, and manage scaling policies—exactly the operational overhead the team wants to avoid.
  • C is Incorrect: A multi-model endpoint still requires provisioning a SageMaker endpoint with instances, hosting a shared serving container, and managing scaling policies. This contradicts the "no endpoint management" requirement.
  • D is Incorrect: Amazon ECS with GPU instances puts container orchestration, cluster scaling, and GPU driver management entirely on the team's plate—maximum operational overhead, opposite of the requirement.
  • E is Incorrect: SageMaker JumpStart is a catalog of pretrained models for quick deployment, but it deploys them to SageMaker endpoints (which you can manage). It does not provide a managed API via Bedrock. Additionally, JumpStart is for deploying pretrained models from its catalog, not for importing your externally fine-tuned weights into a managed API.

Key Takeaway: Amazon Bedrock Custom Model Import converts an externally fine-tuned open-weight model into a managed API served by Bedrock—eliminating endpoint, container, and scaling management—but only for supported model architectures.

Question 3 - Provisioned Throughput for Guaranteed Capacity

A healthcare technology company has built a patient triage chatbot using a foundation model in Amazon Bedrock. The chatbot is used in emergency room intake and must guarantee responses within 2 seconds during peak hours (typically 10 AM–8 PM). The company anticipates sustained high throughput during these hours and requires predictable latency. They do not want to self-host the model.

Which deployment option should the ML engineer configure?

A) Use Amazon Bedrock on-demand pricing with auto scaling B) Deploy the foundation model on a SageMaker real-time endpoint with GPU instances C) Purchase Provisioned Throughput for the Bedrock foundation model D) Use SageMaker Serverless Inference with provisioned concurrency E) Deploy the foundation model on Amazon EKS with a cluster autoscaler

Click to reveal Answer & Explanation

Correct Answer: C

Correct Answer Analysis

  • C is Correct: Amazon Bedrock Provisioned Throughput reserves dedicated model processing capacity (measured in Model Units) for a specified term. This guarantees consistent throughput and predictable latency without requiring the team to operate endpoints or instances. For a healthcare application with strict latency SLAs and sustained high traffic during known hours, Provisioned Throughput provides the required performance guarantees while remaining fully managed.

Distractor Analysis

  • A is Incorrect: Bedrock on-demand pricing does not guarantee throughput or latency. During periods of high demand, on-demand requests may experience throttling or increased latency, which violates the 2-second response requirement for emergency triage.
  • B is Incorrect: A SageMaker real-time endpoint with GPU instances is a valid option but contradicts the "do not want to self-host" requirement. It requires instance management, scaling policies, health checks, and container configuration.
  • D is Incorrect: SageMaker Serverless Inference is designed for intermittent traffic with idle gaps. A chatbot used consistently during 10-hour daily peak windows does not have "idle gaps" that justify serverless. Additionally, serverless inference has cold-start implications unless provisioned concurrency is configured, and it does not provide the same throughput guarantees as Bedrock Provisioned Throughput for foundation models.
  • E is Incorrect: Amazon EKS with a cluster autoscaler places all infrastructure management, Kubernetes configuration, and scaling policy management on the team—maximum operational overhead and the opposite of the "managed API" requirement.

Key Takeaway: For foundation models served through a managed API where guaranteed throughput and predictable latency are required, Provisioned Throughput is the correct capacity commitment—on-demand pricing offers no performance guarantees.

Question 4 - SageMaker Real-time vs Bedrock for Custom FM Fine-Tuning

An AI team at a media company has developed a custom transformer-based model for content moderation. The model requires specialized GPU-accelerated inference, integrates with their existing VPC for security compliance, and must support multi-model hosting to serve region-specific model variants from a single endpoint. The team has deep MLOps expertise and wants full control over scaling policies.

Which deployment option should the ML engineer select?

A) Amazon Bedrock Custom Model Import B) SageMaker Serverless Inference C) SageMaker Real-time Inference with GPU instances and Multi-Model Endpoint D) Amazon Bedrock Provisioned Throughput E) AWS Lambda with GPU layers

Click to reveal Answer & Explanation

Correct Answer: C

Correct Answer Analysis

  • C is Correct: SageMaker Real-time Inference with GPU instances provides the hardware acceleration, VPC integration, and full control over scaling policies that the team requires. The Multi-Model Endpoint capability allows multiple region-specific model variants to be hosted on a single shared endpoint, reducing costs. The team's deep MLOps expertise and desire for full control over infrastructure make SageMaker the appropriate choice over the managed Bedrock API.

Distractor Analysis

  • A is Incorrect: Amazon Bedrock Custom Model Import only supports specific open-weight foundation model architectures (like Llama, Mistral, etc.). A custom transformer-based content moderation model would not be a supported architecture. Additionally, Bedrock does not expose VPC configuration for fine-grained network control.
  • B is Incorrect: SageMaker Serverless Inference does not support GPU-accelerated instances, does not support VPC configuration, and does not support multi-model hosting. All three specific requirements rule out serverless inference.
  • D is Incorrect: Provisioned Throughput in Bedrock is for foundation models served through the Bedrock API. It cannot host an arbitrary custom model, does not offer VPC configuration, and does not support multi-model endpoint patterns.
  • E is Incorrect: AWS Lambda does not support GPU layers. Lambda is designed for lightweight, CPU-based workloads and has a 15-minute maximum execution time and limited memory (up to 10 GB). A GPU-accelerated transformer model for content moderation cannot run on Lambda.

Key Takeaway: When requirements include VPC integration, GPU acceleration, multi-model hosting, or other infrastructure-level controls, SageMaker Real-time endpoints are the correct choice—Bedrock and serverless options are ruled out by these constraints.

Question 5 - Batch vs On-Demand FM Inference for Document Processing

A legal technology company processes thousands of contracts nightly using a foundation model for clause extraction. The documents are already stored in Amazon S3, and the results must be written back to S3 by 6 AM for the legal team's review. Processing time is not interactive—no one is waiting for individual responses. The company wants to minimize costs and avoid paying for idle capacity.

Which deployment option should the ML engineer select?

A) Amazon Bedrock on-demand API invoked by a nightly Step Functions workflow B) SageMaker Batch Transform with a foundation model C) SageMaker Real-time Inference with auto scaling to zero D) SageMaker Asynchronous Inference with scale-to-zero E) Amazon Bedrock Provisioned Throughput with scheduled scaling

Click to reveal Answer & Explanation

Correct Answer: A

Correct Answer Analysis

  • A is Correct: The scenario describes a bulk processing job with no interactive latency requirements. A Step Functions workflow can orchestrate invoking Amazon Bedrock's on-demand API for each document, processing them in parallel using a Map state. Bedrock on-demand requires no endpoint lifecycle management and charges per token processed—the company pays only for what was processed during the nightly run. This achieves "no idle capacity" because Bedrock infrastructure is not provisioned by the customer at all.

Distractor Analysis

  • B is Incorrect: While SageMaker Batch Transform is the classic answer for bulk processing of ML models you host, it requires packaging the foundation model into a SageMaker-compatible container and deploying it to the batch transform job. This introduces unnecessary operational overhead when Bedrock can serve the same foundation model through a managed API without any container work.
  • C is Incorrect: Real-time inference is unnecessary here because no caller is waiting for an answer. Auto scaling to zero on real-time endpoints requires inference components and custom scaling policies, and is designed for synchronous request-response patterns, not bulk overnight processing.
  • D is Incorrect: Asynchronous Inference is for dynamically arriving requests that take longer than 60 seconds each and exceed real-time payload limits. A pre-assembled dataset processed overnight fits the batch pattern, not the asynchronous queue pattern.
  • E is Incorrect: Provisioned Throughput commits to a fixed capacity for a term and is designed for guaranteed low-latency serving. For an overnight batch workload with no latency SLA, reserving capacity is cost-prohibitive and unnecessary.

Key Takeaway: When a foundation model processes a bulk dataset with no interactive latency requirement, orchestrate parallel Bedrock on-demand invocations via Step Functions—this avoids endpoint management entirely while maintaining a pay-per-token cost model.

Question 6 - Bedrock AgentCore for Multi-Step Task Automation

An enterprise is building an AI assistant that must handle multi-step workflows: retrieving customer order data from DynamoDB, checking inventory via an internal REST API, and then drafting a personalized response. The assistant must maintain conversational memory across turns and be deployable without managing any infrastructure. The team wants to use existing internal APIs as callable tools.

Which Bedrock capability should the ML engineer use to deploy this agent?

A) SageMaker Real-time Inference with a multi-container endpoint running the agent and tools B) Amazon Bedrock AgentCore with the integrated gateway to expose existing APIs as tools C) Amazon Bedrock Knowledge Bases with Retrieval Augmented Generation D) Amazon ECS running the agent as a long-lived service with API Gateway E) Lambda functions orchestrated by Step Functions for each agent turn

Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

  • B is Correct: Amazon Bedrock AgentCore provides a serverless runtime for running agents in production, with built-in memory for conversational context, identity management, observability, and—critically—a gateway that exposes existing APIs and functions as tools via open protocols like Model Context Protocol (MCP). This directly satisfies the requirements: multi-step workflows with API calls, conversational memory, no infrastructure management, and existing APIs callable as tools.

Distractor Analysis

  • A is Incorrect: A multi-container endpoint still requires provisioning and managing a SageMaker endpoint with instances. It also does not natively provide the agent runtime capabilities (memory, tool orchestration, protocol gateway) that Bedrock AgentCore offers—the team would need to build these from scratch.
  • C is Incorrect: Knowledge Bases handles RAG (document retrieval and augmentation), not agent orchestration with tool calling across multiple services and APIs. While it might provide relevant context, it does not execute multi-step workflows or maintain agent state.
  • D is Incorrect: Amazon ECS requires managing container clusters, scaling policies, networking, and infrastructure—exactly the operational overhead the team wants to avoid. The agent orchestration logic (memory, tool calling, state management) would also need to be custom-built.
  • E is Incorrect: While Step Functions can orchestrate workflows and Lambda can invoke tools, this approach requires building the agent runtime, conversational memory management, and tool-calling protocol from scratch. It introduces significant custom code and operational overhead compared to Bedrock AgentCore's managed runtime.

Key Takeaway: Amazon Bedrock AgentCore is the managed runtime for deploying production agents that require multi-step tool orchestration, conversational memory, and integration with existing APIs—eliminating the need to build agent infrastructure from scratch.

Question 7 - On-Demand Bedrock vs Provisioned Throughput for Prototype vs Production

A startup has prototyped an AI writing assistant using Amazon Bedrock on-demand. They are now preparing for a product launch with a guaranteed user base from a corporate contract. The contract requires that the assistant respond within 3 seconds for up to 1,000 concurrent users. The startup has a limited DevOps team and cannot manage infrastructure.

What deployment strategy should the ML engineer recommend? (Select TWO)

A) Continue using Bedrock on-demand since Bedrock handles all scaling automatically B) Purchase Bedrock Provisioned Throughput to guarantee capacity and consistent latency for the contracted user base C) Deploy a SageMaker real-time endpoint with auto scaling to handle the concurrent users D) Set up a CloudWatch alarm to monitor throttle events and manually request a service quota increase when throttling occurs E) Use Bedrock on-demand for baseline traffic, supplemented by Provisioned Throughput for peak guaranteed capacity

Click to reveal Answer & Explanation

Correct Answer: B and E

Correct Answer Analysis

  • B is Correct: Bedrock Provisioned Throughput reserves dedicated Model Units, providing guaranteed throughput and consistent low-latency responses regardless of overall Bedrock service demand. For a contractual SLA of 3-second responses with 1,000 concurrent users, Provisioned Throughput is the only Bedrock option that guarantees this level of performance.
  • E is Correct: A hybrid approach using on-demand for baseline/development traffic and Provisioned Throughput for production SLA-traffic is a common and cost-effective pattern. The startup can maintain on-demand for testing and low-traffic periods while keeping Provisioned Throughput active for contracted production workloads.

Distractor Analysis

  • A is Incorrect: While Bedrock on-demand does handle scaling automatically, it does not guarantee throughput or latency. On-demand traffic can be throttled during high demand. A contractual SLA of 3-second responses for 1,000 concurrent users requires reserved capacity to guarantee performance.
  • C is Incorrect: A SageMaker real-time endpoint requires the team to build and maintain inference containers, configure instance types, manage scaling policies, and perform operational tasks. With a limited DevOps team and no infrastructure management allowed, this contradicts the requirements.
  • D is Incorrect: Monitoring and manually requesting quota increases is a reactive approach that does not guarantee SLA compliance. Even with higher quotas, on-demand capacity is shared and can still be throttled or experience latency spikes under high service-wide demand.

Key Takeaway: On-demand Bedrock is sufficient for prototyping and variable workloads, but contractual latency SLAs with guaranteed concurrent users require Provisioned Throughput—the only way to reserve dedicated model-serving capacity in Bedrock.

Question 8 - SageMaker Hosting for Custom Fine-Tuned Model with VPC Requirements

A government contractor has fine-tuned a proprietary foundation model for document classification. The model must be hosted in a VPC with strict network isolation, requires GPU inference acceleration, and the traffic has steady, predictable patterns with no idle periods. The organization has regulatory requirements that prohibit using shared, multi-tenant model APIs.

Which deployment option should the ML engineer select?

A) Amazon Bedrock Custom Model Import B) Amazon Bedrock on-demand with VPC endpoints C) SageMaker Real-time Inference with GPU instances and VPC configuration D) SageMaker Serverless Inference with VPC configuration E) Amazon Bedrock Provisioned Throughput with VPC endpoints

Click to reveal Answer & Explanation

Correct Answer: C

Correct Answer Analysis

  • C is Correct: SageMaker Real-time Inference supports deploying to GPU instances within a customer-specified VPC, providing network isolation and regulatory compliance. Real-time endpoints are appropriate for steady, predictable traffic with no idle periods. This option also satisfies the "prohibits shared multi-tenant model APIs" requirement because SageMaker dedicates instances to the customer.

Distractor Analysis

  • A is Incorrect: Amazon Bedrock Custom Model Import serves models through Bedrock's shared, multi-tenant infrastructure. While Bedrock does support VPC interface endpoints for secure network connectivity to its API, the model execution itself runs on Bedrock's shared infrastructure, which likely violates the regulatory requirement prohibiting multi-tenant model APIs.
  • B is Incorrect: Bedrock on-demand serves foundation models through a shared multi-tenant API. VPC endpoints provide network security but do not change the fact that the model execution infrastructure is multi-tenant. This violates the regulatory constraint.
  • D is Incorrect: SageMaker Serverless Inference does not support VPC configuration. This rule-out alone eliminates this option. Additionally, serverless is designed for intermittent traffic, not steady predictable patterns.
  • E is Incorrect: Provisioned Throughput in Bedrock still serves models through Bedrock's infrastructure. While it provides dedicated capacity in terms of model units, the model is still served through the multi-tenant Bedrock service. If the regulatory requirement prohibits shared model APIs entirely, Bedrock does not qualify.

Key Takeaway: Regulatory requirements prohibiting multi-tenant model APIs override the convenience of managed Bedrock—you must self-host using SageMaker with VPC configuration and dedicated GPU instances to satisfy strict isolation requirements.

Question 9 - SageMaker Serverless for Bursty FM Traffic (Small Workloads)

A small e-commerce company uses a foundation model to generate product descriptions when new items are added to their catalog. Items are added sporadically—sometimes 50 in a day, sometimes none for a week. The company has one part-time developer and no ML infrastructure expertise. Latency requirements are flexible (up to 30 seconds). They want zero infrastructure management and minimal costs.

Which deployment option should the ML engineer select?

A) SageMaker Real-time Inference with GPU instances B) Amazon Bedrock on-demand API invoked by a simple scheduled check C) Amazon Bedrock Provisioned Throughput for guaranteed capacity D) SageMaker Batch Transform triggered by an S3 event E) Amazon EC2 with GPU running a custom inference server

Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

  • B is Correct: Amazon Bedrock on-demand provides a fully managed foundation model API with no infrastructure management—the part-time developer simply calls InvokeModel when a new product is added. With sporadic traffic and flexible latency up to 30 seconds, on-demand pricing is ideal. Costs are per token processed, so weeks with zero items incur zero costs. No instances, containers, or scaling policies need to be managed.

Distractor Analysis

  • A is Incorrect: A SageMaker Real-time Inference endpoint requires GPU instances running 24/7, which would incur costs even during weeks with no catalog updates. The part-time developer would also need to manage the endpoint, patch containers, and configure scaling—unnecessary operational overhead for this use case.
  • C is Incorrect: Provisioned Throughput requires a committed term with fixed monthly costs regardless of usage. For a catalog that may have zero items for a week, paying for reserved capacity during idle periods is cost-prohibitive and unnecessary given flexible latency requirements.
  • D is Incorrect: Batch Transform requires hosting the foundation model in a SageMaker-compatible container and is designed for bulk processing of pre-assembled datasets, not for individual items added sporadically. An S3 event trigger would need to accumulate items before processing, introducing unnecessary complexity and delay.
  • E is Incorrect: Amazon EC2 with GPU requires full infrastructure management—instance provisioning, security patching, GPU driver management, scaling, and 24/7 costs. This is the maximum operational overhead option, directly contradicting the "zero infrastructure management" and "part-time developer" constraints.

Key Takeaway: For sporadic, small-scale foundation model inference with flexible latency and a small team, Amazon Bedrock on-demand is the clear winner—zero infrastructure, pay-per-token, and no capacity commitments.

Question 10 - Deploying Agents with Bedrock AgentCore Gateway

An enterprise is deploying a customer service agent that must call multiple internal systems during a conversation: a CRM API (REST), a billing function (AWS Lambda), and an inventory database (DynamoDB). The agent must be exposed to customers via a web chat widget and communicate with tools using an open protocol standard for interoperability. The organization requires serverless deployment with no container management.

Which implementation should the ML engineer use? (Select TWO)

A) Bedrock AgentCore with the integrated gateway exposing internal APIs as tools using Model Context Protocol (MCP) B) SageMaker Real-time Inference with a multi-container endpoint hosting the agent and all tools C) Bedrock AgentCore serverless runtime with built-in conversational memory and state management D) Amazon ECS with the agent running as a service and tool invocations via API Gateway E) SageMaker Serverless Inference with the agent container calling internal services via VPC endpoints

Click to reveal Answer & Explanation

Correct Answer: A and C

Correct Answer Analysis

  • A is Correct: Bedrock AgentCore includes an integrated gateway that exposes existing APIs and functions as tools through open protocols like the Model Context Protocol (MCP). This allows the CRM REST API, Lambda function, and DynamoDB access to be registered as tools that the agent can invoke, without building custom integration code.
  • C is Correct: Bedrock AgentCore provides a serverless runtime for running agents in production with built-in conversational memory and state management. This eliminates the need for container management, infrastructure provisioning, or building custom state management from scratch.

Distractor Analysis

  • B is Incorrect: A SageMaker Real-time multi-container endpoint requires provisioning and managing SageMaker instances, containers, and scaling policies. It also does not natively provide agent-specific capabilities like conversation memory, tool protocol gateways, or state management—these would all need to be custom-built within the containers.
  • D is Incorrect: Amazon ECS requires managing container clusters, scaling, networking, and infrastructure. While it is serverless in the sense that you don't manage the underlying EC2 instances, you still manage containers, task definitions, cluster scaling, and networking. This contradicts the "no container management" requirement.
  • E is Incorrect: SageMaker Serverless Inference is designed for model inference, not agent orchestration. It does not provide agent runtime features like tool-calling protocol support, conversation memory, or state management. The team would need to build the entire agent framework as part of the inference container.

Key Takeaway: For production agents requiring tool integration across multiple services with open protocols, Bedrock AgentCore provides both the runtime (memory, state, orchestration) and the gateway (MCP-based tool exposure) as fully managed serverless capabilities.