Skip to content

Skill 3.2.1: Optimize resource provisioning between on-demand and provisioned resources for performance and cost efficiency

Question Answer & Explanation
### Question 1 - Provisioned Throughput vs On-Demand for Bedrock Models



A financial services company is deploying an enterprise internal assistant using a custom fine-tuned Amazon Bedrock model. The traffic pattern is highly predictable, sustaining a constant rate of 5,000 requests per minute during business hours. The engineering team must optimize the solution for the lowest possible cost while maintaining strict performance consistency.



Which configuration best meets these requirements? (Select ONE)



- A) Use Amazon Bedrock On-Demand Inference to handle the variable capacity dynamically.


- B) Purchase Provisioned Throughput for Amazon Bedrock sized to the predictable baseline workload.


- C) Deploy the model on Amazon SageMaker Serverless Inference to scale with traffic.


- D) Deploy the model on an Amazon SageMaker Real-Time endpoint with a single large instance and no auto-scaling.


| Correct Answer: B



Correct Option Analysis:


- B is Correct: For predictable, high-volume, and sustained traffic, Provisioned Throughput in Amazon Bedrock offers a discounted hourly rate compared to On-Demand pricing. By provisioning capacity to match the known baseline, the company ensures consistent performance and optimizes cost for heavy, steady workloads.



Distractor Analysis:


- A is Incorrect: On-Demand Inference is ideal for unpredictable or sporadic traffic but is more expensive for sustained, predictable workloads compared to Provisioned Throughput.


- C is Incorrect: SageMaker Serverless Inference is not applicable here because the model is hosted on Amazon Bedrock, not a custom SageMaker container. Additionally, serverless introduces cold starts which violate the strict performance consistency requirement.


- D is Incorrect: Deploying on SageMaker Real-Time with a single instance and no auto-scaling introduces a single point of failure and does not guarantee the same cost-efficiency for Bedrock-native models compared to Bedrock Provisioned Throughput.



> Key Takeaway: For predictable, high-volume generative AI workloads on Amazon Bedrock, Provisioned Throughput provides the most cost-effective and performant solution over On-Demand. | | ### Question 2 - High-Resolution Concurrency Metrics for Long-Streaming Responses



An AI engineering team has deployed a Large Language Model (LLM) on an Amazon SageMaker Real-Time endpoint. The application streams responses back to users over a long window (often 20-30 seconds per request). Traffic spikes occur rapidly in seconds, but the standard Application Auto Scaling policy reacts too slowly, leading to throttling during sudden bursts.



The team must configure a scaling policy that adds capacity almost instantly when concurrent requests surge. Which metric and policy configuration should they use? (Select ONE)



- A) Target tracking on the standard 1-minute resolution SageMakerVariantInvocationsPerInstance metric.


- B) Step scaling on the CPUUtilization metric with a 1-minute alarm period.


- C) Target tracking on the high-resolution ConcurrentRequestsPerModel custom metric.


- D) Scheduled scaling to provision extra capacity 15 minutes before known peak hours.


| Correct Answer: C



Correct Option Analysis:


- C is Correct: High-resolution custom metrics like ConcurrentRequestsPerModel (or ConcurrentRequestsPerCopy for Inference Components) are emitted at sub-minute intervals (e.g., 10 seconds). For long-streaming workloads where the number of in-flight requests accumulates rapidly, scaling on concurrency using a high-resolution metric allows Application Auto Scaling to trigger scale-outs in seconds, preventing throttling.



Distractor Analysis:


- A is Incorrect: SageMakerVariantInvocationsPerInstance tracks completed invocations per instance per minute. For long-streaming responses, completed invocations lag far behind concurrent requests, making this metric too slow to react to sudden traffic spikes.


- B is Incorrect: CPUUtilization is generally a poor scaling metric for LLM inference, as GPU/NPU utilization can spike unpredictably based on sequence length rather than request count, and standard 1-minute resolution is too slow for the described scenario.


- D is Incorrect: Scheduled scaling does not dynamically react to unexpected, sudden traffic spikes in seconds; it only provisions capacity for pre-defined, predictable time windows.



> Key Takeaway: For long-streaming generative AI workloads, scale on high-resolution concurrency metrics to ensure rapid scale-out and prevent request throttling. | | ### Question 3 - Optimizing Baseline and Spike Traffic on SageMaker Real-Time Endpoints



A retail company is running an e-commerce recommendation model on an Amazon SageMaker Real-Time endpoint. The application has a steady baseline traffic of 50 requests per second, but during promotional events, traffic can spike unpredictably to 200 requests per second. The team needs to optimize cost while guaranteeing sub-100ms latency during these spikes.



Which architectural approach provides the MOST cost-efficient solution? (Select ONE)



- A) Provision enough instances to handle 200 requests per second continuously, disabling auto-scaling.


- B) Use Amazon SageMaker Serverless Inference with provisioned concurrency sized for 200 requests per second.


- C) Deploy a SageMaker Real-Time endpoint with a minimum instance count sized for the 50 RPS baseline, and configure Application Auto Scaling to add instances based on InvocationsPerInstance.


- D) Deploy a single large instance and rely on an Application Auto Scaling target tracking policy on MemoryUtilization.


| Correct Answer: C



Correct Option Analysis:


- C is Correct: Provisioning the minimum number of instances to handle the steady baseline traffic (50 RPS) ensures you aren't overpaying for idle capacity during normal hours. Configuring Application Auto Scaling on SageMakerVariantInvocationsPerInstance allows the endpoint to dynamically scale out to handle the unpredictable promotional spikes up to 200 RPS, balancing cost and performance.



Distractor Analysis:


- A is Incorrect: Provisioning for the maximum peak capacity 24/7 is highly cost-inefficient, as you pay for 4x the required baseline capacity during off-peak hours.


- B is Incorrect: Serverless Inference is generally suited for intermittent or sporadic traffic. Sizing provisioned concurrency for 200 RPS on Serverless would eliminate the cost benefits of scale-to-zero and could be more expensive than a baseline real-time instance with auto-scaling.


- D is Incorrect: A single large instance has no fault tolerance (violating best practices) and scaling on MemoryUtilization for inference is often misleading, as inference memory usage doesn't always correlate linearly with incoming traffic volume.



> Key Takeaway: To optimize cost for mixed baseline and spike traffic, provision your real-time endpoint minimum capacity to handle the steady baseline, and use target tracking to scale out for unpredictable spikes. | | ### Question 4 - Multi-Model Endpoints (MME) for Low-Traffic Models



An enterprise SaaS provider hosts 100 custom computer vision models, one for each tenant. Each tenant generates less than 10 inference requests per hour. The company currently runs these models on separate SageMaker Real-Time endpoints, resulting in massive compute wastage and high costs. The company must minimize idle compute costs while maintaining low latency for tenant requests.



Which SageMaker deployment option should the company transition to? (Select ONE)



- A) Amazon SageMaker Asynchronous Inference.


- B) Amazon SageMaker Serverless Inference.


- C) Amazon SageMaker Multi-Model Endpoints (MME).


- D) Amazon Bedrock Provisioned Throughput.


| Correct Answer: C



Correct Option Analysis:


- C is Correct: Amazon SageMaker Multi-Model Endpoints (MME) allow you to deploy thousands of models on a single SageMaker Real-Time endpoint container. The endpoint loads models into memory on-demand and caches the most frequently used ones. This dramatically reduces idle compute costs because you only pay for the underlying endpoint instances, not for 100 separate endpoints.



Distractor Analysis:


- A is Incorrect: Asynchronous Inference is for large payloads or long processing times and queues requests; it does not inherently consolidate multiple models onto a single compute resource to save idle costs across 100 distinct models.


- B is Incorrect: While Serverless Inference scales to zero, managing 100 separate Serverless endpoints introduces operational overhead, and cold starts would heavily impact the low-latency requirement for 100 distinct tenant models.


- D is Incorrect: Bedrock Provisioned Throughput is for foundational or fine-tuned Bedrock models, not for hosting custom computer vision models built by the company.



> Key Takeaway: When hosting many models with low individual traffic volumes, use Multi-Model Endpoints (MME) to share compute resources and drastically reduce idle infrastructure costs. | | ### Question 5 - Eliminating Cold Starts on Serverless Inference



A startup is hosting a lightweight text-classification model on Amazon SageMaker Serverless Inference. Traffic is highly sporadic but latency-sensitive. Users are experiencing delayed responses due to cold starts when traffic drops to zero. The startup wants to maintain the cost benefits of serverless (scaling to zero) but needs to guarantee a fast response time for the first few requests after an idle period.



Which configuration optimizes cost while eliminating the cold start delay? (Select ONE)



- A) Switch to a SageMaker Real-Time endpoint with a single ml.t3.medium instance.


- B) Enable Provisioned Concurrency on the SageMaker Serverless endpoint.


- C) Increase the memory allocation for the Serverless endpoint to 6 GB.


- D) Configure Application Auto Scaling to keep the Serverless endpoint "warm" by sending ping requests via EventBridge every 5 minutes.


| Correct Answer: B



Correct Option Analysis:


- B is Correct: Provisioned Concurrency for SageMaker Serverless Inference allows you to pre-initialize a specific number of execution environments so they are ready to respond immediately. This eliminates cold starts for the provisioned capacity while still allowing the endpoint to scale to zero beyond that provisioned capacity, optimizing cost and performance.



Distractor Analysis:


- A is Incorrect: Switching to a Real-Time endpoint means you are paying for the compute 24/7, which eliminates the cost benefits of Serverless scaling to zero.


- C is Incorrect: Increasing memory does not prevent cold starts; it only provides more memory to the execution environment once it is initialized.


- D is Incorrect: Sending ping requests (keep-alive) is an anti-pattern for serverless architectures, introduces unnecessary operational complexity, and still doesn't guarantee the environment will stay warm under all conditions.



> Key Takeaway: To eliminate cold starts on SageMaker Serverless Inference without abandoning the serverless model, configure Provisioned Concurrency. | | ### Question 6 - Right-Sizing Inference Instances with Inference Recommender



A machine learning team has trained a complex custom PyTorch model locally and needs to deploy it to Amazon SageMaker Real-Time Inference. They are unsure which EC2 instance type provides the best balance of cost and latency. Manual benchmarking across 10 different instance types would take weeks. They need a solution that minimizes operational overhead and objectively identifies the most cost-effective hardware.



Which AWS service or feature should they use? (Select ONE)



- A) Amazon SageMaker Model Monitor.


- B) Amazon SageMaker Inference Recommender.


- C) Amazon EC2 Instance Store benchmarks.


- D) AWS Compute Optimizer.


| Correct Answer: B



Correct Option Analysis:


- B is Correct: Amazon SageMaker Inference Recommender automatically benchmarks your model across different instance types and recommends the best hardware for latency and cost. It eliminates the need for manual benchmarking, saving time and ensuring you select the most cost-effective deployment configuration.



Distractor Analysis:


- A is Incorrect: SageMaker Model Monitor monitors data drift and model performance after deployment; it does not recommend instance types before deployment.


- C is Incorrect: Manually benchmarking on EC2 Instance Stores contradicts the requirement to minimize operational overhead and does not use SageMaker's managed deployment capabilities.


- D is Incorrect: AWS Compute Optimizer provides recommendations for standard compute resources (EC2, ASG) but does not natively benchmark ML models for SageMaker Inference endpoint configurations.



> Key Takeaway: Use Amazon SageMaker Inference Recommender to automate instance type selection and identify the most cost-effective hardware for your model deployment. | | ### Question 7 - Custom GPU Utilization Scaling for High-Variable Workloads



An AI team is hosting an audio-transcription model on a SageMaker Real-Time endpoint using expensive GPU instances (ml.g4dn.12xlarge). Incoming request counts are relatively stable, but the GPU computation required per request varies wildly depending on audio file length. The standard InvocationsPerInstance scaling metric is failing to trigger scale-outs when long audio files cause GPU bottlenecks.



Which scaling configuration will ensure the endpoint scales out appropriately based on actual hardware stress? (Select ONE)



- A) Target tracking on SageMakerVariantInvocationsPerInstance.


- B) Target tracking on a high-resolution ConcurrentRequestsPerModel metric.


- C) Target tracking on the GPUUtilization CloudWatch metric.


- D) Step scaling on DiskThrottleExceeded.


| Correct Answer: C



Correct Option Analysis:


- C is Correct: When request volume is stable but resource utilization per request is highly variable, scaling based on invocation count or concurrency will fail to capture backend bottlenecks. Target tracking on the GPUUtilization metric ensures Application Auto Scaling adds capacity when the actual hardware is stressed by heavy computations, maximizing cost-efficiency by only scaling when truly needed.



Distractor Analysis:


- A is Incorrect: SageMakerVariantInvocationsPerInstance scales based on request count. If request count is stable but GPU usage spikes, this metric will not trigger a scale-out, leading to latency degradation.


- B is Incorrect: High-resolution concurrency metrics track in-flight requests. If requests take longer due to heavy GPU usage, this might eventually trigger a scale-out, but GPUUtilization is the direct measure of the root cause (hardware stress).


- D is Incorrect: Disk throttling is irrelevant to GPU compute bottlenecks for audio transcription.



> Key Takeaway: When resource consumption per request varies heavily, use actual hardware utilization metrics (e.g., GPUUtilization, CPUUtilization) for Application Auto Scaling instead of invocation metrics. | | ### Question 8 - Cost Optimization with Asynchronous Inference for Large Payloads



A media processing company runs a batch image-generation pipeline. Users upload large datasets of text prompts via an API, and the model generates high-resolution images. The generation process takes up to a minute per request. Traffic is highly spiky, and the company wants to minimize compute costs by scaling down to zero when there is no active work.



Which deployment option is the MOST cost-effective while meeting the technical requirements? (Select ONE)



- A) SageMaker Real-Time Inference with Application Auto Scaling configured to scale to zero.


- B) SageMaker Serverless Inference with a large memory configuration.


- C) SageMaker Asynchronous Inference with Application Auto Scaling configured to scale to zero.


- D) SageMaker Batch Transform processing user uploads once per day.


| Correct Answer: C



Correct Option Analysis:


- C is Correct: SageMaker Asynchronous Inference is explicitly designed for large payloads (up to 500MB) and long processing times (up to 1 hour). Crucially, it is the only real-time-facing endpoint that natively supports scaling to zero instances when the queue is empty, making it extremely cost-effective for spiky, long-running workloads.



Distractor Analysis:


- A is Incorrect: SageMaker Real-Time Inference does not support scaling the instance count to zero; the minimum instance count must be 1, incurring continuous costs even when idle.


- B is Incorrect: SageMaker Serverless Inference scales to zero, but it has a maximum payload size limit (10MB) and maximum execution timeout (10 minutes). While the time fits, large media datasets often exceed the payload limit, and async is better suited for queuing large payloads.


- D is Incorrect: Batch Transform is not suitable for API-driven, on-demand user uploads; it is for offline processing of entire datasets at once, not spiky real-time API requests.



> Key Takeaway: For large payloads, long inference times, and the need to scale to zero, SageMaker Asynchronous Inference is the most cost-effective deployment option. | | ### Question 9 - Inference Components for Right-Sizing Model Copies



A logistics company is deploying three separate transformer models on a single SageMaker Real-Time endpoint. The team wants to maximize the utilization of their expensive ml.g5.12xlarge instance. They need to deploy multiple copies of the models on the same instance and scale the number of model copies (not the instances) dynamically based on traffic.



Which feature allows the team to scale individual model copies independently on a shared instance? (Select ONE)



- A) SageMaker Multi-Model Endpoints (MME).


- B) SageMaker Inference Components.


- C) SageMaker Serverless Inference.


- D) Application Auto Scaling on InvocationsPerInstance.


| Correct Answer: B



Correct Option Analysis:


- B is Correct: SageMaker Inference Components allow you to deploy one or more models onto a shared SageMaker Real-Time endpoint. You can specify the compute resources for each model and scale the number of model copies up or down dynamically based on traffic, maximizing instance utilization and reducing costs.



Distractor Analysis:


- A is Incorrect: Multi-Model Endpoints (MME) allow deploying many models to one endpoint, but they dynamically load/unload models into memory rather than allowing you to explicitly scale independent "copies" of specific models on a shared instance with granular compute allocation like Inference Components do.


- C is Incorrect: Serverless Inference does not utilize a shared SageMaker Real-Time endpoint instance.


- D is Incorrect: InvocationsPerInstance scales the underlying EC2 instances, not the individual model copies on a shared instance.



> Key Takeaway: Use SageMaker Inference Components to pack multiple models onto a single endpoint and scale individual model copies to optimize hardware utilization and cost. | | ### Question 10 - Balancing Real-Time and Async Endpoints for Cost Efficiency



A healthcare startup hosts a medical image analysis model. Interactive user requests require sub-second latency, generating high traffic during the day. Additionally, a nightly batch process submits large numbers of high-resolution images that take 5 minutes each to process. The startup wants to optimize costs without degrading interactive performance during the day.



Which architecture provides the MOST cost-effective solution? (Select ONE)



- A) A single SageMaker Real-Time endpoint with large GPU instances to handle both interactive and nightly batch workloads.


- B) A SageMaker Real-Time endpoint for interactive traffic, and a SageMaker Asynchronous endpoint for the nightly batch workloads.


- C) A SageMaker Serverless endpoint for interactive traffic, and a Batch Transform job for the nightly workloads.


- D) A single SageMaker Asynchronous endpoint for all traffic to minimize compute costs.


| Correct Answer: B



Correct Option Analysis:


- B is Correct: Using a Real-Time endpoint for the latency-sensitive interactive traffic ensures sub-second responses. Using an Asynchronous endpoint for the long-running nightly batch workloads allows the batch infrastructure to scale to zero when the batch is complete, avoiding 24/7 hardware costs for large GPU instances.



Distractor Analysis:


- A is Incorrect: Provisioning a single Real-Time endpoint large enough to handle the 5-minute nightly batch jobs without timing out would mean over-provisioning expensive GPUs that sit idle during the day, which is not cost-effective.


- C is Incorrect: Serverless Inference has a 10-minute timeout limit, but it is not ideal for high-resolution image batch processing. Also, Batch Transform is an offline job that must be triggered and managed, unlike an Async endpoint which can queue API requests natively.


- D is Incorrect: An Asynchronous endpoint queues requests, which would introduce latency delays that violate the sub-second requirement for interactive user requests.



> Key Takeaway: Segregate latency-sensitive workloads onto Real-Time endpoints and long-running, heavy batch workloads onto Asynchronous endpoints (which scale to zero) to optimize overall compute costs. | |