Skill 3.2.6: Select specific metrics for auto scaling implementations

Question	Answer & Explanation
### Question 1 - Scaling Metrics for Streaming LLM Responses An AI engineering team has deployed a Large Language Model (LLM) on an Amazon SageMaker real-time endpoint to provide a chatbot interface. The model generates streaming responses that can take 5–10 seconds to complete. Traffic to the endpoint is highly bursty, and the team notices that using standard `SageMakerVariantInvocationsPerInstance` causes the endpoint to scale out too late, leading to high latency and timeouts. Which scaling metric should the team use to ensure the endpoint scales out as quickly as possible when concurrent requests climb? Which ONE metric satisfies this requirement? (Select ONE) - A) `SageMakerVariantInvocationsPerInstance` with a lower target value - B) `ConcurrentRequestsPerModel` with a high-resolution metric - C) `ApproximateBacklogSizePerInstance` - D) `CPUUtilization`	Correct Answer: B Correct Answer Analysis: - B is Correct: For workloads with long-running or streaming responses (like LLMs), standard 1-minute metrics like `InvocationsPerInstance` react too slowly. SageMaker supports high-resolution CloudWatch concurrency metrics (such as `ConcurrentRequestsPerModel` or `ConcurrentRequestsPerCopy` for inference components) emitted at sub-minute intervals (e.g., 10 seconds). Target tracking on this high-resolution metric allows Application Auto Scaling to detect in-flight request surges almost instantaneously and trigger scale-out events in seconds. Distractor Analysis: - A is Incorrect: `SageMakerVariantInvocationsPerInstance` publishes at 1-minute resolution. Lowering the target value would just trigger scaling earlier on the same slow cadence, which still doesn't account for requests already in flight during a sudden burst. - C is Incorrect: `ApproximateBacklogSizePerInstance` is the correct metric for scaling Asynchronous endpoints based on queue depth, not real-time endpoints. - D is Incorrect: `CPUUtilization` is a generic infrastructure metric that may not correlate directly with request concurrency, especially for I/O bound or memory-bound LLM generation tasks. > Exam Tip: For streaming or long-running generative AI workloads on SageMaker real-time endpoints, use high-resolution concurrency metrics (e.g., `ConcurrentRequestsPerModel`) rather than standard invocation metrics to react to traffic spikes in seconds.
### Question 2 - Scaling SageMaker Asynchronous Endpoints A media processing company is using an Amazon SageMaker Asynchronous Inference endpoint to process large video files. The processing time per file ranges from 30 seconds to 2 minutes. Traffic is unpredictable, and the team needs to ensure that the queue does not grow unbounded during traffic spikes while keeping costs low during idle periods. Which auto scaling strategy should the team implement to achieve this with the least operational overhead? Which ONE configuration satisfies this requirement? (Select ONE) - A) Configure target tracking on `SageMakerVariantInvocationsPerInstance`. - B) Configure target tracking on `ApproximateBacklogSizePerInstance`. - C) Deploy a custom CloudWatch metric that tracks queue size and invoke a Lambda function to change instance count. - D) Configure a scheduled scaling policy to scale out during expected business hours.	Correct Answer: B Correct Answer Analysis: - B is Correct: SageMaker Asynchronous Inference endpoints place requests in a queue. The built-in metric `ApproximateBacklogSizePerInstance` monitors the number of requests in the queue per instance. Target tracking on this metric automatically scales out instances when the queue grows and scales in (potentially to zero) when the queue is empty, perfectly matching the unpredictable burst nature of this workload. Distractor Analysis: - A is Incorrect: `SageMakerVariantInvocationsPerInstance` tracks completed invocations per minute, which does not reflect the current backlog of unprocessed video files waiting in the queue. - C is Incorrect: Building a custom metric and using a Lambda function to manage scaling introduces unnecessary custom code and operational overhead, as SageMaker and Application Auto Scaling natively support queue-based scaling. - D is Incorrect: Scheduled scaling cannot react to unpredictable traffic spikes; it only works for predictable, recurring calendar events. > Exam Tip: Use `ApproximateBacklogSizePerInstance` to scale SageMaker Asynchronous Inference endpoints based on the queue depth of pending requests.
### Question 3 - Scaling SageMaker Inference Components A machine learning team has deployed multiple models on a single multi-model Amazon SageMaker real-time endpoint using SageMaker Inference Components. They want to scale the number of model copies dynamically based on traffic, without scaling the underlying hardware instances. The underlying instances have sufficient CPU and memory to support multiple copies. Which metric should the team use in their Application Auto Scaling target tracking policy? Which ONE metric satisfies this requirement? (Select ONE) - A) `SageMakerVariantInvocationsPerInstance` - B) `ConcurrentRequestsPerModel` - C) `SageMakerInferenceComponentInvocationsPerCopy` - D) `CPUUtilization`	Correct Answer: C Correct Answer Analysis: - C is Correct: When using SageMaker Inference Components, the scalable target is the inference component itself (representing a model copy), not the physical instance. `SageMakerInferenceComponentInvocationsPerCopy` tracks the number of invocations directed at each individual copy of the model. Scaling on this metric adjusts the `DesiredCopyCount` without altering the instance count. Distractor Analysis: - A is Incorrect: `SageMakerVariantInvocationsPerInstance` scales the physical instances (the endpoint variant), not the number of model copies deployed via inference components. - B is Incorrect: While `ConcurrentRequestsPerModel` can be used, `SageMakerInferenceComponentInvocationsPerCopy` is the specific predefined metric designed for scaling the number of model copies on an inference component. - D is Incorrect: `CPUUtilization` is an infrastructure-level metric for the underlying EC2 instances, which does not directly scale the Inference Component's copy count. > Exam Tip: For SageMaker Inference Components, scale the `DesiredCopyCount` using the `SageMakerInferenceComponentInvocationsPerCopy` metric, keeping the underlying instance count fixed.
### Question 4 - Scaling Based on GPU Utilization A computer vision team is hosting a complex object detection model on an Amazon SageMaker real-time endpoint using a GPU-accelerated instance type (`ml.g5.xlarge`). The inference workload is highly compute-intensive. The team needs to scale out the endpoint based strictly on GPU usage to prevent GPU OOM (Out of Memory) errors and high latency during heavy processing loads. Which approach should the team take to implement this auto scaling policy? Which ONE approach satisfies this requirement? (Select ONE) - A) Create a target tracking policy on the built-in `CPUUtilization` CloudWatch metric. - B) Configure target tracking on `SageMakerVariantInvocationsPerInstance`. - C) Publish a custom GPU utilization metric to Amazon CloudWatch from the container using a sidecar (like `nvidia-smi`), and use it in the target tracking policy. - D) Use the built-in `GPUUtilization` metric provided natively by SageMaker endpoints.	Correct Answer: C Correct Answer Analysis: - C is Correct: SageMaker real-time endpoints natively emit `CPUUtilization` and `MemoryUtilization`, but they do not automatically emit GPU utilization metrics to CloudWatch. To scale based on GPU usage, the team must implement a script (often using a sidecar container or custom inference code) to query `nvidia-smi` and publish the GPU utilization as a custom metric to CloudWatch, then configure Application Auto Scaling to track that custom metric. Distractor Analysis: - A is Incorrect: `CPUUtilization` does not reflect the load on the GPU, which is the primary bottleneck for a complex computer vision model. - B is Incorrect: Invocations per instance tracks request count, not actual hardware resource utilization. A few complex requests could max out the GPU before the invocation threshold is reached. - D is Incorrect: SageMaker does not natively provide a built-in `GPUUtilization` CloudWatch metric for real-time endpoint variants by default. > Exam Tip: To scale SageMaker endpoints on GPU utilization, you must publish a custom CloudWatch metric (e.g., via a sidecar) because SageMaker does not natively emit GPU metrics to CloudWatch.
### Question 5 - Serverless Inference Auto Scaling A startup is deploying a sentiment analysis model on Amazon SageMaker. The traffic is highly sporadic, sometimes dropping to zero for hours, and other times experiencing brief, massive spikes. The team wants the most cost-effective solution with the absolute least operational overhead for capacity provisioning and scaling. They decided to use a SageMaker Serverless Inference endpoint. How is auto scaling managed for this endpoint? Which ONE approach satisfies this requirement? (Select ONE) - A) Register the endpoint variant in Application Auto Scaling and configure a target tracking policy on `SageMakerVariantInvocationsPerInstance`. - B) Configure target tracking on `ConcurrentRequestsPerModel`. - C) Configure the `MaxConcurrency` setting on the endpoint, and AWS automatically scales the compute capacity based on traffic. - D) Create an AWS Lambda function triggered by CloudWatch Alarms to adjust the instance count.	Correct Answer: C Correct Answer Analysis: - C is Correct: SageMaker Serverless Inference endpoints do not require Application Auto Scaling configurations or instance type selections. You only specify the `MaxConcurrency` (the maximum number of concurrent invocations the endpoint can handle). AWS automatically provisions and scales the compute capacity behind the endpoint in response to traffic, down to zero when idle. Distractor Analysis: - A is Incorrect: Serverless endpoints do not expose variants or instances to register in Application Auto Scaling; scaling is fully managed. - B is Incorrect: While concurrency is tracked, you do not configure a target tracking policy on it for serverless endpoints; the platform handles it natively based on your `MaxConcurrency` limit. - D is Incorrect: Building custom Lambda functions to scale instances is a pattern for real-time endpoints and introduces unnecessary operational overhead, which defeats the purpose of using a serverless architecture. > Exam Tip: SageMaker Serverless Inference scales automatically based on traffic up to your configured `MaxConcurrency` limit, without requiring Application Auto Scaling policies.
### Question 6 - Cost-Optimized Scaling for Predictable Traffic A financial services company hosts a fraud detection model on an Amazon SageMaker real-time endpoint. Inference requests complete in under 50 milliseconds. Traffic is predictable: high during market hours (9 AM to 5 PM) and near-zero otherwise. The team wants to minimize costs while ensuring high availability during market hours with the least operational overhead. Which combination of auto scaling strategies meets this requirement? Which TWO strategies should the team use? (Select TWO) - A) A target tracking policy on `SageMakerVariantInvocationsPerInstance` during market hours. - B) A scheduled scaling action to increase the minimum capacity at 8:30 AM and decrease it at 5:30 PM. - C) A target tracking policy on `ApproximateBacklogSizePerInstance` during market hours. - D) Registering a scalable target with a minimum capacity of 0 instances. - E) A scheduled scaling action to set the minimum capacity to 1 instance during off-hours.	Correct Answers: A and B Correct Answer Analysis: - A is Correct: Because responses are short (50ms) and traffic is predictable, the standard predefined metric `SageMakerVariantInvocationsPerInstance` is the general-purpose, least-overhead choice for scaling out and in based on real-time load. - B is Correct: Scheduled scaling is ideal for predictable traffic patterns. Setting a higher minimum capacity just before market hours ensures the endpoint is ready for the morning spike, and lowering it after hours saves costs. Distractor Analysis: - C is Incorrect: `ApproximateBacklogSizePerInstance` is for Asynchronous Inference, not real-time endpoints. - D is Incorrect: Real-time SageMaker endpoints cannot scale to a minimum capacity of 0 instances; the minimum is 1 to keep the endpoint active. - E is Incorrect: While keeping 1 instance during off-hours is necessary for a real-time endpoint, combining it only with scheduled scaling misses the dynamic load tracking required during the day if traffic varies. Target tracking (A) + Scheduled scaling (B) is the optimal combination. > Exam Tip: Use scheduled scaling for predictable daily traffic spikes and target tracking on `InvocationsPerInstance` for dynamic load handling on real-time endpoints.
### Question 7 - Identifying the Correct Resource ID for Auto Scaling A DevOps engineer is writing an AWS CLI script to configure Application Auto Scaling for a newly deployed SageMaker real-time endpoint. The endpoint is named `prod-endpoint`, and the production traffic variant is named `AllTraffic`. The engineer needs to register the variant as a scalable target using the `register-scalable-target` command. What is the correct `resource-id` format the engineer must use? Which ONE value satisfies this requirement? (Select ONE) - A) `endpoint/prod-endpoint/variant/AllTraffic` - B) `prod-endpoint/AllTraffic` - C) `arn:aws:sagemaker:region:account-id:endpoint/prod-endpoint` - D) `inference-component/prod-endpoint/AllTraffic`	Correct Answer: A Correct Answer Analysis: - A is Correct: For a SageMaker endpoint variant, the Application Auto Scaling `resource-id` follows the specific format `endpoint/ENDPOINT_NAME/variant/VARIANT_NAME`. In this case, it is `endpoint/prod-endpoint/variant/AllTraffic`. Distractor Analysis: - B is Incorrect: This lacks the `endpoint/` and `variant/` prefixes required by the Application Auto Scaling service namespace for SageMaker. - C is Incorrect: Application Auto Scaling uses a specific resource ID string format, not the full Amazon Resource Name (ARN). - D is Incorrect: The `inference-component/` prefix is used when scaling SageMaker Inference Components, not endpoint variants. > Exam Tip: When registering a SageMaker endpoint variant for Auto Scaling via CLI/IaC, the resource ID format is `endpoint/ENDPOINT_NAME/variant/VARIANT_NAME`.
### Question 8 - Preventing Scaling Flapping A machine learning team has configured target tracking auto scaling on their SageMaker real-time endpoint using `SageMakerVariantInvocationsPerInstance`. They notice that during minor traffic fluctuations, the endpoint frequently scales out by adding an instance, and then immediately scales back in, causing instability and higher costs. The team needs to smooth out the scaling behavior. Which configuration change will best resolve this issue? Which ONE approach satisfies this requirement? (Select ONE) - A) Increase the target tracking metric value. - B) Decrease the scale-in cooldown period. - C) Increase the scale-in cooldown period. - D) Switch to a high-resolution concurrency metric.	Correct Answer: C Correct Answer Analysis: - C is Correct: The scale-in cooldown period controls how long after a scale-out event before Application Auto Scaling can scale back in. Increasing the scale-in cooldown period prevents the auto scaler from immediately terminating instances after a brief burst, stabilizing the endpoint capacity and reducing "flapping" (rapid scaling in and out). Distractor Analysis: - A is Incorrect: Increasing the target value would just allow each instance to take more load, potentially delaying scale-out, but it does not directly address the rapid scale-in behavior causing the flapping. - B is Incorrect: Decreasing the scale-in cooldown would make the system scale in faster, worsening the flapping issue. - D is Incorrect: A high-resolution concurrency metric is used for fast scale-out on streaming workloads, not for smoothing out minor fluctuations on standard endpoints. > Exam Tip: To prevent "flapping" (rapid scaling in and out) in SageMaker Auto Scaling, increase the scale-in cooldown period to give instances time to stabilize before being terminated.
### Question 9 - Asynchronous Endpoint Scaling to Zero A data science team has deployed an Amazon SageMaker Asynchronous Inference endpoint to process large batch inference requests that are submitted nightly. The jobs take roughly 1 hour to complete. The team wants to ensure that the endpoint scales down completely when the job is finished to minimize operational costs. Which configuration is required to allow the endpoint to scale down to zero instances? Which ONE configuration satisfies this requirement? (Select ONE) - A) Set the `MinCapacity` of the scalable target to 0 in Application Auto Scaling. - B) Configure a scheduled scaling action to set the instance count to 0 at midnight. - C) Deploy the endpoint using SageMaker Serverless Inference instead of Asynchronous Inference. - D) Asynchronous Inference endpoints automatically scale to zero by default; no configuration is needed.	Correct Answer: A Correct Answer Analysis: - A is Correct: SageMaker Asynchronous Inference endpoints support scaling to zero instances to save costs. However, this is not the default behavior. You must explicitly register the endpoint variant as a scalable target in Application Auto Scaling and set the `MinCapacity` parameter to 0. Distractor Analysis: - B is Incorrect: Scheduled scaling cannot set the instance count to 0 if the scalable target's `MinCapacity` is greater than 0. - C is Incorrect: While Serverless Inference scales to zero, the scenario explicitly states they are using an Asynchronous Inference endpoint, likely because of the long processing time and large payload sizes that Serverless might not handle (Serverless has limits on payload size and invocation time). - D is Incorrect: Asynchronous endpoints do not scale to zero automatically by default; they are provisioned with at least one instance unless Application Auto Scaling is configured with a `MinCapacity` of 0. > Exam Tip: SageMaker Asynchronous Inference endpoints can scale to zero, but you must explicitly set the `MinCapacity` to 0 in Application Auto Scaling.
### Question 10 - Scaling for Background Model Warming A retail company is deploying a new recommendation model to an Amazon SageMaker real-time endpoint. To prevent cold start issues and ensure the model is fully loaded into memory before receiving live traffic, the engineering team provisions the endpoint with 2 instances. They expect traffic to spike immediately upon launch. After the initial spike, traffic will vary but remain steady. The team wants to implement an auto scaling policy that prioritizes fast response times but minimizes unnecessary instance provisioning during the initial launch phase. Which metric is most appropriate for this steady-state real-time endpoint once the initial launch is complete? Which ONE metric satisfies this requirement? (Select ONE) - A) `ApproximateBacklogSizePerInstance` - B) `ConcurrentRequestsPerModel` - C) `SageMakerVariantInvocationsPerInstance` - D) `ModelRetrievalLatency`	Correct Answer: C Correct Answer Analysis: - C is Correct: `SageMakerVariantInvocationsPerInstance` is the standard, general-purpose predefined metric for real-time SageMaker endpoints. It measures the number of invocations processed per instance per minute. For steady-state traffic with standard response times, this metric provides a reliable signal for target tracking to scale capacity proportionally with traffic load without unnecessary overhead. Distractor Analysis: - A is Incorrect: `ApproximateBacklogSizePerInstance` is used for Asynchronous Inference to measure queue depth, not for real-time endpoints. - B is Incorrect: `ConcurrentRequestsPerModel` (or `ConcurrentRequestsPerCopy`) is best suited for long-running or streaming workloads where 1-minute invocation metrics react too slowly. For steady-state traffic, the standard invocation metric is sufficient and simpler. - D is Incorrect: `ModelRetrievalLatency` measures the time it takes to retrieve the model from Amazon ECR or S3, typically during endpoint initialization or updates, not for runtime traffic scaling. > Exam Tip: For standard real-time endpoints with predictable, steady-state traffic and short response times, `SageMakerVariantInvocationsPerInstance` is the ideal default metric for target tracking.