Skill 3.1.3: Select model inference strategies (for example, real-time and batch processing)
| Question | Answer & Explanation |
|---|---|
| ### Question 1 - E-Commerce Fraud Scoring An e-commerce company is deploying a fraud detection model. The model must score every transaction while the customer waits at checkout, requiring strict millisecond latency. Traffic spikes significantly during holiday sales, but baseline traffic is continuous. The solution must balance strict latency requirements with cost optimization. Which inference strategy should the ML engineer choose? (Select ONE) - A) SageMaker Serverless endpoint - B) SageMaker Real-time endpoint with Auto Scaling - C) SageMaker Asynchronous inference - D) SageMaker Batch transform |
Correct Answer: B Correct Answer Analysis: - B is Correct: A Real-time endpoint with Auto Scaling maintains warm compute instances, ensuring immediate processing without cold starts. During holiday sales, Auto Scaling provisions additional instances based on invocation metrics to handle the traffic spikes while maintaining the strict latency budget. Distractor Analysis: - A is Incorrect: Serverless inference scales to zero during idle periods. If a request arrives after an idle period, it encounters a cold start, which violates the strict millisecond latency requirement for checkout. - C is Incorrect: Asynchronous inference queues incoming requests for background processing. Since the customer is blocked waiting for payment confirmation, queuing introduces unacceptable delays. - D is Incorrect: Batch transform processes pre-assembled datasets offline on a schedule. It cannot process individual real-time inference requests as they arrive. > Exam Tip: If a user is actively blocked waiting for a response (like during a payment), use Real-Time endpoints. Serverless is for intermittent traffic where cold starts are tolerable. |
| ### Question 2 - Medical Imaging Analysis A healthcare provider receives large medical imaging files individually throughout the day. Each file takes several minutes to process through a custom computer vision model. No user is blocked waiting for the response, and the results are stored in an Amazon S3 bucket for later review by doctors. The organization wants to minimize idle compute costs. Which SageMaker inference option meets these requirements with the LEAST operational overhead? (Select ONE) - A) Real-time endpoint - B) Serverless endpoint - C) Asynchronous inference - D) Batch transform |
Correct Answer: C Correct Answer Analysis: - C is Correct: Asynchronous inference is designed for large payloads and long processing times (up to 15-60 minutes). It queues incoming requests, returns an immediate acknowledgment, and processes them in the background. Crucially, it can scale the instance count down to zero when the queue is empty, minimizing idle compute costs. Distractor Analysis: - A is Incorrect: Real-time endpoints enforce a strict 60-second timeout. Processing files for several minutes would cause timeouts, and maintaining a warm endpoint for occasional files incurs high idle costs. - B is Incorrect: Serverless inference also enforces a strict 60-second execution timeout, making it unsuitable for multi-minute processing times. - D is Incorrect: Batch transform expects a pre-assembled, bulk dataset on a schedule, not individual dynamic requests arriving throughout the day. > Exam Tip: Use Asynchronous Inference for long-running processes (>60 seconds) or large payloads (>6 MB) where the caller doesn't wait, but requests arrive dynamically. |
| ### Question 3 - Mobile Photo App A startup is launching a new mobile application that uses an ML model to suggest filters for user photos. The application is expected to have unpredictable traffic patterns with long periods of inactivity overnight and occasional bursts of requests during the day. The team has a tight budget and wants to avoid paying for idle compute instances. Cold start latency is acceptable for this use case. Which deployment strategy is MOST cost-effective? (Select ONE) - A) SageMaker Serverless endpoint - B) SageMaker Real-time endpoint with Auto Scaling - C) SageMaker Batch transform - D) SageMaker Asynchronous inference |
Correct Answer: A Correct Answer Analysis: - A is Correct: Serverless Inference is explicitly designed for intermittent and unpredictable traffic with idle periods. It automatically scales compute capacity down to zero when not in use, meaning the startup only pays for the execution time and data transferred, not for idle instances. Distractor Analysis: - B is Incorrect: A Real-time endpoint requires maintaining at least one warm instance at all times to handle requests, leading to unnecessary costs during overnight inactivity. - C is Incorrect: Batch transform processes pre-assembled datasets offline. It cannot handle interactive, real-time requests from a mobile app. - D is Incorrect: While Asynchronous inference can scale to zero, it is optimized for large payloads and long processing times. It queues requests rather than processing them inline, which adds unnecessary queueing latency for a simple photo filter suggestion. > Exam Tip: Choose Serverless Inference for workloads with genuine idle gaps where cold starts are acceptable and you want to scale to zero. |
| ### Question 4 - Insurance Claims Processing An insurance company needs to process millions of scanned claims documents every night. The files are aggregated into an Amazon S3 bucket by 10:00 PM, and the inference results must be available in a reporting database by 8:00 AM the next day. There is no interactive user waiting for the response. The company wants to pay only for the compute resources used during processing. Which approach should the ML engineer implement? (Select ONE) - A) Asynchronous inference - B) Real-time endpoint with scheduled Auto Scaling - C) Batch transform - D) Serverless endpoint |
Correct Answer: C Correct Answer Analysis: - C is Correct: Batch Transform is ideal for processing large, pre-assembled datasets that arrive on a schedule. It spins up the compute cluster, processes the documents directly from Amazon S3, writes the predictions back to S3, and terminates the resources immediately, ensuring the company only pays for the exact processing duration. Distractor Analysis: - A is Incorrect: Asynchronous inference queues individual requests arriving dynamically. It is not optimized for bulk processing a massive dataset that is already assembled. - B is Incorrect: A real-time endpoint waits for incoming API calls. Using it for bulk offline processing requires building custom orchestration to feed the data in, creating unnecessary operational overhead compared to Batch Transform. - D is Incorrect: Serverless endpoints are for interactive, synchronous requests with execution limits (60 seconds) and are not suitable for bulk dataset processing. > Exam Tip: If the whole dataset arrives together on a schedule and nobody is waiting, use Batch Transform. |
| ### Question 5 - Hosting an Externally Trained Model A financial services firm has fine-tuned an open-weight foundation model externally on Amazon EC2 and wants to serve it through a managed API without managing the underlying endpoint infrastructure. The firm requires the ability to invoke the model on demand without committing to reserved capacity. Which AWS service or feature meets these requirements? (Select ONE) - A) SageMaker Real-Time Endpoint - B) SageMaker Asynchronous Endpoint - C) Amazon Bedrock Provisioned Throughput - D) Amazon Bedrock Custom Model Import |
Correct Answer: D Correct Answer Analysis: - D is Correct: Amazon Bedrock Custom Model Import allows organizations to import supported open-weight foundation models that were customized externally into Bedrock. Once imported, the model is served via the standard Bedrock API on an on-demand basis, completely removing the need to manage endpoint infrastructure. Distractor Analysis: - A is Incorrect: SageMaker Real-Time Endpoints require the engineer to choose the instance type, configure auto-scaling, and manage the endpoint infrastructure, violating the requirement. - B is Incorrect: Similar to A, Asynchronous Endpoints require managing the underlying instances and endpoint configurations. - C is Incorrect: Provisioned Throughput reserves dedicated model processing capacity for a set term. The firm specifically requested on-demand invocation without reserving capacity. > Exam Tip: Bedrock Custom Model Import is the path for bringing externally fine-tuned open-weight models into AWS without managing instances. |
| ### Question 6 - High-Volume Text Summarization A global media company is running a high-volume text summarization application using a foundation model on Amazon Bedrock. The application has predictable, continuous traffic during business hours and strict latency requirements. To ensure the application meets its SLA, the company wants to reserve a set level of input and output capacity. Which Bedrock feature should the engineer use? (Select ONE) - A) On-Demand Inference - B) Provisioned Throughput - C) Custom Model Import - D) Bedrock Knowledge Bases |
Correct Answer: B Correct Answer Analysis: - B is Correct: Amazon Bedrock Provisioned Throughput allows customers to reserve a specific level of throughput (measured in model units) for their applications. This guarantees capacity for predictable, high-volume workloads and ensures the strict latency SLA is met without being throttled by on-demand shared limits. Distractor Analysis: - A is Incorrect: On-Demand Inference shares capacity across all AWS customers in a region. During peak regional times, requests might be throttled, which violates the strict latency SLA for this high-volume application. - C is Incorrect: Custom Model Import is a deployment mechanism for bringing outside models into Bedrock, not a capacity management feature for an existing model. - D is Incorrect: Bedrock Knowledge Bases are used for Retrieval-Augmented Generation (RAG) to ingest and search documents, not for reserving model inference capacity. > Exam Tip: Use Provisioned Throughput in Bedrock when you have predictable, high-volume traffic and need guaranteed capacity to meet strict SLAs. |
| ### Question 7 - Managing Many Small Models A retail company hosts 500 small ML models that all use the same XGBoost framework. Traffic is highly uneven: 50 models receive constant traffic, while the remaining 450 are called rarely. The company wants to minimize hosting costs and infrastructure overhead. Which SageMaker deployment strategy is BEST suited for this scenario? (Select ONE) - A) 500 individual single-model endpoints - B) 500 batch transform jobs scheduled hourly - C) A multi-model endpoint - D) A multi-container endpoint |
Correct Answer: C Correct Answer Analysis: - C is Correct: A SageMaker Multi-Model Endpoint (MME) hosts many models on a single endpoint using a shared compute fleet and serving container. SageMaker dynamically caches models in memory, keeping frequently used models warm and loading rarely used models from S3 on demand. This drastically reduces hosting costs and infrastructure overhead compared to managing 500 separate endpoints. Distractor Analysis: - A is Incorrect: Provisioning 500 single-model endpoints means paying for 500 sets of instances, leading to extremely high costs and massive operational overhead, especially for the 450 rarely used models. - B is Incorrect: Batch transform processes static datasets offline on a schedule. It cannot serve dynamic, real-time application traffic. - D is Incorrect: Multi-container endpoints are used to host up to 15 different containers (often using different frameworks or serving as an inference pipeline) on a single endpoint, not for scaling hundreds of models sharing the same framework. > Exam Tip: Use Multi-Model Endpoints (MME) to host many models that share the same framework on a single endpoint to save costs, accepting a slight load penalty for rarely used models. |
| ### Question 8 - Autonomous Vehicle Telemetry An autonomous vehicle startup collects sensor data from its fleet. Every 5 minutes, a vehicle sends a 200 MB payload of sensor logs to an AWS endpoint to be processed by a complex inference model. The processing takes approximately 2 minutes per payload. The vehicle's onboard computer does not block waiting for the response, but it needs confirmation that the payload was received. Which inference option should the company use? (Select ONE) - A) SageMaker Serverless Inference - B) SageMaker Real-Time Inference - C) SageMaker Batch Transform - D) SageMaker Asynchronous Inference |
Correct Answer: D Correct Answer Analysis: - D is Correct: Asynchronous Inference supports large payloads (up to 1 GB) and long processing times (up to 15 minutes). It immediately returns a 202 Accepted acknowledgment to the caller, places the request in a queue, and processes it in the background without blocking the vehicle's onboard computer. Distractor Analysis: - A is Incorrect: Serverless Inference has a 60-second execution timeout and a 6 MB payload limit for synchronous requests, both of which are exceeded by this scenario. - B is Incorrect: Real-Time Inference enforces a 60-second timeout and a 6 MB payload limit, causing the large 200 MB payload to fail or the 2-minute processing to time out. - C is Incorrect: Batch Transform processes pre-assembled bulk datasets offline. It cannot accept individual dynamic requests arriving every 5 minutes from a fleet of vehicles. > Exam Tip: For payloads > 6 MB or processing > 60 seconds where the caller doesn't block, Asynchronous Inference is the required choice. |
| ### Question 9 - Scaling to Zero Without Dropping Requests A media streaming service uses a recommendation model that requires GPU acceleration for real-time inference. Traffic is bursty but continuous, and the team needs to keep costs low by scaling to zero during off-peak hours. However, they realize that scaling a real-time endpoint to zero means requests will fail until an instance is provisioned. They need a solution that allows the endpoint to scale to zero while safely queuing incoming requests during the provisioning window. Which inference strategy meets these requirements? (Select ONE) - A) Serverless Inference - B) Asynchronous Inference - C) Batch Transform - D) Real-Time Inference with Inference Components |
Correct Answer: B Correct Answer Analysis: - B is Correct: Asynchronous Inference uses a managed Amazon SQS queue to hold incoming requests. If the endpoint scales its instance count to zero, incoming requests are safely queued rather than dropped. When the endpoint scales back up, it pulls from the queue and processes the requests, returning an immediate acknowledgment to the caller. Distractor Analysis: - A is Incorrect: Serverless Inference does scale to zero, but it does not use an SQS queue. Requests arriving during a cold start simply experience high latency rather than being explicitly queued, and Serverless does not support GPU acceleration. - C is Incorrect: Batch Transform cannot handle dynamic real-time requests arriving continuously; it only processes pre-assembled static datasets. - D is Incorrect: Real-Time Inference can scale to zero using Inference Components, but if a request arrives when there is zero capacity, the request fails immediately with an error because there is no queue to hold it. > Exam Tip: Asynchronous Inference uses an internal SQS queue, allowing it to scale to zero without dropping incoming requests, unlike Real-Time endpoints which will return an error during provisioning. |
| ### Question 10 - Mitigating Cold Starts on a Budget A startup is deploying a lightweight custom ML model on Amazon SageMaker. Traffic is highly intermittent, arriving in short bursts with long idle stretches. To minimize costs, the team chose SageMaker Serverless Inference. However, they noticed occasional cold starts are degrading user experience during the short bursts. They need to mitigate cold starts while keeping operational overhead to a minimum. Which action should the ML engineer take? (Select ONE) - A) Switch to a Real-Time endpoint with Auto Scaling - B) Configure Provisioned Concurrency on the Serverless endpoint - C) Switch to an Asynchronous endpoint - D) Use a Batch Transform job triggered by an S3 event |
Correct Answer: B Correct Answer Analysis: - B is Correct: Provisioned Concurrency on a SageMaker Serverless endpoint keeps the execution environment warm and ready to respond immediately. This eliminates cold starts during the short bursts of traffic while still leveraging the serverless infrastructure to minimize operational overhead. Distractor Analysis: - A is Incorrect: Switching to a Real-Time endpoint would eliminate cold starts, but it requires managing instances and paying for continuous compute, violating the startup's goal to minimize costs during long idle stretches. - C is Incorrect: Asynchronous endpoints would queue the requests, adding unnecessary latency for the end-user who expects an immediate interactive response. It also does not specifically solve the cold start issue on a serverless scale-to-zero architecture. - D is Incorrect: Batch Transform processes static datasets offline. It cannot serve interactive, real-time requests from users. > Exam Tip: Use Provisioned Concurrency on Serverless Inference to eliminate cold starts for bursty traffic without committing to a 24/7 Real-Time endpoint. |