Skill 2.1.7: Assess tradeoffs between AI model performance, latency, and cost

Question	Answer & Explanation
### Question 1 - Foundation Model Selection for Latency-Sensitive Chatbots A retail company is building a customer service chatbot using Amazon Bedrock. The bot must handle moderately complex customer queries while maintaining an excellent user experience. The business constraint requires the time-to-first-token (TTFT) to be strictly under 1.5 seconds. Cost is a secondary concern, but meeting the latency target and providing high-quality answers are critical. Which foundation model category is the MOST appropriate to satisfy these requirements? - A) Anthropic Claude 3 Opus via Bedrock On-Demand - B) Anthropic Claude 3 Haiku via Bedrock On-Demand - C) Amazon Titan Text Premier via Bedrock On-Demand - D) A custom-trained 7B parameter model deployed on SageMaker Real-Time endpoints	Correct Answer: B Correct Answer Analysis: - B is Correct: Claude 3 Haiku is designed for speed and efficiency, offering the lowest latency and fastest TTFT among the Claude 3 family while still maintaining strong reasoning capabilities suitable for customer service queries. Distractor Analysis: - A is Incorrect: Claude 3 Opus is the most capable model but has the highest latency and TTFT, likely violating the strict 1.5-second requirement for a real-time chat interface. - C is Incorrect: While Amazon Titan Text Premier is capable, it generally has higher latency compared to lighter, speed-optimized models like Haiku, making it less ideal for strict TTFT SLAs. - D is Incorrect: Deploying a custom model on SageMaker introduces massive operational overhead, training costs, and infrastructure management compared to using a pre-trained managed model via Bedrock, violating the implicit "least operational overhead" constraint. > Exam Tip / AWS Best Practice: When strict latency SLAs are specified for GenAI chat interfaces, prioritize speed-optimized foundation models (like Claude 3 Haiku) over heavy, maximum-capability models.
### Question 2 - Knowledge Updates vs. Model Customization Cost A legal tech company needs its GenAI assistant to answer questions based on a frequently updated set of 10,000 legal briefs. The legal team adds new briefs every hour. The solution must reflect these updates within minutes and minimize the compute cost of incorporating new information. Which approach BEST balances the performance requirement for up-to-date information with the cost constraint? - A) Supervised fine-tuning of a foundation model on the 10,000 briefs using Amazon Bedrock - B) Retrieval-Augmented Generation (RAG) using Amazon Bedrock Knowledge Bases - C) Reinforcement Fine-Tuning to reward the model for citing recent briefs - D) Provisioned Throughput on a large foundation model to handle the update frequency	Correct Answer: B Correct Answer Analysis: - B is Correct: Bedrock Knowledge Bases (RAG) stores data externally in a vector store. When new briefs are added, only the new documents need to be parsed and embedded (a cheap, fast operation), and the model immediately uses them at inference time. This satisfies the minute-level update requirement with minimal compute cost. Distractor Analysis: - A is Incorrect: Fine-tuning requires a full training job every time new data is added, incurring massive compute costs and taking hours, violating both the cost and update-latency constraints. - C is Incorrect: Reinforcement fine-tuning teaches behavior, not new factual knowledge. It is also computationally expensive and does not solve the need to ingest new documents in real-time. - D is Incorrect: Provisioned Throughput reserves model capacity for inference; it does not help the model learn new information or reduce the cost of updating its knowledge base. > Exam Tip / AWS Best Practice: For knowledge that changes frequently, use RAG (Bedrock Knowledge Bases) to keep data external to the model, ensuring low update latency and low compute cost compared to fine-tuning.
### Question 3 - SageMaker Endpoint Configuration for Spiky Traffic A gaming company runs a real-time cheating detection model. The traffic is highly spiky, scaling from 10 requests per minute to 5,000 requests per minute within seconds during new game launches. The business requires sub-100ms latency during peaks, but wants to minimize idle compute costs during off-peak hours. Which deployment option provides the BEST balance of latency and cost? - A) SageMaker Serverless Inference - B) SageMaker Batch Transform - C) SageMaker Real-Time Inference endpoint with AutoScaling - D) SageMaker Asynchronous Inference	Correct Answer: C Correct Answer Analysis: - C is Correct: SageMaker Real-Time Inference with AutoScaling can scale out compute instances to handle the 5,000 requests/minute spike while maintaining the strict sub-100ms latency requirement, and scale in to zero (or a minimum instance count) to save costs during off-peak periods. Distractor Analysis: - A is Incorrect: SageMaker Serverless Inference is great for spiky traffic and cost optimization, but it introduces cold-start latency that can easily exceed the strict sub-100ms SLA required for real-time gaming. - B is Incorrect: Batch Transform is for offline, asynchronous processing of large datasets, not for real-time, sub-100ms inference. - D is Incorrect: Asynchronous Inference queues requests, which introduces variable queue latency that violates the strict sub-100ms requirement. > Exam Tip / AWS Best Practice: For strict real-time latency SLAs under spiky traffic, use SageMaker Real-Time Inference with AutoScaling; avoid Serverless if cold starts violate latency constraints.
### Question 4 - Model Distillation for Cost Reduction A research firm uses a large, expensive foundation model via Amazon Bedrock to generate highly detailed summaries of scientific papers. The output quality is excellent, but the per-token inference cost is too high for their budget. They have 100,000 historical examples of paper-summary pairs and want to reduce inference costs by 80% while maintaining similar output quality and latency. Which strategy is the MOST cost-effective way to achieve this? - A) Use Provisioned Throughput for the large model to lower per-token costs - B) Distill the large model's behavior into a smaller, faster model using Amazon SageMaker JumpStart - C) Switch to a larger model and increase the temperature to generate shorter outputs - D) Use Amazon Bedrock Guardrails to filter out unnecessary tokens	Correct Answer: B Correct Answer Analysis: - B is Correct: Model distillation trains a smaller, cheaper "student" model to mimic the behavior of the large "teacher" model using the 100,000 examples. The resulting smaller model runs much faster, dramatically reduces inference costs, and maintains similar task-specific quality. Distractor Analysis: - A is Incorrect: Provisioned Throughput requires a long-term commitment and is for reserving capacity, not for drastically reducing the fundamental per-token cost of a massive model. - C is Incorrect: Switching to a larger model increases cost, and increasing temperature increases randomness; it does not reliably shorten output or maintain quality. - D is Incorrect: Bedrock Guardrails are for content filtering, safety, and PII redaction, not for reducing token generation volume or inference costs. > Exam Tip / AWS Best Practice: When you have a high-performing but expensive large model and a large dataset of its input-output pairs, use distillation to train a smaller, cheaper model for production inference.
### Question 5 - Bedrock Pricing Models for Steady-State Traffic An enterprise internal application sends a steady, predictable 500 requests per second to a foundation model on Amazon Bedrock, 24 hours a day, 7 days a week. The finance team wants to lock in the lowest possible inference cost for this specific workload over a 1-month period, as On-Demand pricing is proving too expensive. Which Bedrock feature should the team utilize to optimize cost without sacrificing latency? - A) SageMaker Batch Transform - B) Bedrock Provisioned Throughput - C) Bedrock Knowledge Bases - D) Amazon Bedrock Guardrails	Correct Answer: B Correct Answer Analysis: - B is Correct: Bedrock Provisioned Throughput allows you to purchase model units for a specific foundation model at a discounted rate compared to On-Demand. It is designed for steady, predictable workloads where you can commit to a baseline level of traffic, optimizing cost while guaranteeing latency. Distractor Analysis: - A is Incorrect: Batch Transform is an offline processing service for SageMaker models, not for real-time Bedrock API inference. - C is Incorrect: Bedrock Knowledge Bases is a RAG service for retrieving external data; it does not change the pricing model of the underlying foundation model inference. - D is Incorrect: Guardrails apply safety and content filters; they do not offer cost discounts for inference. > Exam Tip / AWS Best Practice: For predictable, high-volume, steady-state GenAI workloads on Amazon Bedrock, use Provisioned Throughput to lower per-token inference costs.
### Question 6 - Managed AI Services vs. Foundation Models for OCR An accounts payable department processes 50,000 scanned invoices daily. They need to extract vendor name, total amount, and line items. The solution must be highly accurate, cost-effective, and require the least operational overhead. The team is considering using a multimodal foundation model via Amazon Bedrock versus Amazon Textract. Which approach provides the MOST cost-effective and lowest-latency solution for this specific task? - A) Use a multimodal foundation model in Amazon Bedrock, passing the images as base64 strings - B) Fine-tune a foundation model on past invoices using Amazon SageMaker - C) Use Amazon Textract `AnalyzeExpense` API - D) Use Amazon Comprehend to extract key phrases from the OCR text	Correct Answer: C Correct Answer Analysis: - C is Correct: Amazon Textract `AnalyzeExpense` is a pre-trained, managed AI service specifically optimized for extracting data from invoices and receipts. It is significantly cheaper, faster, and more accurate for structured document extraction than a general-purpose GenAI model. Distractor Analysis: - A is Incorrect: Multimodal foundation models (like Claude 3 Sonnet) are extremely expensive and slow compared to specialized OCR services for high-volume, structured data extraction. - B is Incorrect: Fine-tuning introduces unnecessary operational overhead, compute costs, and data labeling requirements for a task that managed services already solve out-of-the-box. - D is Incorrect: Amazon Comprehend performs NLP (sentiment, entities) on text; it does not extract text from images. It would require Textract anyway as a prerequisite step. > Exam Tip / AWS Best Practice: For standard, solved tasks like document/invoice extraction, use specialized managed AWS AI services (Textract) over general-purpose Foundation Models to optimize both cost and latency.
### Question 7 - Batch vs. Real-Time Inference for Non-Urgent Workloads A marketing company runs a nightly job to generate personalized email drafts for 1 million users based on their browsing history. The job currently uses a SageMaker Real-Time endpoint and takes 4 hours to complete by looping through the users. The company wants to drastically reduce the compute cost of this job, as the drafts do not need to be sent until the next morning. Which architectural change will MOST reduce costs while meeting the business requirement? - A) Use SageMaker Serverless Inference instead of Real-Time endpoints - B) Switch to Amazon Bedrock Provisioned Throughput - C) Use SageMaker Batch Transform to run the inference job offline - D) Use Amazon Bedrock Knowledge Bases	Correct Answer: C Correct Answer Analysis: - C is Correct: SageMaker Batch Transform is designed for offline, high-volume inference. It provisions the necessary compute, processes the dataset in parallel, and shuts down automatically when complete. You only pay for the compute time used, drastically reducing costs compared to keeping a Real-Time endpoint running 24/7 for a nightly job. Distractor Analysis: - A is Incorrect: Serverless Inference is for spiky, intermittent real-time traffic. For 1 million sequential/looped requests, it would still be inefficient compared to a parallel batch job. - B is Incorrect: Provisioned Throughput is for Bedrock steady-state real-time traffic. The scenario mentions a SageMaker endpoint, and the workload is batch-oriented. - D is Incorrect: Bedrock Knowledge Bases is for RAG retrieval, not for managing offline inference compute costs for a custom model. > Exam Tip / AWS Best Practice: If a workload does not require real-time, sub-second latency (e.g., nightly batch processing), use SageMaker Batch Transform to avoid the cost of running a 24/7 Real-Time endpoint.
### Question 8 - Edge Deployment Latency and Memory Constraints An IoT company needs to deploy an image classification model to remote edge devices with strict hardware constraints (under 50MB of available memory). The model is currently 500MB and runs at 20ms latency per frame in the cloud. The devices have intermittent internet connectivity, so cloud inference is not viable. The team must optimize the model for the edge. Which AWS service should they use to meet the memory and latency constraints? - A) Amazon Bedrock Provisioned Throughput - B) Amazon SageMaker Neo - C) AWS Lambda with Serverless Inference - D) Amazon EC2 Inf1 instances	Correct Answer: B Correct Answer Analysis: - B is Correct: Amazon SageMaker Neo compiles machine learning models for edge devices, optimizing them for specific target hardware. It often uses quantization to reduce model size (e.g., from 500MB to under 50MB) and optimizes the computational graph for significantly lower latency on edge devices. Distractor Analysis: - A is Incorrect: Provisioned Throughput is for cloud-based Bedrock APIs; it does not help deploy a model to an edge device. - C is Incorrect: AWS Lambda is a cloud service and requires internet connectivity. It does not solve the 50MB edge device memory constraint. - D is Incorrect: EC2 Inf1 instances are cloud-based hardware accelerators. They do not solve the edge deployment requirement or the 50MB memory limit. > Exam Tip / AWS Best Practice: For models deployed to edge devices with strict memory and latency constraints, use Amazon SageMaker Neo to compile and quantize the model.
### Question 9 - Latency vs. Capability in GenAI Coding Assistants A software company is building an internal coding assistant using Amazon Bedrock. Developers use it to write boilerplate code, generate unit tests, and explain simple functions. The assistant must be highly responsive to fit into developer workflows (under 1 second to first token), but it does not need to solve complex architectural reasoning problems. The company wants to balance cost and performance. Which model selection provides the BEST balance of cost, latency, and performance for this use case? - A) Claude 3 Opus, because it is the most capable model for coding tasks - B) Claude 3 Haiku, because it provides sufficient capability for boilerplate and low latency - C) A custom-trained 70B parameter model on SageMaker Real-Time endpoints - D) Amazon Titan Text Premier for enterprise-grade accuracy	Correct Answer: B Correct Answer Analysis: - B is Correct: Claude 3 Haiku offers a balance of strong coding capability (for boilerplate and tests) with the lowest latency and cost among the Claude 3 family, perfectly fitting the strict sub-1-second TTFT requirement for a responsive developer experience. Distractor Analysis: - A is Incorrect: Claude 3 Opus is highly capable but has much higher latency and cost, which is unnecessary for simple boilerplate generation and would break the 1-second TTFT constraint. - C is Incorrect: Training and hosting a 70B model on SageMaker introduces massive operational overhead and high latency compared to a managed Bedrock model. - D is Incorrect: While capable, Titan Text Premier is generally slower and more expensive than Haiku for tasks that do not require heavy reasoning, making it suboptimal for strict latency SLAs. > Exam Tip / AWS Best Practice: Match the model size to the task complexity; for high-frequency, low-complexity tasks (like boilerplate generation) requiring strict latency, use speed-optimized models (e.g., Haiku) rather than heavy reasoning models (e.g., Opus).
### Question 10 - Performance vs. Cost in Multi-Modal RAG Architectures A media company is building a search engine for their vast library of 1 million video clips. Users search by text, and the system must return the exact timestamp of the matching video frame. The team plans to use a multimodal foundation model to generate embeddings for every frame of every video. The cost of embedding 1 billion frames using a heavy multimodal model is prohibitively expensive. They need a more cost-effective approach that still maintains high search accuracy. Which approach BEST reduces embedding costs while maintaining performance? - A) Use a smaller, faster embedding model (like Amazon Titan Embeddings) on keyframes extracted from the videos, rather than every frame - B) Fine-tune a large language model to output timestamps based on video transcripts - C) Use Claude 3 Opus to analyze the videos at runtime when a user searches - D) Increase the vector database size to handle the 1 billion frames more efficiently	Correct Answer: A Correct Answer Analysis: - A is Correct: Using a specialized, cheaper embedding model (like Titan Embeddings) and applying it only to extracted keyframes (rather than every single frame at 30 FPS) exponentially reduces the compute cost of generating embeddings while still maintaining a high-quality, searchable index. Distractor Analysis: - B is Incorrect: Fine-tuning an LLM to predict timestamps from transcripts is highly complex, ignores visual content, and is not a standard search architecture. - C is Incorrect: Using a massive model like Opus to process entire videos at runtime for every search query would result in extreme latency (minutes per query) and astronomical costs. - D is Incorrect: While a larger vector DB can store 1 billion vectors, it does not reduce the massive inference cost of generating those 1 billion embeddings in the first place. > Exam Tip / AWS Best Practice: To optimize embedding costs in multi-modal RAG, pre-process data to reduce volume (e.g., extract keyframes) and use specialized, lightweight embedding models (e.g., Titan Embeddings) instead of heavy foundation models.