Skill 4.2.7: Evaluate cost implications of using FMs for inference in production
| Question | Answer & Explanation |
|---|---|
| ### Question 1 - Provisioned Throughput for High-Volume Inference An AI engineering team is running a high-traffic customer support chatbot using a foundation model on Amazon Bedrock. The traffic is highly predictable, maintaining a steady, heavy volume 24/7 throughout the year. The team needs to optimize the inference costs and is looking for the most cost-effective purchasing option for this baseline load. Which approach should the team implement to minimize costs while meeting the workload constraints? - A) Configure Amazon SageMaker AI Savings Plans to cover the Bedrock on-demand token usage. - B) Purchase Provisioned Throughput for the foundation model on Amazon Bedrock. - C) Switch the chatbot to use Amazon Bedrock Batch Inference for offline processing. - D) Deploy the foundation model on a SageMaker Real-Time Endpoint and enable auto-scaling. |
Correct Answer: B Correct Answer Analysis: - B is Correct: For predictable, steady, and high-volume inference workloads on Amazon Bedrock, Provisioned Throughput allows you to reserve model units at a fixed hourly rate. This provides a discounted rate compared to On-Demand token pricing, making it the most cost-effective option for a continuous baseline load. Distractor Analysis: - A is Incorrect: SageMaker AI Savings Plans apply exclusively to Amazon SageMaker ML instance usage (compute hours), not to Amazon Bedrock token-based on-demand usage. - C is Incorrect: Batch Inference is designed for asynchronous workloads where immediate responses are not required. A 24/7 customer support chatbot requires real-time, interactive responses. - D is Incorrect: Deploying the model on SageMaker Real-Time Endpoints requires managing the underlying compute infrastructure and does not leverage the fully managed, serverless API of Amazon Bedrock, potentially increasing operational overhead and cost. > Key Takeaway: In the Generative AI world, the equivalent of committing to a steady compute baseline (Savings Plans) is purchasing Provisioned Throughput for token-based foundation model inference. |
| ### Question 2 - Prompt Caching Invalidation A development team is building an internal assistant using Amazon Bedrock and a large language model. To optimize inference costs and latency, they enable Prompt Caching. However, the system prompt includes a dynamically generated timestamp and user ID at the very beginning of the prompt sequence to track sessions. The team notices that cost savings from caching are not materializing. What is the most likely cause of this issue, and how can it be resolved while maintaining the least operational overhead? - A) The cache is invalidated on each request due to the dynamic variables; move the static instructions to the beginning of the prompt and dynamic variables to the end. - B) Prompt caching only works with smaller foundation models; switch from Claude 3 Opus to Claude 3 Haiku. - C) Amazon Bedrock does not support prompt caching for system prompts; use a custom SageMaker endpoint to cache KV states. - D) The maximum response length is too short; increase the max_tokens parameter to allow the cache to build properly. |
Correct Answer: A Correct Answer Analysis: - A is Correct: Prompt caching relies on exact token sequence matching from the beginning of the prompt. Placing dynamic content (like a timestamp) at the start changes the sequence hash on every request, causing continuous cache misses. Moving static content to the beginning and dynamic content to the end allows the static prefix to be cached and reused. Distractor Analysis: - B is Incorrect: Prompt caching is supported across various model sizes, including large models like Claude 3 Opus. The issue is the prompt structure, not the model size. - C is Incorrect: Amazon Bedrock natively supports prompt caching (e.g., for Anthropic Claude models) without requiring custom SageMaker endpoints or manual KV state management. - D is Incorrect: The max_tokens parameter controls the output length and has no bearing on whether the input prompt prefix is eligible for caching.> Key Takeaway: Prompt caching requires a stable, static prefix. Any dynamic content placed at the beginning of the prompt invalidates the cache and eliminates cost savings. |
| ### Question 3 - Batch Inference for Asynchronous Workloads A media company needs to process 10,000 long-form video transcripts daily to generate summaries and extract key topics using a foundation model. The process is entirely asynchronous, and the summaries just need to be available in an Amazon S3 bucket within 24 hours of the video upload. The team must optimize for the lowest cost of token inference. Which Amazon Bedrock invocation method meets these requirements? - A) Provisioned Throughput with a fixed hourly rate. - B) On-Demand API invocation via AWS Lambda. - C) Amazon Bedrock Batch Inference. - D) Amazon SageMaker Asynchronous Inference. |
Correct Answer: C Correct Answer Analysis: - C is Correct: Amazon Bedrock Batch Inference is designed for asynchronous workloads that do not require real-time responses. It allows you to process large datasets offloaded to S3, typically at a lower cost per token compared to real-time On-Demand API invocations, making it ideal for a 24-hour SLA. Distractor Analysis: - A is Incorrect: Provisioned Throughput is for steady, real-time interactive traffic. It commits you to a fixed hourly rate, which is wasteful for an asynchronous batch job that can run once a day. - B is Incorrect: Using On-Demand API invocations via Lambda is a real-time execution model. While it works, it does not offer the cost benefits specifically designed for large, asynchronous batch workloads. - D is Incorrect: SageMaker Asynchronous Inference is for hosting custom models on ML instances, not for invoking foundation models via the fully managed Amazon Bedrock service. > Key Takeaway: For asynchronous workloads with flexible time constraints (e.g., 24-hour SLA), use Amazon Bedrock Batch Inference to significantly reduce token inference costs compared to real-time On-Demand. |
| ### Question 4 - Agent Resource Consumption Tracing An enterprise is deploying a complex Amazon Bedrock Agent that orchestrates multiple foundation model calls, retrieves data from a knowledge base, and invokes external APIs. The monthly token bill has doubled, but the number of end-user requests has remained flat. The team needs to identify exactly which component (planning, tool call, or synthesis) is consuming the most tokens per invocation with the least operational overhead. Which AWS service or feature should they use to gain this visibility? - A) AWS X-Ray distributed tracing. - B) Amazon CloudWatch Log Insights. - C) Amazon Bedrock AgentCore Observability. - D) AWS Compute Optimizer. |
Correct Answer: C Correct Answer Analysis: - C is Correct: Amazon Bedrock AgentCore Observability is specifically built to capture the internal inference calls, tool invocations, and memory operations inside an agent's execution. It allows engineers to see the exact resource consumption pattern and token usage per invocation step (planning, retrieval, synthesis). Distractor Analysis: - A is Incorrect: AWS X-Ray is excellent for tracing distributed microservices and latency bottlenecks, but it does not natively break down the internal token consumption and agent-specific cognitive steps like AgentCore Observability does. - B is Incorrect: While CloudWatch Log Insights can parse logs, it requires extensive custom logging and parsing logic to reconstruct the agent's internal step-by-step token consumption, introducing high operational overhead. - D is Incorrect: AWS Compute Optimizer analyzes compute resource utilization (like EC2 or Lambda) and recommends right-sizing, but it provides no visibility into FM token consumption. > Key Takeaway: Agent cost is amplification; one user request triggers multiple FM calls. Use Amazon Bedrock AgentCore Observability to attribute token costs to specific internal agent steps. |
| ### Question 5 - Model Selection (Right-Sizing FMs) A startup is building an internal FAQ bot for its HR department using Amazon Bedrock. The team initially selected a large, highly capable foundation model (e.g., Claude 3 Opus) to ensure high quality. However, the QA team found that the answers are simple, factual responses retrieved directly from a static document corpus. The team needs to optimize the solution for the lowest cost without sacrificing response accuracy. What is the most effective way to optimize the inference cost for this specific workload? - A) Configure Amazon EC2 Instance Savings Plans for the underlying Bedrock compute. - B) Switch to a smaller, faster foundation model (e.g., Claude 3 Haiku) that meets the quality requirement. - C) Enable auto-scaling on the Amazon Bedrock endpoint to scale to zero overnight. - D) Implement a custom load balancer to route simple queries to a cached response system. |
Correct Answer: B Correct Answer Analysis: - B is Correct: In the world of foundation models, "right-sizing" translates to model selection. If a smaller, less expensive model can achieve the required accuracy for simple, factual FAQs, switching to it will drastically reduce the cost per input and output token compared to using an oversized model. Distractor Analysis: - A is Incorrect: Bedrock is a serverless, managed service; you cannot apply EC2 Instance Savings Plans to it. Token usage is billed on-demand or via Bedrock Provisioned Throughput. - C is Incorrect: Amazon Bedrock is already a serverless API that scales inherently. There is no "endpoint" to configure auto-scaling on in the traditional SageMaker sense; you only pay for tokens processed. - D is Incorrect: While caching common queries is a good practice, it introduces unnecessary operational overhead to build a custom load balancer and cache system when simply selecting a cheaper model that easily handles the queries solves the core cost issue. > Key Takeaway: Right-sizing a traditional ML endpoint means choosing a smaller instance; right-sizing a Foundation Model means choosing a smaller, less expensive model that still meets the quality bar. |
| ### Question 6 - Embedding Cost Optimization A financial services company is building a RAG pipeline using Amazon Bedrock Knowledge Bases. They are processing millions of historical SEC filings, many of which contain duplicate boilerplate text (e.g., standard disclaimers and cover pages). The team is experiencing unexpectedly high costs associated with generating vector embeddings. Which cost optimization strategy should the team implement to reduce embedding computation costs with the least operational overhead? - A) Deduplicate the text content before invoking the embedding model. - B) Purchase Provisioned Throughput for the embedding model. - C) Switch to a larger foundation model for embeddings to process more text per call. - D) Store the embeddings in Amazon DynamoDB instead of OpenSearch Serverless. |
Correct Answer: A Correct Answer Analysis: - A is Correct: Embedding models charge based on the number of input tokens processed. If millions of documents contain identical boilerplate text, deduplicating or removing standard disclaimers before chunking and embedding will drastically reduce the total number of tokens sent to the embedding model, directly lowering costs. Distractor Analysis: - B is Incorrect: Provisioned Throughput is generally suited for high-volume, steady-state generative LLM traffic, not typically for a one-time or batch embedding ingestion pipeline. It would commit the team to a fixed hourly rate they may not need. - C is Incorrect: Switching to a larger model would increase the cost per token, not decrease it. Furthermore, it does not address the root cause of the high cost (processing duplicate text). - D is Incorrect: The choice of vector database (DynamoDB vs. OpenSearch) affects storage and retrieval costs, but has no impact on the compute cost of generating the embeddings in the first place. > Key Takeaway: Embedding costs scale with input token volume. Deduplicating text and removing irrelevant boilerplate before generating embeddings is a critical cost control for RAG ingestion. |
| ### Question 7 - Token Cost Governance and Budgets A company has deployed multiple Generative AI applications using Amazon Bedrock across different departments. The FinOps team needs to proactively monitor token usage to prevent budget overruns. They want to receive an automated alert when a specific department's projected monthly token spend is on track to exceed their allocated budget. Which combination of AWS services should the team use to fulfill this requirement? (Select TWO) - A) AWS Budgets - B) AWS Compute Optimizer - C) Cost Allocation Tags - D) AWS Trusted Advisor - E) Amazon Macie |
Correct Answer: A and C Correct Answer Analysis: - A is Correct: AWS Budgets allows you to set custom cost and usage thresholds and sends proactive alerts via email or SNS when forecasted spending is projected to exceed the defined budget limit. - C is Correct: Cost Allocation Tags must be applied to the Bedrock resources (or IAM roles invoking them) to attribute the token spend to specific departments. Without tags, AWS Budgets cannot filter spend by department. Distractor Analysis: - B is Incorrect: AWS Compute Optimizer recommends right-sized EC2, Lambda, and EBS resources based on utilization. It does not analyze Bedrock token spend or send budget alerts. - D is Incorrect: AWS Trusted Advisor checks accounts against best practices for cost, performance, and security, but it does not provide custom, forecasted budget alerts based on departmental tags. - E is Incorrect: Amazon Macie is a data security service that discovers and protects sensitive data (PII) in S3; it has no cost management or budgeting functionality. > Key Takeaway: To govern token spend by team, apply Cost Allocation Tags to resources/usage and configure AWS Budgets to send alerts on forecasted overages. |
| ### Question 8 - Context Window Optimization A software company has built a coding assistant using Amazon Bedrock. For each user prompt, the application sends the entire chat history (sometimes exceeding 10,000 tokens) to the model to maintain context. The company is seeing extremely high input token costs. The team needs to optimize input token costs without losing the conversational context. Which prompt engineering approach best optimizes the input token cost? - A) Increase the max_tokens parameter for the output generation.- B) Implement summarization of older chat history before passing it in the context window. - C) Switch to Bedrock Batch Inference for all user prompts. - D) Use a larger foundation model that accepts a larger context window. |
Correct Answer: B Correct Answer Analysis: - B is Correct: Summarizing older chat history (e.g., using a smaller, cheaper model or a background process) and passing the summary plus recent messages to the main model drastically reduces the number of input tokens billed per request while preserving the necessary conversational context. Distractor Analysis: - A is Incorrect: The max_tokens parameter controls the maximum number of output tokens the model can generate. It does not affect the number of input tokens sent in the prompt.- C is Incorrect: Batch Inference is for asynchronous, non-interactive workloads. A coding assistant requires real-time, interactive responses. - D is Incorrect: Using a model with a larger context window allows you to send more tokens, but it does not reduce the cost of the tokens you are already sending; in fact, larger models often charge more per token. > Key Takeaway: Prompt discipline is the Generative AI equivalent of auto-scaling. Trimming context windows and summarizing history directly reduces the input tokens billed on every request. |
| ### Question 9 - Monitoring Token Usage Dashboards An ML engineering team wants to investigate a sudden spike in their Amazon Bedrock invoice. They suspect that a recent code change caused the application to send significantly longer prompts to the model, rather than an actual increase in user traffic. Which AWS service should they use to visualize the invocation count alongside token usage to confirm their hypothesis? - A) AWS X-Ray - B) Amazon CloudWatch - C) AWS Cost Anomaly Detection - D) AWS Artifact |
Correct Answer: B Correct Answer Analysis: - B is Correct: Amazon CloudWatch natively collects metrics for Amazon Bedrock, including InvocationCount and InputTokenCount / OutputTokenCount. By building a CloudWatch Dashboard placing these metrics side-by-side, the team can easily see if token usage is rising disproportionately compared to the number of invocations.Distractor Analysis: - A is Incorrect: AWS X-Ray traces requests across components to find latency bottlenecks, but it is not the primary tool for tracking aggregate billing metrics like token counts over time. - C is Incorrect: AWS Cost Anomaly Detection alerts you when your overall spend deviates from expected patterns, but it does not provide the granular operational metrics (Invocation vs. Token count) needed to diagnose the root cause of the spike. - D is Incorrect: AWS Artifact is a service for downloading compliance and audit reports (like SOC 2 or ISO 27001), completely unrelated to operational metrics or token monitoring. > Key Takeaway: Place token usage beside invocation count on a CloudWatch dashboard. If invocations are flat but token usage spikes, the issue is heavier prompts, not more traffic. |
| ### Question 10 - Translating Traditional ML Optimization to GenAI A team of traditional ML engineers is transitioning a legacy recommendation model to a Generative AI summarization pipeline on Amazon Bedrock. The team is accustomed to optimizing costs by sizing SageMaker endpoints to the average utilization and using Spot Instances for baseline traffic. Which concept correctly translates their "SageMaker Spot Instance for baseline" optimization strategy to the Amazon Bedrock token-based billing model? - A) Using Amazon EC2 Spot Instances for the Bedrock API gateway. - B) Enabling Provisioned Throughput for steady, predictable token volume. - C) Configuring Amazon Bedrock to use Spot Token Pricing. - D) Implementing Multi-Model Endpoints (MMEs) for the foundation models. |
Correct Answer: B Correct Answer Analysis: - B is Correct: In traditional ML, committing to a steady baseline with Savings Plans or utilizing steady capacity reduces the hourly compute rate. For Amazon Bedrock, the equivalent for a steady, predictable token volume is purchasing Provisioned Throughput, which commits to a fixed rate for model units, lowering the cost per token compared to On-Demand. Distractor Analysis: - A is Incorrect: Amazon Bedrock is a fully managed serverless service. You do not manage the underlying compute or API gateways via EC2 Spot Instances. - C is Incorrect: There is no such thing as "Spot Token Pricing" in Amazon Bedrock. Token pricing is either On-Demand or Provisioned Throughput. - D is Incorrect: Multi-Model Endpoints (MMEs) are a feature of Amazon SageMaker for hosting multiple custom models on shared compute instances. They are not applicable to invoking managed foundation models via Bedrock. > Key Takeaway: When moving from SageMaker to Bedrock, the concept of committing to a steady baseline changes from SageMaker Savings Plans (compute hours) to Bedrock Provisioned Throughput (token throughput). |