Skill 4.2.9: Manage FM inference costs with usage optimization

Question	Answer & Explanation
### Question 1 - Optimizing Repeating Prompts in Amazon Bedrock An enterprise HR company is using an Amazon Bedrock-powered assistant to analyze employee handbook queries. The system sends a 10,000-token static company policy document as part of the prompt for every user query, along with a short dynamic user question. The team needs to reduce the token processing costs and latency of these requests with the least operational overhead. Which solution will meet these requirements? - A) Configure an AWS Lambda function to summarize the 10,000-token document before sending the request to Bedrock. - B) Use Amazon Bedrock Provisioned Throughput to reserve model units for the baseline traffic. - C) Enable Prompt Caching in Amazon Bedrock to cache the static 10,000-token document prefix. - D) Switch from an on-demand foundation model to a smaller, less capable foundation model. (Select ONE)	Correct Answer: C Correct Answer Analysis: - C is Correct: Amazon Bedrock Prompt Caching allows the system to cache the Key-Value (KV) states of a static prompt prefix. Since the 10,000-token document is repeated across requests, caching it significantly reduces the input token processing cost and latency for subsequent requests, meeting the requirement with no custom code. Distractor Analysis: - A is Incorrect: Summarizing the document dynamically with a Lambda function adds compute costs, latency, and significant operational overhead compared to native caching. - B is Incorrect: Provisioned Throughput secures capacity for predictable workloads by paying a fixed hourly rate, but it does not specifically optimize the cost of repeating tokens in the prompt itself. - D is Incorrect: Switching to a smaller model might compromise the quality of the analysis and does not solve the fundamental inefficiency of repeatedly sending a 10,000-token static context. > Key Takeaway: Prompt Caching reduces cost and latency for workloads that repeatedly send the same static context (like system instructions or large reference documents) by avoiding recomputation of the prompt prefix.
### Question 2 - Asynchronous Document Summarization Cost Optimization A legal firm processes thousands of lengthy court transcripts daily using a large language model on Amazon Bedrock. The workload is highly asynchronous, with no requirement for real-time interactive responses, and the firm wants to optimize the inference costs associated with processing this massive volume of text. Which Amazon Bedrock feature should the firm use to achieve the most cost-effective solution? - A) Amazon Bedrock Batch Inference - B) Amazon Bedrock Provisioned Throughput - C) Amazon Bedrock Guardrails - D) Amazon Bedrock Knowledge Bases (Select ONE)	Correct Answer: A Correct Answer Analysis: - A is Correct: Amazon Bedrock Batch Inference is designed specifically for asynchronous processing of large datasets. It offers a significant cost discount (typically 50%) compared to on-demand inference, making it the most cost-effective solution for non-real-time bulk processing of transcripts. Distractor Analysis: - B is Incorrect: Provisioned Throughput is used to guarantee model capacity for steady, predictable workloads at a fixed hourly rate, but it does not offer the bulk discount benefits of Batch Inference for asynchronous work. - C is Incorrect: Bedrock Guardrails are used for safety, content filtering, and PII redaction, not for reducing inference costs. - D is Incorrect: Knowledge Bases are used for Retrieval-Augmented Generation (RAG) to query existing data, not for generating summaries of newly uploaded bulk data in a batch process. > Key Takeaway: For asynchronous, bulk processing workloads on Amazon Bedrock, Batch Inference provides a substantial cost discount over on-demand token pricing.
### Question 3 - Monitoring Agent Resource Consumption A generative AI application uses an Amazon Bedrock Agent equipped with multiple tools (e.g., database query, web search) to answer complex financial questions. The finance team has noticed that the monthly token bill is rising rapidly even though user traffic has remained steady. The ML engineer needs to understand exactly where the token consumption is occurring within the agent's workflow. Which AWS service or feature provides the granular visibility required to troubleshoot this specific cost spike? - A) AWS Compute Optimizer - B) Amazon Bedrock AgentCore Observability - C) AWS Trusted Advisor - D) AWS Cost Explorer filtered by Bedrock service (Select ONE)	Correct Answer: B Correct Answer Analysis: - B is Correct: Amazon Bedrock AgentCore Observability captures the detailed traces of an agent's invocation, including the planning phase, tool executions, and the final synthesis call. This allows the engineer to see exactly how many tokens are consumed by each step of the agent's process, revealing hidden loops or overly verbose tool responses. Distractor Analysis: - A is Incorrect: AWS Compute Optimizer analyzes compute resource utilization (like EC2 or Lambda) and recommends right-sizing, but it has no visibility into Bedrock token consumption or agent orchestration traces. - C is Incorrect: AWS Trusted Advisor provides high-level account best-practice checks, including identifying idle resources, but does not trace Bedrock agent token execution paths. - D is Incorrect: While Cost Explorer shows the total Bedrock spend, it cannot break down the spend into individual agent steps or tool calls to identify the specific cause of the spike. > Key Takeaway: Agent invocations amplify costs through multiple internal LLM calls (planning, tool calling, synthesis); use Bedrock AgentCore Observability to trace and optimize token consumption per agent step.
### Question 4 - Model Selection for Cost Optimization An e-commerce company is building a customer support chatbot on Amazon Bedrock to classify incoming messages into categories (e.g., "Shipping", "Returns", "Billing"). The team initially selected a large, highly capable foundation model (e.g., Anthropic Claude 3 Opus) for this task. The workload has high volume, and the team wants to drastically reduce inference costs while maintaining acceptable classification accuracy. What is the most appropriate action to optimize costs for this specific task? - A) Implement Amazon Bedrock Provisioned Throughput for Claude 3 Opus. - B) Downsize the model to a smaller, faster, and cheaper model (e.g., Claude 3 Haiku) better suited for classification. - C) Enable Prompt Caching for the classification instructions. - D) Use AWS Batch to process the chat messages at the end of the day. (Select ONE)	Correct Answer: B Correct Answer Analysis: - B is Correct: For simple, high-volume tasks like text classification, using a smaller, lighter-weight model (like Claude 3 Haiku) drastically reduces the cost-per-token compared to a large frontier model, while still easily handling the classification logic. Distractor Analysis: - A is Incorrect: Provisioned Throughput would guarantee capacity for the large model, but the cost per token remains high. It does not address the fundamental overkill of using a large model for a simple task. - C is Incorrect: Prompt caching reduces the cost of the static prompt prefix, but the dynamic input and generation costs still use the expensive large model's per-token rate. - D is Incorrect: A customer support chatbot requires real-time, interactive responses. Delaying classifications to the end of the day using AWS Batch ruins the user experience. > Key Takeaway: "Right-sizing" in the Bedrock world means selecting the smallest model capable of meeting the quality requirement, which is the most effective way to reduce on-demand token costs for high-volume tasks.
### Question 5 - Setting Proactive Cost Alerts for Token Usage A startup is developing a generative AI application using Amazon Bedrock. The application is in the beta phase, and the developers are testing it with a growing number of users. The CTO wants to receive an email notification if the Bedrock token consumption is projected to exceed $500 in the current month, to avoid unexpected overages. Which AWS service can be configured to meet this requirement with the least operational overhead? - A) AWS Cost Explorer - B) AWS Budgets - C) Amazon CloudWatch Alarms - D) AWS Trusted Advisor (Select ONE)	Correct Answer: B Correct Answer Analysis: - B is Correct: AWS Budgets allows users to set custom cost budgets and includes "forecasted" alerting. It will send an email (or SNS notification) when the projected spend is on track to exceed the $500 threshold, giving the CTO time to act before the budget is actually breached. Distractor Analysis: - A is Incorrect: AWS Cost Explorer is used to visualize and analyze historical spend data, but it does not natively generate proactive, threshold-based email alerts. - C is Incorrect: While CloudWatch can alert on token usage metrics, it requires custom metric math and configuration to map token counts to dollar amounts, representing higher operational overhead than AWS Budgets for financial tracking. - D is Incorrect: AWS Trusted Advisor offers pre-built checks for idle resources and service limits, but does not allow for custom financial budget alerts based on forecasted spend. > Key Takeaway: AWS Budgets is the native tool for setting custom financial limits and receiving alerts based on actual or forecasted spend across AWS services, including Amazon Bedrock.
### Question 6 - RAG Embedding Cost Management A financial services firm uses Amazon Bedrock Knowledge Bases to index thousands of daily updated research reports into an Amazon OpenSearch Serverless vector database. The team has noticed that the daily ingestion job is generating unexpectedly high costs. They want to reduce the cost of computing embeddings for this workload. Which action will directly optimize the embedding computation costs without impacting the query performance? - A) Switch to a smaller embedding model (e.g., Amazon Titan Text Embeddings Lite) if the dimensionality and quality meet the search requirements. - B) Enable Provisioned Throughput on the OpenSearch Serverless collection. - C) Increase the chunk size to the maximum limit to reduce the number of OpenSearch documents. - D) Disable the vector index at night when users are not querying the system. (Select ONE)	Correct Answer: A Correct Answer Analysis: - A is Correct: Embedding models are billed per token. If a smaller embedding model (which processes tokens at a lower cost per token) still produces embeddings of sufficient quality for the retrieval task, switching to it directly reduces the daily ingestion costs. Distractor Analysis: - B is Incorrect: Provisioned Throughput on OpenSearch Serverless relates to the capacity allocated for searching the index, not the cost of computing the embeddings themselves via Bedrock. - C is Incorrect: Increasing chunk size to the maximum limit might reduce the number of vectors stored, but it severely degrades the quality of retrieval (making chunks too large to be semantically accurate), and does not necessarily lower the Bedrock embedding processing cost proportional to the tokens read. - D is Incorrect: Disabling the vector index at night would prevent the daily ingestion job from running and complicate availability; it does not optimize the embedding computation cost itself. > Key Takeaway: In a RAG pipeline, embedding computation costs scale with text volume. Selecting a smaller, cheaper embedding model that meets your semantic search quality needs is a direct cost optimization.
### Question 7 - Caching Dynamic Prompts An AI engineer is implementing Prompt Caching in Amazon Bedrock for an application that generates marketing copy. The prompt consists of a 2,000-token static system prompt, followed by a dynamically generated timestamp and user session ID, and then the user's specific request. After a week of testing, the engineer notices that cache hit rates are 0% and costs have actually increased. What is the most likely cause of this issue? - A) The static system prompt is not long enough to meet the minimum token requirement for caching. - B) The dynamic timestamp and session ID are placed before the static prompt prefix, invalidating the cache on every request. - C) The model selected does not support prompt caching. - D) The application is using on-demand inference instead of batch inference. (Select ONE)	Correct Answer: B Correct Answer Analysis: - B is Correct: Prompt caching requires an exact, sequential match of tokens from the beginning of the prompt. If dynamic variables (like a timestamp or session ID) are inserted before or within the static prefix, the sequence hash changes on every request, causing a cache miss. Additionally, writing tokens to the cache often incurs a higher cost than reading uncached input tokens, leading to increased overall costs. Distractor Analysis: - A is Incorrect: While there is a minimum prefix length for caching, 2,000 tokens generally exceeds the minimum (which is often around 1024 tokens depending on the model). The 0% hit rate points to a sequence mismatch, not length. - C is Incorrect: The prompt states caching is implemented but hitting 0%, implying the model supports it but the implementation is flawed. - D is Incorrect: The choice between on-demand and batch inference does not affect the mechanics of prompt caching hit rates within real-time invocations. > Key Takeaway: To achieve prompt cache hits, static content must be placed at the very beginning of the prompt. Any dynamic content must be appended at the end of the prompt.
### Question 8 - Capacity Commitment for FM Inference A media streaming company runs a high-volume, 24/7 recommendation and summary generation workload on Amazon Bedrock. The traffic is highly predictable, with a steady baseline of token generation occurring every hour. The company is currently using on-demand token pricing and wants to optimize costs for this steady baseline. Which Amazon Bedrock pricing model is best suited for this workload? - A) Amazon Bedrock Batch Inference - B) Amazon Bedrock Provisioned Throughput - C) Amazon SageMaker AI Savings Plans - D) Compute Savings Plans (Select ONE)	Correct Answer: B Correct Answer Analysis: - B is Correct: Amazon Bedrock Provisioned Throughput allows you to reserve model units at a fixed hourly rate for a 1-month or 3-month commitment. For a steady, predictable, 24/7 high-volume baseline, this is cheaper than paying the on-demand per-token rate. Distractor Analysis: - A is Incorrect: Batch Inference is for asynchronous, bulk processing of stored data, not for continuous, real-time inference workloads. - C is Incorrect: SageMaker AI Savings Plans apply to SageMaker ML instances (compute hours), not Amazon Bedrock token-based or provisioned throughput usage. - D is Incorrect: Compute Savings Plans apply to EC2, Fargate, and Lambda, not Amazon Bedrock foundation model inference. > Key Takeaway: For predictable, steady-state foundation model inference traffic, Provisioned Throughput is the Bedrock equivalent of a Savings Plan, offering a discounted rate for committed usage.
### Question 9 - Tracing Latency vs. Cost in an Agent An AWS Step Functions state machine orchestrates a complex multi-agent workflow using Amazon Bedrock. The workflow takes an average of 15 seconds to complete, but users are complaining about intermittent latency spikes up to 45 seconds. The ML team needs to identify which specific component or tool call within the agent workflow is causing the delay. Which combination of services should be used to pinpoint the bottleneck? (Select TWO) - A) Amazon CloudWatch Metrics - B) AWS X-Ray - C) AWS Budgets - D) Amazon Bedrock AgentCore Observability (Select TWO)	Correct Answer: B and D Correct Answer Analysis: - B is Correct: AWS X-Ray provides distributed tracing across the Step Functions state machine and the Bedrock API calls, helping visualize the end-to-end request path and identifying which step introduces the latency spike. - D is Correct: Amazon Bedrock AgentCore Observability traces the internal operations of the Bedrock agent, revealing the time spent on planning, retrieval, and specific tool executions (like a database query or API call) that might be hanging. Distractor Analysis: - A is Incorrect: Amazon CloudWatch Metrics provide aggregate time-series data (like average latency or total invocations), which can show that a spike occurred, but not where the specific bottleneck is located within the agent's internal tool calls. - C is Incorrect: AWS Budgets is a financial tracking tool and has no capability to trace or monitor application latency or execution paths. > Key Takeaway: Use AWS X-Ray for tracing across distributed components (like Step Functions to Bedrock), and Bedrock AgentCore Observability for tracing the internal execution steps of the Bedrock agent itself.
### Question 10 - Managing Token Costs via Application Logic A software company has built a coding assistant using Amazon Bedrock. Users frequently paste large codebases into the prompt and ask the model to "explain everything." The team wants to control the rising inference costs by limiting the amount of output the model generates, while keeping the solution completely native to AWS with the least operational overhead. Which approach will effectively manage the output token costs? - A) Implement a custom AWS Lambda proxy to count words in the response and cut off the connection. - B) Set the `max_gen_tokens` (or equivalent max tokens to generate) parameter in the Bedrock `InvokeModel` API request. - C) Use an Amazon Bedrock Guardrail to block long responses. - D) Switch the workflow to Amazon Bedrock Batch Inference. (Select ONE)	Correct Answer: B Correct Answer Analysis: - B is Correct: The `InvokeModel` or `Converse` API in Amazon Bedrock accepts inference configuration parameters, including `maxTokens` (the maximum number of tokens the model can generate). Setting this parameter caps the output length directly at the API level, strictly controlling the output token cost without requiring custom code. Distractor Analysis: - A is Incorrect: Using a Lambda proxy to count words and cut off a streaming connection is complex, prone to errors, and introduces unnecessary operational overhead and latency. - C is Incorrect: Bedrock Guardrails are used for content moderation, PII redaction, and safety filters, not for enforcing strict response length limits for cost control. - D is Incorrect: The coding assistant is an interactive, real-time use case. Batch inference is inappropriate as it does not provide real-time responses. > Key Takeaway: The most efficient way to control output token costs at the application level is to use the native inference configuration parameters (like `maxTokens`) available in the Bedrock API request.