Skill 4.2.10: Monitor AI-specific cost patterns (for example, token usage optimization, embedding computation costs, vector database storage optimization)
| Question | Answer & Explanation |
|---|---|
| ### Question 1 - Token Usage Optimization with Prompt Caching An AI engineering team is using Amazon Bedrock to power an enterprise customer service chatbot. The system prompt includes extensive company policy documents (10,000 tokens) and remains identical across 95% of user requests. The team needs to reduce inference costs while maintaining the exact same response quality and low latency. Which solution is the most cost-effective while meeting all constraints? - A) Decrease the maximum output response length to 50 tokens for all requests. - B) Enable prompt caching for the static system prompt prefix in Amazon Bedrock. - C) Switch to a smaller, less capable foundation model for all customer service requests. - D) Use an AWS Lambda function to truncate the user input before sending it to Amazon Bedrock. |
Correct Answer: B Correct Answer Analysis: - B is Correct: Prompt caching allows the system to store the pre-computed Key-Value (KV) attention states of the repetitive 10,000-token prefix. Subsequent requests reuse this cache, significantly lowering the input token processing costs and reducing latency, all without changing the model or the response quality. Distractor Analysis: - A is Incorrect: Capping the output length controls output token costs but does not address the massive recurring input token cost of the 10,000-token system prompt. It also risks degrading response quality by cutting off complete answers. - C is Incorrect: Switching to a smaller model violates the constraint of maintaining the exact same response quality, as smaller models may struggle with complex policy interpretations. - D is Incorrect: Truncating user input removes context, leading to degraded answer accuracy. It also fails to address the cost of the large static system prompt. > Exam Tip: Prompt caching optimizes costs for static input prefixes by avoiding redundant token processing, without compromising model capability or output quality. |
| ### Question 2 - Monitoring Embedding Computation Costs An enterprise builds a RAG solution using Amazon Bedrock Knowledge Bases. The ingestion pipeline parses thousands of new PDF documents daily from an S3 bucket, chunks them, and generates vector embeddings using a Titan embedding model. The AI team notices a spike in daily Bedrock token costs but cannot easily determine if the spike is from user RAG queries or the background document ingestion process. Which approach has the least operational overhead to monitor and isolate these specific cost patterns? - A) Implement AWS X-Ray to trace the embedding generation calls and manually count tokens in the trace metadata. - B) Use AWS Cost Explorer to analyze Amazon Bedrock usage by API operation and filter by Bedrock usage types (e.g., Ingestion vs. Query). - C) Enable Amazon CloudWatch Logs on the S3 bucket to track the byte size of incoming PDF files and estimate token counts. - D) Create an AWS Budgets report that sends a daily email summary of the total aggregate Bedrock service spend. |
Correct Answer: B Correct Answer Analysis: - B is Correct: AWS Cost Explorer natively tracks AWS service usage and costs. By filtering for Amazon Bedrock and grouping by API operation or Usage Type, the team can objectively distinguish between token usage for Knowledge Base ingestion (embedding generation) and token usage for model invocation (RAG queries) without writing custom code. Distractor Analysis: - A is Incorrect: AWS X-Ray is designed for tracing distributed requests and latency analysis, not for aggregating financial cost or total token consumption metrics across a workload. Implementing custom token counting in X-Ray introduces unnecessary operational overhead. - C is Incorrect: Tracking S3 byte sizes provides a rough estimate of document size but does not equate to exact token counts (which vary by tokenizer) and fails to account for the tokens generated during user RAG queries. - D is Incorrect: AWS Budgets provides alerts on aggregate spend but does not natively break down the spend by specific API operations or isolate ingestion vs. query costs. > Exam Tip: Use AWS Cost Explorer to break down AI service costs by API operation and usage type to identify whether token consumption is driven by ingestion or user queries. |
| ### Question 3 - Vector Database Storage Optimization A retail company is storing millions of product catalog embeddings in an Amazon OpenSearch Serverless collection. As the catalog grows, the storage costs for the vector index become prohibitively expensive. The team needs to reduce the vector database storage costs without dropping products from the catalog or changing the embedding model provider. Which solution is the most cost-effective for vector database storage optimization? - A) Provision a larger OpenSearch Serverless collection to take advantage of bulk storage discounts. - B) Switch from OpenSearch Serverless to a managed Amazon RDS PostgreSQL database using the pgvector extension.- C) Reduce the ingestion frequency of the embedding pipeline from daily to once per month. - D) Configure the OpenSearch k-NN vector index to use lower-precision data types (e.g., byte or bfloat16 instead of float32). |
Correct Answer: D Correct Answer Analysis: - D is Correct: By default, many vector indexes store embeddings as 32-bit floating-point numbers ( float32). Configuring the index to use a lower-precision data type like byte (8-bit) or bfloat16 (16-bit) can reduce the storage footprint of the vector index by 2x to 4x, directly lowering storage costs while keeping the catalog intact.Distractor Analysis: - A is Incorrect: OpenSearch Serverless does not offer "bulk storage discounts" for provisioning larger collections; it charges based on actual storage consumed and OCU compute capacity. - B is Incorrect: Migrating from OpenSearch Serverless to RDS PostgreSQL introduces massive operational overhead for re-architecture and may not support the same advanced approximate nearest neighbor (ANN) algorithms natively. - C is Incorrect: Reducing ingestion frequency does not reduce the storage costs of the existing vectors in the index; it only delays adding new ones. The current catalog size remains unchanged. > Exam Tip: Optimize vector database storage costs by using lower-precision data types (like byte or bfloat16) for your embeddings index instead of float32, which can halve or quarter the storage requirements. |
| ### Question 4 - Batch Inference for Asynchronous Workloads A marketing agency uses an Amazon Bedrock foundation model to generate weekly performance summaries for thousands of clients. The summaries are generated overnight and do not require real-time interactive responses. The team wants to optimize the token costs associated with this recurring workload without altering the prompt sizes or model choice. Which solution is the most cost-effective? - A) Use Amazon Bedrock Provisioned Throughput to reserve model units specifically for the overnight window. - B) Switch the workload to use the Amazon Bedrock Batch Inference API. - C) Enable prompt caching on the static weekly summary template to reduce input tokens. - D) Run the workload during the day and rely on AWS Lambda concurrency limits to stagger the requests. |
Correct Answer: B Correct Answer Analysis: - B is Correct: For asynchronous workloads that do not require real-time responses, Amazon Bedrock Batch Inference provides a significant discount (often up to 50%) compared to On-Demand token pricing. It processes large datasets efficiently without interactive latency requirements. Distractor Analysis: - A is Incorrect: Provisioned Throughput is designed for predictable, high-volume continuous traffic, not a one-off overnight batch job. Reserving model units for a short window is cost-prohibitive and inflexible. - C is Incorrect: While prompt caching reduces input token costs, it does not provide the off-peak batching discount. Also, if the weekly data changes, the cache might miss, negating the savings. - D is Incorrect: Running during the day with Lambda staggering does not lower the Bedrock token costs; it just rate-limits the requests, increasing the total time to completion. > Exam Tip: For asynchronous, non-interactive AI workloads (like bulk summarization or batch processing), use Amazon Bedrock Batch Inference to benefit from discounted token rates. |
| ### Question 5 - Agent Resource Consumption Monitoring A financial services firm has deployed an Amazon Bedrock Agent that orchestrates multiple tool calls (e.g., fetching stock prices, retrieving account balances) and synthesizes a final response. The firm is experiencing unexpectedly high token consumption per user interaction. The team needs to identify exactly which step in the agent's execution is consuming the most tokens. Which AWS service provides the lowest operational overhead to gain this specific visibility? - A) Query AWS Cost Explorer daily and filter by the Bedrock Agent service to find the peak usage hours. - B) Manually instrument the frontend application to send the total API response length to Amazon CloudWatch. - C) Enable Amazon Bedrock AgentCore Observability to capture the token usage of individual model calls, tool invocations, and orchestration steps per request. - D) Configure AWS X-Ray to trace the network latency of the tool calls and correlate it with the total account Bedrock bill. |
Correct Answer: C Correct Answer Analysis: - C is Correct: Amazon Bedrock AgentCore Observability (or Bedrock's native trace logging) is specifically designed to capture the internal mechanics of an agent invocation. It breaks down the trace into individual steps (planning, tool execution, synthesis), allowing the team to see the exact token count and latency for each sub-call without writing custom instrumentation. Distractor Analysis: - A is Incorrect: AWS Cost Explorer provides aggregate account-level billing data. It cannot break down costs to the granularity of individual internal steps within a single agent's execution. - B is Incorrect: Frontend instrumentation only captures the final output given to the user, not the intermediate token consumption of the agent's planning and tool-calling phases. - D is Incorrect: AWS X-Ray traces distributed requests for latency, but it does not natively parse and report Bedrock token usage per agent step without heavy custom instrumentation, which violates the "lowest operational overhead" constraint. > Exam Tip: To debug token amplification in Amazon Bedrock Agents, use native Agent Observability/Tracing to see a step-by-step breakdown of token usage across planning, tool calls, and synthesis. |
| ### Question 6 - Cost Optimization Trade-offs in Model Selection An AI startup is building a document summarization application using Amazon Bedrock. Currently, they use a large, highly capable foundation model (e.g., Anthropic Claude 3 Opus) for every request. The team wants to minimize costs, but summary quality must remain high for complex legal documents, while simpler internal memos can be easily summarized by a less capable, cheaper model. Which solution is the most cost-effective to handle both document types while meeting quality constraints? - A) Route all summarization requests to a smaller model (e.g., Claude 3 Haiku) and increase the temperature parameter to improve quality. - B) Use Amazon Bedrock Guardrails to block simple memos from being processed by the large model. - C) Implement a semantic router or classifier model to classify document complexity, routing to the large model for legal docs and the small model for memos. - D) Use Amazon Bedrock Provisioned Throughput for the large model and On-Demand for the small model. |
Correct Answer: C Correct Answer Analysis: - C is Correct: This approach implements dynamic model routing. By using a cheap, fast classification step to assess the document, the application only invokes the expensive large model when high quality is strictly necessary, drastically reducing the average cost per request while maintaining quality for complex documents. Distractor Analysis: - A is Incorrect: Routing everything to a smaller model violates the quality constraint for complex legal documents. Increasing the temperature does not increase a model's reasoning capability; it only increases randomness. - B is Incorrect: Bedrock Guardrails are used for content filtering, PII redaction, and safety, not for dynamic model routing or cost optimization routing. - D is Incorrect: Provisioned Throughput for the large model locks the startup into a high hourly fixed cost, which is not cost-effective for variable or mixed workloads. > Exam Tip: Optimize FM inference costs by routing requests dynamically: use smaller, cheaper models for simple tasks and reserve larger models only for complex tasks that require advanced reasoning. |
| ### Question 7 - Embedding Model Cost Management An e-commerce platform uses a third-party embedding model via an Amazon SageMaker Real-Time Endpoint to generate product embeddings for a recommendation engine. The data science team updates the entire product catalog embeddings daily, even though only 5% of the catalog changes each day. The team must reduce the embedding computation costs with the least operational overhead. Which solution meets these requirements? - A) Maintain a lookup table (e.g., in DynamoDB) of existing product embeddings and only compute embeddings for products that have been added or modified in the last 24 hours. - B) Switch from a SageMaker Real-Time Endpoint to an Asynchronous Inference Endpoint to lower the hourly compute cost. - C) Convert the SageMaker endpoint to a Serverless Inference Endpoint to scale to zero when the embedding pipeline is not running. - D) Use Amazon EC2 Spot Instances to run the SageMaker endpoint to lower the hourly compute cost. |
Correct Answer: A Correct Answer Analysis: - A is Correct: This is a standard delta processing optimization. By tracking which items have changed and only computing embeddings for those items, the team reduces the volume of inference calls by 95%. This directly lowers the token/compute cost of the embedding generation process while maintaining an up-to-date recommendation index. Distractor Analysis: - B is Incorrect: Asynchronous Inference endpoints are for large payloads or long processing times. Switching to it does not change the fact that 100% of the catalog is being processed unnecessarily; it just changes the endpoint hosting model. - C is Incorrect: Serverless Inference reduces idle compute costs but does not address the core inefficiency: recomputing embeddings for 95% unchanged products. - D is Incorrect: Spot Instances lower the hourly rate but introduce the risk of interruption. More importantly, like the others, it fails to address the redundant embedding computation, which is the primary cost driver. > Exam Tip: Avoid recomputing embeddings for your entire dataset during ingestion updates. Implement delta processing to only compute embeddings for new or modified items. |
| ### Question 8 - Managing FM Inference Costs with Usage Quotas A SaaS company provides an AI writing assistant powered by Amazon Bedrock. They offer a free tier to users, but have noticed a small number of users are abusing the free tier by sending thousands of automated requests per hour, causing the company's Bedrock token costs to skyrocket. The team needs to implement a mechanism to cap the token usage per user to control costs. Which approach has the least operational overhead to enforce per-user token quotas securely? - A) Implement an AWS Budgets alert that triggers an AWS Lambda function to block the user's IAM role when their token spend exceeds $5. - B) Track user token usage in Amazon DynamoDB and use application-level middleware (e.g., API Gateway + Lambda authorizer) to reject requests when a user exceeds their daily token quota. - C) Configure Amazon Bedrock Provisioned Throughput to strictly limit the total tokens the application can process per hour. - D) Use Amazon Bedrock Guardrails to set a maximum input length limit of 100 tokens for all free-tier users. |
Correct Answer: B Correct Answer Analysis: - B is Correct: Bedrock does not natively offer per-user token quotas. The standard, least-overhead architectural pattern to enforce this is to use a fast database (DynamoDB) to track cumulative token usage per user per day, and an API Gateway/Lambda authorizer to intercept requests, check the quota, and reject the request if the user has exceeded their limit. Distractor Analysis: - A is Incorrect: AWS Budgets operates at the AWS account or tag level, not at the individual end-user level within an application. It also updates on a delayed basis (typically every 8-12 hours), making it unsuitable for real-time quota enforcement. - C is Incorrect: Provisioned Throughput reserves capacity for the application but does not distinguish between users or enforce per-user limits. - D is Incorrect: Bedrock Guardrails can limit the size of a single input prompt, but cannot track cumulative usage across multiple requests over time (e.g., a user sending 1,000 requests of 50 tokens each). > Exam Tip: To enforce per-user token quotas in a Bedrock application, implement application-level tracking (e.g., DynamoDB) and middleware to control access before the request reaches the model. |
| ### Question 9 - RAG Context Optimization A healthcare company has a RAG-based search application using Amazon Bedrock and an Amazon OpenSearch vector database. The system retrieves the top 20 most similar documents for every query and passes all 20 documents as context to a large foundation model. The team needs to reduce the input token costs of the foundation model while maintaining the accuracy of the answers. Which solution is the most cost-effective without degrading accuracy? - A) Reduce the number of retrieved documents passed to the model from 20 to 5 by applying a higher relevance score threshold in OpenSearch. - B) Decrease the chunk size of the documents during ingestion from 500 tokens to 50 tokens. - C) Switch the embedding model to a smaller dimension size to lower ingestion costs. - D) Enable Provisioned Throughput on the foundation model to lower the cost per token. |
Correct Answer: A Correct Answer Analysis: - A is Correct: Passing 20 full documents often results in "lost in the middle" phenomena where models ignore information, and it massively inflates input token costs. By filtering for relevance (e.g., only passing documents with a cosine similarity score above 0.85, or just the top 5), you drastically cut input tokens while often maintaining or even improving accuracy by reducing noise. Distractor Analysis: - B is Incorrect: Reducing chunk size to 50 tokens fragments the context, potentially degrading answer quality because the model lacks full sentence/paragraph context. It also increases the number of chunks retrieved, which may not reduce the final context window size. - C is Incorrect: Switching the embedding model dimension lowers the storage and ingestion costs in OpenSearch, but does nothing to reduce the input token cost of the foundation model during RAG query generation. - D is Incorrect: Provisioned Throughput is a commitment discount for high volume; it does not optimize the underlying architecture or reduce the number of tokens processed per request. > Exam Tip: In RAG architectures, input token costs are driven by the context window size. Filter retrieved chunks by relevance score to pass only the most pertinent information to the LLM. |
| ### Question 10 - Infrastructure Cost vs Token Cost Distinction A large enterprise is running two separate AI workloads on AWS. Workload 1 is a traditional machine learning model hosted on an Amazon SageMaker Real-Time Endpoint. Workload 2 is a generative AI chatbot powered by Amazon Bedrock. The finance team has requested that both workloads implement cost-saving measures for their steady, year-round baselines. Which TWO strategies should be implemented to apply the correct cost optimization mechanism for each workload's baseline? (Select TWO) - A) For Workload 1, use Amazon SageMaker Savings Plans to commit to a steady hourly compute spend. - B) For Workload 2, use Amazon Bedrock Provisioned Throughput to reserve model units for the steady inference volume. - C) For Workload 1, use Amazon EC2 Spot Instances to run the SageMaker endpoint. - D) For Workload 2, use AWS Compute Savings Plans to cover the Bedrock token usage. - E) For Workload 1, reduce the instance count on the endpoint to lower the baseline cost regardless of traffic. |
Correct Answers: A and B Correct Answer Analysis: - A is Correct: Amazon SageMaker AI Savings Plans provide a discount in exchange for a commitment to a steady hourly spend on SageMaker compute instances, which is ideal for a 24/7 real-time endpoint baseline. - B is Correct: Amazon Bedrock Provisioned Throughput allows you to reserve model units at a fixed hourly rate. For a steady, year-round generative AI workload with predictable traffic, this provides a discounted rate compared to On-Demand token pricing. Distractor Analysis: - C is Incorrect: Amazon EC2 Spot Instances are for interruptible workloads (like batch training). A SageMaker real-time endpoint requires high availability and cannot tolerate capacity reclamation, making Spot unsuitable for a serving baseline. - D is Incorrect: AWS Compute Savings Plans apply to Amazon EC2, AWS Lambda, and AWS Fargate. They do not apply to Amazon Bedrock token usage. - E is Incorrect: Blindly reducing the instance count violates capacity requirements. If the baseline traffic requires that capacity, reducing it will cause throttling and latency errors. > Exam Tip: Match the commitment mechanism to the billing unit: use SageMaker Savings Plans for instance-hour baselines, and Bedrock Provisioned Throughput for token-volume baselines. |