Skill 4.2.9: Manage FM inference costs with usage optimization

Question 1 - Token Usage Monitoring and Cost Attribution

A financial services company is deploying a customer support assistant built on Amazon Bedrock using Claude models. The billing team reports that Bedrock spend has tripled over the past month, but the number of user requests has remained stable. The ML engineering team needs to identify what is driving the cost increase and wants to monitor token usage patterns per application going forward without building custom metric pipelines.

Which approach should the ML engineer implement FIRST to address these requirements?

A) Query AWS Cost Explorer with the correct filter dimension enabled to analyze Bedrock usage type by invocation count.
B) Create a custom Python script using boto3 to calculate token counts from CloudTrail logs and store results in an S3 bucket.
C) Write a Lambda function that parses Bedrock response headers for InputTokenCount/OutputTokenCount and publishes CloudWatch custom metrics.
D) Enable CloudWatch monitoring and filter metrics by the ApplicationId dimension in CloudWatch Metrics Insights.

Click to reveal Answer & Explanation

Correct Answer: D

Correct Answer Analysis

D is Correct: Amazon Bedrock publishes CloudWatch metrics including InputTokenCount, OutputTokenCount, and InvocationCount, which can be filtered by dimensions such as ModelId and ApplicationId. This enables per-application token usage analysis without custom infrastructure.

Distractor Analysis

A is Incorrect: While AWS Cost Explorer can break down Bedrock costs by usage type, the encryption dimension "filter dimension" must be manually enabled for some services; also it may not provide per-application attribution unless cost allocation tags are already configured. It is not the most direct method for identifying per-request token growth.
B is Incorrect: Building a custom Python script to parse CloudTrail logs introduces unnecessary operational overhead and does not capture real-time token usage patterns. CloudTrail logs also may not include granular token counts for every Bedrock invocation.
C is Incorrect: Writing a custom Lambda to parse Bedrock response headers adds operational complexity (deploying, monitoring, and maintaining Lambda functions) when CloudWatch natively publishes token metrics for Bedrock.

Key Takeaway: Amazon Bedrock natively publishes token usage metrics (InputTokenCount, OutputTokenCount) to CloudWatch, filterable by ModelId and ApplicationId—no custom telemetry pipelines needed.

Question 2 - Prompt Caching with Dynamic Content

An AI development team operates a document analysis assistant on Amazon Bedrock using Anthropic Claude. The system prompt includes legal disclaimers, tool schemas, and a session identifier that is generated fresh for every API invocation. The team wants to implement prompt caching to reduce token costs but is concerned about cache effectiveness.

What should the ML engineer recommend to maximize prompt caching efficiency while preserving functional requirements? (Select TWO)

A) Place the session identifier at the end of the prompt, after all static content.
B) Enable prompt caching with a low TTL value to refresh the cache before cache invalidation occurs.
C) Remove the session identifier entirely to eliminate dynamic content from the prompt.
D) Structure the prompt with the static legal disclaimers and tool schemas at the beginning.
E) Use a larger model with a higher token capacity to accommodate the dynamic prefix.

Click to reveal Answer & Explanation

Correct Answers: A and D

Correct Options Analysis

A is Correct: Prompt caching relies on exact token sequence matching from the start of the prompt. Placing dynamic content (session ID) at the end ensures the static prefix is cacheable without invalidation.
D is Correct: Static content such as legal disclaimers and tool schemas must appear at the beginning of the prompt to be included in the cached prefix, enabling high cache hit rates across requests.

Distractor Analysis

B is Incorrect: As of the latest Bedrock capabilities, users cannot directly configure TTL solely for prompt caching. Repeatedly writing the same prefix to the cache with dynamic content still invalidates on every request; TTL causes cache misses and isn't a solution for dynamic prefix invalidation.
C is Incorrect: Removing the session identifier entirely changes the application's functional design; session tracking may be required for compliance or personalization. The question asks for maximizing efficiency while preserving functional requirements.
E is Incorrect: Model size does not affect cache invalidation behavior. Using a larger model would increase per-token cost, contradicting the cost optimization goal.

Key Takeaway: For prompt caching to be effective, place all static content (system instructions, tool schemas) at the beginning and keep dynamic variables at the end of the prompt.

Question 3 - Provisioned Throughput vs. On-Demand for RAG Application

A healthcare technology company has deployed a clinical documentation assistant using Amazon Bedrock with the Claude 3 Sonnet model. The application experiences highly variable usage—peak usage occurs during hospital shift changes (2 hours in morning and evening), while overnight usage is nearly zero. The current implementation uses on-demand pricing, but the finance team wants to reduce costs. The ML engineer must evaluate whether Provisioned Throughput (PT) would lower the bill.

What is the MOST COST-EFFECTIVE pricing approach for this workload pattern?

A) Purchase a 1-year Provisioned Throughput commitment for the minimum required model units at peak capacity.
B) Purchase Provisioned Throughput for the overnight baseline and use on-demand pricing for the peak periods.
C) Continue using on-demand pricing with prompt optimization and caching to reduce both input and output tokens.
D) Purchase Provisioned Throughput in 1-hour increments to cover only the 4 peak hours each day.

Click to reveal Answer & Explanation

Correct Answer: C

Correct Answer Analysis

C is Correct: On-demand pricing is ideal for variable, unpredictable, or intermittent workloads since you pay only for the exact tokens processed. Provisioned Throughput requires a committed hourly rate over a term (associated with 1-month or 6-month terms), regardless of actual usage. For this traffic pattern, on-demand is the most cost-effective baseline choice; token usage optimization (prompt engineering, caching) is the best way to reduce costs without a PT commitment.

Distractor Analysis

A is Incorrect: Provisioned Throughput committed for peak capacity during all hours of a 1-year term would result in paying for idle capacity overnight and during non-peak hours, sufficient to result in higher costs than on-demand.
B is Incorrect: Purchasing PT for the overnight baseline is counterproductive since overnight usage is nearly zero; PT involves a fixed hourly commitment and you would pay for unused throughput during non-busy hours where on-demand cost is already minimal.
D is Incorrect: Provisioned Throughput is not purchasable in 1-hour increments; it requires term commitments (typically 1-month or 6-month). If you need automatic scaling based on demand over short durations, you should consider on-demand pricing with caching or Amazon Bedrock's incremental provisioning support (via boto3) where supported—but the option as stated is invalid.

Key Takeaway: Provisioned Throughput offers hourly commitments, not flexible hourly scaling—highly variable or intermittent Bedrock workloads are better served by on-demand pricing plus token discipline, with PT reserved for predictable high-volume baselines.

Question 4 - Agent Cost Amplification and Monitoring

A retail company is building an agentic AI assistant using Amazon Bedrock Agents. The agent orchestrates several hundred daily user conversations, each involving multiple internal steps: planning calls, tool invocations (API calls to an order management system), and synthesis calls to generate final responses. The finance team flags that Bedrock costs are rising faster than user traffic. The ML engineer needs to identify which part of the agent workflow is driving the cost increase and monitor it per invocation.

Which service capability should the engineer enable to diagnose the issue most effectively?

A) Use Amazon CloudWatch Logs Insights to correlate agent invocation logs by request ID.
B) Enable AWS X-Ray tracing to sample agent invocations and visualize call sequences.
C) Use Amazon Bedrock AgentCore Observability to capture inference calls, tool invocations, and memory operations per invocation.
D) Create a custom Cost and Usage Report (CUR) to identify spend by Bedrock agent.

Click to reveal Answer & Explanation

Correct Answer: C

Correct Answer Analysis

C is Correct: Amazon Bedrock AgentCore Observability captures inference calls, tool invocations, and memory operations for each agent invocation, providing visibility into the cost amplification of one user request into multiple model and tool calls. This is the native Bedrock feature designed specifically for this requirement.

Distractor Analysis

A is Incorrect: CloudWatch Logs Insights can analyze log entries but doesn't automatically correlate agent-specific sub-operations (planning, tool call, synthesis) per invocation without significant custom instrumentation and log querying overhead.
B is Incorrect: AWS X-Ray can trace distributed requests across AWS services, but agentic FMs have specialized observability with Bedrock AgentCore that captures the specific natural language agent operations; X-Ray alone would miss details such as memory operations and tool schema token consumption.
D is Incorrect: AWS Cost and Usage Report (CUR) provides billing data at service level and can break down usage but does not provide per-invocation cost attribution for the internal agent orchestration steps (planning calls vs. tool calls) within a single agent interaction.

Key Takeaway: Use Bedrock AgentCore Observability to see the resource consumption per agent invocation—separating planning calls, tool invocations, memory operations, and synthesis calls—to pinpoint cost amplification within an agent.

Question 5 - Model Selection for Cost Optimization

A startup is running a text summarization feature on Amazon Bedrock using a large frontier model (e.g., Claude 3 Opus). The feature summarizes support tickets into 3-5 bullet points, and the quality bar is moderate—users need the key facts, not nuanced reasoning. The Finance team reports that per-request costs are 3x higher than budgeted. Quality tests show a smaller model (e.g., Claude 3 Haiku) produces summaries meeting the quality bar in 95% of cases, but the product team is hesitant to switch.

What should the ML engineer recommend to MOST effectively reduce unit costs while maintaining quality?

A) Keep the current model but enable prompt caching to reduce input token costs for repetitive instructions.
B) Switch the production summarization workload to Claude 3 Haiku and route the 5% edge cases to the larger model via a quality evaluation step.
C) Move the summarization workload to Amazon SageMaker with a custom BERT model deployed on an ml.g5 instance.
D) Use Provisioned Throughput to reserve capacity for the larger model at a discounted rate.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

B is Correct: Model selection is the single most impactful lever for foundation model cost control. Using a smaller model that meets the quality requirement for the majority of cases and routing complex edge cases to the larger model via a routing step (model cascade) achieves significant cost savings while maintaining quality.

Distractor Analysis

A is Incorrect: Prompt caching only reduces input token costs for a repeated prefix; it does not address the larger model's higher per-token rate and output token cost. The cost driver here is the model selection, not input token repetition.
C is Incorrect: Moving to a self-hosted SageMaker endpoint with a custom BERT model for text summarization introduces substantial operational overhead (model training, deployment, instance management, scaling) and likely underperforms vs. a smaller FM on quality for a summarization task. This contradicts the "least operational overhead" principle.
D is Incorrect: Provisioned Throughput reduces the hourly rate for committed capacity but still binds the team to a fixed commitment and larger-model pricing. It does not address the root cause: the model is oversized for the task.

Key Takeaway: When optimizing Bedrock costs, start with model selection—using a smaller model that meets quality is the highest-leverage cost optimization before considering caching or purchasing options.

Question 6 - Cost Allocation Tags for Bedrock

A multi-tenant SaaS platform runs several Bedrock applications for different enterprise customers, each with distinct billing agreements. The finance team wants to create per-customer cost reports to bill clients accurately. The ML team currently tags all Bedrock invocations with a CustomerID cost allocation tag. However, Cost Explorer shows all Bedrock costs lumped together with no tags.

What is the MOST LIKELY root cause of this issue?

A) Bedrock does not support cost allocation tags; tags are only supported on Amazon EC2 and S3.
B) The cost allocation tags have not been activated in the AWS Billing console.
C) The tags are applied using aws:ReservedTags which are not available for cost allocation.
D) Cost Explorer requires a 24-48 hour delay before tags appear; the team is viewing data immediately after tagging.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

B is Correct: After applying cost allocation tags to AWS resources, you must explicitly activate them in the AWS Billing and Cost Management console. Until activated, tags are not included in Cost Explorer, Cost and Usage Reports, or AWS Budgets.

Distractor Analysis

A is Incorrect: Amazon Bedrock supports cost allocation tags (via the TagResource API and console). Tags can be applied to Bedrock resources including custom models, agents, and knowledge bases.
C is Incorrect: aws:ReservedTags is not a valid construct. AWS prefixes are reserved for AWS-generated tags (like aws:createdBy), but this does not impede cost allocation activation.
D is Incorrect: While Cost Explorer does have a delay (typically 24 hours) for new data, tags do not appear at all if not activated in the Billing console. The question states "no tags are visible," not a slight delay.

Key Takeaway: Cost allocation tags must first be applied to resources and then explicitly activated in the Billing console before they appear in Cost Explorer reports—tagging alone is not sufficient.

Question 7 - Batch Inference for Non-Real-Time Workloads

A marketing analytics company runs a nightly job that processes 50,000 customer reviews using Amazon Bedrock for sentiment classification and topic extraction. The job runs at midnight and must complete within 6 hours. The current implementation invokes the real-time InvokeModel API for each document, costing approximately $850 per night. The company wants to reduce this cost without sacrificing throughput or data quality.

Which approach provides the MOST cost-effective solution for this workload?

A) Switch to Amazon Bedrock Batch Inference to process the nightly jobs at a 50% discount per token.
B) Provision a larger SageMaker Real-Time endpoint to process all reviews simultaneously.
C) Use prompt caching to cache the sentiment classification instructions across the 50,000 requests.
D) Convert the job to use a serverless inference option with Amazon Bedrock.

Click to reveal Answer & Explanation

Correct Answer: A

Correct Answer Analysis

A is Correct: Amazon Bedrock Batch Inference is designed for asynchronous, non-real-time workloads like processing large document sets. It offers a 50% discount on input/output token rates compared to on-demand pricing and can scale to handle large workloads within the specified time window. The nightly job has no interactive latency requirement and fits perfectly into batch mode.

Distractor Analysis

B is Incorrect: SageMaker Real-Time endpoints require provisioned compute hours and are significantly more expensive for a nightly batch job of this scale compared to using Bedrock Batch Inference. Self-hosting also reintroduces operational overhead (instance management, scaling, and monitoring).
C is Incorrect: Prompt caching reduces input token cost for a stable prefix. For 50,000 documents, the document content itself dominates input tokens, not the instruction prefix. Caching would provide minimal savings compared to switching to Batch Inference.
D is Incorrect: Amazon Bedrock does not offer a "serverless inference option" as a distinct mode. On-demand inference is already serverless. The cost-savings opportunity here is from batch pricing (50% discount), not serverless.

Key Takeaway: Workloads that tolerate latency and process large volumes asynchronously should leverage Amazon Bedrock Batch Inference for a 50% discount on token rates vs. on-demand invocation.

Question 8 - Embedding Cost Optimization

A document search product uses Amazon Bedrock with the Titan Embeddings model to convert thousands of documents into vector embeddings stored in Amazon OpenSearch Serverless. The engineering team observes that embedding computation is the largest cost driver. Documents are stored in S3 and rarely change—only new documents are added weekly. The product manager wants to reduce embedding costs by at least 40% without delaying user-facing search results.

Which TWO strategies should the ML engineer implement? (Select TWO)

A) Reuse previously computed embeddings by tracking document content hashes and only embedding new or changed documents.
B) Switch to a smaller embedding model with lower dimension output (e.g., Titan Embeddings G1 - Text v1.2) if it meets the quality requirement.
C) Move the embedding workload to a SageMaker endpoint with a self-hosted embedding model for lower costs.
D) Store raw text instead of embeddings and compute embeddings at query time to reduce storage costs.
E) Increase the batch size to 100 documents per API call using the BatchEmbedText API.

Click to reveal Answer & Explanation

Correct Answers: A and B

Correct Options Analysis

A is Correct: Embedding computation is billed per token processed. By caching computed embeddings and only embedding new or modified documents (detected via content hash comparison), you eliminate redundant embedding computation for the unchanged document corpus, providing significant cost reduction.
B is Correct: Using a smaller embedding model that meets the quality requirement reduces per-token rates and potentially output token count (dimension size), lowering both computation cost and vector storage requirements.

Distractor Analysis

C is Incorrect: Self-hosting an embedding model on SageMaker introduces substantial operational overhead (instance provisioning, scaling, model deployment) and eliminates Bedrock's serverless benefits. SageMaker's per-instance-hour billing may not be cheaper for low-throughput workloads and violates the least-operational-overhead principle.
D is Incorrect: Computing embeddings at query time would increase latency for user-facing search results, violating the "without delaying user-facing search results" constraint. It also requires repeated computation of the same document embeddings, increasing costs.
E is Incorrect: Amazon Bedrock does not provide a BatchEmbedText API. For embedding workloads, batch processing is done through the standard InvokeModel API with individual requests or via Batch Inference; increasing batch size for embeddings may not be supported in the same manner as the API described.

Key Takeaway: For embedding cost optimization, first eliminate redundant computation by caching embeddings for unchanged documents, then select a smaller model that meets quality requirements.

Question 9 - CloudWatch Dashboard for Token Cost Monitoring

A DevOps team is responsible for monitoring AI application costs across multiple Bedrock models. The team currently reviews monthly Cost Explorer reports but needs a real-time dashboard that shows token usage, invocation counts, and estimated costs side by side for immediate decision-making. The dashboard will be viewed by both engineers and non-technical stakeholders.

Which Amazon CloudWatch capability should the team configure to meet these requirements?

A) Create a CloudWatch dashboard with widgets displaying InvocationCount, InputTokenCount, and OutputTokenCount metrics filtered by ModelId, along with latency metrics.
B) Use AWS Cost Explorer API to feed cost data into a CloudWatch dashboard custom widget.
C) Deploy a QuickSight dashboard connected to Amazon Athena queries over CloudTrail logs for real-time token analysis.
D) Enable CloudWatch Anomaly Detection on Bedrock metrics to alert on token usage spikes.

Click to reveal Answer & Explanation

Correct Answer: A

Correct Answer Analysis

A is Correct: Amazon CloudWatch dashboards natively support visualizing Bedrock metrics like InvocationCount, InputTokenCount, and OutputTokenCount with dimension filters (e.g., ModelId). You can arrange widgets side by side to display token usage, invocations, and operational metrics (latency, error rate) in real time. This requires no custom pipeline and is viewable by engineers and non-technical stakeholders via the AWS console.

Distractor Analysis

B is Incorrect: AWS Cost Explorer API provides billing data, not real-time token metrics. The data is delayed (typically 24 hours) and has a different granularity compared to CloudWatch Bedrock metrics. It's not built for real-time operational dashboards.
C is Incorrect: Deploying QuickSight with Athena over CloudTrail logs is a heavy operational lift and requires ETL pipelines, SQL queries, and separate dashboard infrastructure. CloudTrail logs also don't capture direct token counts from Bedrock API responses unless specifically logged through other mechanisms.
D is Incorrect: CloudWatch Anomaly Detection on Bedrock metrics creates a model for detecting abnormal patterns in a single metric, but it doesn't create a side-by-side dashboard for token usage, invocation count, and cost estimation. It's useful for alerting but not for operational visibility of multiple metrics.

Key Takeaway: For real-time visibility of Bedrock token usage and invocation patterns, build native CloudWatch dashboards using Bedrock's built-in metrics—no custom pipelines or BI tooling required.

Question 10 - Amazon Bedrock Cost Monitoring for Multi-Tenant Platform

A SaaS company provides AI-powered legal document review on Amazon Bedrock to multiple enterprise clients. Each client has a separate Bedrock agent. The billing team wants to ensure that cost quotas are enforced per client so that no single client exceeds their monthly AI budget. The ML platform team must implement a solution with the least operational overhead.

Which solution should the platform team implement to satisfy these requirements?

A) Use AWS Budgets with cost allocation tags applied to each Bedrock agent, alerting on forecasted spend exceeding client-specific thresholds.
B) Create a custom Lambda that reads CloudWatch token metrics, aggregates by client, and terminates client access when token limits are breached.
C) Use AWS Service Catalog to enforce per-client IAM policies that reject API calls when budget thresholds are met.
D) Configure a CloudWatch alarm on TokenUsage per client to automatically disable the client's Bedrock agent via AWS Lambda.

Click to reveal Answer & Explanation

Correct Answer: A

Correct Answer Analysis

A is Correct: AWS Budgets supports cost allocation tags as filters, allowing you to create per-client budgets that monitor the tagged Bedrock agent costs. Budgets can send alerts (via SNS or email) when actual or forecasted spend exceeds the client's threshold. This requires no custom code and directly addresses the cost quota requirement with minimal operational overhead.

Distractor Analysis

B is Incorrect: Writing a custom Lambda to aggregate token metrics and enforce limits introduces unnecessary operational overhead (developing, deploying, maintaining, and monitoring Lambda functions). AWS Budgets already provides threshold-based alerts with minimal setup.
C is Incorrect: AWS Service Catalog manages portfolios of approved resources for provisioning, not runtime budget enforcement. IAM policies cannot reject API calls based on cumulative token costs; IAM conditions are based on request context, not spend.
D is Incorrect: This solution requires custom development (CloudWatch alarm → Lambda → disable agent), adding operational overhead and complexity. Additionally, automatically disabling a client's agent may violate service-level agreements; the requirement is to set a cost quota and alert, not to automatically kill clients.

Key Takeaway: AWS Budgets with cost allocation tags provides built-in per-client cost tracking and threshold alerts for Bedrock workloads without custom code—it's the lowest-operational-overhead solution for enforcing cost quotas.