Skill 4.2.10: Monitor AI-specific cost patterns

Question 1 - Bedrock Token Usage Monitoring

A media analytics company runs a customer support chatbot on Amazon Bedrock using the Claude 3.0 Sonnet foundation model. The finance team notices that the Bedrock invoice has increased by 40% month-over-month, but the number of user conversations has remained flat. The ML engineering team needs to identify whether the cost increase is driven by longer input prompts or heavier output generation, so they can implement the correct optimization.

Which monitoring configuration should the engineering team implement to diagnose this cost driver with the least operational overhead?

A) Enable Amazon Bedrock invocation logging to Amazon CloudWatch Logs and parse the promptTokenCount and completionTokenCount fields from each logged request.
B) Use Amazon Bedrock's built-in CloudWatch metrics (InputTokenCount and OutputTokenCount) and build a dashboard that separates input token usage from output token usage over time.
C) Deploy a custom AWS Lambda function that intercepts Bedrock API calls through Amazon API Gateway, extracts token counts from request/response payloads, and publishes custom CloudWatch metrics.
D) Enable AWS X-Ray tracing on the chatbot frontend and analyze latency histograms to infer whether input or output token generation consumes more time.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

B is Correct: Amazon Bedrock automatically publishes CloudWatch metrics per model, including InputTokenCount and OutputTokenCount (or equivalent token metrics depending on region and model). These metrics allow you to distinguish rising input token costs (longer prompts/context) from output token costs (longer generations) without writing any custom code or parsing libraries.

Distractor Analysis

A is Incorrect: While invocation logging to CloudWatch Logs does capture token counts per invocation, it requires configuration effort and significant log-parsing infrastructure (e.g., CloudWatch Logs Insights or Lambda) to derive aggregate token trends. This introduces more operational overhead than using native CloudWatch metrics.
C is Incorrect: Building a custom interceptor with Lambda and API Gateway requires application code changes and custom metric publishing. This is high operational overhead and unnecessary when Bedrock natively emits token metrics to CloudWatch.
D is Incorrect: X-Ray provides request-level latency tracing across distributed components, but it does not provide token-level billing or usage attribution. Inferring token costs from latency is indirect and misleading because latency can be affected by network conditions, model load, and output streaming behavior.

Key Takeaway: Amazon Bedrock emits separate input and output token metrics to CloudWatch out of the box; use them to differentiate prompt bloat from excessive generation costs without building custom tooling.

Question 2 - Prompt Caching with Dynamic Content

A legal document summarization service uses Amazon Bedrock with Anthropic Claude to summarize contracts. The prompt consists of a large static system instruction containing legal definitions and extraction rules, followed by the document text. The team enables prompt caching to reduce input token costs. However, the engineering team notices that cost savings are not materializing.

What is the most likely cause of the cache misses? (Select ONE)

A) The system instruction includes a dynamically generated timestamp at the top before the static content, causing the entire prefix to be invalidated on every request.
B) Prompt caching is not supported on Anthropic Claude models in Amazon Bedrock.
C) Prompt caching only applies to output tokens, so input token costs remain unchanged.
D) The legal document text exceeds the maximum cacheable prefix length.

Click to reveal Answer & Explanation

Correct Answer: A

Correct Option Analysis

A is Correct: Prompt caching in Bedrock/Anthropic Claude relies on exact token sequence matching from the beginning of the prompt. Any dynamic content (timestamps, request IDs, user metadata) inserted before or within the static prefix changes the sequence hash on every request, causing a cache miss and no savings. Static content must appear at the very beginning.

Distractor Analysis

B is Incorrect: Prompt caching is a supported feature for Anthropic Claude models on Amazon Bedrock (minimum context length requirements apply depending on model version).
C is Incorrect: Prompt caching directly targets input token processing costs, specifically the pre-fill/KV-cache computation for the static prompt prefix. It does not apply to output tokens.
D is Incorrect: While there are maximum context window limits, the question scenario describes the system instruction + document being within typical limits. Cacheable prefix has minimum (not restrictive maximum) length requirements, and document length is unlikely to be the stated failure mode. Dynamically generated content at the start is the archetypal cache-invalidation failure pattern.

Key Takeaway: Prompt caching requires a byte-for-byte stable prefix at the very beginning of the prompt; never insert timestamps, dynamic metadata, or volatile content before or within the cached prefix if you want cache hits and cost savings.

Question 3 - Multi-Model Endpoint vs Individual Endpoints

A fintech company has deployed 80 small fraud detection models on Amazon SageMaker AI real-time endpoints, one endpoint per model. Each model uses the PyTorch framework, has a similar inference latency profile, and is invoked only a few times per hour. The team wants to reduce monthly hosting costs while maintaining the ability to scale horizontally.

Which change is most cost-effective and operationally aligned with these requirements?

A) Convert all 80 endpoints to Amazon SageMaker Serverless Inference with pay-as-you-go pricing per invocation.
B) Consolidate all 80 models onto a single SageMaker Multi-Model Endpoint (MME) with shared instance capacity.
C) Keep individual endpoints but reduce the instance size on each endpoint to the smallest available ml.t3.micro instance.
D) Refactor all 80 models to run as AWS Lambda functions with GPU-optimized runtime.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

B is Correct: Multi-Model Endpoints (MMEs) are specifically designed for scenarios where dozens or hundreds of models are hosted on shared compute resources. Since all models share the same serving framework (PyTorch) and have low invocation rates, consolidating them onto one MME dramatically reduces the number of running instances and the total endpoint hosting cost, while preserving the ability to scale horizontally via Auto Scaling.

Distractor Analysis

A is Incorrect: SageMaker Serverless Inference does offer pay-per-invocation pricing, but it still requires each model to be deployed separately (or as a limited multi-model serverless endpoint). Serverless may actually introduce per-invocation cold-start latency for infrequently-called models and is not designed for 80 distinct artifacts within a single serverless endpoint. An MME on instance-backed hosting remains the more established cost optimization choice for large model counts sharing compute.
C is Incorrect: Reducing instance sizes without reducing the number of endpoints still incurs 80 × instance-hour costs. Many of these small instances will remain idle most of the time, so this approach retains massive fragmentation of compute resources and higher aggregate billing than a single shared MME.
D is Incorrect: AWS Lambda has memory and deployment package size constraints (10 GB [deployment] / 10,240 MB [memory] limits), and GPU-optimized runtimes for Lambda are not natively available in the same way they are on SageMaker. Refactoring 80 PyTorch models to Lambda would introduce significant code changes and potential performance degradations.

Key Takeaway: When many small models share a common serving framework and experience low or intermittent traffic, consolidating them onto a SageMaker Multi-Model Endpoint (MME) dramatically reduces compute hours and hosting costs.

Question 4 - Provisioned Throughput vs On-Demand for Bedrock

A healthcare company runs a patient triage application on Amazon Bedrock using a foundation model for real-time inference. The application processes an average of 5 million input tokens per day, with traffic spread uniformly across 24 hours, 7 days a week. The team wants to reduce per-token inference costs while maintaining consistent low latency for a year-round production workload.

Which purchasing or capacity strategy should the ML engineer recommend?

A) Switch to Amazon Bedrock Batch Inference for all patient requests; latency is non-critical for triage.
B) Implement Amazon Bedrock Provisioned Throughput by purchasing model units for the baseline volume at an hourly rate.
C) Enable Amazon Bedrock Model Distillation to a smaller, faster model and avoid any capacity commitments.
D) Purchase SageMaker AI Savings Plans, since they cover Amazon Bedrock Foundation Model usage.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

B is Correct: For predictable, steady, high-volume production workloads on supported Bedrock foundation models, Provisioned Throughput allows you to reserve dedicated capacity in the form of "model units" purchased at a fixed hourly rate. This provides lower per-token (or per-hour) costs compared to on-demand token pricing and delivers consistent, low-latency responses for production workloads.

Distractor Analysis

A is Incorrect: Batch Inference is designed for asynchronous workloads that tolerate long execution times and require no real-time user response. A patient triage application requires synchronous, low-latency responses, making batch inference unsuitable from an application architecture perspective.
C is Incorrect: Model distillation may reduce model size and token/processing costs, but it does not necessarily provide the lowest-cost or latency-optimal solution for a steady production workload requiring a specific model backbone. Distillation also requires additional engineering (training a student model) and may not fully address inference cost at high volume.
D is Incorrect: SageMaker AI Savings Plans apply exclusively to SageMaker usage (e.g., SageMaker endpoints, training, batch transform). They do not cover Amazon Bedrock foundation model inference. Bedrock has its own Provisioned Throughput mechanism.

Key Takeaway: Steady, predictable foundation model inference on Bedrock should use Provisioned Throughput for cost savings and latency consistency; SageMaker Savings Plans only apply to SageMaker compute, not Bedrock tokens.

Question 5 - Vector Database Storage Optimization

A large e-commerce company stores product embeddings in Amazon OpenSearch Serverless for vector search. The collection has grown to 4 TB, and query performance has deteriorated. Analysis shows that only 30% of the vectors are accessed in a typical month, while the other 70% correspond to seasonal or deprecated products. The team needs to reduce storage and query costs while retaining the ability to search all vectors.

Which TWO strategies should the ML engineer implement? (Select TWO)

A) Migrate infrequently accessed vectors to Amazon S3 Standard-Infrequent Access (S3-IA) and use OpenSearch's remote-backed storage feature to query them on demand.
B) Enable UltraWarm storage for the OpenSearch Serverless collection to move cold data to cheaper S3-backed storage while keeping it searchable.
C) Reduce the number of replicas on the vector collection from 2 to 0 to salve storage costs while accepting data loss risk.
D) Delete all vectors older than 90 days to reduce storage and improve query performance.
E) Use Amazon S3 Intelligent-Tiering to move cold vectors to cheaper storage classes, ignoring the requirement to query them.

Click to reveal Answer & Explanation

Correct Answers: A and B

Correct Option Analysis

A is Correct: OpenSearch supports remote-backed storage for managed clusters, where segment data is stored in Amazon S3 and cached locally. When infrequently accessed data is aged out of the local cache, it can be fetched from S3 upon query. While S3-IA is not directly a remote repository class, the concept of offloading colder shards to S3-backed storage aligns with OpenSearch's storage-tiering features for cost reduction.
B is Correct: UltraWarm storage is an OpenSearch feature that stores read-only, less frequently accessed data on Amazon S3-backed infrastructure using warm nodes. It allows indices to be migrated from hot to warm storage, where they remain queryable (with slightly higher latency) at a fraction of the hot storage cost. This fits perfectly for seasonal or deprecated vectors that still need to be searchable.

Distractor Analysis

C is Incorrect: Reducing replicas reduces redundant copies of the primary shard. However, setting replicas to 0 is risky because if a node fails, you lose data and query availability. For OpenSearch Serverless, replica (or shard) count is often managed automatically and reducing it to zero in an attempt to save storage could lead to catastrophic data loss during node failures. This is not a best practice for cost optimization.
D is Incorrect: Deleting all vectors older than 90 days violates the requirement to retain the ability to search all vectors, including seasonal or deprecated ones. This option trashes data instead of optimizing it.
E is Incorrect: Moving vectors to S3 Intelligent-Tiering would remove them from the OpenSearch vector index entirely, making them unsearchable by OpenSearch. This does not satisfy the requirement to retain queryability.

Key Takeaway: Use OpenSearch UltraWarm tiers or remote-backed storage to shift cold vector data to cheap S3 storage while still allowing queries; avoid deleting data or setting replica counts to zero to solve cost problems.

Question 6 - Token Usage Dashboard Metrics

A generative AI application team uses Amazon Bedrock AgentCore to deliver a multi-agent application that orchestrates retrieval, SQL tool calls, and multiple model invocations. The CFO wants a single dashboard that shows per-agent resource consumption (tokens, tool calls, and memory operations) on a per-invocation basis so the team can identify which agent is inflating the monthly bill.

Which solution provides this visibility with minimal custom development?

A) Use Amazon Bedrock AgentCore's built-in CloudWatch metrics for agent invocations and parse per-invocation token metrics from the AgentCore namespace.
B) Enable Amazon Bedrock AgentCore Observability and configure its trace export to Amazon CloudWatch (or OpenSearch) to capture per-invocation traces with token, tool, and memory details.
C) Embed a custom SDK in each agent that fires CloudWatch custom metrics after every model call, tool call, or memory read.
D) Export all AgentCore logs to Amazon S3, then use an AWS Glue job on a nightly schedule to transform logs into aggregated per-agent cost metrics.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

B is Correct: Amazon Bedrock AgentCore Observability is the native feature designed to capture detailed traces of each agent invocation, including the sequence of model calls, tool invocations, memory operations, and associated token counts. By exporting these traces to CloudWatch or other observability backends, you get per-invocation, per-agent cost and resource breakdowns without building custom instrumentation.

Distractor Analysis

A is Incorrect: While Bedrock does publish CloudWatch metrics, the aggregation and per-agent breakdown of each agent's tokens, tool calls, and memory operations per invocation require the observability traces (spans/events) from AgentCore. CloudWatch metrics alone do not give per-invocation granularity at the agent level.
C is Incorrect: Instrumenting each agent with a custom SDK to record metrics is high operational overhead and error-prone. The whole point of AgentCore Observability is to obtain this data natively.
D is Incorrect: Log export + Glue ETL is a heavy batch processing solution. It adds significant latency (nightly batch) and operational complexity for a use case that can be solved with native AgentCore Observability traces.

Key Takeaway: Use Amazon Bedrock AgentCore Observability to natively capture per-invocation traces of model calls, tool calls, and memory usage, avoiding custom instrumentation and batch pipelines.

Question 7 - SageMaker Auto Scaling vs Savings Plans

A logistics company runs a real-time package-routing model on Amazon SageMaker AI with a backlog of predictions. The endpoint's utilization peaks during business hours (8 AM - 8 PM) and drops to almost zero overnight. The team wants to reduce costs but is unsure whether to use Auto Scaling, a smaller instance family, or a Savings Plan.

What is the optimal combination of changes for this workload?

A) Purchase a SageMaker AI Savings Plan and set the instance count high enough to cover the 12-hour peak permanently.
B) Configure a target-tracking Auto Scaling policy on InvocationsPerInstance to reduce instance count overnight, and purchase a SageMaker AI Savings Plan for the minimum baseline.
C) Right-size the instance family to the smallest possible type and disable Auto Scaling to simplify management.
D) Use a Multi-Model Endpoint (MME) to host unrelated models on the same endpoint, eliminating the need for Auto Scaling.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

B is Correct: The workload has a clear diurnal pattern: high utilization during business hours and near-zero overnight. A target-tracking Auto Scaling policy on InvocationsPerInstance (or CPUUtilization) allows the endpoint to automatically add instances during peak and release them overnight. Pair that with a SageMaker AI Savings Plan for the minimum steady baseline capacity that remains overnight, so you are not paying On-Demand rates even for the reserved minimum.

Distractor Analysis

A is Incorrect: Setting the endpoint to peak capacity permanently means you hold unused instances overnight, defeating the purpose of cost optimization. While a Savings Plan reduces the per-hour rate, it still applies to the inflated instance count.
C is Incorrect: Disabling Auto Scaling on a workload with a drastic diurnal pattern leaves you either over-provisioned (paying for idle instances at night) or under-provisioned (risking timeouts during peak). Right-sizing alone does not solve the daily traffic swing.
D is Incorrect: A Multi-Model Endpoint is designed to consolidate many small, infrequently-called models onto shared compute; it does not address the scaling pattern of a high-traffic single model. Using it with unrelated models can also introduce cold-start latency and resource contention.

Key Takeaway: For diurnal workloads, combine Auto Scaling (to release peak capacity) with SageMaker AI Savings Plans (to discount the steady baseline); right-sizing alone or a static peak config leaves savings on the table.

Question 8 - Foundation Model Selection for Cost Optimization

A gaming analytics team uses a large frontier foundation model on Amazon Bedrock to generate short player-facing notifications like "Level completed, claim your reward." The responses are generated to be no longer than 30 tokens, and the input context is small (under 100 tokens). The large model is costing $0.015 per 1K input tokens and $0.075 per 1K output tokens, and the team wants to cut inference costs by at least 50% without materially affecting quality for these short, templated responses.

Which optimization should the ML engineer recommend?

A) Enable prompt caching for the notification templates to reduce input token costs.
B) Configure Provisioned Throughput so that the high volume of notifications qualifies for a reserved-capacity discount.
C) Switch to a smaller, faster foundation model (e.g., Claude Haiku or an equivalent lightweight model) that meets the short-response quality bar at a lower per-token cost.
D) Increase the max tokens parameter to 100 so the model has more generation capacity, reducing cost through prompt truncation.

Click to reveal Answer & Explanation

Correct Answer: C

Correct Option Analysis

C is Correct: The task is simple (short, templated notifications) and the current model is oversized ("larger than the job needs"). By selecting a smaller foundation model with lower per-token pricing that still meets the quality threshold, the team can cut inference costs substantially (often >50%) without changing application behavior. This is the direct analogue of "right-sizing" from the instance world to the token world.

Distractor Analysis

A is Incorrect: Prompt caching saves input token costs for long, repetitive, static prompt prefixes. For a short prompt (under 100 tokens) and very short outputs, the savings from caching would be minimal or nil, since the cacheable prefix is small relative to the total request.
B is Incorrect: Provisioned Throughput is designed for predictable, high-volume workloads, but it primarily lowers cost per hour for reserved capacity. For a simple short-response task, a smaller model would be a more direct and substantial cost lever than committing to provisioned capacity (which also locks you into hourly minimums).
D is Incorrect: Increasing max tokens increases the maximum possible output length and thus the maximum cost per request. It does not reduce token costs; if anything, it may lead to longer-than-needed outputs and higher costs.

Key Takeaway: The strongest lever for token-cost reduction is often choosing a smaller, sufficient-sized foundation model for simple tasks; prompt caching, provisioned throughput, and max-token tuning are secondary levers for specific conditions.

Question 9 - Cost Attribution with Tags and Budgets

A data platform team runs 15 separate Amazon SageMaker AI endpoints across three workloads: fraud detection, product recommendation, and demand forecasting. Each workload belongs to a different business unit, and monthly spend on SageMaker has been rising. The CFO requests a way to attribute endpoint costs by workload and get an alert before the quarterly SG&A budget for fraud detection specifically is exceeded.

Which TWO AWS services or features should the team implement? (Select TWO)

A) Apply AWS cost allocation tags (e.g., CostCenter=Fraud, Workload=FraudDetection) to each SageMaker endpoint resource.
B) Use AWS Trusted Advisor to send an alert when the fraud-detection endpoint's spend crosses the quarterly threshold.
C) Create an AWS Budget for the fraud-detection workload using the allocated tag filter and define an alert threshold at 80% of the quarterly amount.
D) Use AWS Compute Optimizer to get a recommendation to right-size the endpoints and trigger an SNS alert when recommendations become available.
E) Configure Amazon CloudWatch Logs to export all endpoint logs to Amazon S3, then use AWS Glue to calculate monthly per-endpoint costs.

Click to reveal Answer & Explanation

Correct Answers: A and C

Correct Option Analysis

A is Correct: Cost allocation tags (e.g., Workload, Team, Environment) are the foundational mechanism for attributing AWS costs. By tagging every SageMaker endpoint with meaningful workload/cost-center labels, you can group and filter spend per business unit in AWS Cost Explorer and AWS Budgets.
C is Correct: AWS Budgets allows you to set a custom budget on a tag-filtered subset of your AWS spend (e.g., only resources with Workload=FraudDetection). You can set a threshold alert (e.g., 80% of quarterly amount) that triggers an SNS/email notification when forecasted or actual spend passes that amount. This fulfills the "alert before exceeding" requirement.

Distractor Analysis

B is Incorrect: AWS Trusted Advisor checks accounts against AWS best practices (including underutilized resources), but it does not provide workload-specific budget alerts or tag-filtered spend tracking.
D is Incorrect: AWS Compute Optimizer recommends right-sizing for compute resources, but it does not send alerts about budget thresholds or track spend against a quarterly business-unit budget. It helps optimize workloads, not attribute costs or send budget alerts.
E is Incorrect: While logs and Glue can eventually calculate costs, this requires heavy ETL and is not a dashboard/alerting mechanism. There is no native cost attribution from endpoint logs—tags are the native, direct way to attribute spend.

Key Takeaway: Use cost allocation tags for attribution and AWS Budgets with tag filters for threshold-driven alerts; do not confuse Trusted Advisor (best-practice checks) or Compute Optimizer (right-sizing) with budget management.

Question 10 - Embedding Computation Cost Reduction

A multilingual content platform has an embedding pipeline that processes 3 million new text documents per day. The team uses Amazon Bedrock (Titan Embeddings) to generate embeddings and stores them in Amazon OpenSearch Serverless. The daily batch job runs on Amazon SageMaker and takes 12 hours to complete, leaving little time for re-runs if errors occur. The team wants to reduce per-unit embedding costs while accelerating the pipeline.

Which TWO strategies should the ML engineer implement to reduce costs and improve throughput? (Select TWO)

A) Convert the embedding job to use Amazon Bedrock batch inference (or asynchronous model invocation) which may support discounted pricing or higher throughput quotas for batch workloads.
B) Replace Amazon Bedrock embeddings with a self-hosted embedding model on Amazon SageMaker real-time endpoints so you can avoid token-based pricing.
C) Implement prompt caching on the embedding prompts to reuse embeddings for repeated text patterns.
D) Use Amazon OpenSearch Serverless with vector quantization (e.g., int8 or binary quantization) to reduce vector storage size and query cost without retraining the embedding model.
E) Preprocess and deduplicate the input text corpus to remove redundant or near-identical documents before generating embeddings.

Click to reveal Answer & Explanation

Correct Answers: A and E

Correct Option Analysis

A is Correct: Amazon Bedrock batch inference (or asynchronous invocation) is designed for large-scale embedding jobs that don't require real-time responses. Batch inference can provide higher throughput and in some cases uses a different pricing structure or priority queue that results in lower cost per embedding. This directly addresses the daily batch embedding workload.
E is Correct: Deduplicating the input corpus (e.g., via semantic similarity or hash-based dedup) is the most direct way to reduce embedding computation costs: fewer unique inputs mean fewer embedding calls, fewer tokens processed, and less storage.

Distractor Analysis

B is Incorrect: Self-hosting an embedding model on SageMaker real-time endpoints replaces token-based Bedrock pricing with instance-hour pricing, but for 3M daily documents this likely increases operational overhead and does not necessarily reduce cost per embedding. You would also have to manage GPU inference and scaling on your own.
C is Incorrect: Prompt caching is not applicable to embedding requests in the same way it is to text generation; embeddings are deterministic for a given text and "cache" here typically means you store the input–embedding pair, not rely on KV-cache mechanisms. Caching is better expressed as "deduplicate inputs" (Option E).
D is Incorrect: Vector quantization (e.g., product quantization, int8) reduces storage and accelerates query speed, but it is applied to the vector index after the vectors are generated, not to the embedding computation phase. It does not reduce per-unit embedding computation costs or output size during generation.

Key Takeaway: Deduplicate text before generating embeddings to cut computation cost, and use Bedrock batch inference for large-scale non-real-time embedding workloads; vector quantization reduces storage, but not embedding compute.