Skip to content

Skill 4.2.4: Optimize capacity for cost, performance, and reliability

Question 1 - SageMaker Real-Time Endpoint Auto Scaling vs. Right-Sizing

A financial services company operates a fraud detection model on an Amazon SageMaker AI real-time endpoint using ml.g5.2xlarge instances. CloudWatch metrics reveal the endpoint consistently averages 85% CPU utilization during market hours (9:00 AM - 4:00 PM) but drops to 8% overnight. The current instance count is fixed at 8, and latency during peak hours occasionally exceeds the 50ms SLA due to bursty traffic spikes. The team wants to reduce costs while maintaining the 50ms latency SLA during trading hours.

Which solution BEST optimizes cost and performance under these constraints? (Select ONE)

  • A) Configure SageMaker target-tracking auto scaling on InvocationsPerInstance to scale capacity between 4 and 12 instances, keeping the same instance type.
  • B) Replace the endpoint with a SageMaker Serverless Inference endpoint to eliminate idle capacity costs entirely.
  • C) Migrate the workload to a smaller instance family (ml.g5.xlarge) and set a fixed instance count of 12 to handle peak bursts.
  • D) Move the model to a SageMaker multi-model endpoint shared with unrelated models to reduce idle capacity.
  • E) Convert to Amazon Bedrock with an on-demand fraud detection model to avoid instance management.
Click to reveal Answer & Explanation

Correct Answer: A

Correct Option Analysis

  • A is Correct: Target-tracking auto scaling on InvocationsPerInstance allows the endpoint to dynamically adjust instance count based on actual traffic patterns. During market hours, it scales up (up to 12 instances) to meet the 50ms SLA during bursty spikes; overnight, it scales down (to as low as 4 instances) to release idle compute capacity. This most directly addresses the diurnal traffic pattern while meeting performance requirements.

Distractor Analysis

  • B is Incorrect: SageMaker Serverless Inference is best suited for intermittent or unpredictable low-to-moderate traffic workloads. The traffic pattern described is high and bursty during market hours—serverless would likely throttle or exceed cold-start limits under sustained load, potentially missing SLA targets.
  • C is Incorrect: Moving to a smaller instance family assumes the workload is right-sized incorrectly—but the average 85% CPU during market hours indicates the instance type is appropriate; the problem is the fixed instance count (capacity problem, not sizing problem). Additionally, this introduces a larger fixed capacity that would be wasted overnight.
  • D is Incorrect: While multi-model endpoints (MMEs) can help consolidate many small, intermittently used models, they are not ideal for a core, high-throughput production workload. MMEs introduce cold-start latency and potential resource contention, which risks the 50ms SLA.
  • E is Incorrect: Converting to Amazon Bedrock for fraud detection is a fundamentally different architectural paradigm (token-based FM inference) that may not match the latency, cost, or regulatory requirements of a financial fraud detection model. Bedrock is not a drop-in replacement for a real-time SageMaker endpoint.

Key Takeaway: For workloads with predictable diurnal patterns where instances are correctly sized for peak demand, auto scaling is the right lever—not right-sizing or switching to serverless.

Question 2 - Choosing the Right Cost Management Tool for Anomaly Detection

A large e-commerce company runs a portfolio of generative AI applications on Amazon Bedrock across 12 AWS accounts. They use cost allocation tags to attribute token spend by team, environment (production vs. staging), and model. The finance team wants to be alerted when forecasted monthly Bedrock spending (grouped by tag Team=RecommendationAI) is projected to exceed $15,000, and they need a historical view of token usage trends to identify which model is driving the increase.

Which combination of tools should the ML engineering team configure? (Select TWO)

  • A) AWS Trusted Advisor to send alerts when Bedrock spend crosses the $15,000 threshold.
  • B) AWS Budgets with a cost budget filtered by the tag Team=RecommendationAI and threshold alerts at 80% forecasted and 100% actual.
  • C) AWS Cost Explorer to analyze historical token usage trends by model and identify the cost driver.
  • D) AWS X-Ray with service map tracing to correlate token usage with AWS Cost Anomaly Detection.
  • E) Amazon CloudWatch Logs with a metric filter on token counts to trigger a billing alert.
Click to reveal Answer & Explanation

Correct Answers: B and C

Correct Options Analysis

  • B is Correct: AWS Budgets supports setting custom cost budgets filtered by allocation tags. You can configure forecasted cost alerts—meaning the finance team will be notified while spend is on track to exceed $15,000, giving them time to act before the threshold is breached.
  • C is Correct: AWS Cost Explorer provides historical and forecast spend analysis. Filtering by model (via tag or usage type) allows the team to identify which model is driving cost increases over time.

Distractor Analysis

  • A is Incorrect: AWS Trusted Advisor checks accounts against AWS best practices (idle resources, underutilized EBS volumes, etc.), but it does not support custom tag-based budget alerting or threshold-driven notifications on Bedrock spend.
  • D is Incorrect: AWS X-Ray is a distributed tracing service for performance analysis and latency attribution; it does not help with cost anomaly detection or budget alerting. AWS Cost Anomaly Detection is a separate service.
  • E is Incorrect: CloudWatch Logs with metric filters on token counts can monitor token usage as an operational metric, but it cannot send billing threshold alerts. Billing and cost tracking are managed by AWS Budgets, Cost Explorer, and Cost Anomaly Detection.

Key Takeaway: Use AWS Budgets for proactive threshold-based cost alerts (including forecasted thresholds) and AWS Cost Explorer for historical analysis and cost driver identification.

Question 3 - Prompt Caching for Frequently Reused Prefixes

A customer support team has deployed a RAG-based chatbot using Amazon Bedrock with Anthropic Claude. Each request includes a large system prompt (~4,000 tokens) containing detailed company policies, product schemas, and a few-shot example set. The team observes that 90% of user queries are from 10 common question types, and the prefix is identical for all requests. They want to reduce token costs without changing the model or degrading response quality.

Which approach BEST reduces cost under these conditions? (Select ONE)

  • A) Move the system prompt to the end of each request to minimize input token charges.
  • B) Enable prompt caching for the static prefix, ensuring dynamic user queries are appended after the cached prefix.
  • C) Reduce the number of few-shot examples from the system prompt to shorten the context window.
  • D) Route all requests through a SageMaker real-time endpoint instead of Bedrock to avoid token-based pricing.
  • E) Configure the model to use a smaller maximum output token limit to reduce costs.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

  • B is Correct: Prompt caching is designed for exactly this scenario. A large, static, frequently reused prefix (system prompt, tool schemas, few-shot examples) can be cached and reused across requests, reducing input token prefill costs. Dynamic content (user query) must be appended after the cached prefix to achieve cache hits.

Distractor Analysis

  • A is Incorrect: Prompt caching requires the static (cached) content to appear at the beginning of the prompt. Placing the system prompt at the end would not invalidate caching per se, but it would mean the cacheable prefix is absent—no savings materialize.
  • C is Incorrect: While reducing few-shot examples does lower input tokens, it also degrades response quality and is a prompt engineering trade-off—not the most effective solution when prompt caching can maintain quality while reducing cost.
  • D is Incorrect: Moving from Bedrock to SageMaker real-time endpoints is a significant architectural change that shifts the billing model from per-token to per-instance-hour. This is not appropriate for a RAG chatbot requiring an LLM and introduces unnecessary operational overhead.
  • E is Incorrect: Capping max output tokens only controls output token costs; it does not address the large input token cost from the system prompt.

Key Takeaway: Prompt caching reduces cost by reusing the KV-cache of a static prefix across requests—ensure the static content appears at the beginning and dynamic content is appended at the end.

Question 4 - SageMaker Savings Plans vs. Generic Compute Savings Plans

A global logistics company runs multiple ML workloads on Amazon SageMaker AI:

  • A 24/7 real-time inference endpoint for route optimization (steady, predictable throughput)
  • A nightly batch transform job for demand forecasting (predictable, daily)
  • An experimental training pipeline that runs sporadically during model development (variable, unpredictable)

The team wants to commit to a purchasing option that applies to the SageMaker usage they know will occur year-round, while keeping variable workloads on-demand.

Which purchasing option should they select? (Select ONE)

  • A) Compute Savings Plans (1-year term) covering all compute across the account.
  • B) EC2 Instance Savings Plans for ml.m5 instance families.
  • C) SageMaker AI Savings Plans committing to a specific hourly spend based on the steady-state real-time inference and batch transform usage.
  • D) AWS Reserved Instances for all SageMaker endpoints.
  • E) On-demand pricing with AWS Budgets alerts—no commitment is advisable for SageMaker.
Click to reveal Answer & Explanation

Correct Answer: C

Correct Option Analysis

  • C is Correct: SageMaker AI Savings Plans are specifically designed for SageMaker workloads (real-time inference, batch transform, training, notebooks). By committing to a consistent hourly dollar amount that covers the predictable 24/7 endpoint and nightly batch job, the company receives discounted rates (up to 64%) on that usage. Variable/experimental training can remain on-demand without being covered by the commitment.

Distractor Analysis

  • A is Incorrect: Compute Savings Plans apply to EC2, Lambda, and Fargate—they do not cover SageMaker ML instance usage (ml.* instance types).
  • B is Incorrect: EC2 Instance Savings Plans apply only to standard Amazon EC2 instances within a specific family and region, not to SageMaker-managed instances (ml.* types).
  • D is Incorrect: AWS Reserved Instances (RIs) apply only to EC2 instances, not to SageMaker ML instances. The SageMaker-equivalent commitment is SageMaker AI Savings Plans.
  • E is Incorrect: While on-demand pricing with Budgets is prudent for variable workloads, the question explicitly asks for a purchasing option that reduces rates on the predictable SageMaker usage—SageMaker AI Savings Plans fulfills this.

Key Takeaway: SageMaker AI Savings Plans apply to Amazon SageMaker ML instance usage only; Compute Savings Plans and EC2 Instance Savings Plans do not cover SageMaker.

Question 5 - Monitoring Agent Consumption with Bedrock AgentCore Observability

A healthcare technology company has deployed an Amazon Bedrock Agent for patient triage. Each user request triggers a planning step, multiple knowledge base retrievals, a tool call to query patient records in DynamoDB, and a final synthesis step. The team wants to identify which component of the agent's execution is consuming the most tokens and causing the highest latency, per invocation, to optimize the agent's performance and cost.

Which tool should the team use? (Select ONE)

  • A) Amazon CloudWatch Logs with a custom metric filter on token counts.
  • B) AWS X-Ray with trace maps for the Bedrock Agent.
  • C) Amazon Bedrock AgentCore Observability.
  • D) Amazon CloudWatch Container Insights with EKS-based agent tracing.
  • E) AWS CloudTrail event logging for agent invocations.
Click to reveal Answer & Explanation

Correct Answer: C

Correct Option Analysis

  • C is Correct: Amazon Bedrock AgentCore Observability is purpose-built to capture agent invocation internals—including inference calls, tool invocations, memory operations, and retrieval steps—at the individual invocation level. It provides per-invocation trace data that separates model calls from tool calls and knowledge base retrievals, enabling precise cost and latency attribution.

Distractor Analysis

  • A is Incorrect: CloudWatch Logs with metric filters can aggregate token counts, but they do not provide per-invocation trace data separating planning, tool calls, retrieval, and synthesis steps. This lacks the granular breakdown needed to identify the cost driver.
  • B is Incorrect: AWS X-Ray provides distributed tracing across services, but Bedrock Agents are a managed service—X-Ray tracing does not automatically capture internal agent orchestration steps. AgentCore Observability is the service that exposes agent execution traces.
  • D is Incorrect: Container Insights monitors EKS-based container workloads and is irrelevant for Bedrock Agents, which are managed by AWS and do not run on customer containers.
  • E is Incorrect: CloudTrail logs API-level events (e.g., InvokeAgent, StartIngestionJob), but does not capture internal agent orchestration metrics (token counts, tool call latencies, planning step details).

Key Takeaway: For per-invocation visibility into an agent's token consumption, tool call latency, and orchestration overhead, use Bedrock AgentCore Observability.

Question 6 - Optimizing Vector Database Storage Costs

A media streaming platform stores user preference embeddings in Amazon OpenSearch Serverless for a recommendation retrieval system. The vector index contains 500 million embeddings of dimension 1024 using float32 (4 bytes per dimension). Recent cost analysis shows vector database storage is the second-largest cost driver after Bedrock token spend. The team wants to reduce storage costs without degrading retrieval quality by more than 1% accuracy.

Which approach BEST optimizes storage cost under these constraints? (Select ONE)

  • A) Move to Amazon RDS for PostgreSQL with pgvector to leverage cheaper EBS storage.
  • B) Switch from float32 to int8 scalar quantization for vector embeddings, trading a small accuracy loss for a 4x reduction in storage.
  • C) Delete 50% of the embeddings to halve the storage footprint.
  • D) Move the vector index to Amazon S3 with S3 Select for vector similarity search.
  • E) Increase the embedding dimension to 2048 with float16 to reduce overall vector count.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

  • B is Correct: Scalar quantization (e.g., converting float32 to int8) reduces vector storage by ~4x (32 bits → 8 bits per dimension). This is a well-established technique in vector database optimization and typically results in minimal retrieval accuracy degradation (< 1-2%). It directly addresses the storage cost without changing the infrastructure.

Distractor Analysis

  • A is Incorrect: Moving to RDS with pgvector may change the database engine but does not inherently reduce storage costs—the same embeddings are stored. Additionally, pgvector at 500M vectors with high-dimensional embeddings can underperform and cost more, introducing significant operational overhead.
  • C is Incorrect: Deleting 50% of embeddings would severely degrade retrieval quality—well beyond the 1% accuracy loss tolerance. This is a destructive optimization that does not meet the stated quality constraint.
  • D is Incorrect: S3 Select does not support vector similarity search (nearest neighbor queries). S3 Select is for CSV/JSON/Parquet row-level filtering—not for ANN similarity queries. This option is functionally invalid.
  • E is Incorrect: Increasing embedding dimension from 1024 to 2048 actually doubles the storage requirement per vector, contrary to the goal of reducing storage costs. Even with float16, the total size remains roughly the same (2048 × 2 bytes = 4KB vs. 1024 × 4 bytes = 4KB).

Key Takeaway: Vector database storage optimization typically involves quantization (e.g., float32 → int8 or float16), which reduces bytes per dimension with minimal retrieval accuracy loss.

Question 7 - Provisioned Throughput for Predictable Bedrock Workloads

A legal document processing platform runs an Amazon Bedrock application using Claude for contract summarization. The platform handles approximately 2 million requests per month, with consistent daily volume throughout the week. The team has already optimized prompts (capped output tokens, trimmed context) and is on On-Demand pricing. They want to further reduce cost while maintaining throughput for the predictable, steady-state workload.

Which change should the team make? (Select ONE)

  • A) Switch to SageMaker JumpStart to host an open-source LLM on an EC2 instance.
  • B) Purchase SageMaker AI Savings Plans to cover the Bedrock token usage.
  • C) Migrate to Amazon Bedrock Provisioned Throughput for the specific model and region.
  • D) Implement AWS Budgets alerts to cap spending at the current monthly amount.
  • E) Route requests through Amazon Bedrock Batch Inference to get discounted rates.
Click to reveal Answer & Explanation

Correct Answer: C

Correct Option Analysis

  • C is Correct: Amazon Bedrock Provisioned Throughput allows you to reserve dedicated model units (throughput capacity) for a specific foundation model at a fixed hourly rate. For predictable, high, steady-state token consumption, this can provide significant cost savings compared to On-Demand token pricing.

Distractor Analysis

  • A is Incorrect: Moving to SageMaker JumpStart means self-hosting an LLM on EC2 instances, shifting the billing model from per-token to per-instance-hour. This introduces significant operational overhead (infrastructure management, scaling, security patching) and may not lower costs for a steady 2M requests/month workload.
  • B is Incorrect: SageMaker AI Savings Plans apply only to Amazon SageMaker AI instance usage (endpoints, training, batch transform, notebooks), not to Amazon Bedrock token consumption. Bedrock is a managed FM inference service with its own pricing model.
  • D is Incorrect: AWS Budgets provides alerts and tracking; it does not reduce the per-request cost or offer any discount mechanism. Budgets are useful for visibility, not cost reduction.
  • E is Incorrect: Amazon Bedrock Batch Inference is designed for asynchronous, non-interactive workloads (e.g., document processing pipelines) where latency is not a concern. The platform described handles real-time contract summarization requests, so batch inference would require architectural changes to the client and could introduce unacceptable latency.

Key Takeaway: For predictable, high-volume Bedrock inference workloads, Provisioned Throughput reserves model capacity at a fixed hourly rate, offering savings compared to On-Demand token pricing. For real-time workloads, batch inference is not a substitute.

Question 8 - Right-Sizing SageMaker Instances Based on Utilization

A marketing analytics company runs a customer segmentation model on an Amazon SageMaker AI real-time endpoint using ml.c5.9xlarge instances. CloudWatch shows the endpoint averages 40% CPU utilization and 55% memory utilization over a representative one-month period, with no significant traffic peaks. The p99 latency is 150ms, well below the company's 500ms SLA. The team wants to reduce costs while maintaining the current SLA.

Which action should the team take? (Select ONE)

  • A) Configure SageMaker auto scaling to scale the endpoint to zero instances during periods of low traffic.
  • B) Migrate the workload to a smaller instance family (e.g., ml.c5.4xlarge) to better match the observed utilization.
  • C) Move the workload to a multi-model endpoint shared with 20 other infrequently used models.
  • D) Switch to Amazon Bedrock with a large language model for customer segmentation.
  • E) Increase the instance count to 2 and use a load balancer to distribute requests.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

  • B is Correct: Right-sizing is the correct optimization for an adequately utilized (but oversized) endpoint with stable traffic. Moving from ml.c5.9xlarge to ml.c5.4xlarge halves the compute resources while maintaining sufficient headroom (observed 40% CPU / 55% memory on a 9xlarge suggests ~80% CPU / 110% memory on a 4xlarge—which may be too tight; further analysis needed, but a smaller family is the right direction). The p99 latency is 150ms against a 500ms SLA, indicating substantial headroom for downsizing without breaching the SLA.

Distractor Analysis

  • A is Incorrect: SageMaker real-time endpoints do not support scaling to zero instances. Unlike serverless inference, real-time endpoints always require at least one instance. Scaling to zero also introduces cold-start latency when traffic arrives.
  • C is Incorrect: Multi-model endpoints are best suited for consolidating many small, intermittent models that share a serving container. This core production workload has steady traffic and should retain its dedicated endpoint. MMEs also introduce cold-start latency and potential resource contention that could impact the SLA.
  • D is Incorrect: Amazon Bedrock is a managed FM inference service that is not designed for traditional ML models like customer segmentation algorithms. Bedrock is appropriate for LLM-based workloads, not classical ML models on a SageMaker endpoint.
  • E is Incorrect: Adding more instances would increase costs, not reduce them. The issue is oversized instances, not insufficient capacity; the endpoint already meets SLAs with a single instance.

Key Takeaway: When CloudWatch shows sustained low utilization without traffic peaks, right-sizing (smaller instance family) is the appropriate cost optimization—not auto scaling (which addresses variable traffic) or multi-model endpoints (which target fragmentation).

Question 9 - Analyzing Cost Drivers with AWS Cost Explorer

A healthcare analytics company runs multiple ML workloads on Amazon SageMaker AI and Amazon Bedrock. Their monthly AWS bill increased by 60% over the past 3 months. The ML team suspects token usage on a new chatbot is the primary driver, but they also have 4 SageMaker endpoints and 2 training jobs running. They need to identify the exact driver and attribute spend to specific models and teams.

Which approach should the team use to identify the cost driver? (Select ONE)

  • A) Use AWS Cost Explorer with cost allocation tags filtered by team, service, and usage type to analyze the historical spend trend.
  • B) Use AWS Trusted Advisor to scan for idle resources and underutilized instances.
  • C) Use AWS X-Ray service maps to trace token usage and generate a cost attribution report.
  • D) Use Amazon CloudWatch Alarms to notify the team when any endpoint CPU utilization exceeds 50%.
  • E) Use AWS Config rules to track resource configuration changes and correlate them with cost spikes.
Click to reveal Answer & Explanation

Correct Answer: A

Correct Option Analysis

  • A is Correct: AWS Cost Explorer is the primary tool for analyzing historical and forecast AWS spend. It supports grouping and filtering by tag (e.g., Team=Chatbot, Model=Claude-3), service (SageMaker vs. Bedrock), and usage type (e.g., token usage, instance hours). This allows the team to pinpoint which workload drove the 60% increase.

Distractor Analysis

  • B is Incorrect: AWS Trusted Advisor provides cost checks that surface idle or underutilized resources (e.g., unused EBS volumes, idle load balancers), but it does not provide historical spend trend analysis or attribute costs to specific workloads or teams.
  • C is Incorrect: AWS X-Ray is a performance tracing and latency analysis service; it does not provide cost attribution or billing data. It traces request latency across services, not token spend.
  • D is Incorrect: CloudWatch Alarms notify on metric thresholds (CPU, memory, latency, errors), but they are operational monitoring tools—not cost analysis tools. High CPU on a SageMaker endpoint does not necessarily correlate with the 60% increase, and alarms do not provide historical cost trends.
  • E is Incorrect: AWS Config tracks resource configuration changes and compliance drift, not billing data or spend attribution. It is not a cost management tool.

Key Takeaway: AWS Cost Explorer is the tool for identifying cost drivers and attributing spend to specific teams, models, or environments—especially with proper cost allocation tagging.

Question 10 - Optimizing Token Usage for Embedding Computations

A legal research platform uses Amazon Titan Embeddings on Amazon Bedrock to generate embeddings for legal documents. The system processes approximately 100,000 documents daily, re-embedding every document each day to capture any modifications. Analysis shows 80% of documents are unchanged daily, and the same input text is repeatedly sent for embedding. The team wants to reduce token costs without degrading the embedding quality.

Which strategy should the team implement? (Select ONE)

  • A) Cache the embedding results for unchanged documents and only re-embed modified documents.
  • B) Reduce the embedding dimension from 1536 to 256 to lower compute costs.
  • C) Switch to a smaller embedding model (e.g., from Titan to a smaller open-source model) regardless of quality.
  • D) Use Amazon Bedrock Batch Inference for all embedding requests to get discounted rates.
  • E) Convert the embedding pipeline to use SageMaker real-time endpoints instead of Bedrock.
Click to reveal Answer & Explanation

Correct Answer: A

Correct Option Analysis

  • A is Correct: The core inefficiency is redundant embedding computation for unchanged documents. By caching embedding results (e.g., in Amazon DynamoDB or S3 keyed by document hash) and only re-embedding documents whose content has changed (detected via checksum comparison), the team reduces token consumption by ~80% without any quality degradation.

Distractor Analysis

  • B is Incorrect: Reducing embedding dimension from 1536 to 256 would degrade retrieval quality and is not supported by Amazon Titan Embeddings (which outputs fixed dimensions). Titan typically supports dimension reduction via Matryoshka Representation Learning (MRL) in some models, but this still trades quality for cost.
  • C is Incorrect: Switching to a smaller embedding model may reduce cost per token, but it ignores the fundamental problem—re-embedding 80% unchanged documents daily. This approach also risks quality degradation without assessing whether the smaller model meets the quality requirement.
  • D is Incorrect: Bedrock Batch Inference is appropriate for asynchronous, high-volume processing of large datasets, but the problem described is not about synchronous vs. asynchronous processing—it is about redundant embedding of unchanged content. Batch inference alone would not eliminate the redundancy.
  • E is Incorrect: Moving from Bedrock to SageMaker real-time endpoints shifts billing from per-token to per-instance-hour. This introduces significant operational overhead (instance management, scaling, deployment) and does not address the root cause of redundant embedding.

Key Takeaway: Embedding computation costs can be optimized by caching embeddings for unchanged content and using content hashing to detect modifications—eliminating redundant token usage without quality loss.