Skill 4.2.8: Monitor agent resource consumption patterns

Question	Answer & Explanation
### Question 1 - Tracing Agent Invocations with Bedrock AgentCore An enterprise is deploying a multi-step customer service agent using Amazon Bedrock. The agent coordinates between a foundation model, a knowledge base retrieval tool, and a database query tool. The ML engineering team needs to visualize the exact sequence of tool invocations and model calls for a specific user request that took 15 seconds to complete. The solution must require the least operational overhead and use AWS native observability features. Which approach should the team use to fulfill these requirements? (Select ONE) - A) Enable AWS X-Ray tracing on the Lambda function hosting the agent logic. - B) Use Amazon Bedrock AgentCore Observability to capture the inference calls, tool invocations, and memory operations inside the agent invocation. - C) Configure CloudWatch Container Insights to monitor the CPU and memory utilization of the underlying Bedrock infrastructure. - D) Parse CloudWatch Logs with CloudWatch Logs Insights to manually reconstruct the sequence of tool calls.	Correct Answer: B Correct Answer Analysis: - B is Correct: Amazon Bedrock AgentCore Observability is specifically designed to capture the internal operations of an agent invocation, including inference calls, tool invocations, and memory operations. It provides a per-invocation trace natively without requiring custom instrumentation, satisfying the least operational overhead constraint. Distractor Analysis: - A is Incorrect: AWS X-Ray traces requests across microservices (like Lambda functions), but it does not natively peer inside the agent's orchestration logic to map out the specific sequence of model and tool calls without heavy custom instrumentation. - C is Incorrect: CloudWatch Container Insights provides infrastructure-level metrics (CPU, memory) for containerized workloads, but it does not provide application-level visibility into agent orchestration or tool invocation sequences. - D is Incorrect: Manually parsing CloudWatch Logs to reconstruct the sequence introduces high operational overhead and is prone to errors compared to a purpose-built observability tool. > Key Takeaway: For Amazon Bedrock Agents, AgentCore Observability provides native, per-invocation tracing of internal agent operations (planning, tool calls, synthesis) without custom code.
### Question 2 - Attributing Agent Latency to Specific Components A generative AI application uses an Amazon Bedrock Agent to handle complex financial queries. The team notices that the average response latency has increased from 5 seconds to 12 seconds over the past week. They need to identify whether the bottleneck is due to the foundation model's inference time or an external API call made by one of the agent's tools. The solution must provide a per-invocation breakdown rather than aggregate metrics. Which approach meets these requirements? (Select ONE) - A) Check CloudWatch Metrics for `InvocationsPerInstance` on the SageMaker endpoint. - B) Use AWS Compute Optimizer to analyze the utilization history of the foundation model. - C) Analyze the distributed trace in Amazon Bedrock AgentCore Observability to attribute latency to specific components. - D) Enable AWS CloudTrail to log all API calls made by the agent.	Correct Answer: C Correct Answer Analysis: - C is Correct: AgentCore Observability provides distributed tracing for the agent, allowing engineers to see the exact duration of each segment (planning model, tool call, synthesis model). This per-invocation breakdown pinpoints whether the latency is caused by the external API tool or the FM inference. Distractor Analysis: - A is Incorrect: `InvocationsPerInstance` is an aggregate metric that shows traffic volume, not the latency breakdown of individual components within an agent request. - B is Incorrect: AWS Compute Optimizer recommends instance types based on utilization history, but it does not provide application-level latency tracing for agent tool calls. - D is Incorrect: CloudTrail logs API management events and audit trails, but it is not a performance monitoring tool and cannot measure the latency of internal agent orchestration steps. > Key Takeaway: Use distributed traces (via AgentCore Observability) to attribute latency to specific agent components, rather than relying on aggregate metrics that only show average behavior.
### Question 3 - Tracking Agent Cost Amplification A company's AI engineering team is building a monitoring dashboard in Amazon CloudWatch. They have deployed an agent that performs planning, tool execution, and synthesis. The team needs to track how the cost per invocation changes over time, specifically watching for "cost amplification" where a single user request triggers multiple hidden model calls. Which combination of metrics and observability tools provides this visibility with the least operational overhead? (Select TWO) - A) Amazon CloudWatch metrics for aggregate token count across the entire AWS account. - B) Amazon Bedrock AgentCore Observability for per-invocation token usage breakdown. - C) AWS X-Ray service map to visualize the dependency graph of the agent. - D) CloudWatch Logs Insights to query and aggregate token counts per `sessionId` or `traceId`. - E) AWS Trusted Advisor cost checks to identify idle agent resources.	Correct Answers: B and D Correct Answer Analysis: - B is Correct: AgentCore Observability natively captures the token usage for each model call within an agent invocation, exposing the "cost amplification" effect where one user request generates multiple internal calls. - D is Correct: CloudWatch Logs Insights can be used to query the structured logs generated by the agent's observability data, allowing the team to aggregate and visualize token counts per `sessionId` on a CloudWatch Dashboard. Distractor Analysis: - A is Incorrect: Aggregate account-level token metrics do not provide visibility into individual agent invocations, hiding the amplification effect. - C is Incorrect: The X-Ray service map shows dependencies and architectural flow, but it does not inherently display token consumption or cost metrics for model calls. - E is Incorrect: Trusted Advisor checks for idle resources (like EC2 instances), not for token consumption patterns of serverless AI agents. > Key Takeaway: Agent cost amplification occurs when a single user request triggers multiple model calls; tracking per-invocation token usage via AgentCore and Logs Insights is essential.
### Question 4 - Identifying High Token-Generating Sub-Agents An AI application utilizes multiple specialized agents to complete a single workflow. The team has noticed that a single user request results in an unexpectedly high token consumption. They need to monitor the resource consumption pattern to identify which agent or sub-agent is generating the most tokens. The constraint is to use AWS native tracing to map the request path without writing custom logging code. Which AWS service should the team use to identify the token-generating segments? (Select ONE) - A) AWS X-Ray - B) Amazon CloudWatch Container Insights - C) Amazon Bedrock AgentCore Observability - D) AWS CloudTrail Note: Assume the entire workflow is orchestrated within Amazon Bedrock.	Correct Answer: C Correct Answer Analysis: - C is Correct: Amazon Bedrock AgentCore Observability is purpose-built to capture the internal inference calls, tool invocations, and memory operations of Bedrock agents, allowing teams to see exactly which agent or sub-agent in a multi-agent workflow is consuming the most tokens. Distractor Analysis: - A is Incorrect: While AWS X-Ray traces distributed requests, it does not natively parse and display token consumption metrics for Bedrock agents without custom instrumentation. - B is Incorrect: CloudWatch Container Insights monitors infrastructure metrics (CPU, memory) for containerized applications, not application-level token consumption for AI agents. - D is Incorrect: AWS CloudTrail logs API calls for auditing and compliance, not for performance tracing or token monitoring. > Key Takeaway: When troubleshooting multi-agent workflows in Bedrock, AgentCore Observability maps the request path and token consumption natively, avoiding custom logging code.
### Question 5 - Detecting Agent Tool Retry Loops A team is troubleshooting an Amazon Bedrock agent that occasionally enters a retry loop when calling an external API tool. The retry loop goes unnoticed by users but causes the token bill to spike because the planning model is invoked multiple times. The team needs a way to monitor and detect when an agent invocation includes multiple consecutive tool retries for the same action. Which monitoring approach will best expose this behavior? (Select ONE) - A) Enable AWS X-Ray to monitor the HTTP response codes of the external API. - B) Use Amazon Bedrock AgentCore Observability to inspect the sequence of tool invocations within the agent's trace. - C) Implement an AWS Lambda function to parse CloudTrail logs for API errors. - D) Use AWS Budgets to set an alert for when token usage exceeds a certain threshold.	Correct Answer: B Correct Answer Analysis: - B is Correct: AgentCore Observability provides a per-invocation trace that shows the exact sequence of the agent's actions. Inspecting this trace will reveal consecutive duplicate tool calls and the subsequent repeated planning model invocations, directly identifying the retry loop. Distractor Analysis: - A is Incorrect: While X-Ray can show the API failure, it does not natively correlate it with the agent's internal retry loop and subsequent planning model token consumption. - C is Incorrect: Parsing CloudTrail logs with Lambda introduces high operational overhead and custom code, and CloudTrail does not capture internal agent orchestration logic (like planning model retries). - D is Incorrect: AWS Budgets alerts the team that the bill has spiked, but it does not provide the detailed per-invocation observability needed to diagnose why it spiked (the retry loop). > Key Takeaway: AgentCore Observability exposes the internal orchestration trace of an agent, making it easy to spot hidden retry loops that drive up token costs.
### Question 6 - Diagnosing Ineffective Prompt Caching in Agents A team is troubleshooting an Amazon Bedrock agent that summarizes daily reports but includes a large, volatile context window that changes with every request. The team wants to implement prompt caching to reduce resource consumption but finds that the savings are not materializing. Which monitoring approach will best explain why the cache is missing, and what is the likely root cause? Which approach identifies the issue and the root cause? (Select ONE) - A) Check CloudWatch Metrics for high CacheHitRate; the root cause is that the response length is too long. - B) Inspect the agent traces in AgentCore Observability to view token counts; the root cause is that the volatile context is placed before the static prefix, invalidating the cache. - C) Review CloudTrail logs for API throttling; the root cause is that the model is too small. - D) Monitor CPU utilization using CloudWatch; the root cause is that the vector database is returning too many results.	Correct Answer: B Correct Answer Analysis: - B is Correct: AgentCore Observability allows the team to view the exact input token counts per invocation. If cache is missing, the input token count will remain high. The root cause is that prompt caching requires a stable, exact prefix; placing volatile content (like a timestamp or dynamic report ID) at the beginning invalidates the cache on every request. Distractor Analysis: - A is Incorrect: A high CacheHitRate would indicate caching is working, not failing. Response length does not invalidate input prompt cache. - C is Incorrect: API throttling in CloudTrail would manifest as errors, not as a failure to cache tokens. Model size does not affect cache invalidation. - D is Incorrect: CPU utilization is an infrastructure metric and does not diagnose prompt caching behavior, which is an application-level token optimization feature. > Key Takeaway: Prompt caching requires a stable, static prefix; dynamic content at the start of a prompt invalidates the cache. Use AgentCore to observe the resulting high input token counts.
### Question 7 - Centralized Agent Observability Dashboard An enterprise is running multiple Bedrock Agents in production. The engineering team wants to set up a centralized observability dashboard that displays aggregate latency, error rates, and token consumption side-by-side for all agents. The solution must minimize operational overhead and use native AWS integrations. Which approach meets these requirements? (Select ONE) - A) Export CloudTrail logs to Amazon S3 and query them using Amazon Athena. - B) Configure a CloudWatch Dashboard that pulls metrics from Amazon Bedrock and AgentCore Observability. - C) Use AWS Compute Optimizer to recommend optimal agent configurations. - D) Set up AWS X-Ray to sample traces and visualize the service map.	Correct Answer: B Correct Answer Analysis: - B is Correct: Amazon CloudWatch Dashboards can natively aggregate metrics from multiple AWS services. By pulling metrics from Amazon Bedrock (for invocation and latency metrics) and AgentCore Observability (for token consumption patterns), the team can build a unified view with minimal operational overhead. Distractor Analysis: - A is Incorrect: Exporting CloudTrail logs to S3 and querying with Athena is a heavy operational pattern meant for auditing, not for real-time operational dashboards monitoring latency and token consumption. - C is Incorrect: AWS Compute Optimizer provides recommendations for compute resources (like EC2 instances), not for serverless AI agent token consumption or latency. - D is Incorrect: The X-Ray service map is useful for viewing dependencies, but it is not a customizable dashboard for tracking aggregate token consumption and error rates side-by-side over time. > Key Takeaway: Amazon CloudWatch Dashboards provide a native, low-overhead way to correlate aggregate latency, error rates, and token consumption metrics from Bedrock and AgentCore.
### Question 8 - Segregating Planning vs. Synthesis Latency A financial services company uses an AI agent to process complex transaction disputes. The agent uses a multi-step planning process: it calls a planning model, then a retrieval tool, then a synthesis model. The team needs to determine exactly how much latency is added by the planning model versus the synthesis model for each request. The constraint is to use AWS native observability tools that require zero custom code instrumentation. Which service fulfills this requirement? (Select ONE) - A) Use Amazon Bedrock AgentCore Observability to capture the inference calls and segment the trace by model role. - B) Deploy the agent on a SageMaker endpoint and use CloudWatch metrics for `InvocationsPerInstance`. - C) Manually time the API calls in the application code and log them to CloudWatch Logs. - D) Use AWS X-Ray to trace the Lambda function execution time for the entire agent.	Correct Answer: A Correct Answer Analysis: - A is Correct: AgentCore Observability natively segments the agent invocation into distinct phases (planning, tool calls, synthesis). It shows the exact duration of each inference call without requiring custom timing code, meeting the zero custom instrumentation constraint. Distractor Analysis: - B is Incorrect: `InvocationsPerInstance` on SageMaker only tracks the number of invocations on a hosted endpoint, not the internal segmentation of an agent's planning vs. synthesis phases. - C is Incorrect: Manually timing API calls violates the "zero custom code instrumentation" constraint. - D is Incorrect: AWS X-Ray can trace the overall Lambda execution, but it does not automatically segment the internal Bedrock agent orchestration phases (planning vs. synthesis) without custom sub-segment instrumentation. > Key Takeaway: AgentCore Observability automatically segments agent invocations into planning, tool, and synthesis phases, providing per-phase latency without custom code.
### Question 9 - Evaluating FM Inference Cost Implications An AI engineering team is evaluating the cost implications of using Foundation Models (FMs) for inference in production. They have deployed a Bedrock Agent and noticed that a single user request results in a planning call, three tool calls, and a synthesis call. The team needs to monitor the agent's resource consumption per invocation to understand the total token cost generated by a single user request. Which AWS service is purpose-built to capture this specific agent invocation data? Which AWS service is purpose-built for this task? (Select ONE) - A) AWS Compute Optimizer - B) AWS Trusted Advisor - C) Amazon Bedrock AgentCore Observability - D) Amazon CloudWatch Container Insights	Correct Answer: C Correct Answer Analysis: - C is Correct: Amazon Bedrock AgentCore Observability is specifically designed to capture the internal operations of an agent invocation, including inference calls, tool invocations, and memory operations. It provides the per-invocation token data needed to evaluate the total cost of a single user request that triggers multiple internal calls. Distractor Analysis: - A is Incorrect: AWS Compute Optimizer analyzes utilization history to recommend compute resource configurations, not AI agent token consumption patterns. - B is Incorrect: AWS Trusted Advisor checks accounts against AWS best practices (like idle resources or security gaps), not agent invocation costs. - D is Incorrect: CloudWatch Container Insights monitors infrastructure-level metrics for containerized workloads, not application-level token consumption for AI agents. > Key Takeaway: To evaluate the cost of a single user request that triggers multiple internal FM calls, use AgentCore Observability to capture per-invocation token data.
### Question 10 - Monitoring Context Size in Agent Retrieval A company is building a generative AI application with an agent that retrieves data from an Amazon OpenSearch Serverless vector index before generating a response. The team needs to monitor the agent's resource consumption to ensure that the retrieval tool is not pulling oversized contexts that inflate token usage. They want to view the exact token count consumed by the retrieval step versus the generation step. Which approach satisfies this requirement with the least operational overhead? Which approach should the team use? (Select ONE) - A) Use Amazon Bedrock AgentCore Observability to inspect the per-invocation trace and view token consumption for each tool and model call. - B) Enable CloudWatch Logs for OpenSearch Serverless and manually count the returned documents. - C) Configure an AWS Budgets alert on the Bedrock service usage. - D) Use AWS X-Ray to trace the OpenSearch query latency, which indirectly correlates with token size.	Correct Answer: A Correct Answer Analysis: - A is Correct: AgentCore Observability provides a per-invocation trace that shows the token consumption associated with each step of the agent's execution. This allows the team to directly see how many tokens the retrieval tool's output (the context) added to the final synthesis model call, identifying oversized contexts. Distractor Analysis: - B is Incorrect: Manually counting returned documents in CloudWatch Logs is a high-operational-overhead approach and does not directly translate to token counts (which vary by document content). - C is Incorrect: AWS Budgets tracks aggregate spend but does not provide the per-invocation, per-step visibility needed to identify oversized retrieval contexts. - D is Incorrect: X-Ray query latency correlates with query complexity, not necessarily with the token size of the returned documents, and does not measure token consumption. > Key Takeaway: To identify if an agent's retrieval tool is inflating token costs, use AgentCore Observability to view the token consumption attributed to the tool's context output.