Skip to content

Skill 4.1.5: Monitor and automate the management of agent performance and coordination

An AI engineering team operates a multi-agent orchestration system built on Amazon Bedrock AgentCore, where a coordinator agent distributes tasks to specialized sub-agents for insurance claims processing. The system uses OpenTelemetry-compatible instrumentation and is deployed with Amazon CloudWatch generative AI observability enabled.

The team has recently experienced intermittent coordination failures between the coordinator agent and sub-agents, resulting in incomplete workflows where claims are orphaned without a final disposition. The compliance team requires that every claim must be resolved within 24 hours or automatically escalated. The operations team needs to detect these coordination failures in real time without manually inspecting logs.

Which approach should the engineering team implement to meet these requirements? (Select ONE)

A) Configure AWS CloudTrail to monitor Bedrock AgentCore API calls and set resource-based policies to trigger alerts when a coordination session exceeds 24 hours.

B) Use Amazon CloudWatch generative AI observability to monitor AgentCore trace spans for missing expected downstream tool invocations, then create a CloudWatch alarm that triggers an AWS Step Functions escalation workflow when a session lacks a completion signal within 24 hours.

C) Deploy a separate Amazon SageMaker endpoint that periodically re-scores all in-flight claims using a classification model, then route to human review if the confidence score falls below a threshold.

D) Configure Amazon Bedrock Guardrails to block any agent conversation that does not include a valid claim resolution statement within 24 hours.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

  • B is Correct: Amazon CloudWatch generative AI observability provides built-in trace visualization for Bedrock AgentCore agents, capturing model inference calls, memory operations, and tool invocations as distributed spans. By instrumenting the agent system to emit a distinct span when a claim is completed (or a coordinating handoff succeeds), the team can detect the absence of that span within a time window—a signal for coordination failure. A CloudWatch alarm on the metric or trace absence then triggers Step Functions for escalation, meeting both real-time detection and 24-hour resolution compliance.

Distractor Analysis

  • A is Incorrect: AWS CloudTrail records API activity for audit purposes, but it does not provide the end-to-end execution context (span lifecycle, tool call chains, session state) needed to detect coordination failures in agent orchestration. It also cannot trigger an escalation workflow directly on an "incomplete" state.
  • C is Incorrect: A separate SageMaker classification endpoint introduces unnecessary operational overhead (training, hosting, retraining, two-model consistency) when AgentCore's built-in span tracing already provides the necessary telemetry. It addresses a symptom, not root cause visibility.
  • D is Incorrect: Amazon Bedrock Guardrails enforces content safety (filtering, PII redaction, topic denial) on model input/output—it does not have session-state or coordination-awareness to enforce workflow completion timelines.

Key Takeaway: For agent orchestration failures like orphaned workflows or missed handoffs, leverage CloudWatch generative AI observability's trace-span coverage rather than retrofitting a separate ML model or relying on audit logs.

Question 2 - Truncated Streaming Detection

A financial services company deployed a streaming RAG-based assistant using Amazon Bedrock with response streaming (server-sent events) enabled for low-latency token delivery to traders. Users occasionally experience responses that stop mid-sentence without an explicit error, especially during high market volatility when input contexts are large.

The engineering team wants to automatically detect truncated streams and surface a bilingual (English/Japanese) warning in the UI. They need a solution that captures the streaming event lifecycle and distinguishes truncation from latency spikes or throttling.

Which monitoring configuration should the team implement? (Select TWO)

A) Enable CloudWatch generative AI observability and create a metric filter that flags stream completions where the final SSE event lacks a [DONE] sentinel with a non-empty finish_reason.

B) Configure AWS X-Ray tracing on the Bedrock streaming invocation and set a custom sampling rule that captures partial response segments, then create an alarm when stop_reason is missing from the final trace segment.

C) Use Amazon OpenSearch Ingestion to buffer stream events and apply an anomaly detection model to predict truncation probability based on token velocity.

D) Modify the client to track bytes received versus expected content length and publish a custom CloudWatch metric when the ratio falls below a threshold.

E) Create a CloudWatch Synthetics canary that replays recorded trader queries and asserts that every streaming response ends with an end-of-stream marker.

Click to reveal Answer & Explanation

Correct Answers: A and D

Correct Options Analysis

  • A is Correct: CloudWatch generative AI observability captures model invocation event metadata, including streaming lifecycle events. Filtering on stream endings without a terminal sentinel (e.g., [DONE] or explicit finish_reason=stop) allows precise truncation detection.
  • D is Correct: Client-side tracking of received bytes vs. expected content length (from HTTP headers) provides a direct measurement of stream completion. Publishing a custom CloudWatch metric enables threshold alarms and UI warnings without depending solely on server-side logs.

Distractor Analysis

  • B is Incorrect: AWS X-Ray traces request-level latency and service calls, but it does not capture streaming event granularity (individual SSE chunks or model completion signals). Setting a custom sampling rule does not make X-Ray capable of detecting finish_reason gaps.
  • C is Incorrect: OpenSearch Ingestion buffers and preprocesses data, but building an anomaly detection model from "token velocity" to predict truncation is speculative, resource-intensive, and fails to detect the actual terminal event. It's overengineered for a deterministic signal (missing end-of-stream marker).
  • E is Incorrect: CloudWatch Synthetics canaries are for synthetic user journeys (pinging endpoints, validating HTML responses), not for validating production streaming event protocols. They cannot detect truncation in live user traffic.

Key Takeaway: Stream truncation is a deterministic signal (absence of a terminal marker/ratio mismatch), not a probabilistic inference problem—prefer exact detection via CloudWatch generative AI observability or client-side telemetry over ML-based anomaly models.

Question 3 - Tool Failure Automation

A healthcare AI team runs an Amazon Bedrock AgentCore agent that assists clinicians by calling three tools: a medication lookup tool (via AWS Lambda), a drug interaction checker (via Amazon API Gateway), and a lab results retriever (via AWS Lambda). The agent must handle tool failures gracefully.

Recently, the drug interaction checker began returning intermittent HTTP 503 errors during peak hours. When this happens, the agent sometimes retries indefinitely, causing session timeouts and bad patient experience. The team wants to automate failure handling with bounded retries and fallback to a cached interaction database.

Which combination should the team configure? (Select ONE)

A) Create a CloudWatch alarm on API Gateway 5XX error rate that triggers an AWS Lambda function to modify AgentCore agent configuration (remove the tool, add fallback tool). Then rely on AgentCore's default retry policy.

B) Use AWS Step Functions to wrap the tool invocation and implement an exponential backoff retry with a maximum of 3 attempts, then a fallback task that queries an Amazon S3-backed cache; wire the Step Functions state machine into the Bedrock agent as a single tool.

C) Enable Amazon Bedrock AgentCore Action Groups and configure an action group-level retry policy with backoff and fallback actions, then attach the fallback database as an alternative action group.

D) Deploy the drug interaction checker on Amazon EKS with auto-scaling so it never returns 503 errors, and configure the agent to not retry on client errors.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

  • B is Correct: Wrapping the tool's invocation logic in Step Functions gives explicit control over retry behavior (bounded attempts, backoff), fallback branching, and state persistence. This directly addresses the problem of "indefinite retries" and provides a deterministic fallback path to a cached database. Bedrock agents can call Step Functions synchronously when the state machine is exposed as a tool/API.

Distractor Analysis

  • A is Incorrect: Relying on AgentCore's default retry policy after an alarm-triggered Lambda disrupts the live agent mid-session and introduces configuration-change latency. Alarms are for observability, not as the primary control plane for tool routing.
  • C is Incorrect: Amazon Bedrock AgentCore Action Groups allow defining parameter schemas and OpenAPI specifications for tools, but they do NOT provide explicit bounded retry-with-backoff controls or fallback tool routing at the action group level. This is a non-existent feature for granular retry semantics.
  • D is Incorrect: Auto-scaling on EKS reduces errors under load, but it does not handle all failure modes (e.g., dependency outages, rate limiting by upstream providers). Relying solely on scaling and disabling retries sacrifices resilience and violates the requirement to handle tool failures gracefully.

Key Takeaway: For bounded retries and fallback semantics in agent tool calls, orchestrate them explicitly (Step Functions or equivalent) rather than relying on agent runtime defaults or external scaling.

Question 4 - Coordination Failure Detection

A retail company uses a multi-agent system for order fulfillment, where a supervisor agent coordinates three sub-agents (inventory checker, payment processor, shipping labeler) across Amazon Bedrock AgentCore sessions. Transactions require all three sub-agents to complete their tasks. The system processes ~50,000 orders daily.

The team observed that occasionally the supervisor agent prematurely ends a session after only 2 of 3 sub-agents respond, leading to incomplete orders. They need to detect these "premature session closures" in near real-time and auto-trigger reprocessing.

Which metric and automation strategy should they use? (Select ONE)

A) Configure CloudWatch generative AI observability to track the count of unique sub-agent spans per session, then create a CloudWatch anomaly detection alarm that triggers a Step Functions reprocessing workflow when a session's span count is below 3.

B) Use Amazon Bedrock Guardrails to enforce a minimum of 3 tool calls per session via sensitive information policy, which rejects sessions that don't meet the threshold.

C) Enable AWS Trusted Advisor recommendations for underutilized Bedrock agent sessions and trigger manual review for flagged sessions.

D) Schedule an hourly Amazon Athena query over AgentCore execution logs stored in S3 to identify sessions with fewer than 3 sub-agent responses, then manually reprocess them.

Click to reveal Answer & Explanation

Correct Answer: A

Correct Option Analysis

  • A is Correct: CloudWatch generative AI observability traces each AgentCore session as a distributed transaction, capturing individual sub-agent invocations as spans. By counting unique sub-agent spans per session, the team can define a threshold (e.g., alert when count < 3) and use CloudWatch anomaly detection (or a simple static alarm) to trigger Step Functions for reprocessing. This meets the "near real-time" and automation requirements.

Distractor Analysis

  • B is Incorrect: Amazon Bedrock Guardrails applies content safety and sensitive information filtering on user/assistant text, not on structural counts of tool invocations per session. There is no "minimum number of tool calls" feature in Guardrails.
  • C is Incorrect: AWS Trusted Advisor provides best-practice recommendations for cost, performance, and security—it does not monitor runtime agent execution metrics or detect incomplete transaction workflows.
  • D is Incorrect: Athena-on-S3 (query over AgentCore logs) is correct for post-hoc forensics but introduces an hour-long latency and manual reprocessing step, violating the near real-time and automation constraints. This is a higher-operational-overhead pattern than CloudWatch's native tracing.

Key Takeaway: For agent session integrity, use CloudWatch generative AI observability's span-level telemetry to detect structural anomalies (e.g., missing sub-agent calls) in near real-time—not post-hoc queries or content filters.

Question 5 - Agent Performance Baseline and A/B Comparison

An enterprise has two versions of a Bedrock AgentCore agent for customer support: the production agent (v1) uses a single-tool routing strategy, and the candidate agent (v2) uses a multi-tool orchestration strategy with improved coordination logic. Both agents share the same Bedrock foundation model.

The operations team wants to compare the agents' coordination efficiency—specifically, the time-to-first-tool-call and number of tool calls per conversation—over a two-week period using 20% live traffic on v2. They also need to detect if v2's coordination logic causes increased error rates.

Which configuration supports this requirement? (Select ONE)

A) Deploy both agents as separate Bedrock endpoints and manually configure API Gateway routing with a 80/20 split. Compare metrics by querying CloudWatch Logs manually each week.

B) Use SageMaker endpoint with two production variants (v1 and v2), set variant weights 80/20, and use the generated Invocations, ModelLatency, OverheadLatency, and Invocation4XXErrors metrics for A/B comparison.

C) Configure Amazon Bedrock AgentCore agents with distinct agent IDs but use the same InvokeAgent API gateway fronted by Amazon CloudFront; use CloudFront's built-in A/B testing feature to split traffic 80/20 and compare metrics via CloudFront access logs.

D) Deploy both agents via AgentCore Runtime, instrument both with the OpenTelemetry-compatible AgentCore Observability, route 20% of production traffic to v2 via application logic, and use CloudWatch generative AI observability dashboards to compare span metrics (time-to-first-tool-call, tool count, error rate) between agent IDs.

Click to reveal Answer & Explanation

Correct Answer: D

Correct Option Analysis

  • D is Correct: AgentCore agents are not SageMaker endpoints—they are runtime abstractions invoked via the InvokeAgent API. Using application-level routing (e.g., Lambda or Step Functions dispatching 20% to v2) and relying on AgentCore Observability's span metrics provides the correct per-agent comparison without needing SageMaker variant infrastructure. CloudWatch generative AI observability aggregates these spans and allows side-by-side dashboard comparison.

Distractor Analysis

  • A is Incorrect: AgentCore agents are not deployed as "Bedrock endpoints." This option uses non-existent API constructs (Bedrock endpoint routing for agents) and relies on manual log analysis, violating operational efficiency.
  • B is Incorrect: SageMaker endpoints have production variants for model inference, not for AgentCore agent orchestration. Even if the FMs behind the agents were SageMaker-hosted, variant weight metrics would not capture agent-level coordination efficiency (time-to-first-tool-call, tool count).
  • C is Incorrect: CloudFront does not have a native A/B testing feature for APIs; access logs capture request counts but not agent span metrics like tool invocation count or coordination latency. This is a fabricated service feature.

Key Takeaway: Agent coordination metrics (tool call latency, count, error rate) are captured via AgentCore Observability spans, not via model endpoint metrics—use CloudWatch generative AI observability for side-by-side agent version comparison.

Question 6 - Automated Remediation for Tool Failures

A logistics company's Bedrock AgentCore agent calls a third-party weather API (via AWS Lambda) as part of route planning. The API occasionally times out (30% error rate during storm seasons), causing the agent to produce suboptimal routes.

The team wants to implement a self-healing pattern: when the weather API tool returns errors for 5 consecutive invocations within a 10-minute window, the agent should automatically switch to a secondary weather provider tool. They have budget for minimal ongoing operational overhead.

Which solution achieves this? (Select ONE)

A) Create a CloudWatch alarm on the Lambda function's Errors metric with a composite alarm (consecutive breach over 10 minutes), and configure the alarm action to invoke a Step Functions state machine that updates the Bedrock agent's tool configuration to use the secondary provider.

B) In the Lambda function code, add a try-catch block that tracks consecutive failures in a DynamoDB table and returns a special error response to the agent, prompting it to self-modify its own tool routing based on a pre-authored prompt.

C) Deploy Amazon OpenSearch Ingestion to collect Lambda error logs, run a real-time anomaly detector on error patterns, and send an Amazon SNS notification to a human operator who manually reroutes.

D) Configure an AWS Health Dashboard alert for the Lambda service in the region, then subscribe to that alert channel to trigger automatic tool failover via API Gateway.

Click to reveal Answer & Explanation

Correct Answer: A

Correct Option Analysis

  • A is Correct: A CloudWatch alarm on Lambda Errors (with evaluation periods matching 5 consecutive errors within 10 minutes) triggers a Step Functions state machine that toggles the agent's tool configuration to the secondary provider. This is fully automated, low operational overhead, and uses native AWS services for self-healing.

Distractor Analysis

  • B is Incorrect: While creative, this approach tasks the agent with modifying its own routing logic based on a "special error response." It introduces unbounded complexity and depends on prompt engineering rather than deterministic infrastructure state management. Agents should not self-modify tool configurations.
  • C is Incorrect: OpenSearch Ingestion with anomaly detection aimed at notifying a human operator introduces unnecessary operational overhead and latency (human in the loop), violating the minimal-overhead and automated constraints.
  • D is Incorrect: AWS Health Dashboard reports service-level health events (e.g., AWS Lambda regional outage), not custom Lambda function error metrics from third-party API timeouts. This is the wrong telemetry source.

Key Takeaway: For deterministic, low-overhead tool failover, use CloudWatch alarms → Step Functions → tool configuration update; do not rely on agent self-modification or human-in-the-loop processes.

Question 7 - End-to-End Prompt Tracing for Multi-Agent Debugging

A financial compliance AI team runs a multi-agent pipeline where agent A (document classification) hands off to agent B (sanction screening) before producing a final output. Both agents are deployed on Amazon Bedrock AgentCore and use proprietary prompts.

Users report that some outputs contain correct classification but incorrect screening results, leading to compliance violations. The team's manual review shows agent A's output was correct, but they cannot determine whether agent B failed or the handoff lost information. They need end-to-end visibility of the prompt lifecycle across both agents.

Which solution should they implement? (Select ONE)

A) Enable AWS CloudTrail data event logging for AgentCore InvokeAgent calls, then build a custom correlation map by parsing CloudTrail event json for each session.

B) Configure Amazon CloudWatch generative AI observability with X-Ray trace integration enabled for the AgentCore sessions, then inspect the trace map showing spans for agent A, agent B, the handoff event, and any intermediate prompt transformations.

C) Store every agent prompt and response in an Amazon DynamoDB table via a Lambda function triggered by AgentCore events, then query manually to reconstruct the handoff.

D) Use Amazon QuickSight with S3 log exports to visualize the agent interaction pattern, and create a dashboard for the compliance team.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

  • B is Correct: CloudWatch generative AI observability integrates with X-Ray to provide distributed prompt tracing. The trace map visualizes each model invocation, memory write, tool call, and the explicit handoff spans between agents, allowing the team to pinpoint whether agent B received the correct context or whether information was lost in the handoff. This is the built-in, purpose-designed solution.

Distractor Analysis

  • A is Incorrect: CloudTrail captures API call metadata (caller, timestamp, resource) but does NOT capture the content of prompts, handoff context, or reasoning steps. Parsing CloudTrail JSON cannot reveal the data-flow that caused incorrect screening.
  • C is Incorrect: Building a custom prompt store in DynamoDB requires ongoing code maintenance, is prone to schema drift, and still forces manual correlation across agent sessions rather than providing a native trace view.
  • D is Incorrect: QuickSight is a BI dashboard tool that lacks distributed trace semantics (spans, parent-child relationships, timing). It cannot provide end-to-end execution tracing for multi-agent handoffs.

Key Takeaway: For multi-agent debugging, use CloudWatch generative AI observability's X-Ray integrated prompt tracing—it is the only option that natively captures span-level correlation and handoff context across agent boundaries.

Question 8 - Reducing Measurement Noise for Coordination Monitoring

A media streaming company has a Bedrock AgentCore orchestration for content moderation that makes an average of 12,000 tool calls per hour. They want to detect coordination failures (e.g., agent not completing a moderation verdict within 30 seconds of the final tool call) using CloudWatch alarms.

They initially set a CloudWatch alarm on AgentLatency but observe too many false positives during regular peaks. They need to reduce noise while preserving the ability to catch genuine coordination delays.

Which CloudWatch features should they use? (Select TWO)

A) Set the alarm on a high-resolution custom metric (1-second granularity) so that quick coordination failures are caught before they merge into aggregate averages.

B) Configure Amazon CloudWatch Anomaly Detection to learn the expected latency band from historical data and trigger only on deviations outside the band.

C) Use the p95 or p99 percentile statistic on AgentLatency instead of the Average statistic, reducing sensitivity to regular trace outliers while still flagging extreme coordination delays.

D) Create a composite alarm that combines the latency threshold with a minimum invocation count condition, ensuring the alarm only fires during active traffic.

E) Publish a zero-valued custom metric every hour to ensure CloudWatch doesn't treat the absence of data as an alarm condition.

Click to reveal Answer & Explanation

Correct Answers: B and D

Correct Options Analysis

  • B is Correct: CloudWatch Anomaly Detection uses machine learning on historical metric data to establish a dynamic, adaptive baseline. This reduces false positives from regular daily peaks while flagging genuine outliers that indicate coordination delays.
  • D is Correct: A composite alarm can combine a latency threshold rule with a minimum invocation count rule. This prevents false positives during low-traffic periods where latency outliers are spurious or statistically insignificant.

Distractor Analysis

  • A is Incorrect: High-resolution metrics are for capturing sub-minute data, not for reducing false positives. In fact, 1-second granularity can increase noise by exposing transient fluctuations that are not meaningful coordination failures.
  • C is Incorrect: While p95/p99 on AgentLatency reduces noise compared to Average, it is not the best solution because percentiles alone don't handle time-of-day patterns. Anomaly detection better adapts to regular peaks.
  • E is Incorrect: Publishing a zero-valued metric to prevent "absence of data" alarms is a hack that does not reduce false positives for latency-based coordination failures and can mask genuine outages.

Key Takeaway: Use CloudWatch Anomaly Detection and composite alarm conditions to adapt to regular traffic patterns and separate meaningful coordination delays from expected, higher-latency peaks.

Question 9 - Capturing and Analyzing Tool Failure Patterns

A legal AI company runs a Bedrock AgentCore agent that uses three distinct Lambda functions as tools for case research. The team wants to correlate tool failure rates with specific model invocations to identify whether certain prompt patterns cause the agent to trigger faulty tool calls.

They need a solution that captures per-tool error rates, correlates them with the parent model invocation, and visualizes the failure paths.

Which combination should they implement? (Select TWO)

A) Enable CloudWatch generative AI observability and use the trace view to inspect spans where tool invocation spans are marked with error status, including the parent model invocation span.

B) Use AWS CloudTrail to capture InvokeAgent and InvokeTool API events, then join them on the requestId field in QuickSight.

C) Configure the Lambda tools to emit structured logs with a correlation ID (from the AgentCore session), then use CloudWatch Logs Insights to query and aggregate error rates by tool name and session.

D) Deploy an Amazon Managed Grafana workspace, enable the CloudWatch data source, and build dashboards that filter tool error spans by the agent's session ID and model invocation ID.

E) Enable Amazon Bedrock Guardrails with a sensitive information policy to block tool calls that contain PII, thereby reducing tool error rates.

Click to reveal Answer & Explanation

Correct Answers: A and D

Correct Options Analysis

  • A is Correct: CloudWatch generative AI observability's trace view provides span-level error status on tool invocations, linked to the parent model invocation span. This gives direct correlation between specific prompt invocations and tool failures.
  • D is Correct: Amazon Managed Grafana supports CloudWatch as a data source and enables filtering of trace spans by attributes like session ID and model invocation ID, allowing customized, operational dashboards for failure-path visualization.

Distractor Analysis

  • B is Incorrect: CloudTrail logs API calls but does not capture span relationships (parent-child, session context, error expectations) between model invocations and tool calls. Joining on requestId would not yield meaningful correlation because the field is not a joint correlation key across separate CloudTrail events.
  • C is Incorrect: While structured Lambda logs with correlation IDs are a best practice for deep debugging, they require manual schema design and Logs Insights query management. CloudWatch generative AI observability achieves the same without custom code, making option C a higher-overhead path than necessary.
  • E is Incorrect: Bedrock Guardrails filters content, not tool call behavior. Blocking PII may reduce specific errors, but it does not provide correlation data or visibility into tool failure patterns.

Key Takeaway: For correlating model invocations with tool failures, leverage CloudWatch generative AI observability's native traces or Grafana dashboards—not CloudTrail or content filters.

Question 10 - Proactive Coordination Failure Alerting with Auto-Remediation

A retail AI team operates a Bedrock AgentCore agent that handles product recommendation via four tool calls (user profile lookup, inventory check, pricing lookup, recommendation generation). Coordination failure occurs when the agent skips the "recommendation generation" step, leaving the workflow incomplete.

The team wants a fully automated solution that detects this specific failure mode within 5 minutes and triggers a recovery workflow that re-invokes the missing tool. They want the detection logic to be as simple and deterministic as possible.

Which solution should they choose? (Select ONE)

A) Create a CloudWatch metric filter on AgentCore logs that matches the absence of a "recommendation_generation_completed" event within a session, then use a CloudWatch alarm with a 5-minute evaluation period to trigger a Step Functions workflow that re-invokes the tool.

B) Train a supervised model on historical agent session data to predict the probability of missing the recommendation generation step, then deploy it as a SageMaker endpoint and trigger recovery when probability > 0.9.

C) Configure an Amazon EventBridge rule that listens for the "AgentCore Session Completed" event and checks a DynamoDB table for the count of tool invocations; if count is less than 4, invoke a Lambda function to re-run the session.

D) Implement a canary agent that periodically simulates full user requests and validates the response, then have a CloudWatch Synthetics alarm notify the team to manually re-run failed sessions.

Click to reveal Answer & Explanation

Correct Answer: A

Correct Option Analysis

  • A is Correct: A CloudWatch metric filter matching for a session without the "recommendation_generation_completed" event is deterministic and requires no ML. The alarm's 5-minute evaluation period triggers Step Functions for automated recovery (re-invoking the missing tool). This meets the requirement for simplicity, determinism, and full automation.

Distractor Analysis

  • B is Incorrect: Training a supervised model to predict a deterministic event (presence/absence of a specific tool call) is needless complexity. A simple metric filter is cheaper, faster, more explainable, and easier to maintain.
  • C is Incorrect: While EventBridge + DynamoDB is a valid pattern, it introduces additional state management and potential consistency delays in DynamoDB. The CloudWatch metric filter approach (Option A) is simpler and directly integrates with the agent's session telemetry.
  • D is Incorrect: A canary agent tests synthetic traffic, not production user sessions. It does not catch individual real-user failures and requires manual intervention (notify team), violating the "fully automated" requirement.

Key Takeaway: For known, deterministic agent failure modes (e.g., missing a specific tool call), use metric filters + CloudWatch alarms + Step Functions recovery workflows—not ML models or manual remediation.