Skip to content

Skill 3.2.11: Deploy agentic workflow infrastructure

Task 3.2: Deployment and Orchestration of ML and AI Workflows - Provision and configure resources for ML and AI workloads based on existing architecture and requirements.

Core Objective & Architecture

  • Agentic workflow infrastructure = production runtime + state + tools + identity + observability. Not just a model endpoint.
  • Primary AWS service: Amazon Bedrock AgentCore. Managed components replace hand-built agent infrastructure.
  • Existing architecture constraint: integrate with current VPC, IAM, IdP, API, Lambda, knowledge bases, and model hosting.
AgentCore Component Job Key Exam Focus
Runtime Serverless, session-isolated execution for agent code Session isolation, scaling, container image, IAM role, network mode
Memory Short-term & long-term state STM vs LTM, namespaces, retention, encryption
Gateway Exposes APIs/Lambda/functions as agent tools MCP/OpenAPI, auth, tool discovery, avoid rebuilding services
Identity Agent identity for outbound calls Workload identity, token exchange, least privilege
Observability Traces, metrics, logs for each step Tool-call tracing, CloudWatch integration, debugging
Browser / Code Interpreter Managed tools for web/code tasks Isolated execution, no infra to manage

Runtime Provisioning

  • Runtime is serverless: no instances to provision; scales automatically with sessions.
  • Session isolation: each session gets separate execution environment. One conversation cannot read another’s working state.
  • Container image: deploy agent code as container. Store in Amazon ECR, version tags, pull via execution role.
  • IAM execution role: grants runtime access to models, memory, gateway, logs, secrets, VPC resources.
  • Network mode: public or VPC-attached. For private access, configure VPC subnets, security groups, and VPC endpoints for dependencies.
  • Session timeout: controls how long a session lives. Align with business conversation length and cost.
  • Async / long-running: AgentCore Runtime supports long-running agent tasks; do not force into Lambda 15-minute limits.

Runtime enforces session isolation, not the memory store. Never rely on memory namespaces alone for tenant security.

Memory Management

Memory Type Scope Survives Session? Use Case
Short-term Current conversation / session No Multi-turn follow-ups, working context
Long-term Actor / user / cross-session Yes Returning users, preferences, historical facts
  • Short-term memory is scoped by sessionId. Cleared when session ends or expires.
  • Long-term memory is namespaced by actor/user. Extracts and consolidates facts, summaries, preferences.
  • Memory strategies: semantic, summary, user preference. Choose based on what must persist.
  • Retention & privacy: LTM is an explicit design decision. Set retention, encryption, and access controls.
  • KMS encryption: encrypt memory stores and session data at rest.
  • Anti-pattern: using STM for cross-session context → users repeat themselves.
  • Anti-pattern: storing PII in LTM without retention policy → compliance risk.

STM = conversation; LTM = relationship. If returning users must avoid repeating context, configure LTM.

Gateway & Tool Integration

  • AgentCore Gateway exposes existing services as tools the agent already speaks (MCP/OpenAPI).
  • Targets: Lambda functions, REST APIs, AWS services, internal microservices.
  • Auth: Gateway handles outbound auth to targets. Use least-privilege IAM roles per target.
  • Tool discovery: agents can discover available tools dynamically via Gateway.
  • Do not rebuild APIs as agent code. Gateway is the integration layer.
  • Schema: define input/output schemas so the model can call tools correctly.
  • Anti-pattern: hardcoding API calls inside agent code → brittle, no centralized auth, no observability.

Identity & Security

  • AgentCore Identity: provides agent workload identity for outbound calls. Separate from user identity.
  • Token exchange: agent obtains scoped credentials to call tools on behalf of user or system.
  • IAM roles:
  • Runtime execution role → access to models, memory, logs, VPC.
  • Gateway role → invoke Lambda/API targets.
  • Identity role → outbound auth to external services.
  • KMS: encrypt memory, session data, logs, and secrets.
  • VPC: private subnets + security groups + VPC endpoints for Bedrock, S3, ECR, Secrets Manager, KMS, CloudWatch.
  • Session isolation: runtime-level. Do not share session IDs across users.
  • Least privilege: one identity per agent; no shared admin role across tools.

Identity is for agent-to-service auth, not user auth. User authentication belongs to your IdP/Cognito layer.

Observability

  • AgentCore Observability: records each step, tool call, input, output, and error.
  • CloudWatch integration: traces, metrics, logs for GenAI workloads.
  • Why it matters: agent failures are usually a tool call returning unexpected data. Without traces, undiagnosable.
  • Key metrics: session count, latency, tool error rate, memory hits, token usage.
  • Enable tracing in production. Do not rely on ad-hoc logging.
  • Alarms: on tool errors, latency spikes, memory failures, session timeouts.

Scaling & Capacity

  • AgentCore Runtime: automatic scaling. No manual instance provisioning.
  • Concurrency: scales per session. Check service quotas for concurrent sessions.
  • Model hosting: if agent uses custom models, host on SageMaker AI endpoints. Scale those separately via Application Auto Scaling.
  • GPU workloads: provision SageMaker GPU instances or use Bedrock models. AgentCore runtime itself is not GPU-provisioned.
  • Memory store: managed by AgentCore. For vector-heavy LTM, integrate with OpenSearch or other vector stores via Knowledge Bases.
  • Quotas: session duration, memory retention, gateway targets, concurrent sessions. Verify before production.

Deployment Patterns & Trade-offs

Need Best Fit Why
Simple agent with built-in orchestration, action groups, KB Amazon Bedrock Agents Fast, integrated, less control
Custom agent framework, session isolation, memory, gateway, identity Amazon Bedrock AgentCore Production agent infrastructure, managed components
Full control over runtime, networking, scaling ECS/EKS/Lambda + custom Max control, high ops burden
Private networking required AgentCore Runtime in VPC + VPC endpoints Keeps traffic on AWS private backbone
  • AgentCore vs Bedrock Agents: AgentCore for custom frameworks and production agent infra. Bedrock Agents for simpler, managed orchestration.
  • Serverless vs self-managed: AgentCore removes session isolation, memory, gateway, identity, and observability burden. Self-managed requires building all of it.
  • Anti-pattern: deploying agents on Lambda without session isolation → cross-session state leakage.
  • Anti-pattern: using a single shared IAM role for all tools → privilege escalation risk.

Key Config Checklist

  • Runtime: container image, IAM role, session timeout, network mode, environment variables.
  • Memory: STM retention, LTM strategies, namespaces, KMS encryption.
  • Gateway: tool targets, schemas, auth, MCP/OpenAPI protocol.
  • Identity: agent workload identity, outbound auth, token exchange, least privilege.
  • Observability: tracing enabled, CloudWatch log groups, metrics, alarms.
  • Network: VPC subnets, security groups, VPC endpoints for S3/ECR/Bedrock/KMS/Secrets Manager/CloudWatch.
  • Scaling: runtime auto-scales; scale SageMaker model endpoints separately if used.

Critical gotcha: AgentCore Runtime is serverless — you do not provision instances. If the exam asks about provisioning compute for agentic workflows, the answer is usually “use AgentCore Runtime” or “scale the underlying model endpoint separately.”