Skip to content

Skill 3.2.9: Implement Agent State Management Systems

Task 3.2: Deployment and Orchestration of ML and AI Workflows – Provision and configure resources for ML and AI workloads

Core Objective: Persist and retrieve agent context across turns and sessions while enforcing isolation, compliance, and cost control.

Agent State Taxonomy

State Type Scope AWS Mechanism Primary Use Retention
Short-term memory Single session AgentCore Memory short-term; session attributes Multi-turn dialogue, follow-ups Session lifetime
Long-term memory Cross-session, per actor AgentCore Memory long-term; namespaces Personalization, preferences, history Configurable (days)
Execution state Single session runtime AgentCore Runtime session isolation Tool call state, intermediate results Session lifetime
Tool/identity state Per invocation AgentCore Identity, Gateway Auth tokens, tool access Per call/session

AWS Service Map

Service Role in Agent State Key Config
Amazon Bedrock AgentCore Memory Managed memory store Short-term & long-term strategies, namespaces (actorId, sessionId), extraction/consolidation
Amazon Bedrock AgentCore Runtime Isolated execution per session Session isolation, scaling, VPC endpoints
Amazon Bedrock AgentCore Gateway Expose APIs/functions as tools Protocol translation, auth
Amazon Bedrock AgentCore Identity Agent identity & permissions IAM roles, OAuth, token vending
Amazon Bedrock Agents (legacy) Session attributes sessionId, memory retention
Amazon DynamoDB / ElastiCache Custom state store Table design, TTL, encryption
AWS KMS Encryption at rest CMK for memory stores
Amazon VPC / PrivateLink Private access VPC endpoints for AgentCore, S3, DynamoDB

Key Parameters (Conceptual)

Parameter Purpose Scope
sessionId Identifies a single conversation Short-term memory
actorId Identifies the user/entity Long-term memory
namespace Partitions long-term memory e.g., /preferences/{actorId}/
retentionDays TTL for memory Long-term memory
Memory strategies Extraction, summarization, consolidation Long-term memory

Implementation Patterns

  • Separate memory tiers: Short-term for dialogue; long-term for cross-session facts. Never store long-term data in session-only memory.
  • Namespace by actor: Long-term memory must be scoped by actorId. Use namespaces like /preferences/{actorId}/.
  • Enable extraction & consolidation: Configure strategies to extract facts, summaries, preferences automatically. Consolidation merges redundant memories.
  • Session isolation: Runtime provides separate execution environments per session. Never share working state between sessions.
  • Retention policies: Set TTL/retention days per memory type. Long-term memory must support per-actor deletion for compliance.
  • Observability: Enable traces and step logs. Agent failures often manifest as unexpected tool returns; trace shows which step.
  • Tool exposure via Gateway: Don't rebuild services; expose existing APIs as tools. Gateway handles protocol and auth.
  • Identity: Use AgentCore Identity to vend short-lived credentials for tool calls. Avoid hardcoding secrets.

Security & Compliance

  • IAM roles: AgentCore Runtime assumes a role with least privilege for memory, tools, and model access.
  • Encryption: KMS CMK for memory at rest; TLS for in transit.
  • VPC endpoints: Use AWS PrivateLink for AgentCore, S3, DynamoDB, etc., to keep traffic private.
  • Session isolation: Enforced by runtime; memory namespaces prevent cross-user leakage.
  • Data retention: Configure retention to meet compliance (e.g., GDPR right to erasure). Long-term memory must be deletable per actor.
  • Audit: CloudTrail for API calls; AgentCore observability for agent steps.

Scaling & Performance

  • Concurrency: AgentCore Runtime scales automatically per session. Memory store scales with requests.
  • Latency: Long-term memory retrieval adds latency. Use filters and top-k to limit.
  • Cost: Long-term memory storage and retrieval costs. Use consolidation to reduce size.
  • GPU scaling: If agent uses GPU models, ensure inference endpoint scales with high-resolution concurrency metrics (e.g., ConcurrentRequestsPerModel).

Trade-offs & Anti-Patterns

Decision Option A Option B When to Use
Memory store AgentCore Memory (managed) Custom DynamoDB/ElastiCache Managed for speed; custom for full control/legacy
Memory scope Short-term only Short-term + Long-term Short-term for stateless Q&A; long-term for personalization
Isolation Session isolation Shared memory Always isolate; share only via long-term namespaces
Retention Short TTL Long TTL Short for sensitive data; long for preferences
Retrieval Semantic search Hybrid search Hybrid for exact IDs + semantic
Tool exposure Gateway Direct integration Gateway for reusability and auth

Anti-patterns:

  • Storing long-term memory in session store: Lost on session end.
  • Global namespace: All users share memory → data leak.
  • No retention policy: Unbounded growth and compliance risk.
  • Ignoring observability: Hard to debug tool call failures.
  • Hardcoding credentials: Use AgentCore Identity.
  • Over-retrieving memory: Too many results increases latency and cost.
  • Mixing execution state with memory: Execution state is ephemeral; don't persist as long-term.

Exam Gotchas - Session isolation is runtime-enforced, not memory-store-enforced. Separate sessions get separate execution environments; memory namespaces prevent cross-session data access. - Long-term memory must be scoped by actorId. Using a global namespace leaks context between users. - Short-term memory is not persistent across sessions. Returning users need long-term memory. - AgentCore Memory is not the same as SageMaker endpoint memory. Don't confuse. - Index dimensions must match embedding model, and field mapping must be declared before ingestion (if retrieval is part of agent state). - Use VPC endpoints for private access. Serverless SageMaker endpoints do not accept VPC configuration; AgentCore uses PrivateLink.