Skill 3.2.2: Automate compute resource provisioning with integrated communication between stacks and orchestration services
Task 3.2: Deployment and Orchestration of ML and AI Workflows – Provision and configure resources for ML and AI workloads based on existing architecture and requirements
Core objective: Turn an ML/AI hosting decision into repeatable resources: compute, containers, network, scaling, and generative-AI stores. Automate with IaC; connect stacks without hardcoding; orchestrate deployment order, runtime workflows, and failure handling.
Automation Stack & Exam Roles
| Service | Primary role | Exam differentiator |
|---|---|---|
| AWS CloudFormation | Declarative IaC; provisions stacks | Repeatable environments, drift detection, change sets, rollback, nested stacks, StackSets |
| AWS CDK | Imperative IaC; synthesizes CloudFormation | Use for developer-friendly constructs; still deploys CloudFormation stacks |
| SageMaker AI | Hosts models; holds container, instance, network, scaling config | Real-time endpoints, serverless inference, batch transform, HyperPod |
| Amazon ECR | Stores/versions container images | Endpoint/training pulls image by URI; use immutable tags/digests, not latest |
| Application Auto Scaling | Registers scalable target and applies scaling policy | SageMaker endpoint variants/inference components; target tracking, step, scheduled |
| Step Functions | Orchestrates multi-step ML workflows | Service integrations, retries, catch, wait-for-callback; coordinates training → deploy → update |
| EventBridge | Event-driven triggers and schedules | Decouples stacks/services; reacts to model/endpoint events; cross-account buses |
| CodePipeline / CodeBuild | CI/CD for ML, containers, IaC | Builds images, runs tests, deploys CloudFormation/CDK stacks |
| SSM Parameter Store / Secrets Manager | Externalized config and secrets | Avoid hardcoding subnet IDs, image tags, endpoints; dynamic references |
| Amazon VPC | Private network for endpoints and jobs | Subnets, security groups, VPC endpoints/PrivateLink |
| Bedrock Knowledge Bases | Managed ingestion + retrieval pipeline | Vector store choice, index dimensions, field mapping, resync, hybrid search |
| Bedrock AgentCore | Agent runtime, memory, gateway, identity | Session isolation, short/long-term memory, tool access, observability |
Cross-Stack Communication & Orchestration Patterns
| Pattern | Use when | Key trade-off / trap |
|---|---|---|
| CloudFormation exported outputs + imports | One stack owns a resource (e.g., VPC subnets); another consumes it | Decouples teams; cannot delete exporting stack while imported; export names unique per account/region |
| Nested stacks | Same owner, related resources, reusable components | Tight coupling; parent manages lifecycle; good for modular templates |
| CDK cross-stack references | CDK apps with multiple stacks | Passing props creates deployment dependency; avoid if stacks must deploy independently |
| SSM Parameter Store | Runtime lookup or loose coupling across accounts/services | Values can change without stack update; IAM permissions required; not a hard CloudFormation dependency |
| StackSets | Multi-account, multi-region standard deployment | Centralized admin; target OUs/accounts; use for guardrails and shared ML infrastructure |
| Service Catalog | Governed self-service provisioning | Portfolio/products constrain choices; good for approved ML environments |
| Step Functions | Long-running orchestration with retries/error handling | Coordinates Lambda, SageMaker, ECS, Glue, etc.; supports callback pattern |
| EventBridge | Event-driven, loosely coupled workflows | React to state changes; schedules; cross-account event buses |
| CodePipeline + CloudFormation/CDK | CI/CD for ML infrastructure | Deploys stacks; integrates change sets; manual approval gates |
| SageMaker Pipelines | ML-specific workflow orchestration | CI/CD for ML; tracks lineage; not a general infrastructure orchestrator |
Exam Gotcha: Cross-stack references create hard dependencies. If the network stack updates an exported value, importing stacks may need update. Use SSM Parameter Store for loose coupling.
Exam Gotcha: CDK synthesizes CloudFormation. It is not a separate provisioning engine. Any CDK question ultimately concerns CloudFormation stacks, roles, and resources.
Compute Provisioning Decision Matrix
| Requirement | Choose | Why / Trap |
|---|---|---|
| Private network, no internet | Real-time endpoint with VPC config | Needs subnets in multiple AZs, security groups, VPC endpoints for S3/ECR/Logs/KMS |
| Private network + serverless | Not supported | Serverless inference does not take VPC config; rules out private networking |
| Intermittent/low traffic | Serverless inference or on-demand | Cost-efficient; cold starts; no VPC config |
| Steady predictable traffic | Provisioned capacity / reserved instances | Lower cost; requires forecast |
| Bursty/streaming LLM inference | Target tracking on high-resolution concurrency metric | Standard invocation metrics aggregate ~1 min; concurrency publishes sub-minute |
| Cost driver not request count | Custom resource-utilization metric | Must test metric correlates with capacity |
| Scheduled known peaks | Scheduled scaling | Combine with target tracking for baseline + burst |
| Large-scale distributed training | SageMaker HyperPod | Resilient accelerator clusters; workload manager or Kubernetes; auto-replace faulty hardware |
| Batch scoring | Batch transform | Reads/writes S3; not real-time; can run in VPC |
| Custom inference logic | Bring Your Own Container (BYOC) | Must implement serving contract: health, invoke, model load; store in ECR |
| Framework standard | Prebuilt or extended framework container | Faster; less control; extended adds dependencies |
Exam Gotcha: SageMaker endpoint VPC requires subnets in at least two Availability Zones. Execution role needs permissions to create, describe, and delete network interfaces.
Exam Gotcha: In private subnets, S3 access needs an S3 Gateway VPC endpoint. ECR pulls need interface endpoints for ECR API and Docker, plus S3 Gateway. CloudWatch Logs needs an interface endpoint.
Security & Compliance Checklist
| Area | Requirement | Exam note |
|---|---|---|
| IAM execution role | Endpoint/training job assumes role | Must allow ECR pull, S3 model artifact read, CloudWatch Logs write, VPC ENI management, KMS decrypt |
| KMS encryption at rest | S3, ECR, EBS, CloudWatch Logs, Secrets Manager | Use customer managed keys for compliance; grant role decrypt permission |
| Encryption in transit | TLS for API calls and data movement | VPC endpoints use TLS; PrivateLink keeps traffic off internet |
| VPC endpoints | S3 Gateway, ECR Interface, CloudWatch Logs, SageMaker API, Secrets Manager, KMS | Gateway endpoints for S3/DynamoDB; interface endpoints for most others |
| Network isolation | Enable on training/inference containers | Prevents container outbound calls; requires VPC endpoints for dependencies |
| Cross-account access | Resource policies + IAM roles + PrivateLink | Avoid public endpoints; use VPC endpoint services |
| Secrets | Secrets Manager or SSM Parameter Store | Never hardcode credentials in templates or containers |
Exam Gotcha: Network isolation + private subnets = container cannot reach internet. All dependencies must be reachable via VPC endpoints or NAT. If no NAT and no endpoint, artifact download fails.
Knowledge Bases & Agentic Infrastructure Provisioning
| Component | Provisioning decision | Trap |
|---|---|---|
| Vector store | OpenSearch (serverless/managed), Aurora, Neptune, S3 Vectors | OpenSearch supports hybrid search + filterable text field; dimensions must match embedding model |
| Index mapping | Declare vector, text chunk, metadata fields before ingestion | Cannot change dimensions/field mapping without reindexing |
| Ingestion job | Initial sync + resync | Resync processes only changed documents since last sync |
| Retrieval pipeline | Combine search type, top-k, metadata filters, reranking | Metadata filters narrow candidates before similarity; reranking improves precision |
| Agent runtime | AgentCore runtime | Isolates each session; separate execution environments |
| Short-term memory | Conversation within a session | Lost when session ends |
| Long-term memory | Persists across sessions; can be shared between agents | Requires explicit retention/privacy design; scoped to actor/user |
| Gateway | Exposes APIs/functions as tools | Agent uses existing services instead of rebuilding |
| Observability | Trace each agent step/tool call | Failures often show as unexpected tool return; trace identifies which call |
Exam Gotcha: If returning users must not repeat context, choose long-term memory. Session isolation is a security boundary, not persistence.
Anti-Patterns & Distractor Traps
- Hardcoding subnet IDs, security groups, image tags → Use CloudFormation exports/imports, CDK props, or SSM Parameter Store.
- Manual console changes after IaC deployment → Drift; use CloudFormation drift detection, change sets, and CI/CD.
- Using
latestcontainer tag → Non-reproducible; use immutable tags or image digests. - Choosing serverless inference for private networking → Serverless does not support VPC config; use real-time endpoint.
- Scaling on average invocations for streaming LLM → Too slow; use high-resolution concurrency target tracking.
- Ignoring cooldown → Flapping; scale-in cooldown should be longer than scale-out.
- Changing embedding model without reindexing → Dimension mismatch; rebuild index.
- Treating agent memory as one thing → Separate short-term vs long-term; design retention.
- Using Step Functions for simple event routing → EventBridge is lighter; Step Functions for long-running orchestration.
- Using CloudFormation for runtime orchestration → CloudFormation provisions; Step Functions orchestrates runtime workflows.
- Forgetting execution role ENI permissions → VPC endpoint deployment fails to create network interfaces.
- Forgetting S3 Gateway endpoint in private subnets → Endpoint cannot pull model artifacts.
Quick Decision Flow
- Private networking required? → Real-time endpoint + VPC config + VPC endpoints. Not serverless.
- Need repeatable environment? → CloudFormation or CDK. Cross-stack via exports/imports or SSM.
- Need runtime orchestration? → Step Functions (long-running, retries) or EventBridge (event-driven).
- Need scaling? → Application Auto Scaling. Metric choice: invocations (general), concurrency (fast/streaming), custom (cost driver), scheduled (known peaks).
- Need container? → Prebuilt/extended/BYOC. Store in ECR. Immutable tags.
- Need retrieval? → Bedrock Knowledge Bases. Vector store choice + dimensions + mapping + resync.
- Need agent? → AgentCore runtime + memory + gateway + observability. Session isolation by runtime.