Domain 3: Deployment and Orchestration of ML and AI Workflows
3.1 Manage deployment infrastructure for ML and AI model types
| Trigger Keywords & Scenario Constraints | Lock-In Architecture | Immediate Rejection / Trap to Avoid | Explanation |
|---|---|---|---|
| "Millisecond / sub-second latency" + "sustained / predictable traffic" + REST HTTPS endpoint | SageMaker Real-Time Inference endpoint | Serverless Inference (cold start) or Batch Transform (no live endpoint). | Persistent instance-backed REST endpoint; payload typically \(\le 25\text{ MB}\), timeout \(\le 60\text{s}\) (streaming up to \(8\text{ min}\)). |
| "Intermittent / unpredictable / spiky traffic" + "pay only for inference" + no idle cost | SageMaker Serverless Inference | Real-time endpoint (pay for idle instances) or Lambda (no SageMaker model hosting features). | SageMaker manages scale-to-zero; pay for compute used. Payload \(\le 4\text{ MB}\), timeout \(\le 60\text{s}\), no GPU. |
| "Offline / bulk predictions" + "data already in S3" + "no persistent endpoint" | SageMaker Batch Transform | Real-time endpoint (always-on cost) or Async (still an endpoint). | Job-based inference on S3 in/out; GB-scale datasets; runtimes of hours–days; no endpoint to manage. |
| "Large payload (up to 1 GB)" + "long processing (\(\le 1\) hour)" + "queue requests" | SageMaker Asynchronous Inference | Real-time (6–25 MB / 60s cap) or Serverless (4 MB / 60s). | Requests land in SQS-backed queue; results to S3; SNS completion; can scale to 0. |
| "Scale to zero when idle" + near-real-time / queued work (not sub-second) | Async Inference auto scaling MinCapacity=0 | Real-time endpoint (cannot scale to 0) or Serverless (4 MB / no GPU). | Async is the instance-backed option that can scale instance count to zero. |
| "Thousands of similar models" + "same framework" + "infrequent / sparse invocations" + cost | SageMaker Multi-Model Endpoint (MME) | One endpoint per model (cost explosion) or Multi-Container (max 15, all in memory). | Shared container dynamically loads/unloads models from S3; tolerate cold-start on rare models. |
| "Different ML frameworks / containers" + "all models always in memory" + "direct invoke" + \(\le 15\) | SageMaker Multi-Container Endpoint (MCE) | MME (same-framework / dynamic load) or 15 separate endpoints. | Up to 15 containers on one endpoint; no load/unload cold start; invoke a specific container. |
| "Preprocess → model → postprocess" + single low-latency call + chained containers | SageMaker Serial Inference Pipeline | Client-side chaining of endpoints (extra hops/latency) or Step Functions per request. | Containers invoked in order on one endpoint; typical Feature Store / sklearn preprocess + XGBoost/PyTorch. |
| "A/B test two models" + "split live traffic" + "both return predictions" | Production variants on one real-time endpoint (traffic weights) | Shadow variant (does not return to client) or two fully separate apps. | Weighted routing (e.g., 90/10); compare live metrics; both variants serve users. |
| "Shadow test new model" + "mirror production traffic" + "do not return shadow output" | SageMaker Shadow variant | Production variant A/B (users see new model) or Batch Transform replay. | Shadow gets copied traffic; production response unchanged; analyze logs/metrics offline. |
| "Need instance type / cost / latency recommendation" before production | SageMaker Inference Recommender (model in Model Registry) | Manual guess of ml.g5 / ml.c5 or Compute Optimizer (not ML-endpoint specific). |
Load-tests instance types; returns cost, MaxInvocations, ModelLatency, env tunables. |
| "Pre-trained / JumpStart FM" + "host on SageMaker GPU endpoint" + full control of instance | SageMaker JumpStart deploy to real-time endpoint | Amazon Bedrock on-demand (no instance control) when they explicitly need SageMaker hosting. | JumpStart packages model + container; you choose instance, autoscaling, VPC. |
| "Invoke FM via API" + "no infra to manage" + "pay per token" | Amazon Bedrock on-demand inference | SageMaker real-time GPU endpoint (idle instance cost) unless custom hosting is required. | Serverless FM invoke (InvokeModel / Converse); default for most GenAI apps. |
| "Guaranteed throughput / dedicated capacity" + "consistent FM latency" + enterprise SLA | Amazon Bedrock Provisioned Throughput | On-demand (throttles under burst) or SageMaker endpoint when Bedrock PT is enough. | Purchased model units; required for some customized Bedrock models; predictable latency. |
| "Model trained outside AWS / SageMaker" + "use Bedrock APIs, KB, Guardrails, Agents" | Amazon Bedrock Custom Model Import | Re-train inside Bedrock or self-manage GPU on EC2/EKS. | Import weights from S3; invoke via Bedrock on-demand; no instance management. |
| "Customized Bedrock model" + "on-demand invoke (no PT)" | Bedrock Custom Model Deployment (on-demand) | Buying Provisioned Throughput when on-demand custom deploy is available. | Deploy custom/imported model; invoke via deployment ARN; pay per use. |
| "Open-source / custom LLM" + "GPU control / vLLM / tensor parallel" + not Bedrock-supported | SageMaker real-time GPU endpoint (or EKS only if k8s is mandated) | Bedrock on-demand (model not available) or Serverless Inference (no GPU). | Host Hugging Face / custom containers on ml.g5/ml.g6/ml.p4/ml.inf2. |
| "Lowest inference $ for high-volume NLP/CV" + "AWS silicon" | SageMaker Inf1/Inf2 (Inferentia) endpoints | GPU p4/p5 for inference (training-class cost) or CPU when throughput is high. |
Inferentia is the inference-optimized accelerator family; Trainium (trn1) is for training. |
| "Simple classification / regex-like" + sporadic + no GPU + <15 min | AWS Lambda (+ API Gateway) | SageMaker real-time (overkill/idle cost) or ECS. | Cheap event-driven inference for tiny models; 15-min cap, no GPU. |
| "Existing Kubernetes / Kubeflow / custom GPU serving (vLLM, Triton)" | Amazon EKS GPU node groups (+ GPU Operator) | SageMaker endpoint when they already standardized on k8s control. | Use EKS only when k8s portability/control is the constraint; otherwise SageMaker. |
| "Containerized inference" + "no k8s skill" + Fargate/EC2 tasks | Amazon ECS | EKS (k8s overhead) or SageMaker when you need Model Registry/MME/guardrails. | Simpler than EKS; still you own scaling, health, rolling updates. |
| "Managed agent" + "action groups (Lambda/OpenAPI)" + "KB + Guardrails" + little custom code | Amazon Bedrock Agents | AgentCore/custom LangGraph on ECS when a fully managed agent suffices. | Built-in orchestration, tool calling, KB retrieval, session; fastest path for standard agents. |
| "Custom agent framework" (Strands, LangGraph, CrewAI) + "MCP / A2A protocols" + production runtime | Amazon Bedrock AgentCore Runtime | Hand-roll ECS/EKS + ALB or Bedrock Agents (limited to its orchestration). | Managed runtime for your agent container; MCP on :8000/mcp, A2A on :9000. |
| "Expose APIs/Lambda as MCP tools" + "agent-to-agent (A2A)" + unified ingress/auth | Amazon Bedrock AgentCore Gateway | Custom API Gateway + auth glue or Bedrock Agent action groups only. | Converts APIs/Lambda to MCP tools; A2A passthrough; ingress/egress auth. |
| "Agent discovery card" + JSON-RPC + port 9000 | AgentCore Runtime with A2A protocol | MCP protocol (port 8000 /mcp) or HTTP /invocations (8080). |
A2A: port 9000, path /, Agent Card at /.well-known/agent-card.json. |
| "MCP tools / streamable HTTP" + port 8000 | AgentCore Runtime with MCP protocol | A2A (9000) or Bedrock Agents action groups. | MCP servers at 0.0.0.0:8000/mcp; choose stateful vs stateless HTTP. |
| "RAG fully managed" + "S3/SharePoint/Confluence/web" + embeddings + retrieval | Amazon Bedrock Knowledge Bases | DIY: Lambda + OpenSearch + embeddings (undifferentiated heavy lifting). | Managed ingest, chunk, embed, retrieve; Retrieve / RetrieveAndGenerate. |
| "Hybrid search (keyword + vector)" + low-latency RAG | Bedrock KB + OpenSearch Service (hybrid) | S3 Vectors (semantic only) or pgvector when hybrid/lexical is required. | Hybrid needs a store with filterable text (OpenSearch, Aurora/RDS, MongoDB). |
| "Rerank retrieved chunks" + improve RAG precision | Bedrock KB reranker model (rerank config) | Increase numberOfResults only or re-embed with a larger model first. |
Two-stage retrieve-then-rerank; higher quality, extra latency/cost. |
| "Semantic search only" + "lowest vector $ at huge scale" | Bedrock KB + Amazon S3 Vectors | OpenSearch Serverless (higher $ ) when sub-20ms hybrid is not required. | Cheap vector store; sub-second; no hybrid search; no binary embeddings. |
| "Join vectors with relational rows" + SQL + RAG metadata | Bedrock KB + Aurora PostgreSQL pgvector | OpenSearch (no relational joins) or S3 Vectors. | Only vector option that natively joins embeddings to OLTP data. |
| "Sub-20ms / hybrid / aggregations" RAG | OpenSearch Service (managed or Serverless) k-NN | Aurora pgvector (~50–100ms) or S3 Vectors (sub-second, semantic only). | In-memory HNSW; best latency + hybrid. |
| "Programmatic deploy from notebook / SDK" | SageMaker Python SDK Model.deploy() |
Only Console clicking when automation is required. | Fastest path for data scientists; wraps endpoint config + endpoint APIs. |
| "Production IaC / repeatable endpoint" | CloudFormation or AWS CDK (AWS::SageMaker::Endpoint) |
One-off SDK deploy() with no stack. |
Versioned infra; rollback the stack; required for MLOps environments. |
| "CLI / CI job creates endpoint" | AWS CLI / Boto3 create_model → create_endpoint_config → create_endpoint |
Console-only or SDK when pipeline is shell/Python boto3. | Explicit three-resource flow used in CodeBuild/CodePipeline. |
3.2 Provision and configure resources for ML and AI workloads
| Trigger Keywords & Scenario Constraints | Lock-In Architecture | Immediate Rejection / Trap to Avoid | Explanation |
|---|---|---|---|
| "Target tracking autoscaling" + real-time endpoint + CPU/small models | Application Auto Scaling + SageMakerVariantInvocationsPerInstance |
CPUUtilization as first choice, or Step Scaling without load test. | Predefined metric; keep invocations/instance near target (e.g., 70). Load-test the target. |
| "GPU-bound LLM / utilization-based scale" | Custom metric GPUUtilization (or concurrent requests per model) |
InvocationsPerInstance alone (hides GPU saturation). |
GPU workloads stall before invocation count spikes; scale on GPU % or concurrency. |
| "Async endpoint scale on queue depth" | ApproximateBacklogSizePerInstance |
InvocationsPerInstance (does not reflect queue). |
Scale out when S3/SQS backlog per instance exceeds target; scale to 0 when empty. |
| "Serverless endpoint cold-start too slow" + still intermittent | Serverless Inference Provisioned Concurrency | Switch immediately to real-time (idle cost) if a few warm workers suffice. | Keeps a floor of warm workers; still no GPU / 4 MB limit. |
| "On-demand vs provisioned FM capacity" + bursty chat vs SLA | Bedrock on-demand (burst) vs Provisioned Throughput (SLA) | Always PT (waste) or always on-demand (throttles). | Match purchasing model to traffic shape; mix both if needed. |
| "Pack multiple models on one GPU endpoint" + independent scaling | SageMaker Inference Components | One model per GPU endpoint (underutilized) or MME when you need dedicated allocation. | Multiple models share hardware with per-component autoscaling/min copies. |
| "Private ML endpoint" + "no internet" + "VPC only" | SageMaker endpoint in VPC + Interface VPC endpoints (sagemaker.runtime, sagemaker.api) + S3 gateway |
Public endpoint + security group only, or Serverless Inference (no VPC). | Place ENIs in private subnets; lock SG; PrivateLink; enable network isolation if required. |
| "Endpoint must not reach public AWS APIs" | VPC + EnableNetworkIsolation |
NAT Gateway “to be safe” when isolation is the requirement. | Containers cannot make outbound calls; dependencies must be in the image. |
| "Build custom inference/training image" + store | CodeBuild → Amazon ECR → SageMaker job/endpoint | Push from laptop or store images in S3. | ECR is the SageMaker image source; scan with Inspector (Domain 4). |
| "Prefer not to maintain Dockerfiles" | SageMaker pre-built DLC / framework containers (extend if needed) | Immediate BYOC for sklearn/XGBoost/PyTorch/TF. | Use prebuilt first; extend for extra deps; full BYOC only when runtime is unsupported. |
| "Automate stacks talking to pipelines" + env promotion | CloudFormation/CDK nested stacks + SageMaker Projects / Service Catalog | Manual Console create of endpoints per env. | Projects bootstrap multi-account MLOps; Service Catalog standardizes products. |
| "Event-driven provision/teardown of endpoints" | EventBridge → Step Functions / Lambda → SageMaker APIs | Long-running always-on real-time for a nightly job. | Spin batch/async/real-time only when a model is Approved or a schedule fires. |
| "KB vector store choice: cheap, huge, semantic only" | S3 Vectors behind Bedrock KB | OpenSearch Serverless for archival RAG. | Up to ~90% storage savings; warm ~100ms; no hybrid. |
| "KB vector store: hybrid + lowest latency" | OpenSearch Serverless or managed cluster | pgvector when hybrid/sub-20ms is required. | k-NN + BM25 hybrid; more $ and ops than S3 Vectors. |
| "KB vector store: existing Aurora + SQL filters" | Aurora PostgreSQL + pgvector | Stand up OpenSearch just for vectors if Aurora already holds the data. | HNSW/IVFFlat; cosine/L2/IP; join metadata in SQL. |
| "Refresh / re-index knowledge base on new S3 docs" | KB StartIngestionJob on schedule (EventBridge) or S3 event |
Recreate the entire KB or re-embed manually from a notebook. | Incremental sync of data sources; Domain 3.3.11 also tests this in pipelines. |
| "Retrieve only (no generation)" vs "retrieve + answer" | Retrieve API vs RetrieveAndGenerate |
Always RetrieveAndGenerate when the app needs citations/chunks only. |
Split retrieval from generation when you own the prompt/FM call. |
| "Agent session / conversation state" + durable, multi-turn | Amazon DynamoDB (session table) or AgentCore Memory | S3 per turn (latency) or in-container memory (lost on scale-in). | DynamoDB for session keys/history; ElastiCache only if sub-ms hot state is required. |
| "Stateful MCP elicitation / sampling" | AgentCore MCP stateless_http=False + session header |
Default stateless MCP (no elicitation/progress). | Stateful mode preserves MCP session; client must send Mcp-Session-Id. |
| "Low-latency feature lookup at inference" | SageMaker Feature Store Online Store in serial pipeline / container | Offline Feature Store (Athena/Glue) or scanning S3 at request time. | Online store = DynamoDB-backed GetRecord; pair with real-time endpoint. |
| "GPU inference autoscaling" + fast scale-out | Autoscaling + Inference Components + GPUUtilization; pre-warm min instances | MinCapacity=0 on GPU real-time (cold start minutes). | Keep min warm GPUs for SLA; use Inf2/G6 and load-test. |
| "Fargate vs EC2 for agent containers" + no GPU | ECS/EKS Fargate or AgentCore Runtime | GPU EC2 when the agent only calls Bedrock APIs. | Agent orchestrators are usually CPU; GPUs belong on the model endpoint, not the agent. |
| "Host models programmatically in CI" | Boto3 / SageMaker SDK inside CodeBuild | Click-ops in Studio as the production path. | Same APIs as notebooks; credentials via CodeBuild role. |
| "Need HTTPS API in front of endpoint" + auth/throttle | Amazon API Gateway → SageMaker Runtime | Expose raw SageMaker endpoint URL to browsers. | API Gateway adds API keys, WAF, usage plans; watch 29s REST integration timeout (use async if longer). |
| "Streaming LLM tokens to client" | Bedrock InvokeModelWithResponseStream / Converse stream or SageMaker streaming |
API Gateway REST (buffers) for token streaming. | Use WebSocket API, App/CloudFront, or direct SDK stream; REST API Gateway is a trap for streams. |
| "Nightly scoring of the data lake" | SageMaker Batch Transform or AWS Batch on S3 | Always-on real-time endpoint. | No need for an endpoint if all inputs are known ahead of time. |
| "Notifications when async inference finishes" | SNS topic on async endpoint + S3 output | Client polling the endpoint (real-time pattern). | Async is designed for S3 + SNS/EventBridge completion. |
3.3 Automated orchestration and CI/CD for MLOps and AI workloads
| Trigger Keywords & Scenario Constraints | Lock-In Architecture | Immediate Rejection / Trap to Avoid | Explanation |
|---|---|---|---|
| "ML-native DAG" + processing/train/eval/register + lineage + step cache | Amazon SageMaker Pipelines | Step Functions (no native lineage/cache) or Airflow for a pure SageMaker flow. | First-class Processing/Training/Tuning/Condition/RegisterModel steps; $0 orchestration. |
| "Orchestrate many AWS services" + human approval + event-driven, serverless | AWS Step Functions | SageMaker Pipelines (limited non-SageMaker steps) or MWAA (always-on env $). | Callback/wait-for-task-token approvals; 200+ service integrations. |
| "Existing Airflow DAGs / Python operators" + long batch + resume from failure | Amazon MWAA | Step Functions when the team already lives in Airflow. | Managed Airflow; you size the env; best for complex stateful data DAGs. |
| "Software release pipeline" (build → test → deploy infra) | AWS CodePipeline | SageMaker Pipelines as the git/build/deploy engine. | CodePipeline orchestrates CI/CD; SageMaker Pipelines orchestrates ML steps. |
| "GitHub/GitLab/Bitbucket as source" for CodePipeline | AWS CodeConnections | CodeCommit-only assumption, or embed PATs in env vars. | Managed third-party git connection (successor to CodeStar Connections). |
| "Build images / run unit tests / package models" | AWS CodeBuild | Lambda (15 min / size limits) as the image builder. | Buildspec runs tests, docker push to ECR, start Pipeline/training. |
| "AWS-hosted git" (if scenario still uses it) | AWS CodeCommit | GitHub without CodeConnections when the item specifies AWS git. | Still in-scope; many templates still show CodeCommit. |
| "Deploy endpoint/app artifacts" (EC2/ECS patterns) | AWS CodeDeploy | CodeDeploy to update a SageMaker endpoint (wrong tool). | SageMaker endpoints update via UpdateEndpoint / guardrails, not CodeDeploy. |
| "Blue/green cutover of endpoint" + baking period + auto-rollback | SageMaker Deployment Guardrails – blue/green | In-place UpdateEndpoint with no alarms. |
New (green) fleet; CloudWatch alarms during bake; auto-rollback to blue. |
| "Shift a small % first" then rest if healthy | Canary traffic shifting (canary \(\le 50\%\)) | All-at-once blue/green when blast radius must be tiny. | Two-step shift; failures affect only the canary slice. |
| "N equal traffic steps" (e.g., 10/20/…/100) | Linear traffic shifting | Canary (only 2 steps) when they ask for n steps. | Configurable step count and % per step. |
| "Replace instances in batches" + capacity control | Rolling deployment | Blue/green (2 full fleets = 2× capacity $) when capacity is tight. | Incremental instance replacement; cheaper than full green fleet. |
| "Auto-rollback on error metric" | Guardrails + CloudWatch alarms on bake time | Manual rollback playbooks as the only control. | Alarms on 4xx/5xx/ModelLatency trip rollback automatically. |
| "Approve model before prod" + audit trail | SageMaker Model Registry (PendingManualApproval → Approved) |
Deploy straight from training job artifacts. | Model package group versions; EventBridge on status change kicks deploy. |
| "Experiment tracking + metrics/params/artifacts" | MLflow on SageMaker AI | Model Registry as an experiment tracker (it is a catalog, not a tracker). | MLflow = runs/experiments; Registry = production packages/approvals. Use both. |
| "Register then auto-deploy on approval" | EventBridge rule on Model Registry → CodePipeline / Step Functions / CFN | Polling Lambda every 5 min. | Event-driven promotion staging → prod is the MLOps default. |
| "Retrain when new data lands in S3" | S3 Event → EventBridge → SageMaker Pipeline | Cron-only retrain ignoring new data, or Glue workflow as the ML trainer. | Standard continuous-training trigger; pass new URI as pipeline parameter. |
| "Retrain on a schedule" (weekly/nightly) | EventBridge Scheduler → SageMaker Pipeline | Long-running Step Functions loop sleeping 7 days. | Time-based orchestration; combine with quality gates in ConditionStep. |
| "Retrain when quality/drift degrades" | Model Monitor / eval job failure → EventBridge → Pipeline | Always-on hourly retrain (cost) with no signal. | Close the loop; actual drift detectors are Domain 4, the mechanism is 3.3. |
| "Fail pipeline if metric below threshold" | SageMaker Pipelines ConditionStep (e.g., Accuracy \(\ge 0.9\)) |
Register every trained model regardless of eval. | Evaluation processing step → JSON metrics → condition → RegisterModel or Fail. |
| "Unit + integration tests in CI" for training/inference code | CodeBuild test stage before training/deploy | Testing only in production shadow. | pytest on preprocessing/inference handlers; block pipeline on failure. |
| "Prompt versioning / A-B prompts" + audit | Amazon Bedrock Prompt Management | Prompts hardcoded in Lambda/agent source. | Versioned prompts independent of app deploys; rollback a prompt without redeploying code. |
| "Automated prompt/FM quality tests in CI" | Amazon Bedrock Evaluations (or LLM-as-judge job) in CodeBuild/Pipeline | Only BLEU/ROUGE on a chat agent, or no prompt tests. | Run eval suite on prompt+model combo before promoting an agent/prompt version. |
| "Agent versioning + automated deploy" | CI: build agent image → ECR → AgentCore Runtime update + tagged versions | Mutate a live agent container by hand. | Treat agents like microservices: git tag, image digest, staged runtime. |
| "Fine-tuned FM versioning + automated host" | Model Registry (SageMaker host) or Bedrock custom model + PT/on-demand deploy in pipeline | Overwrite the same unlabeled endpoint/model name. | Immutable versions; Provisioned Throughput update or new custom-model deployment ARN. |
| "RAG index refresh in the pipeline" | Step Functions / SageMaker Pipeline step: ingest → StartIngestionJob → wait → eval retrieval |
Training pipeline with no KB sync, or rebuild OpenSearch from scratch every time. | KB refresh is a first-class CD step; add retrieval-accuracy eval before prod cutover. |
| "End-to-end MLOps template (code + train + deploy)" | SageMaker Projects (CodePipeline or Jenkins templates) | Assemble 10 services from scratch when a project template fits. | Seeds git repos, Model Registry, pipelines, staging/prod accounts. |
| "Human-in-the-loop deploy gate" | Step Functions wait-for-callback or Model Registry manual approval | Automatic prod deploy with no gate when the scenario requires review. | Two standard gates: registry approval and Step Functions task token. |
| "Multi-account / multi-env promotion" | CodePipeline + CloudFormation/CDK (dev/stage/prod) + IAM roles | Single shared “prod” endpoint updated from a notebook. | Artifact promotion across accounts; least-privilege deploy roles. |
| "Lineage / repeatability / audit of models" | SageMaker Model Registry + Pipelines lineage | S3 folder named model-v3.pkl as the registry. |
Packages capture container URI, metrics, data lineage, approval history. |
| "Parameterize pipeline runs" (dataset URI, instance type) | SageMaker Pipeline Parameters | Clone the entire pipeline definition per run. | Same pipeline definition, new execution with parameters. |
| "Skip unchanged steps" (cache training if data/code same) | SageMaker Pipelines step caching | MWAA/Step Functions custom cache. | Native cache keys; saves training $ on CI reruns. |
| "Glue vs ML orchestration" | SageMaker Pipelines / Step Functions for ML; Glue only for ETL | AWS Glue workflow as the model trainer/deployer. | Glue is data prep (Domain 1); not the MLOps orchestrator. |
| "Jenkins already enterprise standard" | SageMaker Projects Jenkins template or Jenkins → SageMaker APIs | Force CodePipeline when the constraint is “use existing Jenkins”. | Exam honors existing CI constraints; still register models in Model Registry. |
| "Rollback a bad model quickly" | Flip production variant weights or redeploy previous Model Package + guardrail rollback | Retrain from scratch as the rollback plan. | Keep N-1 package Approved; variant weight 100% back to old model. |
| "Configure training/inference jobs in CI" | Pipeline TrainingStep / TransformStep / TuningStep with IAM role + VPC config |
Notebook estimator.fit() as the production trainer. |
Jobs must be declarative, retryable, and role-scoped inside the pipeline. |
| "Notify on pipeline/endpoint events" | EventBridge → SNS/Slack/Lambda | Busy-wait in CodeBuild for 2 hours. | Event-driven success/fail; start deploy only on Approved or pipeline succeeded. |
| "Service Catalog for approved ML products" | AWS Service Catalog product (endpoint/pipeline stack) | Each team creates raw CFN with no governance. | Central portfolio: approved instance types, encryption, VPC. |
| "IaC for agents + KB + IAM" | CDK/CloudFormation including Bedrock KB, agent action groups, roles | Console-built agents with no template. | Required for repeatable agentic infra (3.2.11 + 3.3.8). |
| "Code artifact / pip wheels for pipelines" | AWS CodeArtifact | pip install from public PyPI at training time with no lock. |
Private, versioned dependencies for reproducible training/inference images. |
| "Connect SageMaker Studio to Git" | CodeConnections (or CodeCommit) | Copy-paste notebooks as version control. | Source of truth is git; pipelines start from commit. |
| "FM deploy automation after fine-tune" | Fine-tune job → register → EventBridge → Bedrock PT or SageMaker endpoint update | Manual console “Set up inference” as the only path. | Same CD pattern as classical ML; version the custom model ARN. |
| "Don't use SageMaker Neo / edge compile" | N/A (removed in MLA-C02) | SageMaker Neo as the answer for edge optimization. | Edge compilation was deleted from MLA-C02 Task 3.1 — skip it. |
Cross-cutting lock-ins vs traps (Domain 3)
| Trigger Keywords & Scenario Constraints | Lock-In Architecture | Immediate Rejection / Trap to Avoid | Explanation |
|---|---|---|---|
| "Need GPU inference" | Real-time / Async / Batch on GPU or Inf2 | Serverless Inference (no GPU) or Lambda. | GPU is available on real-time, async, and batch — not serverless. |
| "Payload > 4 MB" and interactive | Real-time (\(\le 25\text{ MB}\)) or Async (\(\le 1\text{ GB}\)) | Serverless (4 MB hard cap). | Payload size is the fastest discriminator among hosting options. |
| "Processing time > 60s" | Async (\(\le 1\text{ h}\)) or Batch (days) | Real-time / Serverless (60s). | Timeout is the second discriminator. |
| "Cost-sensitive, idle most of the day" | Serverless or Async scale-to-0 or Batch | Real-time min instances = 1 (idle $). | Do not leave GPU/CPU endpoints up “just in case”. |
| "Same framework, many models, cold start OK" | MME | MCE (all resident) or 1k endpoints. | MME wins on $ when models are similar and sparse. |
| "Different frameworks, no cold start, few models" | MCE (\(\le 15\)) | MME (single container/framework). | MCE keeps every container warm. |
| "Users must not see candidate model" | Shadow variant | Production variant 10% canary (users see it). | Shadow = observe only; canary/A-B = live answers. |
| "Users see a small % of new model" | Canary guardrail or production variant weights | Shadow variant (users never see it). | Pick based on whether the new model answers clients. |
| "Pure SageMaker ML lifecycle" | SageMaker Pipelines | MWAA hourly environment cost. | Pipelines is free orchestration + lineage. |
| "Cross-service workflow + approval" | Step Functions | SageMaker Pipelines CallbackStep hacks as first choice. | Approvals and non-ML services are Step Functions’ home turf. |
| "Airflow already in production" | MWAA | Rewrite DAGs in Step Functions “because serverless”. | Honor existing Airflow skill/DAG investment. |
| "Git → build → deploy" | CodePipeline + CodeBuild + CodeConnections | SageMaker Pipelines replacing git/CI. | Two layers: CI/CD (code) and ML DAG (training). |
| "Catalog / approve models" | SageMaker Model Registry | MLflow stages alone, or S3 object versions. | Registry is the deploy contract; MLflow is experiment tracking. |
| "Version prompts" | Bedrock Prompt Management | Git-only strings with no Bedrock version IDs. | Prompts are first-class production artifacts in MLA-C02. |
| "Managed RAG" | Bedrock Knowledge Bases | Custom LangChain + self-managed OpenSearch unless constraints demand it. | KB is the exam default RAG architecture. |
| "Hybrid RAG search" | OpenSearch (or Aurora/RDS with text field) | S3 Vectors (semantic only). | Hybrid support is store-dependent. |
| "Cheapest huge RAG corpus" | S3 Vectors | OpenSearch Serverless for cold corpus. | Tier: S3 Vectors for full corpus, OpenSearch for hot subset. |
| "Managed tools/agents" | Bedrock Agents | AgentCore unless custom framework/MCP/A2A is stated. | Agents = managed; AgentCore = custom runtime. |
| "Custom agent + MCP/A2A" | AgentCore Runtime + Gateway | Bedrock Agents action groups only. | Protocols and custom frameworks require AgentCore. |
| "Import external FM into Bedrock" | Custom Model Import | Re-host on SageMaker when they want Bedrock KB/Guardrails/Agents. | Single API with other Bedrock FMs. |
| "Guaranteed FM capacity" | Provisioned Throughput | On-demand under a strict latency SLA. | PT is the capacity reservation. |
| "VPC isolation for hosting" | Real-time/Async/Batch in VPC | Serverless Inference (no VPC/PrivateLink). | Feature matrix: serverless lacks VPC, GPU, MME, guardrails, shadow. |
| "Autoscaling metric not specified" (generic real-time) | InvocationsPerInstance target tracking |
RequestCount on the ALB or CPU by default. | Exam default metric for SageMaker variants. |
| "Queue-based scale" | Async + ApproximateBacklogSizePerInstance |
InvocationsPerInstance on async. | Backlog is the async-native signal. |
| "Safe production model swap" | Deployment guardrails (canary/linear/blue-green) + alarms | Direct UpdateEndpoint all-at-once with no bake. |
Guardrails + rollback are the 3.3.1 answer. |
| "Trigger on model Approved" | EventBridge + deploy pipeline | Cron that redeploys whatever is latest. | Registry status change is the promotion event. |