Skill 3.1.5: Deploy Models Built Outside AWS into AWS Environments
Task 3.1: Deployment and Orchestration of ML and AI Workflows – Manage deployment infrastructure for ML and AI model types
Objective: Move externally trained/customized models into AWS with minimal operational overhead while meeting latency, scale, security, and compliance requirements.
1. Deployment Path Decision Matrix
| Path | Best For | Model Types | Ops Overhead | Invocation | Scaling | Key Limits / Gotchas |
|---|---|---|---|---|---|---|
| Bedrock Custom Model Import | Supported open-weight FMs customized outside AWS; no endpoint ops | Llama, Mistral, Mixtral, Flan, etc. | Low (serverless) | Bedrock InvokeModel / Converse; on-demand | Managed | Only supported architectures/file layouts; import job async; S3 same region; IAM role for S3/KMS; no arbitrary models |
| SageMaker BYOM | Any model/framework; need GPU, VPC, autoscaling, custom containers | XGBoost, PyTorch, TF, scikit-learn, custom | Medium–High | Real-time, serverless, async, batch, MME, MCE | Auto Scaling; scale-to-zero for serverless/async | Manage endpoints, containers, monitoring; execution role; KMS; VPC |
| ECS / EKS | Existing container/K8s control plane; custom networking/sidecars | Any containerized model | High | Service / task | Custom | You manage orchestration, scaling, networking |
| EC2 | Full control, special hardware, licensing | Any | Highest | Direct | Custom | You manage everything |
| Lambda | Event-driven glue / light inference | Small models only | Low | Event / API | Managed | No GPU, 15-min timeout, 10 GB memory; not for large models |
2. Bedrock Custom Model Import — Deep Dive
- What it is: Bring externally fine-tuned/customized open-weight FMs into Bedrock; served via the same Bedrock APIs.
- Import source: S3 bucket in the same region; model artifacts + config. Import job is asynchronous.
- Supported models: Only specific open-weight architectures and file layouts. Validate before migration.
- IAM: Bedrock service role needs S3 read and KMS decrypt permissions on the source bucket/key.
- Encryption: KMS CMK for S3 at rest; TLS in transit.
- Networking: VPC endpoint for Bedrock (PrivateLink); S3 Gateway endpoint for private access.
- Invocation: On-demand; no endpoint sizing. Use Bedrock InvokeModel or Converse.
- Monitoring: CloudTrail, CloudWatch.
Gotcha: CMI is not a general-purpose model host. If the model is not a supported open-weight FM (e.g., custom XGBoost, proprietary architecture), use SageMaker instead.
Gotcha: CMI is on-demand. Provisioned Throughput is for base/fine-tuned models in Bedrock; do not assume reserved capacity for imported models.
3. SageMaker BYOM — Deep Dive
- Artifacts: Model artifact in S3; container image in ECR (BYOC or prebuilt: PyTorch, TensorFlow, Hugging Face, XGBoost, scikit-learn).
- Deployment targets:
- Real-time: ms latency, steady/spiky traffic, 60s timeout, 6 MB payload. Autoscaling on invocation metric. User is waiting.
- Serverless: intermittent traffic with idle gaps, scales to zero, cold starts, 60s, 4 MB. No GPU, no VPC, no MME.
- Async: queued requests, large payloads up to 1 GB, up to 15 min, scales to zero, SNS notification. No user waiting.
- Batch: pre-assembled dataset, S3 in/out, no endpoint. Scheduled bulk.
- MME: many models, same framework, shared fleet, load/unload from S3. Cold start on rare models.
- MCE: up to 15 containers, different frameworks, pipeline or direct invocation.
- Security: SageMaker execution role (ECR pull, S3 read, KMS decrypt); KMS CMK for EBS/S3; VPC config; network isolation; PrivateLink.
- Orchestration: SageMaker Pipelines, Step Functions, Model Registry, CI/CD.
4. Decision Signals → Best Fit
| Requirement | Best Fit |
|---|---|
| Supported open-weight FM, no ops | Bedrock Custom Model Import |
| Custom model/framework, need GPU/VPC/autoscaling | SageMaker endpoint |
| Millisecond latency, user waiting | SageMaker real-time |
| Intermittent, idle gaps, cold start OK | SageMaker serverless |
| Large payload/long processing, no user waiting | SageMaker async |
| Bulk dataset, scheduled | SageMaker batch |
| Many models same framework | SageMaker MME |
| Different frameworks/containers | SageMaker MCE |
| Kubernetes-native orchestration | EKS |
| Simple container service | ECS |
| Event-driven glue/light inference | Lambda |
| Full control, special hardware | EC2 |
5. Security & Compliance Checklist
| Area | Requirements |
|---|---|
| IAM | Least privilege; SageMaker execution role; Bedrock model import role; trust policies. |
| Encryption | KMS CMK for S3, ECR, EBS; TLS in transit. |
| Network | Private subnets; VPC endpoints: S3 Gateway, Bedrock PrivateLink, ECR; security groups. |
| Audit | CloudTrail, CloudWatch Logs, Model Monitor. |
6. Exam Gotchas & Anti-Patterns
Bedrock CMI ≠ general hosting. Only supported open-weight FMs. Custom XGBoost/scikit-learn/PyTorch → SageMaker.
Serverless SageMaker limitations: no VPC, no GPU, no multi-model hosting. If requirement names any, serverless is out.
Real-time 60s timeout. Multi-minute jobs → async (up to 15 min, 1 GB) or batch.
Async vs Batch: Async for individual dynamic requests queued; Batch for pre-assembled datasets on a schedule.
MME vs MCE: MME = many models, same framework, shared container. MCE = up to 15 different containers, pipeline/direct.
CMI import: S3 same region; IAM role with S3 read + KMS decrypt; async job; unsupported architecture fails.
Lambda anti-pattern: no GPU, 15-min timeout, limited memory. Not for large model inference.
VPC endpoints: required for private subnets to reach S3/ECR/Bedrock without NAT/IGW.