Skill 4.2.6: Optimize infrastructure costs by selecting purchasing options
Task 4.2: Operating, Monitoring, and Securing ML and AI Solutions - Optimize and manage ML and AI infrastructure costs and performance.
Core objective: Match the purchasing model to demand predictability and interruption tolerance. Commit only for steady, predictable usage; stay on-demand for variable/experimental; use spare capacity only for fault-tolerant ML jobs.
Purchasing Options Decision Matrix
| Option | Applies To | Best For | Commitment / Term | Key Trade-off |
|---|---|---|---|---|
| On-Demand | EC2, SageMaker AI, Bedrock on-demand, Lambda, Fargate | Variable, spiky, experimental, short-lived workloads | None | Highest unit rate; maximum flexibility |
| SageMaker AI Savings Plans | SageMaker ML instances (training, inference, processing, notebooks) | Steady, predictable SageMaker usage | 1- or 3-year, hourly $ commitment | Up to 64% savings; does not cover non-SageMaker compute |
| Compute Savings Plans | EC2, Lambda, Fargate | Steady compute across those services | 1- or 3-year, hourly $ commitment | Flexible across families/regions; does NOT cover SageMaker |
| EC2 Instance Savings Plans | EC2 only, specific family + region | Steady EC2 usage with known family | 1- or 3-year, hourly $ commitment | Higher discount than Compute SP; less flexible; does NOT cover SageMaker |
| Standard Reserved Instances | EC2, RDS, Redshift, ElastiCache | Steady, long-running, predictable instance usage | 1- or 3-year | Highest RI discount; least flexible; not for SageMaker |
| Convertible Reserved Instances | Same as Standard RI | Steady usage needing instance-type changes | 1- or 3-year | Lower discount; exchangeable; not for SageMaker |
| Spot Instances / Managed Spot Training | EC2, SageMaker training/processing, EMR, Batch | Interruptible, fault-tolerant training & batch jobs | None | Up to 90% discount; can be reclaimed with 2-min notice; requires checkpointing |
| Bedrock Provisioned Throughput | Amazon Bedrock foundation models | Predictable high-volume token inference | 1 month / 6 months / 1 year (model-dependent) | Fixed hourly model units; dedicated throughput; no auto-scaling |
| Bedrock Batch Inference | Amazon Bedrock | Large, delay-tolerant async jobs | Per-job | ~50% lower than on-demand; no real-time latency SLA |
Coverage Mapping — What Covers What
| Service | Valid Purchasing Options | Common Distractor Trap |
|---|---|---|
| Amazon SageMaker AI | On-Demand, SageMaker AI Savings Plans, Managed Spot (training/processing only) | Compute SP and EC2 Instance SP do not cover SageMaker ML instances |
| Amazon Bedrock | On-Demand (tokens), Batch Inference, Provisioned Throughput | No Savings Plans or RIs apply to Bedrock token usage |
| Amazon EC2 (self-managed ML) | On-Demand, Spot, Compute SP, EC2 Instance SP, RIs | Spot breaks real-time inference; use only for interruptible jobs |
| AWS Lambda / Fargate | On-Demand, Compute Savings Plans | EC2 Instance SP and SageMaker SP do not apply |
| Amazon EMR | On-Demand, Spot, EC2 purchasing options | EMR uses EC2 billing; match EC2 purchasing model |
Commitment Mechanics
| Option | Payment Options | Flexibility | Typical Max Savings |
|---|---|---|---|
| SageMaker AI Savings Plans | All / Partial / No Upfront | Across SageMaker instance families & sizes | Up to 64% |
| Compute Savings Plans | All / Partial / No Upfront | Across EC2, Lambda, Fargate; any family/region/OS | Up to 66% |
| EC2 Instance Savings Plans | All / Partial / No Upfront | Specific EC2 family in a region | Up to 72% |
| Standard RI | All / Partial / No Upfront | Specific instance type/region | Up to 72% |
| Convertible RI | All / Partial / No Upfront | Exchangeable for different instance types | Up to 54% |
| Spot | Variable market price | None; interruption risk | Up to 90% |
| Bedrock Provisioned Throughput | Hourly model-unit commitment | Fixed throughput; no auto-scale | Varies by model & term |
Workload-to-Purchasing Mapping
| Workload Pattern | Recommended Purchasing Option | Why |
|---|---|---|
| Steady 24/7 SageMaker real-time endpoint | SageMaker AI Savings Plans | Commit to baseline; reduce hourly rate |
| Diurnal SageMaker endpoint (peak day, idle night) | On-Demand + Auto Scaling; SP for baseline only | Capacity follows demand; don't commit peak |
| Interruptible SageMaker training job | SageMaker Managed Spot Training | Up to 90% off; use checkpoints to resume |
| Real-time inference endpoint | On-Demand or SageMaker AI Savings Plans | Never Spot — interruption breaks serving |
| Many small models, intermittent calls | Multi-Model Endpoint on shared compute + SP for baseline | Consolidate instances; still commit steady baseline |
| Steady high-volume Bedrock inference | Provisioned Throughput | Fixed hourly model units; lower than on-demand |
| Variable / experimental Bedrock inference | On-Demand | No commitment; pay per token |
| Large, delay-tolerant Bedrock jobs | Batch Inference | ~50% discount; no latency SLA |
| Self-managed EC2 ML training | Spot for fault-tolerant; Compute SP for steady; EC2 Instance SP for known family | Match commitment to predictability |
Exam Gotcha: SageMaker AI Savings Plans cover SageMaker ML instances. Compute Savings Plans and EC2 Instance Savings Plans do NOT cover SageMaker.
Exam Gotcha: Spot is for interruptible training/processing only. Real-time inference requires on-demand or committed capacity.
Exam Gotcha: Bedrock Provisioned Throughput is a fixed hourly commitment to model units. It does not auto-scale. Over-provisioning wastes money; under-provisioning throttles.
Exam Gotcha: Batch inference is asynchronous and cheaper, but it is not for real-time latency-sensitive applications.
Exam Gotcha: Cost allocation tags are the prerequisite for attributing spend by model, team, or environment. Without them, cost reports show only service totals.
Cost Governance & Monitoring (Brief)
- AWS Cost Explorer: Analyze historical/forecasted spend; group by tag, service, or usage type to find cost drivers.
- AWS Budgets: Set custom spend/usage thresholds; receive forecasted alerts before overages occur.
- AWS Trusted Advisor: Surfaces idle and underused resources across cost checks.
- Amazon CloudWatch: Provides utilization metrics needed for right-sizing and auto-scaling decisions.
- Tagging: Enforce cost allocation tags for chargeback and attribution.
Anti-Patterns
- Using Compute Savings Plans for SageMaker — no coverage; use SageMaker AI Savings Plans.
- Using Spot for real-time inference — interruption breaks serving guarantees.
- Committing to peak capacity — commit only to steady baseline; let auto scaling handle peaks.
- Buying RIs for SageMaker — SageMaker uses Savings Plans, not RIs.
- Over-provisioning Bedrock Provisioned Throughput — no auto-scale; variable traffic belongs on-demand.
- Skipping checkpoints with Spot — interrupted jobs cannot resume without them.
- Applying endpoint right-sizing to Bedrock token bills — token-based billing has different levers (model selection, prompt discipline, caching, batch).