Skip to content

Skill 4.3.6: Create VPCs, subnets, and security groups to securely isolate ML and AI systems

Task 4.3: Operating, Monitoring, and Securing ML and AI Solutions - Secure ML and AI workloads and model endpoints

Core Objective & Mental Model

  • Goal: Place ML/AI data, training jobs, notebooks, and endpoints in private subnets; allow only required AWS service calls via private paths; block internet ingress/egress unless explicitly needed.
  • Separate controls: VPC/subnet/route = network path; SG/NACL = traffic filter; IAM = identity/permission; KMS = encryption; Guardrails = content. No single control substitutes for another.
  • Exam lens: If a private ML workload can’t reach S3, ECR, KMS, CloudWatch, or SageMaker/Bedrock APIs, the missing piece is usually a VPC endpoint, endpoint SG rule, private DNS, or route table entry—not a broader IAM policy.

VPC & Subnet Design

Element High-yield exam points
VPC Regional; plan CIDR to avoid overlap with on-prem/peered VPCs. Enable DNS resolution + DNS hostnames for interface endpoints.
Public subnet Route to Internet Gateway (IGW). Use for NAT Gateway, bastion, public ALB. Avoid for training jobs, notebooks, model endpoints.
Private subnet No IGW route. Place SageMaker jobs/endpoints/notebooks, Bedrock callers, data stores. Egress via NAT or VPC endpoints.
Route table Controls subnet egress. Gateway endpoints (S3/DynamoDB) require route table entries.
Availability Zone Spread subnets and interface endpoints across ≥2 AZs for HA. Interface endpoints are AZ-specific.
IP sizing Each interface endpoint consumes an ENI/IP per subnet; each SageMaker endpoint instance consumes an ENI. Under-sized subnets cause deployment failures.

Gotcha: Private subnet + no NAT + no VPC endpoint = AWS API calls fail. Fix the private path, not the security group.

Security Groups vs NACLs

Control Scope State Rules Exam use
Security Group Instance/ENI/interface endpoint Stateful Allow only; default deny inbound, allow all outbound Primary micro-segmentation. Reference source SG instead of CIDR.
Network ACL Subnet Stateless Allow + deny; numbered order, first match wins Subnet guardrail. Must explicitly allow return traffic incl. ephemeral ports.
  • SG best practice: Inbound 443 from application SG only; restrict egress to required endpoint SGs/prefix lists. Avoid 0.0.0.0/0 inbound/outbound.
  • Interface endpoint SG: Must allow inbound 443 from the workload SG. If workload egress is restricted, allow outbound to endpoint SG.
  • NACL gotcha: Return traffic is not tracked. Allow ephemeral ports (1024–65535) both directions or connections drop asymmetrically.
  • Default SG trap: Default outbound allows all. For strict isolation, tighten egress to prevent data exfiltration.

Private Access to AWS Services

Path Services How it works Key config
Gateway endpoint S3, DynamoDB Route-table target; private IP Endpoint policy; no SG; free
Interface endpoint (PrivateLink) SageMaker, Bedrock, KMS, Secrets Manager, ECR, CloudWatch Logs, etc. ENI in subnet with private IP SG inbound 443 from workload; enable private DNS; one per AZ
NAT Gateway Any internet/AWS public API Public NAT in public subnet Broad egress; not private; use only when endpoint unavailable
VPC peering / Transit Gateway Cross-VPC Private IP routing Non-transitive peering; route tables + SGs both sides

Gotcha: Gateway endpoints do not use security groups. Interface endpoints do. Both IAM policy and VPC endpoint policy are evaluated; a deny in either blocks access.

ML/AI Workload VPC Attachment

Workload VPC configuration Required private path
SageMaker training job Subnets + SGs; optional network isolation S3 gateway, KMS, ECR, CloudWatch Logs, SageMaker API
SageMaker processing job Subnets + SGs S3, ECR, KMS, CloudWatch Logs, SageMaker API
SageMaker notebook/Studio VPC + subnets + SGs SageMaker API/Studio, S3, ECR, KMS, Secrets Manager, CloudWatch Logs
SageMaker endpoint Subnets + SGs; use ≥2 AZs for HA SageMaker runtime, S3 (model download), ECR, KMS, CloudWatch Logs
Bedrock invocation Interface endpoint for Bedrock runtime IAM role + endpoint policy; TLS
Custom container ECR interface endpoints ECR API + ECR DKR + S3 gateway for image layers
  • Network isolation: SageMaker training/endpoint option that blocks outbound network calls from the container. Use when code/model must not exfiltrate; may break direct S3/KMS/external calls.
  • Credential type: Production FM access uses IAM roles with temporary credentials, not long-term API keys. Static secrets belong in Secrets Manager with rotation.

Security & Compliance Overlay

  • IAM: Least privilege on execution roles and VPC endpoint policies. Separate human identity from workload identity.
  • KMS: Encrypt artifacts, volumes, logs at rest. Use KMS interface endpoint for private decryption. TLS in transit via endpoints.
  • Secrets Manager: Interface endpoint for private secret retrieval; rotate credentials.
  • CloudTrail: Management events by default. S3 object access and model invocation are data events and must be enabled separately.
  • VPC Flow Logs: IP traffic metadata to CloudWatch/S3. Use for troubleshooting and anomaly detection; does not capture payload.
  • AWS Config: Detect SG/NACL/endpoint drift. Amazon Inspector: vulnerability + network exposure scans. CodeGuru Security: static code analysis.
  • Bedrock Guardrails: Content filters, denied topics, word filters, sensitive info filters, contextual grounding, Automated Reasoning. Filters content—does not replace network or IAM controls.

Decision Matrix: Private Subnet Egress

Need Use Why
Call AWS service privately Interface endpoint (PrivateLink) Stays on AWS backbone; endpoint policy; no internet
Call S3/DynamoDB privately Gateway endpoint Route-table target; no SG; free
Call external internet/package repo NAT Gateway Broad egress; less isolated; monitor
Fully block container egress SageMaker network isolation No outbound calls; may break direct S3/KMS
Cross-VPC access Peering/Transit Gateway Private IP; route tables + SGs both sides

Anti-Patterns & Distractor Traps

  • Public subnet for endpoints/training: exposes model/data path. Use private subnets.
  • 0.0.0.0/0 inbound on endpoint SG: defeats isolation. Restrict to app SG.
  • Missing endpoint SG inbound rule: endpoint exists but calls time out.
  • Private DNS disabled: interface endpoint not used; traffic still tries public IP.
  • Single AZ endpoint/subnet: AZ failure. Create subnets/endpoints in each used AZ.
  • NAT for all AWS calls: works but not private. Exam prefers PrivateLink when supported.
  • NACL missing ephemeral return: asymmetric traffic dropped; SG alone is stateful.
  • Confusing VPC isolation with IAM/KMS/Guardrails: VPC controls path, not identity, encryption, or content.
  • Assuming CloudTrail records data events by default: model invocation and S3 object events need explicit data-event configuration.

Troubleshooting Quick Hits

  • Timeout from private subnet: check route table, endpoint, endpoint SG inbound, DNS resolution, NACL.
  • AccessDenied: IAM/endpoint policy, not network.
  • Endpoint policy denies: even if IAM allows, endpoint policy must allow.
  • Container can’t pull image: missing ECR API/DKR interface endpoints and S3 gateway for layers.
  • Notebook can’t reach SageMaker API: missing SageMaker interface endpoint.
  • Bedrock call fails from VPC: missing Bedrock runtime interface endpoint or SG rule.
  • Training job can’t write logs: missing CloudWatch Logs interface endpoint.