Skip to content

Skill 2.2.2: Configure SageMaker AI Script Mode with Supported Frameworks

Task 2.2: ML Model and Foundation Model (FM) Development – Train, fine-tune, and customize models for ML and AI solutions

Core Concept: Three Routes Into a Managed Training Job

Script Mode = you author the training loop, AWS maintains the container. The framework Deep Learning Container (DLC) supplies the runtime (CUDA, cuDNN, framework binaries, SageMaker Training Toolkit); your entry_point supplies the model, loss, data loading, and loop.

Route Training Code Container Image Maintenance Burden Use When
Built-in algorithm AWS AWS Lowest Standard tabular/CV/NLP problem; only data + HPs to supply
Script Mode (framework estimator) Yours AWS DLC Low Custom architecture/loss/loop in a supported framework
BYOC (bring your own container) Yours Yours (Amazon ECR) Highest Unsupported framework, exotic OS-level dependency, or pinned proprietary stack

Exam rule: Prefer Script Mode over BYOC whenever the framework is supported. "Team does not want to build or maintain container images" ⇒ Script Mode is the answer. BYOC is the distractor for "custom code" — custom code does not require a custom image.

Supported Frameworks & Their Estimators

Framework Estimator Family Notes
PyTorch PyTorch Most common MLA-C02 target; pairs with torch_distributed / smdistributed
TensorFlow TensorFlow Supports native TF distribution strategies
Hugging Face HuggingFace Requires three version pins: transformers_version + (pytorch_version OR tensorflow_version); HF Trainer/TrainingArguments honored
Scikit-learn SKLearn Tabular/classical ML; also the generic "plain Python" container
XGBoost XGBoost Available both as built-in algorithm and script mode
MXNet / Chainer MXNet Legacy; low exam weight

Gotcha: Built-in algorithms can run in script mode internally while still shipping their own script. Do not blur "built-in algorithm" with "script mode" — the discriminator is who writes the training logic.

Configuration Surface (the parameters that get tested)

Parameter Purpose Exam Note
entry_point The Python training script The definition of script mode
source_dir Folder uploaded to /opt/ml/code Where helpers + requirements.txt live
requirements.txt Extra pip deps installed at job start Adds startup latency; avoid by using DLC-provided libs
framework_version + py_version Selects the exact DLC Must be a published pair; unsupported combos fail at job creation
hyperparameters Dict passed to the job Delivered as JSON → script must argparse them
metric_definitions Regex/JSON to scrape stdout Required for AMT and early stopping
instance_type / instance_count Compute shape count > 1 needs a distribution strategy
role Execution IAM role Needs S3 (and ECR/ECR-pull if BYOC)
output_path / checkpoint_s3_uri Artifacts / mid-run checkpoints Checkpointing enables spot resumption
use_spot_instances + max_wait Managed spot training Cost lever; needs checkpoints to be safe
subnets + security_group_ids VPC placement Required for private jobs
volume_size EBS for the job Under-sizing is a classic training failure

Container Contract & Environment (what the DLC expects)

  • Directory contract: /opt/ml/input/data/<channel> (training data), /opt/ml/model (final artifact → tarred to S3), /opt/ml/output/data, /opt/ml/code.
  • Environment variables: SM_MODEL_DIR, SM_CHANNEL_<NAME>, SM_OUTPUT_DATA_DIR, SM_NUM_GPUS, SM_HOSTS, SM_CURRENT_HOST, SM_HPS — use these instead of hardcoded paths.
  • SageMaker Training Toolkit is the open-source shim inside every DLC that reads /opt/ml/input/config/hyperparameters.json and forwards HPs as CLI args.
  • Local mode (local / local_gpu): same script, runs in Docker on the notebook instance — for fast iteration only, never for production scale.

Gotcha: A script that ignores argparse will run with default hyperparameters and silently produce a mediocre model. Tuning jobs will "succeed" while optimizing nothing.

Metrics, Hyperparameters, and Tuning Hand-off

  • Hyperparameters are a search problem, not a computed value. AMT runs many training jobs across declared ranges and keeps the best objective-metric score.
  • Objective direction matters: minimize MSE/ RMSE/loss; maximize AUC/accuracy/F1. A tuning job pointed the wrong way optimizes toward a worse model.
  • Early stopping shortens a single run; warm start reuses a previous search; adding instances shortens elapsed time only while communication stays cheaper than the work saved.
  • Fundamental HPs to reason about: epochs (passes over data), batch size (records per weight update), learning rate (step size — governs whether the run settles at all).

Performance & Training-Time Reduction

Technique Mechanism Trade-off
Data parallelism Replicate model per instance, split each batch, sync gradients Near-linear speedup when model fits on one device
Model parallelism Partition the model itself across devices For models too large for one device's memory
Managed spot training Discounted capacity + checkpoint resume Requires checkpoint_s3_uri; interruption risk
Warm pools / keep-alive Reuse instances across jobs Cuts repeated provisioning/provisioning-pip time
Fast File mode / Pipe mode Stream S3 data instead of full download Reduces startup on large datasets
Amazon FSx for Lustre High-throughput shared filesystem For very large or multi-epoch datasets
Right-size instance family GPU/accelerated compute, larger EBS volume Cost

Gotcha: Distributed training only helps if the model fits. If the model exceeds single-device memory, data parallelism will OOM — the answer is model parallelism (or sharded data parallel).

Security, Compliance & Networking

  • IAM execution role is assumed by the training container: scope it to the specific S3 bucket/prefix, plus ECR pull and CloudWatch Logs.
  • Encryption: S3 SSE-KMS at rest; encrypt_inter_container_traffic=True for inter-node traffic in transit.
  • VPC-only jobs: supply subnets + security_group_ids and an S3 gateway VPC endpoint — otherwise the job cannot reach training data.
  • Network isolation (enable_network_isolation=True): blocks outbound internet — requirements.txt installs will fail unless dependencies are pre-baked or vendored into source_dir.
  • CloudWatch Logs is where the loss curve lives; a stalled curve is the first diagnostic for overfitting/plateau.

Decision Matrix: Which Route / Which Lever

Situation Correct Move
Custom PyTorch loop, no Docker ownership desired Script Mode + PyTorch estimator
Standard tabular classification, no custom code Built-in algorithm (e.g., XGBoost, Linear Learner)
Unsupported framework or OS-level dependency BYOC
Training scores high, validation low Overfitting → regularization, dropout, early stopping, more data
Both scores low Underfitting → more capacity/epochs, higher LR
Fine-tune degrades prior general capability Catastrophic forgetting → blend broader data into the mixture
Job converges but takes too long, fits on one instance Data parallelism
Job OOMs on one instance Model parallelism
Tuning job can't find a best model Missing/misconfigured metric_definitions

Anti-Patterns & Distractor Traps

  • Rebuilding a DLC to add one pip package → use requirements.txt in source_dir instead.
  • Assuming the service picks an algorithm → that is Autopilot (AutoML), not script mode.
  • Believing script mode fixes hyperparameters → HPs remain fully passable and tunable.
  • Believing script mode silences metrics → stdout/stderr stream to CloudWatch automatically.
  • Choosing BYOC for "custom code" → custom logic is the entire point of script mode.
  • Baking training data into the image → data belongs in S3 channels, not the container.
  • Mismatched framework_version/py_version → job creation fails before any compute runs.