Skip to content

Skill 1.1.1: Extract data from data sources

Question 1 - Distributed Training: Data Parallelism vs Model Parallelism

A financial services company is training a large transformer-based fraud detection model on Amazon SageMaker AI. The model has 8 billion parameters and does not fit into the memory of a single ml.p4d.24xlarge instance. Additionally, the training dataset contains 40 TB of transaction data, and the team needs to complete training within 48 hours to meet regulatory compliance deadlines.

The ML engineering team wants to reduce both memory constraints and overall training time using SageMaker's distributed training libraries.

Which TWO strategies should the team implement to satisfy these requirements? (Select TWO)

  • A) Use SageMaker Model Parallelism (SMP) to partition the model across multiple GPUs within a single instance using pipeline parallelism.
  • B) Use SageMaker Data Parallelism (SDP) alone, splitting each mini-batch across multiple instances with AllReduce gradient synchronization.
  • C) Combine SageMaker Model Parallelism (SMP) across multi-instance clusters with SageMaker Data Parallelism (SDP) to shard both the model and the dataset.
  • D) Increase the number of epochs from 3 to 10 to ensure convergence before the 48-hour deadline.
  • E) Use Amazon SageMaker Automatic Model Tuning (AMT) to find the optimal batch size for distributed training.
Click to reveal Answer & Explanation

Correct Answers: A and C

Correct Options Analysis

  • A is Correct: SageMaker's Model Parallelism (SMP) library supports pipeline parallelism, which partitions transformer layers or tensor operations across GPUs within an instance, directly addressing the memory limitation when the model exceeds single-GPU capacity.
  • C is Correct: For very large models and massive datasets, combining SageMaker Distributed Data Parallel (SDP) with SageMaker Model Parallel (SMP) enables sharding both the dataset across nodes and the model across GPUs, optimizing both throughput and memory usage simultaneously.

Distractor Analysis

  • B is Incorrect: Data parallelism alone replicates the entire model on each worker. If the model doesn't fit in a single GPU's memory, SDP cannot solve the memory constraint—it only splits data, not the model.
  • D is Incorrect: Increasing epochs increases total training time directly, violating the 48-hour deadline constraint rather than helping meet it.
  • E is Incorrect: Hyperparameter tuning finds optimal settings but does not reduce memory footprint or fundamentally alter how the model and data are distributed across infrastructure.

Key Takeaway: When a model exceeds a single device's memory, use model parallelism; when the dataset is too large for one worker to process quickly, use data parallelism; combine both when both constraints exist simultaneously.

Question 2 - Early Stopping in SageMaker Training

A retail company is training a product recommendation model using Amazon SageMaker AI with a PyTorch script in Script Mode. The model has shown a history of overfitting during previous training runs. The ML engineer wants to automatically stop training when validation accuracy stops improving for a sustained period, minimizing wasted compute resources.

The validation metric is logged to CloudWatch at the end of each epoch. The engineering team wants minimal operational overhead.

Which solution should the engineer implement? (Select ONE)

  • A) Manually monitor the CloudWatch dashboard and terminate the training job when validation accuracy plateaus.
  • B) Modify the training script to implement a custom early stopping callback that monitors validation loss and calls sagemaker.estimator.Estimator.stop_training_job() when no improvement is seen for a configured number of epochs.
  • C) Set up an AWS CloudWatch Alarm on the validation accuracy metric with a LessThanLowerOrGreaterThanUpperThreshold threshold action to invoke a Lambda function that stops the SageMaker training job.
  • D) Use SageMaker Automatic Model Tuning (AMT) with early_stopping_type set to Auto and define a max_num_jobs limit.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

  • B is Correct: Implementing a custom early stopping callback within the training script (using PyTorch Lightning's EarlyStopping callback, TensorFlow tf.keras.callbacks.EarlyStopping, or a manual PyTorch loop) directly checks the validation metric after each epoch and terminates training via the SageMaker training job API when the metric plateaus. This is the least overhead solution for script-mode training, as the logic runs natively within the training loop.

Distractor Analysis

  • A is Incorrect: Manual monitoring is reactive, introduces human error, and wastes compute resources since a human may not notice the plateau in real-time. It also violates the "minimal operational overhead" requirement.
  • C is Incorrect: CloudWatch Alarms have a minimum evaluation period (typically 1 minute) but cannot elegantly interrupt SageMaker training mid-epoch; setting up a Lambda to stop training based on alarm thresholds is more complex and introduces unnecessary infrastructure compared to a native callback.
  • D is Incorrect: AMT's early_stopping_type=Auto is designed for tuning jobs—it stops launching new tuning evaluation jobs when a search space isn't yielding improvement, not to stop individual training jobs early. Using a full tuning job when you're not searching hyperparameters is overkill.

Key Takeaway: Early stopping is implemented within the training script itself (not via external alarms) for Script Mode; it halts a single run when validation performance plateaus, directly reducing wasted training time.

Question 3 - Data Parallel Training with SageMaker DDP

An e-commerce company is using Amazon SageMaker AI to train a recommendation system with a ResNet50 model on a dataset of 10 million product images. The model fits comfortably in a single ml.g4dn.xlarge GPU instance's memory, but training takes 72 hours on a single instance, which is too slow for the team's weekly retraining cycle.

The team wants to reduce training time to under 12 hours while maintaining minimal code changes and using the built-in SageMaker distributed training capabilities.

Which approach should the team use? (Select ONE)

  • A) Use SageMaker Data Parallelism (SDP) with the distribution parameter set to {"smdistributed": {"dataparallel": {"enabled": True}}}, and let SageMaker handle gradient synchronization across multiple ml.g4dn.xlarge instances.
  • B) Use SageMaker Model Parallelism (SMP) to split the ResNet50 architecture across multiple GPUs, even though the model fits on one GPU.
  • C) Rewrite the training script to use Horovod with MPI for custom gradient synchronization across instances.
  • D) Use SageMaker Pipeline Mode to stream data directly from S3 without downloading it to local storage, reducing per-epoch I/O time.
Click to reveal Answer & Explanation

Correct Answer: A

Correct Options Analysis

  • A is Correct: Since the model fits entirely within a single GPU's memory, the bottleneck is data throughput. SageMaker Data Parallelism (SDP) keeps the model replicated on each worker, splits the dataset across workers, and uses the efficient AllReduce gradient synchronization algorithm. Setting distribution={"smdistributed": {"dataparallel": {"enabled": True}}} requires minimal code changes—sometimes just one line.

Distractor Analysis

  • B is Incorrect: Model parallelism is specifically for models that do not fit on a single device. Splitting ResNet50 across devices when it already fits wastes inter-device communication and adds unnecessary complexity, potentially lowering throughput.
  • C is Incorrect: While Horovod works, integrating it manually introduces significant code changes (importing Horovod, modifying optimizer, syncing gradients manually) compared to SageMaker's built-in data parallel library, violating the "minimal code changes" constraint.
  • D is Incorrect: Pipeline Mode is a legacy feature that only works with certain built-in algorithms and file formats (protobuf, TFRecord). It reduces I/O but doesn't parallelize compute across multiple GPUs—it does not solve the core training speed bottleneck.

Key Takeaway: When the model fits on a single device, use data parallelism (SDP) to spread the dataset across multiple instances; this is the lowest-effort path to scaling training speed in SageMaker.

Question 4 - Hyperparameter Tuning Best Practices

A healthcare startup is building a XGBoost model with SageMaker AI to predict patient readmission risk from tabular clinical data. The team wants to find the optimal combination of max_depth, eta, subsample, and num_round that maximizes the ROC-AUC score. They are working with a fixed budget and must control costs.

The team decides to use SageMaker Automatic Model Tuning (AMT).

Which configuration should they use? (Select ONE)

  • A) Run a single large training job with max_depth=10, eta=0.01, subsample=0.5, and num_round=1000, ensuring convergence by manual inspection.
  • B) Define a tuning job with Bayesian optimization, specify objective_metric_name="validation:auc" with objective_type="Maximize", and set max_jobs=50 and max_parallel_jobs=5 to bound the search budget.
  • C) Use Grid Search over all possible combinations of hyperparameters to guarantee finding the absolute best configuration, regardless of job count.
  • D) Use random search with no resource limits, allowing AMT to run indefinitely until it finds the best combination.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

  • B is Correct: AMT uses Bayesian optimization to intelligently explore the search space, evaluating multiple hyperparameter combinations across parallel training jobs. Setting objective_metric_name="validation:auc" with Maximize ensures the correct metric direction is optimized. max_jobs=50 and max_parallel_jobs=5 provide budget control and resource utilization balance.

Distractor Analysis

  • A is Incorrect: A single training job does not perform hyperparameter tuning. Without search, there's no mechanism to determine whether the chosen hyperparameters are optimal for maximizing AUC.
  • C is Incorrect: Grid search exhaustively tests all combinations, which becomes computationally prohibitive with 4 hyperparameters across continuous ranges. AMT's Bayesian optimization finds near-optimal solutions with significantly fewer jobs. Grid search violates the fixed budget constraint.
  • D is Incorrect: AMT always requires users to specify a completion criterion (max_jobs or max_time). "Running indefinitely" is not a valid configuration and would result in uncontrolled costs.

Key Takeaway: In AMT, always set a numeric objective metric with the correct direction (Minimize for loss, Maximize for scores like AUC/recall) and bound the search with max_jobs/max_parallel_jobs to control cost.

Question 5 - Early Stopping vs Warm Starting

An ML engineering team at a logistics company has successfully completed a hyperparameter tuning job using SageMaker Automatic Model Tuning for a XGBoost delivery-time prediction model. Recently, they received a new delivery region's dataset that has a similar statistical distribution to the original training data. The team wants to tune hyperparameters for a model on this new data without starting the search from scratch, leveraging results from the previous tuning job.

Which technique should the engineer implement? (Select ONE)

  • A) Enable early stopping in the new tuning job to stop individually poor training jobs early.
  • B) Use warm start by referencing the previous tuning job's name in the warm_start_config parameter when creating the new AMT job.
  • C) Re-run the exact same tuning job with the original data to confirm the best hyperparameters before using the new dataset.
  • D) Use transfer learning to copy the best hyperparameter values directly to the new model, skipping the tuning process entirely.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

  • B is Correct: SageMaker AMT supports warm starting, where a new tuning job begins from the hyperparameter combinations that performed well in a previously completed tuning job. This dramatically reduces the time and number of jobs needed to find optimal settings for similar datasets. You set this via warm_start_config with the previous job's tuning_job_name.

Distractor Analysis

  • A is Incorrect: Early stopping shortens a single training run when validation metrics plateau. It does not leverage historical tuning search information across different tuning jobs.
  • C is Incorrect: Re-running the old tuning job on the original dataset wastes compute and provides no new information about optimal hyperparameters on the new region's data.
  • D is Incorrect: Transfer learning refers to reusing pre-trained weights from one model as initialization for another task, not to hyperparameter search. Hyperparameters cannot be "transferred"—they must be tuned per dataset, though warm start intelligently reuses search history.

Key Takeaway: Use warm start (warm_start_config) in AMT to reuse prior search knowledge when training on similar datasets, reducing the number of tuning jobs and overall search cost; early stopping shortens individual runs, not the search itself.

Question 6 - Distributed Training: Choosing the Right Strategy

A research institute is training a large language model (LLM) with 175 billion parameters on Amazon SageMaker AI using model parallelism. The entire model requires multiple instances, and parts of the model must be split across GPUs because no single device can hold even a single transformer block's parameters.

The team must choose between pipeline parallelism and tensor parallelism in SageMaker Model Parallelism (SMP).

Which statement is TRUE about these two parallelism strategies? (Select ONE)

  • A) Pipeline parallelism partitions individual weight matrices (layers) across devices; tensor parallelism splits the sequence of layers into stages processed sequentially.
  • B) Pipeline parallelism splits the model layer-by-layer across devices with micro-batches flowing through the stages; tensor parallelism splits individual tensor operations (e.g., matrix multiplications) across multiple devices.
  • C) Both pipeline and tensor parallelism use AllReduce gradient synchronization after each step.
  • D) Tensor parallelism is used only in data parallelism mode; pipeline parallelism is exclusive to model parallelism mode.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

  • B is Correct: Pipeline parallelism divides the model by layers (or blocks) into stages, with each device holding one stage and processing data sequentially. Tensor parallelism splits individual operations (e.g., a single large matrix multiplication) across multiple devices simultaneously, reducing memory footprint per device for very large tensors.

Distractor Analysis

  • A is Incorrect: This option reverses the definitions. Pipeline parallelism splits the layer sequence vertically; tensor parallelism partitions tensor operations within a layer horizontally.
  • C is Incorrect: SDP (Data Parallel) uses AllReduce gradient synchronization across model replicas; SMP (Model Parallel) uses point-to-point communication between adjacent pipeline stages or tensor slices, not a collective AllReduce across all workers.
  • D is Incorrect: Tensor parallelism and pipeline parallelism are both exclusively model-parallel techniques. Data parallelism is a separate paradigm where the entire model is replicated across workers.

Key Takeaway: Pipeline parallelism splits layers across devices sequentially; tensor parallelism splits individual operations across devices simultaneously—choose based on whether the model is deep (pipeline) or has very wide weight matrices (tensor).

Question 7 - Distributed Training Failure Diagnosis

An ML team at a large social media company is training a large BERT-based model using SageMaker Data Parallelism (SDP). The model fits on each individual ml.p3dn.24xlarge instance. After scaling from 1 to 8 instances, the team observes that training speed increases only by 2x instead of the expected near-linear 8x speedup.

The team suspects a synchronization bottleneck.

Which change would MOST likely improve distributed training efficiency? (Select ONE)

  • A) Increase the batch size on each worker proportionally to the number of workers.
  • B) Reduce the number of instances back to 1 to eliminate communication overhead.
  • C) Increase the number of instances further, since the bottleneck likely stems from underutilized compute.
  • D) Increase the learning rate to accelerate gradient descent convergence.
Click to reveal Answer & Explanation

Correct Answer: A

Correct Options Analysis

  • A is Correct: When scaling data parallel jobs, a common error is keeping the global batch size constant, which means each worker's per-worker batch shrinks as workers increase. This leads to underutilized compute (each worker processes a tiny batch) relative to communication overhead. Increasing the per-worker batch size (or keeping the global batch size proportional to number of workers) allows each worker to do more compute per gradient synchronization cycle, improving scaling efficiency. In practice, scaling to 8 workers with the same per-worker batch size gives a global batch 8x larger, requiring a learning rate warm-up/scale-up—but it achieves near-linear speedup.

Distractor Analysis

  • B is Incorrect: Reducing to 1 instance eliminates speedup entirely; the goal is to improve distributed training, not revert to serial training.
  • C is Incorrect: If scaling from 1 to 8 already shows diminishing returns, adding more instances will worsen the communication-to-computation ratio, not improve it. The bottleneck is likely network synchronization, not underutilization.
  • D is Incorrect: Learning rate affects convergence speed, not distributed training throughput. Increasing learning rate without adjusting warm-up or batch size can cause unstable convergence.

Key Takeaway: In data parallel training, scale per-worker batch size with the number of workers to maintain a healthy compute-to-communication ratio; diminishing returns from distributed training usually signal synchronization overhead dominating worker compute.

Question 8 - Automatic Model Tuning: Early Stopping Type

A fintech company is running an Amazon SageMaker Automatic Model Tuning (AMT) job for a random forest classifier. They want to conserve compute resources during hyperparameter search by automatically stopping training jobs that are unlikely to perform well based on their objective metric (validation:accuracy).

Which AMT configuration parameter should they use, and how does it work? (Select ONE)

  • A) Set early_stopping_type to Off to prevent AMT from spending compute on unpromising jobs.
  • B) Set early_stopping_type to Auto, which causes AMT to stop individual training jobs that show poor performance relative to the best-performing job before they complete.
  • C) Set max_parallel_jobs=1 so AMT evaluates only one hyperparameter combination at a time, inherently preventing wasted compute.
  • D) Set strategy to Random to ensure only random, potentially high-performing jobs are selected, minimizing wasted resources.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

  • B is Correct: AMT's early_stopping_type=Auto (the default for many algorithms) monitors the objective metric of each running training job during its execution. If a job's intermediate performance (e.g., validation accuracy at the current checkpoint) is significantly below the best job seen so far, AMT stops the job early, saving compute resources that would have been wasted on a clearly unpromising configuration.

Distractor Analysis

  • A is Incorrect: Setting early_stopping_type=Off disables early stopping, causing every job (including poor performers) to run to completion—the opposite of what the team wants.
  • C is Incorrect: max_parallel_jobs=1 controls concurrency (how many jobs run simultaneously), not whether individual unpromising jobs are terminated early. It slows the search but does not prevent waste within individual jobs.
  • D is Incorrect: Random search still launches all jobs without early stopping unless early_stopping_type=Auto is configured; the search strategy doesn't inherently reduce per-job compute waste.

Key Takeaway: AMT's early_stopping_type=Auto saves compute within a tuning job by halting individual training jobs that fall well below the best observed performance—set it to Auto (unless you have a specific reason to disable it) to reduce tuning costs.

Question 9 - Overfitting Mitigation: Early Stopping vs Regularization

A data science team is training a neural network in Amazon SageMaker AI using PyTorch Script Mode. After the first training run, they observe that training loss continues to decrease while validation loss plateaus then starts increasing after epoch 25, while the model achieves 99% training accuracy but only 82% validation accuracy.

The team wants to prevent this pattern in the next run using the most operationally simple approach.

Which technique should they use? (Select ONE)

  • A) Increase model complexity (add layers) so the model can better represent the training patterns.
  • B) Implement early stopping with a patience of 5 epochs monitoring the validation loss to terminate training before overfitting worsens.
  • C) Increase the number of training epochs to 100 so the model can learn more general patterns from the training data.
  • D) Set the learning rate to zero after epoch 25 to prevent further weight updates.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

  • B is Correct: The observed pattern—training loss decreasing while validation loss increases—is classic overfitting. Early stopping halts training when validation loss stops improving for a configured number of epochs (patience), preventing the model from continuing to memorize training-specific patterns. This is straightforward to implement via standard callbacks in PyTorch Lightning (EarlyStopping(monitor="val_loss", patience=5)) or TensorFlow (tf.keras.callbacks.EarlyStopping).

Distractor Analysis

  • A is Incorrect: Adding layers increases model capacity, which exacerbates overfitting rather than mitigating it. The solution to overfitting is often to reduce capacity, add regularization, or stop earlier—not to increase complexity.
  • C is Incorrect: Increasing epochs gives the model more time to memorize training noise, worsening validation performance. The validation loss is already increasing; more training makes the gap wider.
  • D is Incorrect: Abruptly setting learning rate to zero is a brittle, rule-based approach that doesn't generalize across model architectures or training runs. Early stopping is a principled, widely supported technique that monitors the exact condition of interest.

Key Takeaway: When validation loss diverges from training loss (the signature of overfitting), early stopping with a patience parameter is the most operationally simple and effective remediation; increasing capacity or training duration will worsen the problem.

Question 10 - Distributed Training Library Selection

An autonomous vehicle company is using Amazon SageMaker AI for two workloads:

  1. Workload A: Training a predictive maintenance model (XGBoost) on 200 GB of sensor data.
  2. Workload B: Fine-tuning a 20-billion-parameter vision transformer that does not fit in a single ml.p4d.24xlarge instance's memory.

The team wants to accelerate both workloads using SageMaker's native distributed training options with minimal custom code.

Which TWO configurations should they use? (Select TWO)

  • A) For Workload A, use SageMaker Data Parallelism (SDP) with the XGBoost built-in container and multiple ml.m5.4xlarge instances.
  • B) For Workload A, use SageMaker Model Parallelism (SMP) to split the XGBoost tree ensemble across multiple GPUs.
  • C) For Workload B, use SageMaker Model Parallelism (SMP) with the ml.p4d.24xlarge multi-GPU instances to shard the model across devices.
  • D) For Workload B, use SageMaker Data Parallelism (SDP) only, ensuring each worker holds the full model on a single GPU by reducing the batch size.
  • E) For both workloads, disable distributed training entirely and use the largest single instance available (ml.p4de.24xlarge for Workload A and ml.p4de.24xlarge for Workload B).
Click to reveal Answer & Explanation

Correct Answers: A and C

Correct Options Analysis

  • A is Correct: XGBoost's built-in SageMaker algorithm supports data parallelism natively. With 200 GB of tabular sensor data and a model that fits easily in memory on CPU instances, data parallelism across multiple ml.m5.4xlarge instances splits the dataset, accelerates gradient-boosted decision tree training, and requires minimal additional code.
  • C is Correct: When a model (especially transformers) exceeds a single device's memory, SageMaker Model Parallelism (SMP) partitions the model across GPUs (pipeline and/or tensor parallelism), enabling training of models that otherwise cannot fit. This is the native SageMaker solution for Workload B.

Distractor Analysis

  • B is Incorrect: XGBoost is a tabular ensemble algorithm that does not benefit from model parallelism (splitting tree ensembles across GPUs). Data parallelism is the correct paradigm for tabular algorithms on large datasets.
  • D is Incorrect: Data parallelism replicates the full model on each worker. If the model does not fit on a single GPU even with reduced batch size, SDP alone is insufficient. Model parallelism is required.
  • E is Incorrect: Disabling distributed training means Workload B remains impossible to train (the model exceeds single-instance memory even on the largest instance) and Workload A remains slow. This option ignores the team's goal of acceleration and feasibility.

Key Takeaway: Match the parallelism strategy to the algorithm type: tabular built-in algorithms (XGBoost, Linear Learner) scale with data parallelism; large neural networks and foundation models exceeding device memory require model parallelism via SMP.