Skip to content

Skill 2.2.7: Adjust fundamental hyperparameters (for example, epoch, steps, batch size)

Question 1 - Epoch Selection for Overfitting Prevention

A financial services company is training a fraud detection model using Amazon SageMaker's built-in XGBoost algorithm. The model achieves 0.98 AUC on the training set but only 0.72 AUC on the validation set after running for 300 epochs. The data science team wants to prevent overfitting while minimizing training costs and avoiding unnecessary compute time.

Which hyperparameter adjustment should the team make to address this generalization gap? (Select ONE)

  • A) Increase the number of epochs to 500 to allow the model more opportunities to learn generalizable patterns.
  • B) Decrease the number of epochs and enable early stopping with a validation metric to halt training when performance plateaus.
  • C) Increase the learning rate to make larger weight updates per epoch, reducing the total epochs needed.
  • D) Increase the batch size so the model sees more data at once and learns more stable patterns.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

  • B is Correct: The 0.26 gap between training AUC (0.98) and validation AUC (0.72) is a clear symptom of overfitting. Reducing epochs limits the model's exposure to training-specific noise, while early stopping automatically halts training when the validation metric stops improving—addressing both the generalization problem and the cost/time constraint.

Distractor Analysis

  • A is Incorrect: Increasing epochs would exacerbate overfitting by giving the model more passes to memorize training data specifics rather than learning generalizable features.
  • C is Incorrect: A higher learning rate typically increases the risk of instability and overshooting optimal weights—it does not directly address the generalization gap and may worsen overfitting.
  • D is Incorrect: While larger batch sizes can sometimes regularize through gradient averaging, they do not directly address overfitting and may require more memory, potentially increasing infrastructure costs.

Key Takeaway: When training performance greatly exceeds validation performance, reduce epoch count and enable early stopping—this directly targets overfitting while minimizing unnecessary training time.

Question 2 - Batch Size and Memory Constraints

A retail analytics startup is training an image classification model on product photos using Amazon SageMaker with a PyTorch deep learning container. The training job runs on a single ml.g5.2xlarge instance (single GPU with 24 GB memory). The current batch size is 256, and the job frequently fails with out-of-memory errors during forward passes. The team wants to resolve this without modifying the model architecture.

Which adjustment should the engineer make first? (Select ONE)

  • A) Increase the batch size to 512 to reduce the number of backward passes and memory overhead.
  • B) Decrease the batch size to 64 to reduce peak GPU memory consumption per forward pass.
  • C) Switch from data parallelism to model parallelism to distribute weights across devices.
  • D) Increase the learning rate proportionally to compensate for the memory issue.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

  • B is Correct: Batch size directly scales with GPU memory usage—larger batches mean more activations and intermediate tensors stored simultaneously during forward and backward passes. Reducing batch size from 256 to 64 is the most direct, zero-additional-cost fix for out-of-memory errors without changing architecture or infrastructure.

Distractor Analysis

  • A is Incorrect: Increasing batch size increases memory consumption further, making the OOM condition worse.
  • C is Incorrect: Model parallelism is for models too large to fit on a single device at all—not for batch-induced memory spikes. Moreover, it adds complexity (pipeline partitioning, communication overhead) that violates the simplicity implied here.
  • D is Incorrect: Learning rate has no bearing on GPU memory usage; it governs weight update magnitude, not tensor storage footprint.

Key Takeaway: Out-of-memory on a single GPU is typically caused by batch size exceeding device memory—reduce batch size as the first remediation step.

Question 3 - Steps vs. Epochs in Distributed Training

A healthcare technology firm is fine-tuning a large language model on clinical notes using Amazon SageMaker distributed training with data parallelism across 8 instances. The dataset contains 1,000,000 training examples. The team sets batch_size=128 globally, meaning each of the 8 instances processes a batch of 128 examples per step. The engineer is unsure whether the tuning configuration should use "steps" or "epochs."

What is the correct relationship between total training steps and epochs in this distributed setup? (Select ONE)

  • A) One epoch equals 1,000,000 / 128 steps, because batch size is a global value.
  • B) One epoch equals 1,000,000 / (128 × 8) steps, because each step processes one batch per instance.
  • C) One epoch equals exactly 8 steps, because there are 8 instances.
  • D) One epoch equals the number of steps where the validation loss stops improving.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

  • B is Correct: In data-parallel distributed training, each instance (worker) processes its own local batch of 128 examples per step. With 8 workers, each step consumes 128 × 8 = 1,024 examples. Therefore, one full epoch (one complete pass over all 1,000,000 examples) requires 1,000,000 / 1,024 ≈ 977 steps.

Distractor Analysis

  • A is Incorrect: This formula assumes no distribution—equivalent to a single-instance scenario. It ignores that 8 workers each process 128 examples per step simultaneously.
  • C is Incorrect: The number of steps per epoch is determined by examples-per-epoch divided by total-epoch-batch-size, not by the raw instance count.
  • D is Incorrect: Early stopping determines when training terminates, not the mathematical steps-to-epochs relationship.

Key Takeaway: In distributed training, steps-per-epoch equals total examples divided by (batch_per_worker × num_workers); batch size is per-device unless specified otherwise.

Question 4 - Learning Rate Effectiveness

A logistics company is training a regression model to predict delivery delays using Amazon SageMaker's built-in Linear Learner. After the first few training jobs, the team observes the training loss oscillates widely and never converges. The current hyperparameters are: learning_rate=0.5, batch_size=64, epochs=50, and optimizer=sgd.

Which hyperparameter adjustment would most directly stabilize convergence? (Select ONE)

  • A) Increase the learning rate to 1.0 to accelerate convergence and escape local minima.
  • B) Decrease the learning rate to 0.01 to reduce the step size per weight update.
  • C) Increase the batch size to 512 to reduce gradient noise.
  • D) Increase the number of epochs to 100 to give the model more chances to settle.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

  • B is Correct: A learning rate of 0.5 is extremely high for SGD-based optimization and will cause the loss to diverge or oscillate by overshooting minima. Reducing to 0.01 (or lower) shrinks the weight update per step, enabling stable convergence.

Distractor Analysis

  • A is Incorrect: Increasing learning rate from an already-too-high value will amplify weight overshoot and further destabilize loss.
  • C is Incorrect: While larger batch sizes may reduce per-step gradient noise, they do not fix a fundamentally excessive learning rate. In fact, with larger batch, the effective learning rate per example ratio changes, but raw 0.5 remains too aggressive.
  • D is Incorrect: More epochs with an unstable learning rate will result in continued oscillation; convergence requires reducing step size, not increasing iterations.

Key Takeaway: If training loss oscillates or diverges (not just fluctuates), the learning rate is too high—lower it first before adjusting any other hyperparameter.

Question 5 - Batch Size and Training Throughput

A video streaming company is fine-tuning a recommendation model using PyTorch in SageMaker script mode on a cluster of 4 GPU instances. The team wants to reduce total training time. They have observed that each data loading step (I/O) takes 100 ms, while the GPU forward+backward pass takes 200 ms, and gradient synchronization across 4 instances takes 75 ms per step. Current configuration: batch_size_per_worker=32.

Which adjustment would most improve training throughput? (Select ONE)

  • A) Decrease batch size per worker to 16 to reduce GPU memory and increase speed.
  • B) Increase batch size per worker to 256 to amortize data loading and synchronization overhead relative to compute.
  • C) Decrease the number of epochs to improve per-epoch completion.
  • D) Switch from data parallelism to single-instance training to eliminate synchronization overhead.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

  • B is Correct: With 32-example batches, the network/synchronization cost (75 ms) is 37.5% of compute time (200 ms), and I/O (100 ms) is 50%. Increasing batch size to 256 means more computation (proportional GPU work) happens per I/O read and per synchronization round, amortizing fixed overhead costs and improving effective throughput.

Distractor Analysis

  • A is Incorrect: Decreasing batch size further increases the relative weight of I/O and synchronization overhead, making throughput worse per sample processed.
  • C is Incorrect: Reducing epochs shortens total runtime but does not improve throughput per example—it simply does less training, reducing model quality.
  • D is Incorrect: Single-instance eliminates synchronization but reduces total compute power available; if the job was intentionally distributed, consolidating to one device likely increases total runtime.

Key Takeaway: Increase batch size when I/O or network overhead dominates per-step time; this improves throughput by amortizing fixed costs over more computation.

Question 6 - Epochs and Underfitting

A non-profit organization is training a document classification model using SageMaker's BlazingText algorithm on a large corpus of 500,000 text documents. After initial training with epochs=2, both training and validation accuracy hover at 0.68. The model is expected to reach at least 0.85 based on benchmarks. The dataset is verified to be large and diverse enough.

Which adjustment would most likely improve both training and validation accuracy? (Select ONE)

  • A) Decrease the batch size to create more frequent weight updates per epoch.
  • B) Decrease the learning rate from 0.01 to 0.001 to make finer adjustments.
  • C) Increase the number of epochs from 2 to 15 to give the model more passes over the data.
  • D) Increase the batch size to reduce gradient noise and stabilize training.
Click to reveal Answer & Explanation

Correct Answer: C

Correct Answer Analysis

  • C is Correct: Both training and validation accuracy are stuck at 0.68—this is a classic underfitting symptom. Two epochs are insufficient for the model to learn the underlying signal from 500,000 documents. Increasing to 15 epochs (or using a validation_accuracy-driven early stopping threshold) gives the model more learning passes.

Distractor Analysis

  • A is Incorrect: Decreasing batch size increases gradient noise and more frequent updates, but with only 2 epochs, the model will still under-learn the dataset; the problem is insufficient passes, not update frequency.
  • B is Incorrect: Lowering learning rate for an underfitting model slows learning further and may make underfitting worse; the correct direction is either more epochs or higher learning rate.
  • D is Incorrect: Larger batch size may stabilize gradients but does not increase the number of learning passes—2 epochs with a 512-batch still only goes through the dataset twice.

Key Takeaway: When both training and validation metrics are unacceptably low, underfitting is the culprit—increase epochs or model capacity before tuning for stability.

Question 7 - Warm Starting Hyperparameter Tuning

A series of failed SageMaker Automatic Model Tuning (AMT) jobs has caused a team at an e-commerce company to run out of tuning budget. They need to run a new AMT job with a slightly different search range for learning_rate and batch_size. Previous tuning jobs covered the ranges learning_rate [0.001–0.1] and batch_size [32–256]; the new job needs to search learning_rate [0.0005–0.01] and batch_size [64–512].

Which AMT feature should the team enable to reduce total tuning time? (Select ONE)

  • A) Warm start with the previous tuning job's metadata to avoid re-evaluating combinations that were already tested.
  • B) Disable model tuning and manually pick the best hyperparameters.
  • C) Run the new AMT job with fewer maximum parallel training jobs.
  • D) Increase the total number of training jobs in the new search to ensure coverage.
Click to reveal Answer & Explanation

Correct Answer: A

Correct Answer Analysis

  • A is Correct: SageMaker AMT supports warm start , which uses results from a prior tuning job (or jobs) to seed the Bayesian optimization search, preventing time and budget being wasted re-testing already-explored hyperparameter ranges.

Distractor Analysis

  • B is Incorrect: Manual hyperparameter selection defeats the purpose of AMT and would not meet the team's intention of automated search.
  • C is Incorrect: Reducing parallelism slows the tuning job's total runtime; it does not efficiently reuse previous knowledge.
  • D is Incorrect: Increasing the total number of training jobs without warm start wastes budget on re-evaluating already-known configurations.

Key Takeaway: When previous AMT results exist, use warm start to seed new search ranges—it reuses past tuning outcomes and avoids re-exploration.

Question 8 - Batch Size and Generalization Trade-off

A research team is training a computer vision model on a synthetic dataset of satellite images using SageMaker's Image Classification built-in algorithm. They currently use batch_size=8, which yields high validation accuracy but produces noisy gradient estimates in training. They want to maintain high generalization performance while smoothing gradient steps.

What is the best approach for adjusting batch size? (Select ONE)

  • A) Gradually increase batch size (e.g., 8 → 16 → 32) while monitoring validation accuracy for degradation.
  • B) Jump directly to a batch size of 512 to eliminate gradient noise immediately.
  • C) Decrease batch size to 2 to create more stochastic exploration and improve final accuracy.
  • D) Keep batch size at 8 and set early_stopping=True without any other change.
Click to reveal Answer & Explanation

Correct Answer: A

Correct Answer Analysis

  • A is Correct: Batch sizes of 8 are typical in research but may be too small for production. However, exploding to 512 can harm generalization (sharp minimas). A gradual increase (progressive batch size scaling) is a recognized technique: smooth gradients while watching for validation accuracy drop, and stop increasing if metrics degrade.

Distractor Analysis

  • B is Incorrect: Jumping to 512 creates very stable gradients but tends to converge to sharp minima that generalize worse; a sudden dramatic increase often causes accuracy drops.
  • C is Incorrect: Decreasing to 2 further reduces batch-normalization efficiency and may magnify gradient noise; it offers no guaranteed improvement.
  • D is Incorrect: Early stopping does not change gradient noise levels; it only halts training when validation metrics plateau.

Key Takeaway: Increase batch size incrementally while tracking validation accuracy—large abrupt increases often converge to sharper, less-generalizable minima.

Question 9 - Batch Size and Steps Interplay

A machine learning team is using PyTorch with Amazon SageMaker to train a deep neural network on 5,000 labeled examples. They set epochs=100 and batch_size=50, giving 100 steps per epoch. After monitoring, they realize training is excessively slow, and each epoch takes 3 minutes. They decide to change to batch_size=100 to halve the steps per epoch.

What effect does this change have on total training time if they keep epochs=100? (Select ONE)

  • A) Training time decreases proportionally because there are half as many steps per epoch.
  • B) Training time remains roughly the same because the number of epochs is unchanged.
  • C) Training time increases because each step processes twice as many samples.
  • D) Training time is halved regardless of dataset size.
Click to reveal Answer & Explanation

Correct Answer: A

Correct Answer Analysis

  • A is Correct: With 5,000 examples and batch_size=100, there are 50 steps per epoch instead of 100. If each step's compute time stays approximately constant, each epoch runs in fewer GPU steps, reducing total training time proportionally (approximately 50% less).

Distractor Analysis

  • B is Incorrect: While the number of epochs is unchanged, the amount of work per epoch is reduced, so total time decreases.
  • C is Incorrect: Processing more samples per step increases per-step time, but the total data processed across an epoch is identical—only the number of optimization steps changes.
  • D is Incorrect: Halving steps per epoch reduces epoch-level time, but total training time depends on per-step compute and I/O; it’s not automatically halved for every dataset.

Key Takeaway: Increasing batch size reduces steps per epoch; if epoch count stays constant and per-step time does not scale linearly with batch size, total training time shrinks.

Question 10 - Early Stopping Configuration for SageMaker

A start-up is training a multilabel classification model on Amazon SageMaker using a TensorFlow framework container. They need to configure early stopping to halt training when the validation macro_f1_score has not improved for 10 consecutive epochs. The estimator has an early_stopping=True flag but the team also needs to specify the metric and patience.

Where should the team set these values in SageMaker script mode? (Select ONE)

  • A) In the estimator object as metric_definitions and early_stopping_mini_batch_window arguments.
  • B) In the training script's training loop, using a callback or manual check that calls stop_training when the metric plateaus.
  • C) In a separate SageMaker Automatic Model Tuning job that does not use early stopping.
  • D) In the model_fn function to stop initialization when the metric is low.
Click to reveal Answer & Explanation

Correct Answer: A

Correct Answer Analysis

  • A is Correct: In SageMaker script mode, early stopping is configured via the estimator's metric_definitions (which tells SageMaker which metric to track and its direction) and early_stopping_mini_batch_window (which specifies how many recent mini-batch evaluations to average to decide if the metric has plateaued).

Distractor Analysis

  • B is Incorrect: While manually calling stop_training from within the training script is possible, it does not leverage SageMaker's managed early stopping feature; the question asks for estimator-level configuration.
  • C is Incorrect: Automatic Model Tuning has its own early stopping (early_stopping_type), but that is for tuning jobs, not a standard single training job.
  • D is Incorrect: model_fn is part of the inference serving script and is never called during training—it has no role in early stopping configuration.

Key Takeaway: SageMaker early stopping requires both metric_definitions and early_stopping_mini_batch_window in the estimator object; the training script itself only needs to emit the metric to stdout/logs.