Skill 2.2.7: Adjust fundamental hyperparameters (for example, epoch, steps, batch size)
| Question | Answer & Explanation |
|---|---|
| ### Question 1 - Resolving Out-of-Memory Errors via Batch Size A machine learning engineer is training a large deep learning model on Amazon SageMaker using a single ml.p3.2xlarge instance. During training, the script fails with a CUDA out-of-memory error during the backward pass. The team needs to resolve this issue with the least operational overhead and without changing the underlying model architecture or provisioning new infrastructure.Which action should the engineer take to resolve the issue? - A) Switch to a larger instance type (e.g., ml.p3.8xlarge) to increase GPU memory.- B) Decrease the batch size hyperparameter so that fewer samples are loaded into memory at once. - C) Implement model parallelism using SageMaker Distributed Training Libraries. - D) Implement gradient accumulation in the custom training script to simulate a larger batch size. |
Correct Answer: B Correct Option Analysis: - B is Correct: The out-of-memory error occurs because the batch size dictates how many samples (and their corresponding intermediate activations) are held in GPU memory simultaneously. Reducing the batch size directly lowers the memory footprint of the forward and backward passes, resolving the OOM error without architectural or infrastructural changes. Distractor Analysis: - A is Incorrect: While this would solve the memory issue, it violates the constraint of not provisioning new infrastructure and introduces higher operational overhead and cost. - C is Incorrect: Implementing model parallelism is a highly complex engineering task reserved for models too large to fit in memory even with a batch size of 1. It introduces massive operational overhead compared to simply changing a hyperparameter. - D is Incorrect: Gradient accumulation keeps the memory benefits of a small batch size but requires rewriting the custom training loop to accumulate gradients over multiple forward passes before stepping the optimizer, violating the "least operational overhead" constraint. > Key Takeaway: Batch size directly correlates with peak GPU memory usage; reducing it is the simplest way to resolve OOM errors without code changes. |
| ### Question 2 - Preventing Overfitting via Epochs and Early Stopping A data science team is training an image classification model using a SageMaker built-in algorithm. After 30 epochs, the training loss continues to decrease, but the validation loss has started to increase. The team must prevent the model from memorizing the training data while minimizing compute waste. Which hyperparameter adjustment strategy should the team implement? - A) Increase the number of epochs to allow the model to find a global minimum. - B) Decrease the batch size to introduce more noise into the gradient updates. - C) Reduce the maximum number of epochs or enable early stopping to halt training when the validation metric degrades. - D) Increase the learning rate to force the model to jump out of the overfitting local minimum. |
Correct Answer: C Correct Option Analysis: - C is Correct: The scenario describes classic overfitting (training loss decreasing while validation loss increases). By reducing the max epochs or using early stopping, the training job will terminate when the validation metric stops improving, preventing wasted compute and further memorization of the training data. Distractor Analysis: - A is Incorrect: Increasing epochs will exacerbate the overfitting problem, leading to worse generalization and more wasted compute. - B is Incorrect: While batch size affects gradient noise, simply decreasing it will not stop the overfitting process over 30+ epochs and will increase training time per epoch. - D is Incorrect: Drastically changing the learning rate mid-training to escape a minimum is not a standard, reliable hyperparameter adjustment strategy and will likely cause divergence (loss spikes) rather than solving overfitting. > Key Takeaway: A divergence between training and validation loss indicates overfitting; early stopping is the most direct way to halt this behavior and save compute resources. |
| ### Question 3 - Configuring Training Duration for Streaming Data A financial services company is training a fraud detection model on a continuously growing dataset that is streamed via Amazon Kinesis Data Firehose to Amazon S3. The data science team is using Amazon SageMaker Pipe mode to train directly from the stream. Because the stream is continuous, the dataset size is effectively infinite. They need to specify the training duration for this job. Which fundamental hyperparameter configuration is most appropriate for this scenario? - A) Configure the training job to run for a specific number of epochs. - B) Configure the training job to run for a specific number of steps. - C) Increase the batch size to process more data per second. - D) Set the learning rate to decay based on the volume of data ingested. |
Correct Answer: B Correct Option Analysis: - B is Correct: In scenarios involving streaming data or infinitely large datasets where an "epoch" (one full pass over the data) is impossible or undefined, training duration is configured using "steps". A step represents one weight update (processing one batch). Setting a max step count allows the model to train for a specific amount of compute/work regardless of total dataset size. Distractor Analysis: - A is Incorrect: An epoch requires a finite dataset to complete a full pass. With a continuous, infinite stream, an epoch can never complete, making this metric invalid. - C is Incorrect: Adjusting the batch size changes the amount of data processed per step, but it does not define the overall termination criteria for the training job on an infinite stream. - D is Incorrect: Learning rate decay schedules can be tied to steps, but setting the decay schedule alone does not define the total training duration or when the job should terminate. > Key Takeaway: Use "steps" to define training duration on continuous or infinite datasets where calculating a full "epoch" is not possible. |
| ### Question 4 - Scaling Learning Rate with Batch Size An ML engineering team is scaling up a training job to use multiple GPUs on a single SageMaker instance. To fully utilize the GPUs, they increase the global batch size from 32 to 256. However, the model fails to converge and the loss spikes early in training. They need to adjust fundamental hyperparameters to stabilize training while keeping the larger batch size for throughput. Which adjustment is most appropriate? - A) Decrease the learning rate significantly to compensate for the larger batch size. - B) Scale the learning rate linearly with the batch size or use a learning rate warmup strategy. - C) Increase the number of epochs to allow the model more time to recover from the loss spike. - D) Reduce the batch size back to 32 and increase the number of data parallel workers. |
Correct Answer: B Correct Option Analysis: - B is Correct: When scaling batch size up significantly, the gradient variance decreases, meaning the updates are more stable but fewer per epoch. If the learning rate remains the same, the model may diverge because the larger, smoother updates push it too far. The standard practice (e.g., Linear Scaling Rule) is to scale the learning rate proportionally to the batch size, often combined with a warmup phase to stabilize early training. Distractor Analysis: - A is Incorrect: Significantly decreasing the learning rate will cause the model to train too slowly and likely underfit within the given epochs. The goal is to scale it appropriately, not just shrink it. - C is Incorrect: Increasing epochs will not solve the divergence issue (loss spikes); the model has already destabilized and requires a learning rate adjustment. - D is Incorrect: This violates the scenario constraint of keeping the larger batch size to fully utilize the GPUs for throughput. > Key Takeaway: When increasing batch size, the learning rate often needs to be scaled up (e.g., linearly) to maintain convergence speed and stability. |
| ### Question 5 - Escaping Local Minima via Gradient Noise A team is training a custom PyTorch script in SageMaker Script Mode. The model is getting stuck at a suboptimal local minimum, resulting in high bias on the training set. The team wants to add more noise to the gradient updates to help the model escape the local minimum and find a better solution. Which fundamental hyperparameter adjustment directly achieves this goal? - A) Increase the batch size to stabilize the gradients. - B) Decrease the batch size to increase the variance of the gradient updates. - C) Increase the number of steps per epoch. - D) Decrease the learning rate to take smaller steps. |
Correct Answer: B Correct Option Analysis: - B is Correct: A smaller batch size means the gradient is calculated on a smaller subset of data, making the updates "noisier" (higher variance). This stochastic noise can help push the model out of sharp, suboptimal local minima and into wider, flatter minima that generalize better. Distractor Analysis: - A is Incorrect: Increasing the batch size makes gradients smoother and more accurate, which is the opposite of what is needed to escape a local minimum via noise. - C is Incorrect: Steps per epoch is determined by dataset_size / batch_size. Increasing it doesn't directly add noise to the gradient updates; it just changes how often gradients are applied.- D is Incorrect: Decreasing the learning rate makes updates smaller, making it harder to escape a local minimum, not easier. > Key Takeaway: Smaller batch sizes introduce gradient noise, which can act as a form of regularization and help escape sharp local minima. |
| ### Question 6 - Defining Epochs for Tabular Data A startup is using SageMaker XGBoost to predict customer churn on a tabular dataset. The dataset contains 50,000 rows. The team wants to configure the training job so that the model iterates over the entire dataset exactly 100 times during the training process. Which fundamental hyperparameter should the team set to achieve this requirement? - A) Set the batch_size hyperparameter to 50,000.- B) Set the num_round (or num_epochs) hyperparameter to 100.- C) Set the max_steps hyperparameter to 100.- D) Set the learning_rate hyperparameter to 100. |
Correct Answer: B Correct Option Analysis: - B is Correct: An epoch is defined as one complete pass over the entire training dataset. In the context of XGBoost (a boosting algorithm), this is typically controlled by the num_round parameter, which specifies the number of boosting rounds (effectively, iterations over the data or trees built). Setting it to 100 ensures the data is processed 100 times.Distractor Analysis: - A is Incorrect: Batch size determines how many rows are processed before a weight update, not how many times the full dataset is passed over. - C is Incorrect: A "step" is one weight update (one batch processed). If the batch size is 500, 100 steps would only process 50,000 rows (1 epoch). Setting max_steps to 100 would stop training after just 1 epoch in this case.- D is Incorrect: The learning rate controls the size of the weight updates, not the duration or number of iterations over the dataset. > Key Takeaway: Epochs define the number of full passes over the training dataset, typically configured via parameters like num_epochs or num_round. |
| ### Question 7 - Defining Weight Update Frequency A company is using Amazon SageMaker to train a large language model (LLM). The training script uses a custom DataLoader. The team needs to ensure that the model's weights are updated after processing every 64 sequences, regardless of the total dataset size or the number of epochs. Which fundamental hyperparameter must be configured to 64? - A) steps- B) batch_size- C) epochs- D) eval_steps |
Correct Answer: B Correct Option Analysis: - B is Correct: The batch size determines the number of samples processed before the model's internal parameters (weights) are updated. By setting the batch size to 64, the DataLoader will yield 64 sequences, the model will perform a forward and backward pass, and the optimizer will step (update the weights). Distractor Analysis: - A is Incorrect: "Steps" refers to the total number of weight updates across the entire training run, not the frequency of updates per dataset pass. - C is Incorrect: "Epochs" defines how many times the entire dataset is iterated over, not the chunk size per weight update. - D is Incorrect: eval_steps defines how many batches are processed during an evaluation phase, not the training update frequency.> Key Takeaway: Batch size is the hyperparameter that directly controls the number of samples processed per single weight update (step). |
| ### Question 8 - Frequent Checkpointing on Massive Datasets An enterprise is training a model on a massive dataset where one full epoch takes several days to complete. The team wants to evaluate the model's performance and save checkpoints frequently without waiting for a full epoch to complete. They need to adjust fundamental hyperparameters to achieve this with the least operational overhead. Which approach should they implement? - A) Decrease the batch size so that evaluations occur more frequently per epoch. - B) Increase the number of epochs to run more evaluations over time. - C) Configure the training script to evaluate and checkpoint based on steps rather than epochs.- D) Use SageMaker Automatic Model Tuning to find the optimal evaluation frequency. |
Correct Answer: C Correct Option Analysis: - C is Correct: When an epoch is too large (taking days), evaluating only at the end of each epoch is inefficient. By defining the evaluation and checkpointing frequency in terms of "steps" (e.g., evaluate every 500 steps), the team can trigger checkpoints multiple times within a single epoch, regardless of the dataset size. Distractor Analysis: - A is Incorrect: Decreasing the batch size would increase the number of steps per epoch, but if the script is hardcoded to evaluate "per epoch," it will still wait days to evaluate. It's an indirect and inefficient way to solve the problem. - B is Incorrect: Increasing epochs increases the total training time (days to weeks), making the evaluation frequency even worse. - D is Incorrect: Using AMT is for hyperparameter optimization, not for defining checkpointing or evaluation frequency logic within a single training run. > Key Takeaway: For massive datasets where epochs take too long, use "steps" to schedule evaluations, checkpoints, and learning rate decay schedules. |
| ### Question 9 - Categorical Hyperparameter Tuning for Batch Size A data scientist is setting up an Amazon SageMaker Automatic Model Tuning (AMT) job to optimize a deep learning model. The team has a strict hardware constraint: the batch size must be a power of 2 (e.g., 16, 32, 64) to ensure compatibility with the GPU tensor core architecture. The AMT job must only test these specific values. How should the data scientist define the batch size hyperparameter in the tuning job? - A) Define batch size as a ContinuousParameter ranging from 16 to 64.- B) Define batch size as an IntegerParameter ranging from 16 to 64.- C) Define batch size as a CategoricalParameter with values ['16', '32', '64'].- D) Define batch size as a FreeTextParameter and let the tuning job generate the values. |
Correct Answer: C Correct Option Analysis: - C is Correct: A CategoricalParameter in SageMaker AMT is used to define a list of discrete, specific values that do not have a natural ordering or scaling relationship. By explicitly listing ['16', '32', '64'], the tuning job is constrained to only testing these exact values, satisfying the power-of-2 hardware constraint.Distractor Analysis: - A is Incorrect: A ContinuousParameter uses floating-point numbers (e.g., 24.5), which is invalid for batch size and would break the power-of-2 constraint.- B is Incorrect: An IntegerParameter allows any integer in the range (e.g., 17, 20, 33), violating the strict power-of-2 hardware constraint.- D is Incorrect: FreeTextParameter is not a valid hyperparameter type in Amazon SageMaker Automatic Model Tuning.> Key Takeaway: Use CategoricalParameter in SageMaker AMT when you need to restrict a hyperparameter (like batch size) to a specific list of valid discrete values. |
| ### Question 10 - Maximizing Compute Utilization with Early Stopping A retail company is training a forecasting model using Amazon SageMaker. The team sets the maximum epochs to 50. They notice that the model typically converges around epoch 15, but running until epoch 50 wastes compute resources and sometimes degrades performance. They want to stop training automatically once the model has converged to save cost. Which configuration should they apply? - A) Configure early stopping in the SageMaker training job with a patience parameter. - B) Hardcode the epoch count to 15 in the training script. - C) Increase the batch size so the model converges faster. - D) Switch from epochs to steps and manually guess the stopping point. |
Correct Answer: A Correct Option Analysis: - A is Correct: SageMaker Automatic Model Tuning and standard training jobs support early stopping. By configuring early stopping with a patience parameter (e.g., waiting 3 epochs for improvement), the training job will automatically terminate when the objective metric (like validation loss) stops improving. This saves compute resources and prevents overfitting. Distractor Analysis: - B is Incorrect: Hardcoding to 15 is brittle. The model might converge at epoch 12 or epoch 20 depending on the data and initialization. It does not provide an automatic, adaptive stopping mechanism. - C is Incorrect: Increasing the batch size might change the convergence speed, but it does not automatically stop the job when convergence is reached. The job would still run for the remaining epochs unless early stopping is configured. - D is Incorrect: Manually guessing the stopping point in steps is a poor operational practice and defeats the purpose of automated, managed training features. > Key Takeaway: Early stopping is a native SageMaker feature that monitors objective metrics and automatically halts training to save compute costs when convergence stalls. |