2.2.3

Question	Answer & Explanation
### Question 1 - AMT Search Strategies for Limited Budgets An enterprise ML team is tuning an XGBoost model using SageMaker Automatic Model Tuning (AMT). They have a strict budget allowing a maximum of only 10 training jobs. They need the highest probability of finding the optimal hyperparameters within this limited budget. Which search strategy should the team use? (Select ONE) - A) Random Search - B) Bayesian Optimization - C) Hyperband - D) Grid Search	Correct Answer: B Correct Option Analysis: - B is Correct: Bayesian Optimization uses the results of previous training jobs to inform the hyperparameter selection for subsequent jobs. It is the ideal strategy when you have a limited budget (number of training jobs) because it intelligently narrows down the search space to find the best parameters efficiently. Distractor Analysis: - A is Incorrect: Random Search selects hyperparameter combinations completely at random. It does not learn from previous jobs, making it less efficient for small budgets. - C is Incorrect: Hyperband is designed for massive budgets and uses aggressive early stopping to kill underperforming jobs. It is not optimal for a strict, small budget of only 10 jobs. - D is Incorrect: Grid Search evaluates every possible combination of a predefined grid. SageMaker AMT does not natively support Grid Search because it scales poorly and is highly inefficient for large or continuous search spaces. > Key Takeaway: Use Bayesian Optimization for small tuning budgets as it learns from previous training job evaluations to find the best hyperparameters faster.
### Question 2 - Configuring Objective Metric Direction A data scientist is configuring SageMaker AMT for a regression model. The objective metric is Mean Squared Error (MSE), which is emitted to CloudWatch. The business requirement is to find the model with the lowest possible error. How should the tuning job be configured? (Select ONE) - A) Set the ObjectiveType to Maximize. - B) Set the ObjectiveType to Minimize. - C) Use a custom Python script to invert the MSE metric (e.g., 1/MSE) before logging it to CloudWatch. - D) Select the Hyperband search strategy, which automatically minimizes all error metrics.	Correct Answer: B Correct Option Analysis: - B is Correct: SageMaker AMT allows you to specify the ObjectiveType as either Minimize or Maximize. Since Mean Squared Error (MSE) is an error metric where lower values are better, setting the objective type to Minimize will direct the tuning job to favor hyperparameter combinations that produce the lowest MSE. Distractor Analysis: - A is Incorrect: Maximizing MSE would intentionally drive the model to produce higher errors, resulting in a worse model. - C is Incorrect: While mathematically possible, writing a custom script to invert metrics is unnecessary operational overhead. SageMaker natively supports minimizing metrics. - D is Incorrect: The search strategy (Hyperband, Bayesian, Random) determines how the search space is explored, not the direction of the objective metric. Hyperband does not automatically assume metric direction. > Key Takeaway: Always verify the ObjectiveType (Minimize vs Maximize) in AMT based on whether your metric is an error (lower is better) or a score (higher is better).
### Question 3 - Reusing Prior Tuning Knowledge An ML team ran a SageMaker AMT job to optimize a PyTorch model's learning rate and batch size. The team has now acquired new domain-specific data and wants to expand the search range for the learning rate while reusing the learnings from the previous tuning job to reduce compute time and cost. Which configuration meets these requirements? (Select ONE) - A) Create a new tuning job with a WarmStartConfig pointing to the previous tuning job. - B) Clone the previous tuning job and manually edit the hyperparameter ranges in the console. - C) Use the Hyperband search strategy with the previous best model as the estimator. - D) Configure the max_parallel_jobs to match the previous tuning job to resume the search.	Correct Answer: A Correct Option Analysis: - A is Correct: SageMaker AMT supports Warm Start, which allows you to use the results from previous tuning jobs to start a new tuning job. By configuring WarmStartConfig, the new job leverages the knowledge already gathered (e.g., which regions of the search space are promising) to find better hyperparameters faster, saving both time and compute costs. Distractor Analysis: - B is Incorrect: Simply cloning and editing the job would start a brand new search from scratch, losing the prior learnings and wasting compute. - C is Incorrect: Hyperband is a search strategy for aggressive early stopping; it does not inherently import state or learnings from a previously completed tuning job. - D is Incorrect: Matching max_parallel_jobs controls the concurrency of the new job but does not link the new search to the previous job's evaluations. > Key Takeaway: Use WarmStartConfig in SageMaker AMT to transfer hyperparameter tuning knowledge from past jobs to new ones, accelerating convergence and reducing costs.
### Question 4 - Metric Extraction in Script Mode A machine learning engineer is using SageMaker Script Mode to train a custom TensorFlow model and wants to optimize it using Automatic Model Tuning (AMT). The training script prints loss values to standard output as: Epoch 1 - Loss: 0.45. The engineer needs SageMaker to parse this output to track the objective metric. What is the MOST operationally efficient way to achieve this? (Select ONE) - A) Enable CloudWatch Logs Insights and create a custom dashboard for AMT to read from. - B) Define a regular expression in the metric_definitions parameter of the TensorFlow Estimator. - C) Use the sagemaker.TensorFlow built-in algorithm instead of Script Mode to get automatic metrics. - D) Write the metric directly to a custom CSV file in Amazon S3 for AMT to read at the end of the job.	Correct Answer: B Correct Option Analysis: - B is Correct: In SageMaker Script Mode, the training script runs inside an AWS-managed container. To allow AMT to track custom metrics (like Loss), the engineer must define a metric_definitions parameter in the Estimator. This parameter uses a regular expression (e.g., Loss: ([0-9\\.]+)) to parse the standard output streams and send the metric data to CloudWatch, which AMT then monitors. Distractor Analysis: - A is Incorrect: CloudWatch Logs Insights is a query tool for analyzing logs after the fact; AMT cannot use it to track objective metrics in real-time during a training job. - C is Incorrect: Switching to a built-in algorithm limits the custom architecture capabilities of Script Mode. If using Script Mode, metric definitions are the standard way to capture metrics. - D is Incorrect: SageMaker AMT relies on CloudWatch metrics emitted in real-time to evaluate jobs and make early-stopping decisions. Writing to a CSV in S3 is not a supported mechanism for objective metric tracking. > Key Takeaway: When using Script Mode with AMT, always define metric_definitions with regular expressions to parse stdout and send custom metrics to CloudWatch.
### Question 5 - Maximizing Tuning Throughput A retail company needs to complete a SageMaker hyperparameter tuning job in the shortest possible time. They have confirmed they have a service quota of 50 concurrent training instances. The total number of training jobs to run is 100. Which configuration will finish the tuning job the fastest? (Select ONE) - A) Set MaxParallelJobs to 50 and MaxTrainingJobs to 100. - B) Set MaxParallelJobs to 100 and MaxTrainingJobs to 50. - C) Set MaxParallelJobs to 1 and MaxTrainingJobs to 100. - D) Use the Bayesian search strategy, which automatically scales parallel jobs up to the quota.	Correct Answer: A Correct Option Analysis: - A is Correct: Setting MaxParallelJobs to 50 allows SageMaker to run 50 training jobs simultaneously, taking full advantage of the available instance quota. Setting MaxTrainingJobs to 100 limits the total search to the desired 100 jobs. This configuration ensures the 100 jobs are completed in just two sequential batches, minimizing the overall wall-clock time. Distractor Analysis: - B is Incorrect: MaxParallelJobs cannot exceed the actual service quota (50 in this case), and MaxParallelJobs (100) cannot be greater than MaxTrainingJobs (50) in valid configurations. - C is Incorrect: Setting MaxParallelJobs to 1 means jobs will run sequentially, resulting in 100 sequential training runs. This would take an incredibly long time. - D is Incorrect: The search strategy does not automatically configure the parallelism. Parallelism is explicitly controlled by the MaxParallelJobs parameter set by the user. > Key Takeaway: To minimize tuning wall-clock time, set MaxParallelJobs as high as your instance quota allows, up to the total MaxTrainingJobs limit.
### Question 6 - Automatic Early Stopping in AMT An ML team is running SageMaker AMT across a wide range of hyperparameters. Many of the training jobs use combinations that perform poorly from the beginning. The team wants to automatically terminate these underperforming jobs to save compute costs without writing custom early stopping logic in their training scripts. Which AMT feature should the team enable? (Select ONE) - A) Enable EarlyStopping in the TrainingJobDefinition of the tuning job. - B) Set MaxTrainingJobs to a very low number to limit wasted compute. - C) Use the Random search strategy to avoid poor combinations. - D) Implement a custom SageMaker Pipelines step to check CloudWatch metrics and kill the job.	Correct Answer: A Correct Option Analysis: - A is Correct: SageMaker AMT has a built-in automatic early stopping feature. By setting EnableEarlyStopping: True in the TrainingJobDefinition (or AlgorithmSpecification), SageMaker monitors the objective metric. If a training job is underperforming compared to the median performance of completed jobs at the same step, SageMaker automatically terminates the job, saving compute time and cost. Distractor Analysis: - B is Incorrect: Lowering MaxTrainingJobs limits the overall search space and the chance of finding a good model, rather than specifically stopping bad jobs early. - C is Incorrect: Random search does not evaluate job performance during the run; it only selects parameters randomly and will run jobs to completion unless early stopping is explicitly enabled. - D is Incorrect: Building a custom SageMaker Pipelines step to monitor and kill jobs introduces unnecessary operational overhead and complexity when AMT provides this natively. > Key Takeaway: Enable automatic early stopping in AMT (EnableEarlyStopping: True) to halt underperforming training jobs automatically based on their objective metric trajectory.
### Question 7 - Defining Hyperparameter Ranges A data scientist is configuring SageMaker AMT for a neural network. The optimizer can be either 'adam', 'sgd', or 'rmsprop'. The learning rate needs to be tested uniformly between 0.0001 and 0.01. Which combination of parameter ranges is correct for this scenario? (Select ONE) - A) Set optimizer as IntegerParameterRange and learning rate as CategoricalParameterRange. - B) Set optimizer as CategoricalParameterRange and learning rate as ContinuousParameterRange. - C) Set optimizer as ContinuousParameterRange and learning rate as IntegerParameterRange. - D) Set optimizer as CategoricalParameterRange and learning rate as CategoricalParameterRange.	Correct Answer: B Correct Option Analysis: - B is Correct: The optimizer is a discrete set of specific string choices ('adam', 'sgd', 'rmsprop'), which maps perfectly to a CategoricalParameterRange. The learning rate is a floating-point number that can take any value between two bounds (0.0001 and 0.01), which requires a ContinuousParameterRange. Distractor Analysis: - A is Incorrect: The optimizer is a string categorical, not an integer. The learning rate is a float, not a categorical range. - C is Incorrect: The optimizer is not a continuous number. The learning rate is not an integer (whole number). - D is Incorrect: While the optimizer is correctly categorical, defining the learning rate as categorical would require manually listing every possible float value between 0.0001 and 0.01, which is impossible and defeats the purpose of continuous tuning. > Key Takeaway: Use CategoricalParameterRange for discrete strings/ints, IntegerParameterRange for whole numbers, and ContinuousParameterRange for floats within a min/max bound.
### Question 8 - Handling Multiple Business Objectives An AI team wants to optimize a model for an e-commerce recommendation engine using SageMaker AMT. They need to maximize the Click-Through Rate (CTR) while keeping the prediction latency under 50ms. They want to tune hyperparameters to satisfy both goals natively within the AMT job. How should they configure the objective metric? (Select ONE) - A) Configure AMT with a composite objective metric that mathematically blends CTR and latency. - B) Define two separate ObjectiveMetric entries in the tuning job configuration. - C) SageMaker AMT only supports a single objective metric; optimize for CTR and filter for latency post-tuning. - D) Use SageMaker Hyperband to optimize multiple metrics simultaneously.	Correct Answer: C Correct Option Analysis: - C is Correct: SageMaker AMT natively supports only one objective metric per tuning job. To handle multiple business requirements, the standard approach is to choose the primary metric (e.g., CTR) as the objective metric to be maximized, and use constraints or post-processing to ensure latency requirements are met (e.g., by analyzing the latency metrics logged by the completed jobs and selecting the best model that meets the 50ms threshold). Distractor Analysis: - A is Incorrect: While you could technically log a custom composite metric from your training script, SageMaker AMT still only optimizes a single metric. Creating a composite metric requires custom logic and may obscure the actual CTR performance. - B is Incorrect: The TuningJobDefinition in SageMaker AMT schema strictly accepts only one ObjectiveMetric. - D is Incorrect: Hyperband is a search strategy for resource allocation and early stopping, not a multi-objective optimization algorithm. > Key Takeaway: SageMaker AMT is a single-objective optimization tool. To manage multiple goals, optimize for the primary metric and use the secondary metrics as filtering criteria when selecting the final model.
### Question 9 - Built-in Algorithms and Metric Definitions A junior ML engineer is using the SageMaker built-in XGBoost algorithm to train a binary classification model. They want to use AMT to maximize the Area Under the Curve (AUC). The engineer is concerned they need to write custom regex patterns to capture the AUC metric. What guidance should be given to the engineer? (Select ONE) - A) Custom regex is required for all AMT jobs to parse CloudWatch logs. - B) SageMaker built-in algorithms automatically emit standard metrics, so no custom regex is needed. - C) They must switch to Script Mode to define metrics manually. - D) CloudWatch Container Insights must be enabled to parse XGBoost metrics.	Correct Answer: B Correct Option Analysis: - B is Correct: SageMaker built-in algorithms (like XGBoost) are pre-configured to automatically compute and emit standard metrics (like validation:auc, validation:accuracy, etc.) directly to CloudWatch in a format SageMaker natively understands. Therefore, when using a built-in algorithm with AMT, you simply specify the metric name (e.g., validation:auc) as the objective, and no custom regex metric definitions are required. Distractor Analysis: - A is Incorrect: Custom regex is only needed when using Script Mode with custom training scripts that print non-standard outputs. Built-in algorithms handle this natively. - C is Incorrect: Switching to Script Mode would require writing custom metric definitions, increasing operational overhead unnecessarily. - D is Incorrect: CloudWatch Container Insights is for infrastructure monitoring (CPU, memory), not for parsing ML training objective metrics for AMT. > Key Takeaway: Built-in SageMaker algorithms automatically emit standard ML metrics to CloudWatch, eliminating the need for custom metric_definitions regex when using AMT.
### Question 10 - Selecting Hyperband for Massive Budgets A research team is training a massive deep learning model for image classification. Each training job is computationally expensive and takes several hours. They have a large budget allowing up to 500 training jobs, but they want to aggressively stop underperforming trials early and allocate more resources to the promising ones. Which search strategy is BEST suited for this scenario? (Select ONE) - A) Bayesian Optimization - B) Random Search - C) Hyperband - D) Grid Search	Correct Answer: C Correct Option Analysis: - C is Correct: Hyperband is a search strategy specifically designed for large-scale hyperparameter tuning. It uses aggressive early stopping to dynamically allocate resources. It starts by running many configurations for a short time, discards the underperforming ones, and progressively runs the promising ones for longer. This is ideal for computationally expensive jobs where you have a large budget and want to maximize the discovery of top-performing models without wasting hours on bad parameter combinations. Distractor Analysis: - A is Incorrect: While Bayesian optimization is good for limited budgets, it does not natively implement the aggressive, multi-stage early stopping of Hyperband. For 500 expensive jobs, Bayesian might waste significant compute on early epochs of bad models before learning to avoid them. - B is Incorrect: Random Search runs jobs blindly without learning from previous trials and does not natively feature the dynamic resource allocation that makes Hyperband efficient for large numbers of expensive jobs. - D is Incorrect: Grid Search is not supported in SageMaker AMT and would be exponentially expensive for a 500-job budget across continuous parameters. > Key Takeaway: Use the Hyperband search strategy for massive tuning budgets with computationally expensive jobs, as it aggressively stops bad jobs early and reallocates resources to promising candidates.