Skill 1.3.3: Identify and mitigate sources of bias in data by using AWS tools and techniques
| Question | Answer & Explanation |
|---|---|
| ### Question 1 - Mitigating Collection Order Bias with Shuffling A retail company is building an ML model to predict customer churn. The dataset was collected sequentially over six months, and early data contains mostly high-value customers, while later data contains mostly low-value customers. The data science team must prepare the dataset for training while ensuring that mini-batch gradient descent does not learn temporal ordering artifacts. The solution must require the least operational overhead and use native AWS tooling. Which ONE action should the team take to fulfill these requirements? - A) Write a custom Python script in a SageMaker Processing job to sort the dataset by customer value. - B) Use Amazon SageMaker Data Wrangler to shuffle the dataset randomly before splitting it into training and validation sets. - C) Deploy an AWS Lambda function to randomly sample records and write them back to S3. - D) Use Amazon SageMaker Ground Truth to label the low-value customers with higher confidence weights. |
Correct Answer: B Correct Option Analysis: - B is Correct: Shuffling the dataset randomly breaks the alignment between collection order and the mini-batches fed into the training algorithm. Using SageMaker Data Wrangler provides a native, low-operational-overhead way to perform this transformation visually without writing custom code. Distractor Analysis: - A is Incorrect: Sorting by customer value would artificially group records by value rather than removing the temporal ordering artifact, potentially introducing a new ordering bias. - C is Incorrect: Writing a custom Lambda function to randomize records in S3 introduces unnecessary operational overhead and is not a best practice for large ML datasets. - D is Incorrect: SageMaker Ground Truth is used for data labeling, not for re-weighting classes or mitigating ordering bias during data preparation. > Key Takeaway: Shuffling is the primary technique to remove bias introduced by the order in which data was collected or ingested, ensuring mini-batches are representative of the overall dataset. |
| ### Question 2 - Addressing Image Representation Bias via Augmentation An autonomous vehicle startup is training an object detection model. The image dataset is heavily biased towards daytime images, with only 5% nighttime images. The team must improve model performance on nighttime conditions without deploying a new fleet to collect more data, and must do so natively within Amazon SageMaker. The solution must not reduce the overall size of the dataset. Which ONE technique should the team use to mitigate this bias? - A) Use SageMaker Data Wrangler to drop daytime images until the dataset is balanced. - B) Use Amazon Rekognition to label the nighttime images and increase their confidence scores. - C) Apply image augmentation techniques (e.g., brightness/contrast adjustment, synthetic noise) to the nighttime images to synthetically increase their representation and diversity. - D) Use SageMaker Ground Truth to generate synthetic daytime images from the nighttime images. |
Correct Answer: C Correct Option Analysis: - C is Correct: Data augmentation modifies existing images to create new, synthetic variations. By applying augmentations to the underrepresented nighttime images, the team increases their representation and diversity without discarding valuable daytime data or deploying new collection mechanisms. Distractor Analysis: - A is Incorrect: Dropping daytime images (undersampling) reduces the overall size of the dataset, which violates the stated constraint and wastes valuable data. - B is Incorrect: Amazon Rekognition provides labels and confidence scores but does not inherently solve representation bias; re-labeling does not increase the model's exposure to nighttime visual features. - D is Incorrect: SageMaker Ground Truth is for human annotation workflows, not for synthetic image generation. > Key Takeaway: Augmentation is the preferred method to address representation bias in image datasets when collecting new data is not feasible, as it increases minority class representation without reducing dataset size. |
| ### Question 3 - Bias Detection and Stratified Splitting with SageMaker Clarify A financial institution is preparing a dataset for a loan approval model. The dataset contains a sensitive attribute gender which is highly imbalanced, leading to potential representation bias. The team needs to evaluate the dataset for bias before training and split the data ensuring the exact same proportion of genders in the training and test sets. They want to use a managed AWS service designed specifically for this task.Which ONE approach should the team take? - A) Use Amazon SageMaker Clarify to generate bias metrics and apply stratified splitting based on the sensitive attribute. - B) Use AWS Glue DataBrew to profile the dataset and manually create train/test splits using a custom PySpark script. - C) Use Amazon Comprehend to detect PII in the gender column and redact it before splitting. - D) Use Amazon SageMaker Feature Store to store the features, which will automatically balance the gender distribution during retrieval. |
Correct Answer: A Correct Option Analysis: - A is Correct: Amazon SageMaker Clarify is specifically designed to detect bias in datasets and models using metrics like class imbalance (CI) and difference in positive proportions. It also provides built-in functionality for stratified splitting to ensure that sensitive attributes are proportionally represented across training, validation, and test sets. Distractor Analysis: - B is Incorrect: AWS Glue DataBrew is excellent for profiling and cleaning but lacks native, purpose-built bias detection metrics and stratified splitting based on sensitive attributes without custom scripting. - C is Incorrect: Redacting the gender column with Amazon Comprehend removes the feature but does not evaluate or mitigate the underlying distribution bias in the dataset prior to training. - D is Incorrect: SageMaker Feature Store is a repository for ML features; it does not automatically balance distributions or perform bias-aware stratified splits. > Key Takeaway: SageMaker Clarify is the go-to AWS service for both identifying bias via pre-training metrics and executing stratified splits to maintain demographic proportions across data partitions. |
| ### Question 4 - Mitigating Multimodal Representation Bias (Select TWO) A healthcare company is building a model to predict disease progression. The dataset includes both numeric patient vitals (tabular) and X-ray images (image). The dataset exhibits representation bias across different age groups in both modalities. The team must mitigate this bias with minimal custom code using AWS services. Which TWO techniques should the team employ to address the bias across both modalities? - A) Use Amazon SageMaker Data Wrangler to oversample the underrepresented age groups in the tabular data. - B) Apply image augmentation techniques (e.g., random rotations, cropping) to the X-ray images of the underrepresented age groups. - C) Use AWS Glue Data Quality to drop the age group feature entirely to prevent the model from learning bias. - D) Replace all missing numeric vitals with the global average using DataBrew. - E) Deploy an AWS Lambda function to resize all images to a standard resolution. |
Correct Answers: A and B Correct Options Analysis: - A is Correct: Oversampling the underrepresented group in the tabular data increases their weight during training without losing data from other groups. SageMaker Data Wrangler provides built-in transformations for balancing datasets. - B is Correct: For image modalities, augmentation is the standard technique to increase the representation and diversity of an underrepresented group without collecting new raw images. Distractor Analysis: - C is Incorrect: Dropping the age group feature may hide the bias but does not fix the underlying data representation issue; the model will still underperform for that demographic if their features are not well-represented. - D is Incorrect: Mean imputation addresses missing data, not representation bias or class imbalance across demographic groups. - E is Incorrect: Resizing images is a standard preprocessing step but does nothing to address demographic representation bias. > Key Takeaway: When dealing with multimodal datasets, bias mitigation techniques must be applied per modality: oversampling/SMOTE for tabular data and augmentation for image data. |
| ### Question 5 - Fixing Mini-Batch Ordering Artifacts An e-commerce platform is training a recommendation model. The training data was ingested chronologically and grouped strictly by user ID. When training without shuffling, the model learns the preferences of users at the beginning of the dataset and overwrites those weights with users at the end of the dataset. The team must fix this ordering artifact with the least computational overhead. Which ONE action should the team take? - A) Implement stratified sampling based on user ID. - B) Apply shuffling to the entire dataset before feeding it into the training algorithm. - C) Apply Synthetic Minority Over-sampling Technique (SMOTE) to the user features. - D) Normalize the numerical features using Amazon SageMaker Data Wrangler. |
Correct Answer: B Correct Option Analysis: - B is Correct: Shuffling the dataset randomly distributes the records so that mini-batches contain a diverse mix of user IDs, preventing the model's weights from being sequentially overwritten by the grouped ordering of users. Distractor Analysis: - A is Incorrect: Stratified sampling ensures proportional representation of classes, but it does not inherently break the within-group ordering artifacts that cause mini-batch gradient descent to learn sequentially. - C is Incorrect: SMOTE is used to generate synthetic examples for an underrepresented class, not to fix the ordering of records within mini-batches. - D is Incorrect: Normalization scales feature values but does not change the order in which the records are fed to the model. > Key Takeaway: If data is ingested in batches grouped by a specific category (e.g., user ID, region, date), shuffling is mandatory to break the alignment between collection order and mini-batch composition. |
| ### Question 6 - Mitigating Class Imbalance in Text Data A media company is preparing a dataset for a sentiment analysis model. The text data contains a disproportionately high volume of positive reviews compared to negative reviews. The team needs to mitigate this class imbalance to ensure the model accurately detects negative sentiment. They want to use an AWS tool that provides built-in transformations for this specific purpose with the least operational overhead. Which ONE approach should the team take? - A) Use Amazon SageMaker Data Wrangler to apply SMOTE (Synthetic Minority Over-sampling Technique) directly to the raw text features. - B) Use Amazon SageMaker Data Wrangler to randomly undersample the majority class (positive reviews). - C) Use AWS Glue Data Quality to enforce a rule that limits the number of positive reviews. - D) Use Amazon Comprehend to translate negative reviews to multiple languages to artificially increase their count. |
Correct Answer: B Correct Option Analysis: - B is Correct: SageMaker Data Wrangler includes built-in transformations to balance data, such as randomly undersampling the majority class. This requires no custom code and directly addresses the class imbalance in the training dataset. Distractor Analysis: - A is Incorrect: SMOTE operates on continuous numeric feature vectors. It cannot be directly applied to raw text strings without first vectorizing the text, which adds operational overhead. - C is Incorrect: AWS Glue Data Quality enforces data quality rules (like dropping rows that fail a condition), but it is not designed as an ML data balancing tool for class imbalance. - D is Incorrect: Translating negative reviews into multiple languages changes the feature distribution and introduces multilingual noise; it is not a valid technique for resolving class imbalance in a monolingual model. > Key Takeaway: For text classification datasets with class imbalance, random undersampling of the majority class or oversampling of the minority class are standard, low-overhead techniques available in SageMaker Data Wrangler. |
| ### Question 7 - Preventing Data Leakage and Distribution Bias in Splits An enterprise is preparing a training dataset for a binary classification model. The data scientists want to ensure that their train, validation, and test sets do not suffer from data leakage and that each set maintains the exact same distribution of a sensitive demographic feature (e.g., age_group). They want to accomplish this using an AWS native service without writing custom splitting scripts.Which ONE approach fulfills these requirements? - A) Use a random split via AWS Glue DataBrew. - B) Use Amazon SageMaker Clarify to perform a stratified split based on the demographic feature. - C) Use Amazon SageMaker Ground Truth to manually separate the records into different S3 buckets. - D) Use AWS Glue Data Quality to validate the distributions after a manual split. |
Correct Answer: B Correct Option Analysis: - B is Correct: Amazon SageMaker Clarify supports stratified splitting, which divides the dataset while preserving the percentage of samples for each target class or sensitive attribute. This ensures the distribution is consistent across all splits and prevents the model from being trained on a biased subset. Distractor Analysis: - A is Incorrect: A standard random split via Glue DataBrew may result in uneven distributions of the sensitive feature across the train, validation, and test sets, especially if the feature is a minority class. - C is Incorrect: Manual separation via Ground Truth is highly inefficient and prone to human error, violating the requirement for an automated, scriptless AWS native approach. - D is Incorrect: Validating distributions after a manual split does not proactively prevent the bias during the split; it only reports on it after the fact. > Key Takeaway: Stratified splitting is essential when dealing with sensitive attributes or imbalanced classes, ensuring that every split accurately reflects the overall population distribution. |
| ### Question 8 - Resolving Class Imbalance vs. Geographic Representation Bias (Select TWO) A machine learning team is using Amazon SageMaker to train a model. The dataset is highly imbalanced regarding a specific outcome label (1% positive, 99% negative). Additionally, the dataset exhibits representation bias where one geographic region is severely underrepresented. The team must resolve both the class imbalance and the geographic representation bias before training. Which TWO techniques should the team employ? - A) Apply shuffling to the entire dataset to randomize the order of records. - B) Use oversampling or class re-weighting to increase the impact of the rare outcome label. - C) Apply data augmentation or collect more data to increase the representation of the underrepresented geographic region. - D) Drop the geographic region feature from the dataset to eliminate bias. - E) Convert the dataset to Parquet format to optimize the distribution of data. |
Correct Answers: B and C Correct Options Analysis: - B is Correct: Class imbalance (where one outcome label dominates) is resolved by adding examples of the rare class (oversampling), reducing the majority class, or applying class weights during training so the model pays more attention to the rare class. - C is Correct: Geographic representation bias (where a specific demographic or group is missing) is resolved by increasing the representation of that group, either through data augmentation or collecting additional real-world data. Distractor Analysis: - A is Incorrect: Shuffling reorders existing records but does nothing to change the underlying mathematical ratios of class imbalance or representation bias. - D is Incorrect: Dropping the geographic feature prevents the model from explicitly using it, but does not fix the fact that the model lacks training data for that region's unique characteristics. - E is Incorrect: Converting to Parquet optimizes storage and query performance but has zero effect on the statistical distribution or bias of the records. > Key Takeaway: Class imbalance (outcome label proportions) and representation bias (group inclusion) are distinct problems requiring different remedies; rebalancing labels does not automatically fix demographic underrepresentation. |
| ### Question 9 - Handling Chronological Batch Bias A gaming company is building a churn prediction model. The raw data consists of player event logs collected sequentially by timestamp. The team noticed a recent game patch caused a spike in churn events. The team decides to treat this as a standard classification problem rather than a time-series forecasting problem, and must split the data such that both the training and test sets reflect the overall distribution of churn events, minimizing bias. Which ONE splitting technique should they use? - A) Perform a chronological split to preserve the time-series nature of the data, accepting the bias in the training set. - B) Perform a standard random split without considering the churn label to ensure a 80/20 ratio of total records. - C) Use stratified splitting on the churn label to ensure the ratio of churn events is consistent across training and test sets.- D) Filter out the churn events from the recent game patch to make the dataset uniform. |
Correct Answer: C Correct Option Analysis: - C is Correct: Because the churn events are clustered chronologically due to the game patch, a chronological split would place all churn events in the test set, starving the training set of positive examples. Stratified splitting on the churn label ensures both sets receive a proportional amount of churn and non-churn events, mitigating the temporal cluster bias.Distractor Analysis: - A is Incorrect: A chronological split would result in a training set with very few churn events, causing the model to predict poorly on the test set which contains the spike. - B is Incorrect: A standard random split might still result in an uneven distribution of the rare churn events, whereas stratified splitting guarantees the distribution is maintained. - D is Incorrect: Filtering out the recent churn events removes valuable data and prevents the model from learning the patterns associated with the game patch. > Key Takeaway: When a rare event is clustered temporally or sequentially, standard random splits may not be sufficient; stratified splitting on the target label ensures both training and test sets contain representative examples. |
| ### Question 10 - Bias Mitigation in Image Data with Least Overhead A global logistics company is training an object detection model to identify packages on conveyor belts. The dataset contains 100,000 images from their US facilities and only 2,000 images from their European facilities. The images from Europe have slightly different lighting conditions. The team must mitigate this geographic bias without collecting new images from Europe, ensuring the model generalizes well across both regions. The solution must require the least operational overhead using AWS services. Which ONE approach should the team take? - A) Use Amazon SageMaker Data Wrangler to apply image augmentation (e.g., brightness adjustment, contrast variation) to the European images to increase their diversity and representation. - B) Use AWS Glue Data Quality to define a rule that drops US images until the counts match. - C) Use Amazon SageMaker Ground Truth to label the US images with higher confidence weights. - D) Use Amazon Rekognition Custom Labels to automatically balance the dataset during training. |
Correct Answer: A Correct Option Analysis: - A is Correct: Image augmentation modifies existing images to create new synthetic variations. By applying augmentations (like adjusting brightness or contrast) to the European images, the team increases the representation of the European lighting conditions without dropping valuable US data or deploying new physical collection infrastructure. Distractor Analysis: - B is Incorrect: Dropping 98,000 US images to match the 2,000 European images is highly inefficient, discards massive amounts of valuable training data, and would severely degrade model performance. - C is Incorrect: Ground Truth is used for labeling annotations, not for re-weighting data distributions or applying transformations to mitigate image bias. - D is Incorrect: Amazon Rekognition Custom Labels does not automatically balance datasets; data preparation for bias mitigation must occur before the data is fed into the training algorithm. > Key Takeaway: In computer vision tasks, augmentation is the standard, low-overhead method to synthetically boost underrepresented visual characteristics (like lighting or angles) without collecting new raw data. |