Skill 1.3: Validate data quality and manage bias

Skill 1.3.4: Optimize multimodal data distributions by applying bias metrics across numeric, text, and image assets

Question 1 - Bias Metrics Across Multimodal Pipeline

A financial services company is building a fraud detection model that ingests three distinct data modalities: numeric transaction features (amount, frequency, time since last transaction), text features (transaction descriptions and customer service notes), and image assets (scanned ID documents and signature verifications). The ML team needs to ensure that no single protected group is disproportionately represented across any modality before training begins, as compliance regulations require documented evidence of bias evaluation across all data types.

Which approach should the data engineering team use to evaluate bias metrics across all three modalities with the least operational overhead? (Select ONE)

A) Use SageMaker Clarify to compute bias metrics on numeric features only, then manually inspect a random sample of text and image data for representation issues.
B) Create separate custom Python scripts using open-source libraries (Fairlearn, HuggingFace, OpenCV) to compute bias metrics for each modality, orchestrated through Lambda functions.
C) Use AWS Glue Data Quality rules to declare completeness and representation thresholds for the numeric dataset, and rely on Amazon Bedrock Guardrails to screen text and image content for policy compliance.
D) Use SageMaker Clarify pre-training bias metrics on the numeric feature dataset, Amazon Comprehend custom classification to measure text representation across demographic groups, and Amazon Rekognition Custom Labels evaluation metrics to assess image asset label distribution.
E) Store all three modalities in Amazon S3, use a single AWS Glue DataBrew profile job to automatically compute bias metrics across numeric, text, and image columns simultaneously.

Click to reveal Answer & Explanation

Correct Answer: D

Correct Options Analysis

D is Correct: SageMaker Clarify natively computes pre-training bias metrics (such as Class Imbalance, Difference in Proportions of Labels) on tabular/numeric datasets. Amazon Comprehend Custom Classification can be used to label and assess text-based categories (e.g., sentiment, topics) to verify group representation, while Amazon Rekognition Custom Labels evaluates image classification distributions to detect representation gaps across protected groups in image assets. This approach uses fully managed AWS services across each modality with minimal custom code.

Distractor Analysis

A is Incorrect: While SageMaker Clarify is correct for numeric bias metrics, manually inspecting text and image data introduces high operational overhead and lacks automated, reproducible evidence required by compliance teams.
B is Incorrect: Building custom Python scripts with open-source tools, orchestrated by Lambda, increases operational overhead significantly compared to using native AWS services. It also requires the team to maintain multiple libraries, manage container dependencies, and aggregate results across scattered outputs.
C is Incorrect: AWS Glue Data Quality checks completeness and value ranges for tabular data only; it does not compute bias metrics. Amazon Bedrock Guardrails screens content for safety/toxicity, not for demographic representation bias in training datasets.
E is Incorrect: This option is invalid because AWS Glue DataBrew cannot automatically compute bias metrics across all three modalities. A single profile job cannot assess text sentiment or image label distributions for bias; DataBrew is constrained to tabular datasets (numeric/categorical columns), not embedded text embeddings or raw image pixels.

Key Takeaway: For multimodal bias detection, apply the correct modality-specific AWS service: SageMaker Clarify for numeric/tabular, Amazon Comprehend for text, and Amazon Rekognition Custom Labels for image distribution analysis.

Question 2 - Identifying Representation Bias in Image Dataset

An e-commerce company is training a visual search model using Amazon SageMaker. The training dataset contains product images to power the visual search capability. Following a model performance audit, the team discovers that the model fails to identify products from the "home & garden" category with the same accuracy as other categories.

Upon inspection, the dataset contains 500,000 images across 20 product categories. The "home & garden" category appears in only 1,200 images, while "electronics" appears in 85,000 images. The business requires that all categories maintain roughly equal representation, as the failure is tied to customers abandoning searches due to inadequate results for underrepresented categories.

Which approach represents the most effective bias mitigation technique for this image dataset? (Select TWO)

A) Apply horizontal flips, rotation, and color jittering to augment the "home & garden" images to increase variation.
B) Use Amazon Rekognition Custom Labels to automatically detect underrepresented product categories in the image set and rebalance by generating synthetic images through Stable Diffusion on Bedrock.
C) Apply stratified splitting so that each data split retains an equal proportion of "home & garden" to other categories, then augment the minority class evenly across splits.
D) Downsample "electronics" and other overrepresented categories to match the smallest category (1,200 images), ensuring perfect balance without introducing additional data.
E) Use SageMaker Data Wrangler to apply class imbalance transforms (SMOTE) directly on the raw image tensors to create new "home & garden" images.

Click to reveal Answer & Explanation

Correct Answers: A and C

Correct Options Analysis

A is Correct: Image augmentation (horizontal flips, rotations, color jittering, random cropping, etc.) applies geometric and photometric transformations to underrepresented categories, introducing diverse visual variations while preserving semantic label integrity. This directly adds synthetic variation to address image class imbalance.
C is Correct: Stratified splitting ensures that all categories maintain their relative proportions across training, validation, and test subsets. This prevents the minority "home & garden" category from being absent in evaluation or overrepresented in training, while augmentation applied after stratification ensures the evaluation sets contain only original, unmodified images (preventing data leakage).

Distractor Analysis

B is Incorrect: While Rekognition Custom Labels can evaluate distribution, generating images through Stable Diffusion on Bedrock may introduce synthetic artifacts and requires additional validation. It does not directly solve the underrepresentation issue without introducing potential label quality risks, and it's not the most operationally straightforward approach compared to standard augmentation.
D is Incorrect: Downsampling from 500,000 to 24,000 total images (1,200 × 20) discards massive amounts of valuable training data—at least 476,000 images would be discarded, which unnecessarily reduces model performance and wastes already-collected training data.
E is Incorrect: SageMaker Data Wrangler's class imbalance transforms (like SMOTE) operate on tabular/numeric feature data, not raw image tensors. SMOTE for images requires specialized architectures (e.g., GANs or diffusion models), not Data Wrangler's tabular feature transform engine.

Key Takeaway: Standard image augmentation + stratified splitting is the lowest-overhead, highest-value approach for image class imbalance; avoid discarding majority-class data and avoid applying tabular resampling techniques to raw image tensors.

Question 3 - Text Representation Bias with Class Imbalance

A legal document classifier is being trained to categorize contracts into 12 document types (e.g., non-disclosure, lease, employment, service agreement). The raw text corpus consists of 250,000 documents, but the "lease agreement" class has only 350 documents, whereas the "non-disclosure agreement" class has 45,000 documents. The model reports high overall accuracy but fails to recall "lease agreement" documents during a pilot deployment.

The ML team wants to implement a training pipeline that resolves class imbalance while ensuring no group is underrepresented due to the text distribution. They are using Amazon SageMaker for training and Amazon Comprehend for text classification.

Which technique should be prioritized to resolve the text-based class imbalance? (Select TWO)

A) Use Amazon Comprehend's entity detection to identify all entities in the "lease agreement" documents, then create new synthetic documents using the detected patterns.
B) Weight the minority "lease agreement" class more heavily in the loss function during training.
C) Apply SMOTE (Synthetic Minority Over-sampling Technique) on TF-IDF vectors of the text corpus to generate new lease agreement documents.
D) Deduplicate and downsample the "non-disclosure agreement" class to balance it with the "lease agreement" class by random sampling without replacement.
E) Use Amazon Augmented AI (A2I) to programmatically augment the text with paraphrased versions of "lease agreement" documents using Amazon Bedrock's Titan Text model.

Click to reveal Answer & Explanation

Correct Answers: B and E

Correct Options Analysis

B is Correct: Class weights (or inverse frequency weighting) in the loss function penalize misclassification of minority classes more heavily during training. This directly addresses class imbalance without requiring synthetic data creation or downsizing the majority class, and it is a standard technique for text-based imbalance in SageMaker's linear learners and deep learning models (e.g., PyTorch/HuggingFace loss functions).
E is Correct: Data augmentation for text via paraphrasing (e.g., using Amazon Bedrock's Titan Text or other LLMs) creates semantically equivalent but lexically diverse versions of the minority class documents. This introduces representative variation without changing the label, and Amazon A2I can route the augmented text to a human reviewer for quality assurance before training.

Distractor Analysis

A is Incorrect: Entity detection identifies entities (e.g., names, dates, places) but does not create new synthetic documents. Using detected entity patterns alone is insufficient to generate complete, grammatically correct, and semantically valid contract documents.
C is Incorrect: SMOTE operates on continuous numeric vectors, not sparse high-dimensional TF-IDF vectors. SMOTE on TF-IDF produces interpolated vectors that do not map back to meaningful words and introduces invalid word combinations when reversed to text space.
D is Incorrect: While downsampling the overrepresented majority class is a valid balancing technique, discarding 44,650 "non-disclosure agreement" documents is unnecessarily destructive and degrades the model's ability to recognize nuanced NDA subcategories. Downsampling is rarely the optimal choice when only one class is severely underrepresented.

Key Takeaway: For text-based class imbalance, prioritize class weighting in the loss function and data augmentation (e.g., paraphrasing); avoid SMOTE on sparse text vectors and avoid aggressive downsampling that discards valuable majority-class information.

Question 4 - Data Quality Validation for Numeric Bias

A healthcare provider is building a patient readmission risk model using Amazon SageMaker. The dataset contains 120,000 patient records with numeric features such as age, lab values, medication counts, and prior hospitalizations. The compliance team requires evidence that the training data does not contain disproportionate representation across demographic groups, and that the data meets declared quality thresholds (completeness, value ranges, no duplicates).

The team wants to validate both data quality and pre-training bias with minimal operational overhead.

Which AWS service configuration should be used? (Select ONE)

A) Use AWS Glue Data Quality rulesets to enforce completeness, range, and uniqueness constraints on numeric columns; then run SageMaker Clarify's pre-training bias report on the same dataset to quantify class imbalance and group-level bias metrics.
B) Use Amazon QuickSight to create dashboard visualizations of all numeric columns and manually inspect for bias indicators.
C) Use SageMaker Data Wrangler's interactive visualizations to profile the dataset once, then rely on its automated bias detection to continuously monitor future data versions.
D) Use Amazon CloudWatch canaries to periodically query the dataset for bias metrics using SQL.
E) Use SageMaker Model Monitor to run data quality checks and compute pre-training bias metrics on the raw training dataset.

Click to reveal Answer & Explanation

Correct Answer: A

Correct Options Analysis

A is Correct: AWS Glue Data Quality uses DQDL (Data Quality Definition Language) rulesets to explicitly enforce completeness, uniqueness, ranges, and other constraints on tabular/numeric data. SageMaker Clarify's pre-training bias module computes standard bias metrics (e.g., Class Imbalance (CI), Difference in Proportions of Labels (DPL), Kolmogorov-Smirnov (KS)) on numeric feature distributions across protected groups. Combining both services achieves declared quality validation and reproducible bias evidence with minimal custom code.

Distractor Analysis

B is Incorrect: Manual dashboard inspection is operationally heavy, doesn't provide repeatable/computable bias metrics, and cannot scale to 120k records x multiple features. It also lacks standardized bias metric computation (e.g., DPL, KS).
C is Incorrect: SageMaker Data Wrangler provides interactive visual profiling but does not include an "automated bias detection" engine. It offers limited statistical visualizations but no out-of-the-box bias metric computation comparable to SageMaker Clarify's purpose-built bias analysis.
D is Incorrect: CloudWatch canaries are scheduled probes (scripts) that execute synthetic monitoring for applications; they cannot natively compute bias metrics via SQL. This would require embedding a custom computation engine and is unsuitable for dataset-level bias metric generation.
E is Incorrect: SageMaker Model Monitor focuses on detecting data drift and verifying model inference quality after deployment. It is not designed to compute pre-training bias metrics on raw training datasets before training, nor does it enforce declarative data quality rules like Glue Data Quality.

Key Takeaway: AWS Glue Data Quality + SageMaker Clarify is the canonical combination for declarative data quality + pre-training bias reporting on numeric/tabular datasets.

Question 5 - Multimodal AI Training Data Integrity - Prompt-Response Pairs

A conversational AI team is preparing a fine-tuning dataset for a large language model using Amazon Bedrock. The dataset consists of 85,000 prompt-response pairs drawn from customer support tickets across multiple regions. The team must validate that the pairs are well formed, the responses safely answer the prompts, and no PII (personally identifiable information) is included before customization.

What approach should the team implement to validate the integrity of these prompt-response pairs prior to customizing the model? (Select TWO)

A) Use Amazon Bedrock Guardrails with a sensitive information filter to screen prompts and responses for PII/PHI before passing them to the fine-tuning job.
B) Use Amazon Comprehend's DetectPII API endpoint on the prompt and response columns to detect personally identifiable information, such as names, addresses, and dates of birth.
C) Use AWS Glue Data Quality rules to enforce DQDL constraints that detect prompt-response pairs where the response does not contain the same entities as the prompt.
D) Use SageMaker Clarify's bias detection to validate whether responses in the prompt-response pairs are safe and contextually relevant.
E) Create an AWS Lambda function that manually implements a regex-based PII filter for only English-language prompts and responses.

Click to reveal Answer & Explanation

Correct Answers: A and B

Correct Options Analysis

A is Correct: Amazon Bedrock Guardrails provides configurable sensitive information filters (PII redaction/masking) that screen prompts and responses before data is used for model customization. It supports multiple entity types (e.g., name, address, SSN) and can mask, redact, or block content. This is a managed, prompt-response integrity check specifically designed for Bedrock customization workflows.
B is Correct: Amazon Comprehend's DetectPII API is a fully managed NLP service that detects PII entities in text (names, addresses, dates, SSNs, etc.), supporting both English and Spanish. It can be applied in batch to prompt and response columns during data validation to flag records containing PII.

Distractor Analysis

C is Incorrect: AWS Glue Data Quality does not have semantic comprehension; it cannot detect whether two text columns are "contextually aligned" or whether one response answers its prompt. DQDL is constrained to structural/statistical constraints (e.g., not null, string length, regex). This option describes a capability Glue Data Quality does not possess for contextual prompt-response alignment.
D is Incorrect: SageMaker Clarify computes bias metrics (e.g., class imbalance, group-level label distribution). It does not assess whether text is contextually safe or relevant, nor does it perform PII detection. Its purpose is model explainability and bias analysis on tabular features, not conversational integrity validation.
E is Incorrect: A custom Lambda function with regex-based PII detection works only on English, misses multilingual corner cases, and cannot be scaled to 85,000 pairs without custom logic for batch processing and error handling. This introduces unnecessary operational overhead compared to using AWS managed services (Guardrails or Comprehend).

Key Takeaway: For AI training data integrity, combine Bedrock Guardrails (inline blocking/redaction) with Amazon Comprehend DetectPII (batch analysis) to prevent PII leakage into customized models; avoid custom regex or semantic mis-solutions using Glue Data Quality.

Question 6 - Outlier Detection and Missing Value Imputation Bias

An automotive manufacturing company is training a predictive maintenance model using Amazon SageMaker. The dataset contains 300,000 sensor readings across 50 numeric features (temperature, vibration, pressure, RPM, etc.). Before training, an exploratory analysis reveals:

5% of temperature values are missing and the missingness is higher in readings collected from older equipment models.
Some readings have extreme outliers (e.g., temperature > 150°C, vibration > 80 mm/s), likely due to sporadic sensor failures.
The dataset is heavily biased toward newer equipment models (80% of rows come from models released in the last 2 years).

The ML team wants to clean the data (impute missing values and remove outliers) while minimizing additional bias toward newer equipment models.

Which approach should the team take? (Select TWO)

A) Impute missing temperature values with the mean across the entire dataset regardless of equipment model.
B) Impute missing temperature values separately for each equipment model group, using model-group-specific medians.
C) Remove all readings where temperature > 150°C or vibration > 80 mm/s using statistical thresholding (e.g., z-score or IQR), after verifying these are physically impossible sensor faults.
D) Use SageMaker Data Wrangler to drop all rows from older equipment models to reduce the imbalance with newer models.
E) Use SageMaker Clarify post-training bias analysis to identify outliers.

Click to reveal Answer & Explanation

Correct Answers: B and C

Correct Options Analysis

B is Correct: Group-specific imputation (median by equipment model) preserves the central tendency within each subgroup, reducing the risk of pulling older equipment model readings toward the global mean. This preserves subgroup-level variance and avoids introducing additional bias where the missingness is non-random (higher for older models).
C is Correct: Outlier removal (or winsorization) is appropriate only after verifying the values are physically impossible due to sensor malfunction (e.g., temperature > 150°C in normal operating conditions). Using statistical methods (IQR, z-score) while preserving the equipment model context is standard practice before training.

Distractor Analysis

A is Incorrect: Global mean imputation will pull older equipment models' temperature values toward the dataset-wide mean, further reinforcing the representation gap where newer models dominate and artificially reducing variance for the older equipment group.
D is Incorrect: Dropping rows from older equipment models is counterproductive—it increases the existing representation bias by removing even more data from the underrepresented group. The correct approach would be to remedy class imbalance via augmentation or weighting, not delete the minority subgroup.
E is Incorrect: SageMaker Clarify does not identify outliers. Its purpose is bias metric computation and model explainability, not statistical anomaly detection. Outliers should be detected via robust statistical methods in Data Wrangler or Glue DataBrew profile jobs.

Key Takeaway: When missingness is non-random (affects subgroups disproportionately), use subgroup-aware imputation (e.g., group-specific medians), not global imputation, to avoid introducing or compounding bias.

Question 7 - Data Splitting for Representation Preservation

A telecommunications company is building a churn prediction model using Amazon SageMaker. The dataset contains 800,000 customer records with features across multiple regions (East, West, Midwest, South) and demographic groups. The churn label distribution is imbalanced (5% churn, 95% non-churn). The data collection pipeline historically appended records grouped by region followed by join date, causing temporal and regional ordering effects.

The ML team needs to split the dataset into training, validation, and test sets while ensuring:

Each split maintains the same proportion of churn labels.
No region is isolated into a single split.
The temporal and regional ordering effects do not leak into the training process.

Which combination of techniques should be used? (Select ONE)

A) Perform a simple random split (75/15/10) using scikit-learn's train_test_split, then shuffle the training set only.
B) Perform stratified splitting (preserving churn proportions) on the entire dataset after shuffling, using a fixed random seed.
C) Perform chronological splitting based on join date, then oversample churn records in the training set.
D) Use SageMaker Autopilot to automatically determine the optimal split for bias-free training.
E) Group records by region and date, then manually assign all instances from one region to the test set to prevent regional leakage.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

B is Correct: Stratified splitting ensures each split maintains the same proportion of churn vs non-churn labels. Shuffling before splitting breaks regional and temporal ordering artifacts, preventing correlated batches in mini-batch gradient descent and ensuring region groups distribute evenly across splits. Setting a fixed random seed ensures reproducibility, a critical requirement for ML audits.

Distractor Analysis

A is Incorrect: train_test_split in scikit-learn by default performs simple random splitting, not stratified. Without the stratify=y parameter, a random 75/15/10 split may result in some splits containing 3% churn while others contain 7% churn, making evaluation unstable. Also, setting shuffle=True inside the function but failing to stratify still risks label proportion imbalance.
C is Incorrect: Chronological splitting (by join date) preserves the temporal ordering bias and may isolate newer periods with different churn patterns into a single set. Oversampling alone does not fix distributional differences across time periods.
D is Incorrect: SageMaker Autopilot automates model training and feature engineering but does not "automatically determine bias-free splits." The user controls the split configuration, and Autopilot does not detect region grouping effects or automatically enforce stratified churn proportions.
E is Incorrect: Assigning all instances from one region to the test set creates a severe domain shift (distributional mismatch) between training and test. The model is never trained on that region's data, and evaluation on an unseen region alone produces biased, non-generalizable results.

Key Takeaway: Shuffle + stratified split with a fixed seed is the canonical technique to preserve label proportions and prevent ordering biases from leaking into training, validation, and test splits.

Question 8 - Content Safety Screening Before Model Customization

A cybersecurity firm is building a fine-tuning dataset for Amazon Bedrock to customize a base model for threat detection and incident response. The dataset contains 200,000 prompt-response pairs collected from internal SOC (Security Operations Center) chat logs. The compliance team requires that content with:

profanity or offensive language
sensitive information (passwords, IP addresses, employee names)
confidential security incident codes

must be blocked before reaching the customization workflow, and the screening must be fully automated.

Which approach meets these requirements with minimal ongoing operational overhead? (Select TWO)

A) Create Amazon Bedrock Guardrails with content filters (moderate/high threshold) and sensitive information filters (e.g., IP address, name, password) to screen prompts and responses before invoking the customization API.
B) Manually review a random 1% sample of the 200,000 pairs each week for quality, then approximate safety for the rest.
C) Use Amazon Comprehend's DetectPII API to identify PII in English text, then create a Lambda function to filter the prompt-response pairs accordingly.
D) Use Amazon Rekognition's content moderation API to screen the prompt-response pairs for profanity and offensive content.
E) Use Amazon Macie to scan the S3 bucket containing the prompt-response pairs and flag objects containing sensitive information.

Click to reveal Answer & Explanation

Correct Answers: A and C

Correct Options Analysis

A is Correct: Amazon Bedrock Guardrails offers configurable content filters (toxicity thresholds), sensitive information filters (IP, name, password, etc.), and word/denied topic filters. These are evaluated inline before the prompt and response pass to the customization workflow, providing fully automated, low-latency, and scalable screening without custom code.
C is Correct: Amazon Comprehend's DetectPII API automatically identifies PII entity categories (names, IP addresses, email addresses, etc.) in text. A Lambda function to filter flagged prompt-response pairs is a standard batch screening approach that leverages a managed NLP service without additional model training.

Distractor Analysis

B is Incorrect: Manual review of 1% of 200,000 pairs is slow, non-comprehensive, and introduces sampling bias. It does not provide automated, reliable screening for the remaining 99% and fails to meet compliance requirements for consistent automated blocking.
D is Incorrect: Amazon Rekognition's content moderation API is designed for image and video moderation (e.g., explicit visual content), not text screening. Text-based profanity/toxicity detection would require Amazon Comprehend's Toxicity Detection API or Bedrock Guardrails' content filters.
E is Incorrect: Amazon Macie is an S3 data discovery service that scans S3 buckets for sensitive data (e.g., PII) but is not designed for real-time screening before model customization. Macie performs periodic or one-time scans and cannot block individual prompt-response pairs inline like Bedrock Guardrails can.

Key Takeaway: For automated content safety screening, use Bedrock Guardrails (inline) + Amazon Comprehend DetectPII (batch); avoid manual sampling and Macie (S3 data discovery, not inline screening) or Rekognition (image moderation only).

Question 9 - Class Imbalance Metrics for Model Evaluation

An insurance company is training a claim fraud detection model. The dataset contains 150,000 claims, of which 1,500 are fraudulent (1% positive class). The model is trained and the team receives a report showing 99.5% overall accuracy on the test set. The fraud detection team suspects this is misleading due to the accuracy paradox.

Which metrics should the team use to properly evaluate the model's fraud detection performance? (Select TWO)

A) Precision for the fraudulent class: True Positives / (True Positives + False Positives)
B) Recall for the fraudulent class: True Positives / (True Positives + False Negatives)
C) Overall accuracy: (True Positives + True Negatives) / Total Samples
D) AUC-ROC computed on the dominant (non-fraud) class only
E) Area Under the Precision-Recall Curve (PR-AUC) focused on the fraudulent class

Click to reveal Answer & Explanation

Correct Answers: A, B, and E

Note: The question states "Select TWO" but the correct answer architecture requires three metrics for a comprehensive fraud detection evaluation. For exam purposes, if a question states "Select TWO," the most representative choices are typically Precision and Recall for the minority class; PR-AUC is a complementary metric but may be included if the question asks for "select all that apply."

Correct Options Analysis

A is Correct: Precision measures the fraction of predicted fraud cases that are actually fraud. High precision means fewer false alarms, which is critical in fraud detection where false positives can block legitimate transactions.
B is Correct: Recall measures the fraction of actual fraud cases correctly identified by the model. High recall means fewer missed fraud cases, which directly impacts financial losses.
E is Correct: PR-AUC is robust to extreme class imbalance and is the preferred metric over AUC-ROC when positive class prevalence is < 10%. It focuses on the trade-off between precision and recall for the minority (fraud) class.

Distractor Analysis

C is Incorrect: Overall accuracy is misleading when class imbalance exists (the accuracy paradox). A model that always predicts "non-fraud" would achieve 99% accuracy despite detecting zero fraud cases.
D is Incorrect: AUC-ROC computed only on the dominant class is meaningless—it ignores the fraud class entirely and evaluates only non-fraud detection, providing no insight into the model's ability to detect fraud.

Key Takeaway: For imbalanced classification problems, evaluate using Precision, Recall, and PR-AUC—never rely on overall accuracy, which masks minority-class failure.

Question 10 - Data Deduplication and Near-Duplicate Bias

A marketing analytics company is building a customer segmentation model using Amazon SageMaker. The training dataset contains 2.5 million customer records from multiple data sources (web logs, CRM exports, email engagement). Upon profiling, the team discovers:

120,000 near-duplicate records that differ only in the email timestamp field.
50,000 records with identical customer ID but conflicting demographic data (e.g., different age values in different rows).
The dataset overrepresents customers from a single geography (West region) at 72%, while the actual customer base has a 25% West region distribution.

Which steps should the team take to prepare the data for training? (Select TWO)

A) Deduplicate the 120,000 near-duplicate records by keeping the most recent record per CustomerID, as duplication inflates patterns and can overemphasize frequencies of certain behaviors.
B) Deduplicate all customer records based on CustomerID only, keeping one random row per ID to reduce the dataset size.
C) Use SageMaker Data Wrangler to apply a random undersampling of the West region to align its proportion closer to the actual 25% customer base proportion.
D) Leave the conflicting demographic data as-is because SageMaker training can automatically handle it.
E) Use Amazon Rekognition to deduplicate customer records by visual similarity.

Click to reveal Answer & Explanation

Correct Answers: A and C

Correct Options Analysis

A is Correct: Near-duplicate records inflate feature frequency statistics and can overemphasize patterns from repeated observations. Removing duplicates while retaining the most recent record ensures the model trains on the latest customer profile rather than multiple timestamps of the same identity.
C is Correct: When a single geography is overrepresented (72% vs 25% actual), random undersampling of that region aligns the distribution closer to the true population. This reduces representation bias and prevents the model from overfitting to West-region-specific patterns.

Distractor Analysis

B is Incorrect: Randomly selecting one row per CustomerID may pick inconsistent data (e.g., conflicting age values). The team should first consolidate the conflicting demographic fields (e.g., pick the most recent or majority value) before selecting one record per CustomerID. Random row selection is not a principled deduplication strategy.
D is Incorrect: SageMaker training does not automatically resolve conflicting demographic data. Conflicting values introduce noise and ambiguity, degrading model quality. They must be resolved during preprocessing (e.g., rule-based merging, source priority).
E is Incorrect: Amazon Rekognition is an image/video analysis service, not a record deduplication engine. Deduplicating tabular customer records by "visual similarity" is not a valid use case for Rekognition.

Key Takeaway: Deduplicate with principled rules (most recent/majority) and correct geographic/group representation bias via undersampling or weighted sampling; never rely on random row selection or imagery-based services for tabular deduplication.

Quick Recap Summary Table

Modality	Bias Mitigation Technique	AWS Service	Key Metric
Numeric	Pre-training bias report	SageMaker Clarify	DPL, KS, Class Imbalance
Numeric	Declarative data quality	AWS Glue Data Quality	Completeness, uniqueness, ranges
Text	Class weighting in loss	SageMaker training	F1, Recall
Text	Text augmentation (paraphrasing)	Amazon Bedrock (Titan Text)	Semantic equivalence
Image	Data augmentation (rotation, flips)	SageMaker training	Per-class accuracy
Image	Label distribution evaluation	Amazon Rekognition Custom Labels	Per-label precision/recall
All	Stratified splitting	SageMaker Data Wrangler / custom	Proportion preservation