Skip to content

Skill 1.3.1: Validate data quality (for example, by using DataBrew, AWS Glue Data Quality)

Question 1 - Declarative Data Quality Rules for Automated Pipeline Validation

A financial services company is building an automated ML pipeline to predict loan default risk. The data engineering team ingests daily transaction records from multiple source systems into an Amazon S3 staging bucket, which then feeds an AWS Glue ETL job that transforms data into the AWS Glue Data Catalog.

The compliance team requires that on every pipeline run, the following checks must pass before model training:

  • Customer ID completeness must be at least 99%
  • Transaction amount must be between $0 and $1,000,000
  • No duplicate transaction records are permitted

The ML engineering team must implement these rules with minimal operational overhead and no custom validation code.

Which solution should the ML engineer implement?

  • A) Create an AWS Lambda function that uses Pandas to check for null values, validate ranges, and drop duplicates before writing to the Data Catalog.
  • B) Author a Data Quality Definition Language (DQDL) ruleset in AWS Glue Data Quality and attach it to the Glue ETL job so rules are evaluated after every transformation run.
  • C) Create a DataBrew recipe that visually removes null values and duplicates, and schedule the recipe job to run daily before training.
  • D) Enable Amazon Macie on the S3 bucket to detect sensitive data and automatically enforce completeness and range checks on every object upload.
  • E) Configure Amazon CloudWatch alarms to monitor the S3 bucket for missing transaction records and invoke a step function when anomalies are detected.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

  • B is Correct: AWS Glue Data Quality uses the Data Quality Definition Language (DQDL) to define declarative, repeatable rules such as Completeness "customer_id" >= 0.99, ColumnValues "transaction_amount" between 0 and 1000000, and Uniqueness "transaction_id" = 1.0. These rulesets are evaluated natively within AWS Glue ETL jobs on every run, producing data quality scores and preventing bad data from propagating downstream—without any custom code.

Distractor Analysis

  • A is Incorrect: Writing a custom Lambda function with Pandas introduces unnecessary operational overhead (code maintenance, error handling, scaling) compared to AWS Glue Data Quality's built-in declarative ruleset evaluation.
  • C is Incorrect: AWS Glue DataBrew provides interactive data profiling and cleaning, but it does not support declarative, automated DQDL rules evaluated on every pipeline run. DataBrew recipes are transformation workflows, not automated quality gates with explicit rule definitions.
  • D is Incorrect: Amazon Macie is a data security and privacy service that discovers and protects sensitive data (e.g., PII) using machine learning and pattern matching. It does not validate completeness, value ranges, or uniqueness for ML data quality purposes.
  • E is Incorrect: CloudWatch alarms can notify on metrics, but they do not natively enforce data quality rules against dataset columns. This approach would require building custom monitoring logic and lacks the declarative rule evaluation of AWS Glue Data Quality.

Key Takeaway: AWS Glue Data Quality with DQDL is the native AWS service for declarative, repeatable data quality rule evaluation integrated directly into ETL pipelines—no custom validation code required.

Question 2 - Data Profiling Before Cleaning Strategy Selection

A healthcare analytics team has received a new dataset containing patient readmission records from three different hospital systems. The team notices that some columns have unexpected data formats, and they suspect missing values and outliers exist across multiple features.

Before selecting any data cleaning approach, the team needs to automatically profile the dataset to understand:

  • The percentage of missing values per column
  • Value distributions and unexpected ranges
  • Data type inconsistencies

The team wants a visual interface with minimal setup and no need to write profiling code.

Which AWS service should the team use to profile this dataset?

  • A) AWS Glue DataBrew - Create a profile job to run data profiling and generate a visual summary of missing values, distributions, and format inconsistencies.
  • B) AWS Glue Data Quality - Author a DQDL ruleset with Completeness and ColumnValues rules to evaluate the dataset.
  • C) Amazon QuickSight - Import the dataset as a data source and create visual dashboards showing missing values and outliers.
  • D) Amazon SageMaker Data Wrangler - Import the dataset into a Data Wrangler flow and use the built-in data quality report to view profiles and distributions.
  • E) AWS Glue Studio - Create a visual ETL job that automatically profiles data before transformation.
Click to reveal Answer & Explanation

Correct Answer: A

Correct Option Analysis

  • A is Correct: AWS Glue DataBrew is specifically designed for visual data profiling and preparation. A DataBrew profile job automatically scans the dataset and generates a comprehensive profile report showing missing value counts, outliers, data type inconsistencies, and value distributions—all without writing any code. This aligns perfectly with the requirement to understand the defect profile before selecting a cleaning strategy.

Distractor Analysis

  • B is Incorrect: AWS Glue Data Quality evaluates rules you already know and declare (e.g., "Completeness > 0.95"). It does not help you discover the profile or understand what cleaning is needed—you must already know what defects exist to write the rules.
  • C is Incorrect: Amazon QuickSight is a business intelligence service for creating dashboards and visualizations. While it can show data distributions, it does not provide automated data profiling results (e.g., built-in missing value percentage summaries, format inconsistencies) specifically designed for data preparation workflows.
  • D is Incorrect: Amazon SageMaker Data Wrangler provides data quality reports, but it is primarily designed for ML feature engineering and transformation workflows. While Data Wrangler can profile data, the question specifies "minimal setup and no code" for a team that needs visual profiling—DataBrew is more directly suited for profiling without a model training context.
  • E is Incorrect: AWS Glue Studio is a visual interface for building ETL jobs, but it does not generate automated data profiling reports. It focuses on transformation orchestration, not exploratory data profiling.

Key Takeaway: Profiling comes before cleaning—you cannot choose a repair strategy before understanding the defect. AWS Glue DataBrew's profile job provides automated, code-free data profiling to surface missing values, outliers, and format inconsistencies.

Question 3 - Mean Imputation Effect on Data Distribution

A data scientist is preparing a training dataset for a predictive maintenance model. The dataset contains a numeric column sensor_temperature with approximately 8% missing values distributed randomly across the dataset. The data scientist decides to fill the missing values with the column's mean value.

What is the MOST ACCURATE description of the effect this choice has on the dataset?

  • A) It maintains the exact original distribution of the column while preserving all rows for training.
  • B) It preserves all rows for training but artificially reduces the variance of the feature, pulling its distribution toward the middle.
  • C) It removes 8% of rows from the dataset, which reduces the training data available but preserves the original distribution.
  • D) It converts the numeric column into a categorical feature because the imputed value is a constant.
  • E) It increases the variance of the feature because imputed values add noise to the distribution.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

  • B is Correct: Mean imputation replaces missing values with the average of observed values (\(\mu\)). This keeps all rows usable for training, but it reduces the overall variance and standard deviation of the feature because multiple records now have identical central values. The distribution becomes artificially concentrated toward the mean, which can bias downstream model estimates and narrow confidence intervals.

Distractor Analysis

  • A is Incorrect: Mean imputation does not preserve the original distribution exactly. It reduces variance, changes covariance with other features, and overstates the frequency of the mean value in the data.
  • C is Incorrect: Removing rows with missing values describes listwise deletion (dropping rows), which is the opposite of imputation. Mean imputation explicitly retains all rows.
  • D is Incorrect: Mean imputation on a numeric column retains its continuous numeric data type. The imputed value is still a meaningful numeric value on the same scale as the rest of the column.
  • E is Incorrect: Mean imputation does not add noise or increase variance. It does the opposite—it artificially reduces variance by injecting repeated central values.

Key Takeaway: Mean imputation trades variance integrity for row retention; the choice matters more when missing values are not randomly distributed, as systematic missingness can amplify bias.

Question 4 - Inter-Annotator Disagreement Root Cause Analysis

A team is building a supervised ML model to classify customer support tickets into priority categories (P1-Critical, P2-High, P3-Medium, P4-Low). The team has engaged two experienced annotators to label 10,000 tickets. During a quality review, the team discovers that the annotators consistently disagree on the boundary between P2-High and P3-Medium categories, but their agreement on other boundaries is high.

What is the BEST first action for the team to take?

  • A) Remove one of the annotators because their labeling quality is inconsistent with the other.
  • B) Collect more unlabeled support tickets to provide additional training data.
  • C) Review and clarify the labeling instructions and provide concrete examples of the P2-P3 boundary to both annotators.
  • D) Switch from human labeling to automated labeling using a pre-built Amazon SageMaker model.
  • E) Exclude all tickets near the P2-P3 boundary from the training dataset.
Click to reveal Answer & Explanation

Correct Answer: C

Correct Option Analysis

  • C is Correct: Persistent inter-annotator disagreement at a specific category boundary indicates ambiguous labeling instructions rather than individual annotator error. When multiple trained annotators consistently diverge at a boundary, the definitions, inclusion criteria, or examples in the labeling guide are unclear. Refining the instructions and adding clear positive/negative examples for edge cases directly addresses the root cause.

Distractor Analysis

  • A is Incorrect: Removing an annotator treats a symptom rather than the root cause. Consistent disagreement across experienced labelers points to systemic ambiguity in the labeling instructions, not individual worker error.
  • B is Incorrect: Additional unlabeled data will simply produce more inconsistently labeled examples. The bottleneck is label quality and boundary definition clarity, not data quantity.
  • D is Incorrect: Using automated labeling for the ambiguous boundary would propagate the same uncertainty into model-generated labels, potentially compounding the problem. Supervised model labels are only as good as their training data.
  • E is Incorrect: Excluding boundary cases avoids the issue but creates a model that cannot handle these real-world edge cases, which are often exactly where model errors occur most frequently. The model needs clear labels for these cases to learn the correct boundary.

Key Takeaway: Label quality sets the ceiling on model quality—when annotators persistently disagree, refine the labeling instructions, not the annotator roster.

Question 5 - Shuffling vs. Augmentation for Representation Bias

A machine learning team is building a multi-class image classification model for identifying plant species. The dataset was collected by a field team that captured images in sequential batches, grouped by geographic region. One region (Region X) is severely underrepresented, containing only 2% of the total images, despite representing 15% of the target deployment population.

The team is considering two techniques:

  1. Shuffling the dataset before creating training/validation splits
  2. Augmenting Region X images using rotation, flipping, and color jittering

Which statement BEST describes the effectiveness of these techniques for the underrepresentation problem?

  • A) Both shuffling and augmentation equally address the underrepresentation because both change the distribution of the training data.
  • B) Shuffling alone addresses the underrepresentation because it redistributes Region X images throughout the training batches.
  • C) Augmentation addresses the underrepresentation by generating additional variation for Region X, but shuffling alone only reorders existing records without adding any new data.
  • D) Neither technique is appropriate; the team must collect additional real-world data from Region X before any model training.
  • E) Shuffling is necessary and sufficient; augmentation would introduce synthetic data that violates the integrity of the dataset.
Click to reveal Answer & Explanation

Correct Answer: C

Correct Option Analysis

  • C is Correct: The underrepresentation is a data composition problem: Region X has too few images relative to its true population proportion. Augmentation addresses this specifically by generating additional synthetic variations of Region X images (increasing both quantity and diversity within the class). Shuffling, by contrast, is purely a permutation operation—it reorders existing records to randomize batch order but does not change class proportions or add new data. Shuffling is still valuable for breaking batch-order bias, but it cannot fix underrepresentation on its own.

Distractor Analysis

  • A is Incorrect: Shuffling and augmentation address different problems. Shuffling randomizes order; augmentation adds data. They are not equally effective for underrepresentation—only augmentation addresses the gap.
  • B is Incorrect: Shuffling does not "redistribute" Region X images in terms of quantity or proportion. The ratio remains 2% regardless of how the data is shuffled.
  • D is Incorrect: While collecting more real-world data is ideal, it may be impractical or expensive. Data augmentation is a recognized and effective approach for addressing underrepresentation, and it would be unreasonable to refuse all model training until more field data is collected.
  • E is Incorrect: Data augmentation is a standard, valid technique for introducing diversity into underrepresented classes. It does not violate dataset integrity when applied appropriately (e.g., with transformations that preserve label semantics). The statement that shuffling is "sufficient" is incorrect because shuffling alone cannot fix underrepresentation.

Key Takeaway: Shuffling fixes order bias; augmentation fixes underrepresentation. They are complementary, not interchangeable—reach for the technique that addresses the specific defect you have identified.

Question 6 - Class Imbalance Evaluation Metric Selection

A fraud detection team is training a binary classification model on credit card transaction data. The dataset contains 200,000 transactions, of which only 2,000 (1%) are fraudulent. The team evaluates the model and finds that it achieves 99% overall accuracy. However, upon further investigation, the model correctly identifies only 200 of the 2,000 fraudulent transactions.

Which metric should the team prioritize to properly evaluate the model's fraud detection capability? (Select TWO)

  • A) Overall accuracy
  • B) Precision for the fraudulent class
  • C) Recall for the fraudulent class
  • D) F1-score for the non-fraudulent class
  • E) Precision-Recall Area Under the Curve (PR-AUC)
Click to reveal Answer & Explanation

Correct Answers: C and E

Correct Option Analysis

  • C is Correct: Recall for the fraudulent class measures the proportion of actual fraud cases that the model correctly identifies (True Positives / (True Positives + False Negatives)). In fraud detection, catching fraud (even with some false positives) is typically more critical than avoiding false alarms. The current recall is only 10% (200/2000), which is unacceptable despite the high overall accuracy.
  • E is Correct: PR-AUC (Precision-Recall Area Under the Curve) is specifically designed for imbalanced datasets. It focuses on the positive (minority) class's trade-off between precision and recall, ignoring the overwhelming true negatives that dominate ROC-AUC and accuracy in imbalanced settings.

Distractor Analysis

  • A is Incorrect: Overall accuracy is the accuracy paradox: the model can achieve 99% accuracy by predicting "not fraud" for every transaction (since 99% of transactions are legitimate). This masks the model's failure to detect fraud.
  • B is Incorrect: Precision alone can be misleading. If the model predicts only 100 transactions as fraud and 90 are correct, precision is 90%—but it would miss 1,900 fraudulent transactions. Precision must be considered alongside recall.
  • D is Incorrect: The non-fraudulent class (majority class) is already well detected. Focusing on F1-score for the non-fraudulent class does not help evaluate the model's ability to detect fraud (the minority class of interest).

Key Takeaway: In imbalanced datasets, overall accuracy is misleading—prioritize recall for the minority class and PR-AUC to evaluate model performance on the class that actually matters.

Question 7 - Stratified Splitting for Group Representation

A healthcare AI team is building a model to predict patient readmission risk. The dataset contains patient records from five different hospital systems, and the team needs to split the data into training, validation, and test sets. One hospital system (Hospital E) has significantly fewer records than the others, representing only 5% of the dataset.

The team wants to ensure that every split contains a proportional representation of patients from all five hospital systems, so the model's evaluation is not biased by a hospital system being absent from a particular split.

Which technique should the team use for the data split?

  • A) Simple random splitting using an 80/10/10 ratio
  • B) Stratified splitting on the hospital_system column to maintain proportional representation across all splits
  • C) Time-based splitting by admission date
  • D) Group-based splitting where patients from the same hospital are assigned to the same split
  • E) Random shuffling followed by a deterministic split
Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

  • B is Correct: Stratified splitting ensures that each split (training, validation, test) maintains the same proportion of records from each hospital system as the original dataset. This prevents scenarios where Hospital E (5% of data) might be entirely absent from the validation or test set, which would leave the team unable to evaluate the model's performance on patients from that system. This is especially important when group representation matters for evaluation bias.

Distractor Analysis

  • A is Incorrect: Simple random splitting without stratification may randomly concentrate or exclude Hospital E from certain splits. With only 5% representation, there is a meaningful chance that validation or test sets contain too few or zero records from Hospital E, making evaluation unreliable for that group.
  • C is Incorrect: Time-based splitting is appropriate for temporal forecasting and prevents data leakage across time periods, but it does not guarantee proportional representation of hospital systems in each split. If Hospital E's data is concentrated in certain time ranges, time-based splitting could exacerbate underrepresentation.
  • D is Incorrect: Group-based splitting where each hospital is assigned wholly to a single split is the opposite of what is needed. This would mean the model trains on Hospitals A-D and is evaluated on Hospital E, making evaluation impossible for the training hospital population and unreliable for Hospital E.
  • E is Incorrect: Shuffling followed by a deterministic split is still a random split (just with a fixed seed). It does not guarantee proportional representation of hospital systems in each split.

Key Takeaway: Stratified splitting preserves group proportions across training, validation, and test splits—ensuring each group is properly represented for fair, unbiased evaluation.

Question 8 - Data Quality Ruleset for GenAI Prompt-Response Validation

A company is building a custom generative AI model using Amazon Bedrock model customization. The training data consists of 50,000 prompt-response pairs. Before training begins, the ML team must validate the integrity of the training dataset. The data validation must check:

  • Each prompt has a non-empty, well-formed response
  • No prompt-response pair is duplicated
  • Response length is within an expected range (10-2000 characters)
  • No pair contains PII or unsafe content

Which combination of AWS services BEST satisfies these requirements with minimal operational overhead? (Select TWO)

  • A) Author a DQDL ruleset in AWS Glue Data Quality with Completeness, Uniqueness, and ColumnLength rules for the prompt-response dataset in the Data Catalog.
  • B) Create an AWS Lambda function that manually checks each pair for formatting, duplicates, and length, and use Amazon Comprehend to detect PII.
  • C) Configure Amazon Bedrock Guardrails with sensitive information filters and content filters to screen for PII and unsafe content.
  • D) Use AWS Glue DataBrew to interactively review each prompt-response pair visually.
  • E) Use Amazon OpenSearch Serverless to index all prompt-response pairs and query for anomalies.
Click to reveal Answer & Explanation

Correct Answers: A and C

Correct Option Analysis

  • A is Correct: AWS Glue Data Quality's DQDL supports structural integrity rules such as Completeness "response" > 0.99, Uniqueness "prompt_id" = 1.0, and ColumnLength "response" between 10 and 2000. These declarative rules are evaluated automatically and repeatably, ensuring well-formed, non-duplicated data without custom code.
  • C is Correct: Amazon Bedrock Guardrails provides built-in sensitive information (PII) filters and content filters (toxicity, violence, etc.) that screen prompts and responses before they reach model customization. This is the natively integrated AWS solution for content safety in Bedrock workflows.

Distractor Analysis

  • B is Incorrect: A custom Lambda function for formatting, duplicates, and length checks introduces unnecessary operational overhead compared to AWS Glue Data Quality's declarative rules. While Amazon Comprehend can detect PII, the combination of custom code + Comprehend is less integrated than Guardrails for Bedrock-specific use cases.
  • D is Incorrect: AWS Glue DataBrew provides visual data profiling and cleaning, but manually reviewing 50,000 prompt-response pairs visually is impractical and violates the "minimal operational overhead" requirement. DataBrew does not provide automated structural validation rules or content safety screening.
  • E is Incorrect: OpenSearch Serverless is a search and analytics service. It is not designed for data quality validation or content safety screening for prompt-response pair integrity. This would require building custom indexing and anomaly detection logic.

Key Takeaway: For GenAI training data integrity, use AWS Glue Data Quality for structural validation (completeness, uniqueness, length) and Amazon Bedrock Guardrails for content safety screening (PII, toxicity)—both are natively integrated AWS services with no custom code required.

Question 9 - Deduplication Edge Case in Near-Duplicate Records

A customer analytics team is cleaning a dataset that contains customer interaction records. During data profiling, they discover that several records are near-duplicates: they share identical customer IDs, timestamps, and interaction types, but differ slightly in one text field (e.g., "preferred contact method": "email" vs. "Email" vs. "e-mail").

What is the BEST approach for handling these near-duplicate records?

  • A) Treat all records as distinct because they differ in at least one field, and proceed with the full dataset.
  • B) Drop all records containing text fields to eliminate the ambiguity.
  • C) Standardize text fields (e.g., lowercase, format normalization) and re-evaluate which records are true duplicates before deciding whether to drop or merge.
  • D) Retain only the most recent record for each customer and discard all others.
  • E) Impute the text field with the most common value across all records.
Click to reveal Answer & Explanation

Correct Answer: C

Correct Option Analysis

  • C is Correct: Near-duplicate records with slight text variations are ambiguous cases that require data normalization before deduplication decisions. By standardizing text fields (lowercasing, removing hyphens, consistent formatting), the team can determine which records truly represent duplicate interactions and which are genuinely distinct. This avoids falsely inflating patterns by treating the same interaction as multiple distinct events or incorrectly dropping valid unique records.

Distractor Analysis

  • A is Incorrect: Treating near-duplicates as distinct inflates whatever pattern they share (customer activity volume, contact preferences), leading to biased analytics and model training. The slight text variation does not guarantee genuine distinction.
  • B is Incorrect: Dropping all records containing text fields is destructive and unnecessary. Text fields often contain valuable information for analysis and ML models. A targeted deduplication approach is appropriate, not blanket removal of text data.
  • D is Incorrect: Keeping only the most recent record is appropriate for certain time-series deduplication scenarios, but it is not the correct general approach for near-duplicate records that may represent different interaction types. This approach loses valid data.
  • E is Incorrect: Imputing text fields with the most common value loses meaningful distinctions in the data. This is not a deduplication strategy; it is a value replacement that dilutes the signal in the text field.

Key Takeaway: Near-duplicate records are the ambiguous case—standardize first to identify true duplicates, then decide whether to drop or merge. Treating near-duplicates as distinct inflates the patterns they share.

Question 10 - Bias Metrics Across Multimodal Datasets

An AI research team is building a multimodal model that processes customer interactions including:

  • Numeric customer demographic data
  • Text transcripts from customer service calls
  • Image data from documents submitted by customers

The team wants to ensure the dataset does not introduce bias across any of these modalities. They plan to measure bias metrics across the full multimodal dataset.

Which statement BEST describes the correct approach for multimodal bias measurement?

  • A) Bias metrics computed on the numeric (tabular) portion of the dataset are sufficient, because numeric features are the primary drivers of model predictions.
  • B) Bias metrics must be applied separately to each modality—numeric, text, and image—because bias can manifest differently in each data type and must be checked in each.
  • C) Bias metrics should only be applied to the text and image modalities, because numeric data is inherently unbiased.
  • D) A single bias metric computed across the combined dataset is sufficient for all modalities.
  • E) Bias metrics are not necessary during data preparation; they should be applied only after the model is deployed.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

  • B is Correct: In multimodal datasets, bias can manifest differently in each modality. For example, numeric data may have representation bias (underrepresented demographic groups in tabular features), text may have language or sentiment bias (e.g., certain accents or dialects underrepresented in transcripts), and images may have visual representation bias (e.g., certain skin tones or document types underrepresented). Therefore, bias metrics must be applied separately to each modality to ensure comprehensive bias coverage and mitigation.

Distractor Analysis

  • A is Incorrect: Numeric bias metrics alone miss bias in text and image modalities. Each modality contributes to model decisions in a multimodal system, and bias in any one modality can propagate to model outputs even if numeric data is unbiased.
  • C is Incorrect: Numeric data is not inherently unbiased. Tabular data can contain representation bias (underrepresented groups), selection bias (systematic exclusion), and measurement bias (inconsistent data collection across groups). Ignoring numeric bias metrics would leave a significant blind spot.
  • D is Incorrect: A single combined metric would obscure modality-specific biases. Different modalities require different bias measurement techniques (e.g., demographic parity for tabular data, word association tests for text, visual representation analysis for images). Combining them dilutes the signal.
  • E is Incorrect: Bias detection starts before training and continues after deployment. The data preparation stage is where bias begins to be identified and mitigated. Waiting until post-deployment would be too late to correct systemic data biases without significant rework.

Key Takeaway: Multimodal datasets require bias checks in every modality—numeric, text, and image—because each data type can harbor distinct sources of bias that propagate into the model's learned representations.