Skip to content

Skill 1.1.1: Extract data from data sources

Question 1 - Class Imbalance in Numeric Datasets

A financial services company is building a fraud detection model using Amazon SageMaker. Their transaction dataset contains 5 million records, with fraudulent transactions representing only 0.3% of the data. The ML team initially trained a logistic regression model, but it achieved 99.7% accuracy while catching virtually no fraud. They need to handle this numeric class imbalance without collecting new data and with minimal architectural changes to their existing SageMaker training pipeline.

Which approach should the team implement? (Select ONE)

  • A) Apply Synthetic Minority Over-sampling Technique (SMOTE) to the minority class using SageMaker Data Wrangler's built-in transforms, then retrain the model.
  • B) Shuffle the training dataset before training to distribute fraudulent transactions evenly across batches.
  • C) Use AWS Glue Data Quality to set a rule that flags rows where the class label is fraudulent, automatically making the model pay more attention to them.
  • D) Create a new S3 bucket, manually duplicate all fraudulent transaction records to achieve 50/50 balance, and retrain using the combined dataset.
Click to reveal Answer & Explanation

Correct Answer: A

Correct Options Analysis

  • A is Correct: SageMaker Data Wrangler provides built-in transforms that include SMOTE for oversampling the minority class in tabular/numeric datasets. This generates synthetic samples of the rare class using nearest-neighbor interpolation, improving minority-class recall without requiring new data collection or significant pipeline changes.

Distractor Analysis

  • B is Incorrect: Shuffling only rearranges the order of existing records. It does not add or generate new fraudulent samples, and the fundamental imbalance ratio (0.3% vs 99.7%) remains unchanged.
  • C is Incorrect: AWS Glue Data Quality evaluates rules for data validation—it does not modify data distributions, change model training weights, or influence how a model learns. Declaring a rule that flags fraud records does not make the model attend to them more.
  • D is Incorrect: Manually duplicating records leads to severe overfitting because the model memorizes exact copies of rare class samples rather than generalizing patterns. The 50/50 ratio is also arbitrary and fails to address representational diversity.

Key Takeaway: SMOTE via SageMaker Data Wrangler is the standard, low-overhead approach for numeric class imbalance. Rely on precision, recall, F1, and PR-AUC—not raw accuracy—when evaluating imbalanced datasets.

Question 2 - Class Imbalance in Text Datasets

A healthcare analytics startup is building a text classification model using Amazon Comprehend custom classification to categorize clinical notes into medical specialties. The dataset contains 120,000 text records, where cardiology notes account for 45%, neurology for 35%, and oncology for only 5%. The business needs high sensitivity for oncology case identification due to triage implications. The team must resolve this text class imbalance using managed AWS services with minimal custom code.

Which approach is most appropriate? (Select ONE)

  • A) Use Amazon SageMaker Ground Truth to create new synthetic oncology notes by asking human labelers to write additional clinical note samples.
  • B) Downsample the cardiology and neurology classes randomly until all three classes are equal, then train the Comprehend custom classifier.
  • C) Use Amazon Augmented AI (A2I) to automatically generate paraphrased oncology notes and upsample the minority text class.
  • D) Use Amazon Bedrock with a text generation model to generate synthetic oncology notes, then add these to the training data before custom classifier training.
Click to reveal Answer & Explanation

Correct Answer: D

Correct Options Analysis

  • D is Correct: Amazon Bedrock provides access to foundational models (like Claude, Titan, etc.) capable of generating context-aware synthetic text data. Generating additional oncology notes through a well-prompted LLM is a scalable, managed approach to augmenting an underrepresented text class with minimal custom code.

Distractor Analysis

  • A is Incorrect: SageMaker Ground Truth is a labeling/annotation service, not a synthetic data generation tool. Asking human labelers to write new notes does not leverage ML augmentation and is time-consuming, expensive, and does not scale well for 120,000+ records.
  • B is Incorrect: While downsampling majority classes is a valid imbalance technique, randomly discarding 75% of cardiology and neurology data wastes valuable information. This method is counterproductive when you already have abundant high-quality data.
  • C is Incorrect: Amazon Augmented AI (A2I) is a human-review workflow service for validating model predictions, not a text generation or augmentation service. It has no paraphrasing or synthetic text generation capabilities.

Key Takeaway: For text class imbalance, generative models via Amazon Bedrock offer a managed way to create synthetic minority-class samples. Balance evaluation sensitivity with data preservation—downsampling discards valuable signal when you have abundant data.

Question 3 - Class Imbalance in Image Datasets

A manufacturing company is building an automated visual defect detection system using Amazon SageMaker with a convolutional neural network. Their image dataset contains 80,000 normal product images and only 500 defect images representing 8 different defect categories. The team needs to boost defect detection performance with least operational overhead and must preserve the integrity of the original defect samples.

Which combination of actions should the team take? (Select TWO)

  • A) Apply data augmentation techniques (rotation, flipping, cropping, brightness adjustment) only to the defect images using SageMaker Data Wrangler or SageMaker image classification built-in algorithms.
  • B) Randomly oversample the defect images by duplicating them 160 times each to achieve class balance, then train the model.
  • C) Use AWS Glue DataBrew to perform image augmentation and generate synthetic defect samples from the existing defect image set.
  • D) Use weighted loss functions during SageMaker training to assign higher misclassification costs to the minority defect classes.
  • E) Shuffle the combined image dataset thoroughly before creating training and validation splits to address the class imbalance.
Click to reveal Answer & Explanation

Correct Answers: A and D

Correct Options Analysis

  • A is Correct: Image augmentation (rotation, flipping, scaling, color jitter) applied to the minority defect class creates varied synthetic samples from existing images, improving generalization without requiring new data. SageMaker built-in image classification algorithms support augmentation, or you can use SageMaker Data Wrangler or preprocessing jobs.
  • D is Correct: Weighted loss functions (e.g., class-weighted cross-entropy) directly address imbalance by penalizing the model more heavily for misclassifying minority class samples. This requires no data modification and preserves original sample integrity.

Distractor Analysis

  • B is Incorrect: Duplicating the same 500 defect images 160 times creates near-identical training samples, causing severe overfitting. The model memorizes specific defect instances rather than learning generalizable defect features.
  • C is Incorrect: AWS Glue DataBrew is a visual data preparation tool for tabular datasets (CSV, JSON, Parquet). It does not support image data transformation or augmentation.
  • E is Incorrect: Shuffling changes only the order of data, not the underlying class proportions. It does not alter the fundamental imbalance between 80,000 normal and 500 defect images.

Key Takeaway: Image imbalance is best addressed with augmentation on the minority class and/or class-weighted loss functions. Never duplicate minority samples—synthetic augmentation preserves diversity while boosting representation.

Question 4 - Bias vs. Imbalance Distinction

An e-commerce company is building a product recommendation model using customer demographic and purchase history data. During data profiling, the team discovers that while the dataset has equal numbers of male and female customer records, over 90% of purchase history records belong to customers from urban regions, despite rural customers making up 35% of their actual customer base.

The team must select the correct remediation strategy. What is the most appropriate approach? (Select ONE)

  • A) Apply SMOTE to oversample rural customer records to balance urban and rural purchase history classes.
  • B) Shuffle the dataset to remove any ordering bias introduced during data collection.
  • C) Collect additional purchase history data from rural customers or apply targeted augmentation to rural customer profiles.
  • D) Remove urban customer records randomly until the rural-to-urban ratio matches the actual customer base distribution.
Click to reveal Answer & Explanation

Correct Answer: C

Correct Options Analysis

  • C is Correct: This is a representation bias problem, distinct from class imbalance. The dataset misrepresents the actual population (35% rural but only 10% in data). Remediation requires either collecting additional real-world rural customer data or targeted augmentation for that underrepresented group. Shuffling, SMOTE, or downsampling of urban records does not solve the underlying representational gap.

Distractor Analysis

  • A is Incorrect: SMOTE is designed for outcome class imbalance (e.g., fraud vs. non-fraud), not representation bias. Using SMOTE to generate synthetic rural customers still relies only on the 10% of rural data available, which may be narrow and fail to capture true population diversity.
  • B is Incorrect: Shuffling reorders records only—it does not add missing population data. A dataset that genuinely underrepresents a group remains underrepresentative regardless of row order.
  • D is Incorrect: Randomly removing urban records discards valuable data and does not improve representation quality. This reduces model performance without solving the underlying issue of rural customer underrepresentation.

Key Takeaway: Class imbalance concerns outcome labels; representation bias concerns who/what is in the data. Imbalance remedies (SMOTE, downsampling) don’t fix population misrepresentation—you need more data from the underrepresented group.

Question 5 - Evaluation Metrics for Imbalanced Data

A telecommunications company is building a customer churn prediction model using Amazon SageMaker XGBoost. Their training data shows 92% of customers did not churn and only 8% churned. The business stakeholder is concerned because initial evaluation reported 92% model accuracy and wants to verify whether this is actually a strong predictor of churn.

Which evaluation approach should the ML team use to properly assess model performance? (Select TWO)

  • A) Report only the overall accuracy metric, since 92% demonstrates the model is correctly predicting most customers.
  • B) Evaluate the model using precision, recall, and F1-score, with particular attention to recall on the churn class.
  • C) Use a confusion matrix to check whether the model is simply predicting "no churn" for all customers.
  • D) Use the Area Under the Precision-Recall Curve (PR-AUC) instead of ROC-AUC, because PR-AUC is more informative for imbalanced datasets.
  • E) Balance the test dataset to a 50/50 split before evaluating the model.
Click to reveal Answer & Explanation

Correct Answers: B, C, D

Correct Options Analysis

  • B is Correct: Precision, recall, and F1 break down performance by class, showing whether the rare churn class is correctly identified. For imbalance problems, recall on the minority class is often the critical business metric.
  • C is Correct: A confusion matrix reveals whether the model has fallen into the accuracy paradox—predicting "no churn" for all examples and achieving 92% accuracy while catching zero churn cases.
  • D is Correct: PR-AUC is more sensitive to minority class performance than ROC-AUC when datasets are heavily imbalanced. It focuses on the positive (minority) class precision-recall tradeoff.

Distractor Analysis

  • A is Incorrect: Overall accuracy on an imbalanced dataset is misleading. A model that predicts "no churn" for all 100% of cases would achieve 92% accuracy but be useless for the business objective.
  • E is Incorrect: Altering the test dataset to 50/50 distorts the real-world class distribution. You should evaluate on a test set that maintains the original population distribution, not manipulate it.

Key Takeaway: For imbalanced datasets, use precision, recall, F1, confusion matrices, and PR-AUC. Accuracy is a misleading metric when one class dominates.

Question 6 - Pipeline-Level Imbalance Handling with SageMaker

A healthcare insurance company has a SageMaker training pipeline processing medical claims for early detection of rare chronic conditions. The pipeline ingests data via Amazon Kinesis and trains models using SageMaker Pipeline. The ML team needs to institutionalize class imbalance handling as part of a repeatable, automated workflow rather than a one-time manual fix.

Which architectural pattern should the team adopt? (Select ONE)

  • A) Use SageMaker Data Wrangler's built-in imbalance transforms in an interactive Data Wrangler Flow, then export the processed dataset to S3 for training. Run this manually before each training pipeline execution.
  • B) Embed a SageMaker Processing Job in the pipeline that applies imbalance correction logic (oversampling, class weights, or synthetic generation) using a custom script, ensuring the treatment runs automatically on every pipeline execution.
  • C) Use AWS Glue Data Quality after each training run to detect whether the model output was biased toward the majority class, and manually intervene when bias is detected.
  • D) Store the few minority class samples in a separate S3 bucket and have an AWS Lambda function duplicate them at the start of each training run.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

  • B is Correct: Embedding a SageMaker Processing Job in the pipeline applies imbalance treatment (SMOTE, class weights, augmentation, or undersampling) as a repeatable, automated step. This ensures consistency across every run without manual intervention and follows MLOps best practices.

Distractor Analysis

  • A is Incorrect: This approach requires manual intervention before each training run, violating the "repeatable, automated workflow" requirement. It also does not guarantee consistency across pipeline executions.
  • C is Incorrect: AWS Glue Data Quality validates data against declared rules and quality standards. It does not perform imbalance correction, and post-training detection with manual intervention is a reactive—not proactive—approach.
  • D is Incorrect: Using Lambda to duplicate minority samples is an unsophisticated approach that can lead to overfitting. Lambda also has runtime and memory constraints not suitable for large-scale synthetic data generation like SMOTE.

Key Takeaway: For automated, repeatable ML pipelines, integrate imbalance handling directly into SageMaker Pipeline via Processing Jobs. This ensures consistent, versioned data treatment with no manual step.

Question 7 - Image Class Imbalance with SageMaker Built-in Algorithms and Augmentation

A retail autonomous checkout startup is developing a computer vision model to identify produce items at self-checkout kiosks. Their image dataset contains 200,000 images of common produce (apples, bananas, tomatoes) but only 2,000 images for rare produce types (dragon fruit, persimmon). They use Amazon SageMaker Image Classification built-in algorithm.

The team must address the image imbalance while minimizing compute cost and avoiding custom code.

Which approach is most appropriate? (Select ONE)

  • A) Use the SageMaker Image Classification built-in algorithm's built-in augmentation capabilities, which automatically apply random perturbations (crop, flip, color jitter) to all classes equally during training, reducing the effective imbalance.
  • B) Use Amazon Rekognition Custom Labels to automatically balance the dataset by generating synthetic images for the rare produce classes.
  • C) Configure SageMaker Data Wrangler to apply SMOTE to the rare produce images, generating new images via pixel interpolation.
  • D) Use AWS Lambda with Pillow to programmatically augment rare produce images by applying rotations, flips, and brightness adjustments, then store the augmented images in S3 before training.
Click to reveal Answer & Explanation

Correct Answer: A

Correct Options Analysis

  • A is Correct: SageMaker built-in Image Classification algorithm automatically applies data augmentation (random cropping, flipping, color jitter, scaling) during training. This synthesizes variations across all classes but disproportionately benefits the minority class by exposing it to exponentially more variations, partially mitigating imbalance without custom code.

Distractor Analysis

  • B is Incorrect: Amazon Rekognition Custom Labels is a managed service for training custom image classification/detection models, but it does not generate synthetic images or automatically balance datasets. It also doesn't expose class-level balancing controls as a primary feature.
  • C is Incorrect: SageMaker Data Wrangler is designed for tabular data, not image data. SMOTE on images via pixel interpolation would create nonsensical, unrecognizable images because SMOTE is designed for numerical feature vectors, not high-dimensional pixel spaces.
  • D is Incorrect: While Lambda with Pillow can perform image augmentation, this introduces custom code and operational complexity. Lambda also has a 15-minute execution timeout and ephemeral storage limits, making it unsuitable for augmenting large image datasets. Contrast with Option A, which requires no custom code.

Key Takeaway: SageMaker built-in Image Classification algorithms automatically apply data augmentation during training. This provides inherent imbalance mitigation with zero custom code and no additional compute cost beyond normal training.

Question 8 - Generative AI Prompt-Response Pair Integrity Validation

A software company is building a conversational AI assistant using Amazon Bedrock with a custom fine-tuning dataset of 50,000 prompt-response pairs. Before fine-tuning their base model, the ML team must validate that all prompt-response pairs are well-formed, contextually aligned, and safe for training. They want to automate this validation with minimal operational overhead.

Which combination of AWS services should the team use? (Select TWO)

  • A) Use Amazon Bedrock Guardrails to screen both prompts and responses for harmful content (violence, hate speech, sexual content, PII) before data is used for fine-tuning.
  • B) Use Amazon SageMaker Ground Truth to have human reviewers manually inspect all 50,000 samples for contextual alignment and data quality.
  • C) Use Amazon Comprehend to detect personally identifiable information (PII) in prompt-response pairs and flag records containing sensitive data.
  • D) Use AWS Glue Data Quality to define rules that check for null or empty prompt/response fields, validate character length constraints, and identify duplicate prompt-response pairs.
  • E) Use Amazon Textract to extract structured information from the prompt-response pairs and automatically correct malformed responses.
Click to reveal Answer & Explanation

Correct Answers: A, C, D

Correct Options Analysis

  • A is Correct: Amazon Bedrock Guardrails provides configurable safety filters (content filters, denied topics, PII redaction) that automatically screen prompt and response pairs for harmful or unwanted content before they reach model customization. This is fully managed and requires no custom code.
  • C is Correct: Amazon Comprehend has built-in PII detection and redaction capabilities for English and Spanish text. It can automatically identify sensitive information (SSN, credit cards, names) in training data, supporting data safety compliance.
  • D is Correct: AWS Glue Data Quality supports declarative rules to check for null values, string length bounds, and uniqueness—validating structural integrity of prompt-response pairs (e.g., both fields non-empty, response length within token limits, no exact duplicates).

Distractor Analysis

  • A is fully correct: This is one of the valid approaches. Important note: no refutation needed here since it is selected. (Note to study guide: A, C, D combination is correct.)
  • B is Incorrect: Manually inspecting 50,000 samples via Ground Truth is extremely expensive, time-consuming, and violates the "minimal operational overhead" constraint. Ground Truth is for initial labeling/annotation, not large-scale validation of already-labeled data.
  • E is Incorrect: Amazon Textract is an OCR service for extracting text from images and PDFs. It has no capability to correct malformed prompt-response pairs or validate conversational data quality.

Key Takeaway: For GenAI training data integrity, combine Bedrock Guardrails (safety), Comprehend (PII), and Glue Data Quality (structural validation) for a fully managed, automated validation pipeline.

Question 9 - Missing Value Imputation Strategy for Biased Data

A credit scoring startup is building a loan approval model using Amazon SageMaker. During exploratory data analysis, the team discovers that 15% of income values are missing, and these missing values are disproportionately concentrated in low-income applicants (a historically underrepresented and economically disadvantaged group).

The team must choose an imputation strategy that does not systematically distort the income distribution for this underrepresented group.

Which approach is most appropriate? (Select ONE)

  • A) Use mean imputation by replacing all missing income values with the overall column average to maintain dataset completeness.
  • B) Use SageMaker Data Wrangler to apply multiple imputation by chained equations (MICE) or model-based imputation, which accounts for correlations between income and other variables (education, employment, region) when estimating missing values.
  • C) Drop all rows with missing income values to preserve the original distribution of the remaining data.
  • D) Replace all missing income values with 0, since these applicants likely have no measurable income.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

  • B is Correct: Multiple imputation or model-based imputation (e.g., regression imputation, k-NN imputation) uses relationships between income and other correlated features to estimate more realistic missing values. This prevents the systematic pull toward the middle caused by mean imputation and better preserves subgroup distributions. SageMaker Data Wrangler supports model-based imputation transforms.

Distractor Analysis

  • A is Incorrect: Mean imputation pulls the distribution toward the center by replacing all missing values with a single value. When missingness is concentrated in underrepresented groups, this artificially narrows variance and can mask the group's true economic profile.
  • C is Incorrect: Dropping 15% of rows significantly reduces dataset size and, if missingness is concentrated in one demographic, discards valuable data from an already underrepresented group. This exacerbates representation bias.
  • D is Incorrect: Imputing missing income as 0 assumes no income, which may be true in some cases but is a strong assumption that will systematically undervalue applicants with unreported income or those whose income data was lost. This hard-codes bias into the dataset.

Key Takeaway: When missing values are not randomly distributed (MNAR—missing not at random), use model-based imputation or multiple imputation, not simple mean/median substitution, to avoid distorting underrepresented groups.

Question 10 - Integrated Data Quality and Imbalance Framework

A large financial institution is modernizing its ML data preparation framework for multiple fraud detection, credit risk, and customer segmentation models. They want to implement a holistic, automated, and declarative approach to validate data quality and manage class imbalance across all ML projects, leveraging AWS-native services with centralized visibility.

Which architectural combination best meets these requirements? (Select TWO)

  • A) Use AWS Glue Data Quality with Data Quality Definition Language (DQDL) rulesets to declare completeness, uniqueness, and range validation rules for every dataset, executed automatically within AWS Glue jobs.
  • B) Use Amazon SageMaker Data Wrangler Data Quality and Insights (DQI) reports to automatically generate data quality profiles and recommend imbalance handling transforms for each dataset.
  • C) Use Amazon CloudWatch dashboards and alarms to monitor validation job successes/failures and data quality metrics, providing centralized operational visibility across all ML pipelines.
  • D) Use Amazon SageMaker Clarify during model training to automatically rebalance datasets and generate synthetic minority class samples for all imbalanced datasets.
  • E) Use Amazon Bedrock Guardrails to validate tabular data integrity and detect class imbalance in numerical datasets before model training.
Click to reveal Answer & Explanation

Correct Answers: A and C

Correct Options Analysis

  • A is Correct: AWS Glue Data Quality uses DQDL rulesets to declaratively define data validation rules (completeness, uniqueness, value ranges) that run automatically every time a Glue job executes. This provides systematic, repeatable validation across all ML datasets.
  • C is Correct: Amazon CloudWatch collects logs and metrics from Glue jobs, DataBrew, and other AWS services. Centralized dashboards and alarms ensure data quality failures are visible immediately rather than silently propagating into model training.

Distractor Analysis

  • B is Incorrect: While SageMaker Data Wrangler DQI reports are useful for exploratory analysis and interactively applying transforms, they do not provide fully automated, declarative, cross-project governance. Data Wrangler is designed for interactive data preparation rather than centralized validation across multiple ML projects.
  • D is Incorrect: Amazon SageMaker Clarify is designed for bias detection and model explainability—it analyzes trained models and datasets for bias metrics, feature attribution, and drift detection. It does not automatically rebalance datasets or generate synthetic minority samples.
  • E is Incorrect: Amazon Bedrock Guardrails is designed for screening generative AI prompts and responses (content filtering, PII redaction, denied topics). It cannot validate tabular data integrity or detect class imbalance.

Key Takeaway: For enterprise-scale data quality, combine Glue Data Quality (declarative rules) with CloudWatch (centralized monitoring) for automated, visible validation across all ML pipelines.