Skip to content

Skill 1.1.1: Extract data from data sources

Question 1 - AI Training Data Integrity Validation

A financial services company is building a custom generative AI model using Amazon Bedrock to answer customer service questions about investment products. The training dataset consists of 50,000 prompt-response pairs collected from historical chat logs. Before fine-tuning the model, the ML team must validate that each prompt-response pair is well-formed, screen out any toxic or inappropriate content, and ensure no personally identifiable information (PII) remains in the training data. The team wants to minimize operational overhead while meeting these compliance requirements.

Which solution should the ML team implement to validate the AI training data integrity? (Select ONE)

  • A) Create an AWS Glue DataBrew recipe that profiles the prompt and response columns, detects missing values, and flags malformed rows before the data reaches the fine-tuning job.
  • B) Create an AWS Glue Data Quality ruleset using Data Quality Definition Language (DQDL) to evaluate prompt length, response non-emptiness, and PII presence on every data pipeline run.
  • C) Configure Amazon Bedrock Guardrails with content filters, sensitive information filters, and word filters to screen prompts and responses before the data reaches the fine-tuning job.
  • D) Use Amazon Comprehend to detect PII in the prompt-response pairs and use Amazon SageMaker AI Ground Truth to manually validate each pair's alignment before fine-tuning.
  • E) Ingest the dataset into Amazon SageMaker Data Wrangler and apply a custom bias report to detect malformed prompt-response pairs.
Click to reveal Answer & Explanation

Correct Answer: C

Correct Options Analysis

  • C is Correct: Amazon Bedrock Guardrails provides configurable filters—content filters (toxicity, harmful content), sensitive information filters (PII detection and masking), word filters (blocked profanity), and denied topics—that can screen both prompts and responses before they are used for model customization. This directly addresses structural and content safety requirements with minimal operational overhead since Guardrails is a managed service.

Distractor Analysis

  • A is Incorrect: AWS Glue DataBrew is designed for data profiling, cleaning, and normalization of structured datasets. While it can profile the prompt-response columns for missing values and basic anomalies, it does not screen for toxic content or validate semantic pairing between prompts and responses. It also requires additional recipe setup and does not provide content safety screening.
  • B is Incorrect: AWS Glue Data Quality evaluates declarative rules against structured datasets but cannot detect toxicity, inappropriate content, or PII in unstructured text. DQDL rules focus on completeness, uniqueness, and value ranges for columns—not content safety screening for generative AI training data.
  • D is Incorrect: While Amazon Comprehend can detect PII and Amazon SageMaker AI Ground Truth can provide human review workflows, this solution introduces significant operational overhead. Setting up a human review pipeline for 50,000 pairs is costly and slow. The scenario specifies minimizing operational overhead and wants an automated solution.
  • E is Incorrect: SageMaker Data Wrangler is designed for structured dataset transformation and feature engineering. It does not provide content safety screening or prompt-response pairing validation for generative AI training data.

Key Takeaway: Amazon Bedrock Guardrails is the purpose-built service for screening generative AI training data (prompt-response pairs) for toxicity, PII, and denied topics before model customization—it eliminates the need for custom validation pipelines.

Question 2 - Prompt-Response Pair Validation for Foundation Model Fine-tuning

A healthcare technology startup is preparing a dataset of 10,000 prompt-response pairs to fine-tune a foundation model on Amazon Bedrock for a medical triage assistant. The ML team discovers that approximately 15% of the pairs have structural issues: some responses do not answer their prompts, some prompts are empty, and some responses contain reference numbers that do not exist in the prompt context. The team wants to automatically detect these malformed pairs and quarantine them for review before fine-tuning.

Which combination of AWS services should the team use to validate the AI training data integrity with the least operational overhead? (Select TWO)

  • A) Use AWS Glue Data Quality with a DQDL ruleset to check that prompt_length > 0, response_length > 0, and response_not_null on every pipeline run.
  • B) Use Amazon Bedrock to generate embeddings for each prompt and response, then compute cosine similarity scores to detect pairs with low semantic alignment, quarantining pairs below a threshold.
  • C) Use Amazon Comprehend PII detection on all prompt-response pairs and remove any containing PII, ensuring the fine-tuning dataset is clean.
  • D) Use Amazon SageMaker Data Wrangler's built-in bias detection transforms to detect prompt-response misalignment and shuffle the dataset before training.
  • E) Configure Amazon Bedrock Guardrails with denied topics and sensitive information filters before the data reaches the fine-tuning job.
Click to reveal Answer & Explanation

Correct Answers: A and B

Correct Options Analysis

  • A is Correct: AWS Glue Data Quality provides declarative DQDL rules that can validate structural integrity—checking that prompt and response fields are non-empty and meet minimum length requirements. This is a repeatable, automated check that runs on the data pipeline before fine-tuning with minimal operational overhead.
  • B is Correct: Using Amazon Bedrock to generate embeddings and computing similarity scores (e.g., cosine similarity) is a valid technique to detect prompt-response pairs where the response semantically does not align with the prompt. Low similarity pairs can be quarantined for human review.

Distractor Analysis

  • C is Incorrect: Amazon Comprehend PII detection removes personal information but does not address the structural integrity issue of misaligned prompt-response pairs or empty prompts. PII screening is necessary but not sufficient for validating training data integrity in this scenario.
  • D is Incorrect: SageMaker Data Wrangler's bias detection and shuffling transforms are designed for structured dataset distribution analysis. Shuffling reorders existing records but does not detect or quarantine misaligned prompt-response pairs.
  • E is Incorrect: Amazon Bedrock Guardrails screens for toxicity and PII but does not detect whether a response semantically answers its prompt. Content safety screening and structural integrity validation are distinct requirements.

Key Takeaway: Validating AI training data integrity requires both structural checks (non-empty fields, format validation via Glue Data Quality) and semantic checks (embedding similarity between prompt-response pairs) to catch all malformed pairs before they reach fine-tuning.

Question 3 - Content Safety Screening for Training Data

An e-commerce company is building a product review summarization model using Amazon Bedrock to generate automated summaries of customer reviews. Before fine-tuning the model, the Data Science team discovers that some historical chat logs contain profanity, hate speech, and references to violence that could cause the model to produce inappropriate outputs. The team must ensure that all training data is screened for these safety concerns and that any filtered content is excluded from the fine-tuning dataset.

The solution must be fully managed and require minimal code development.

Which AWS service should the team use to screen the training data for content safety? (Select ONE)

  • A) Write an AWS Lambda function using Amazon Comprehend Custom Classification to detect toxic content in the training data.
  • B) Use Amazon SageMaker AI Ground Truth with a human review workflow to manually inspect all 50,000 prompt-response pairs for toxic content.
  • C) Configure Amazon Bedrock Guardrails content filters with appropriate toxicity thresholds and apply them to the training data before fine-tuning.
  • D) Use AWS Glue Data Quality with a custom DQDL ruleset containing regex patterns to flag prohibited words in the prompt and response columns.
  • E) Use Amazon OpenSearch Serverless with a vector index of known toxic phrases to detect and filter inappropriate content in the training dataset.
Click to reveal Answer & Explanation

Correct Answer: C

Correct Options Analysis

  • C is Correct: Amazon Bedrock Guardrails content filters are specifically designed to screen text for harmful content categories (hate speech, violence, profanity, etc.) using configurable thresholds. Guardrails is a managed service, requires no custom code development, and can be applied before data reaches a fine-tuning job—satisfying all requirements for minimal operational effort and effective content safety screening.

Distractor Analysis

  • A is Incorrect: Writing a custom Lambda function with Amazon Comprehend Custom Classification introduces unnecessary operational overhead, requiring model training, endpoint management, and code development—violating the minimal code development constraint. Amazon Comprehend also focuses on NLP tasks like PII detection and sentiment, not toxicity screening.
  • B is Incorrect: Human review of 50,000 prompt-response pairs via SageMaker AI Ground Truth is extremely time-consuming, costly, and operationally heavy. The scenario specifies a fully managed automated solution with minimal code, not a manual workflow.
  • D is Incorrect: AWS Glue Data Quality DQDL supports regex-based pattern matching but is not designed for semantic toxicity detection. A regex approach would miss variations of toxic language, require constant maintenance of pattern lists, and cannot detect nuanced harmful content. Bedrock Guardrails uses built-in content classifiers for more robust screening.
  • E is Incorrect: Amazon OpenSearch Serverless with a vector index is a search infrastructure solution, not a content screening service. Building a custom toxicity detection system on top of OpenSearch requires significant engineering effort and provides inconsistent detection quality compared to Bedrock Guardrails' purpose-built content filters.

Key Takeaway: Amazon Bedrock Guardrails content filters provide fully managed, code-free content safety screening for training data—always prefer this over building custom toxicity detection systems (Lambda, Comprehend, Glue, OpenSearch) when the goal is minimal operational overhead.

Question 4 - PII Detection and Redaction for Fine-tuning Data

A global logistics company is preparing a dataset of 100,000 customer support email threads to fine-tune a Amazon Bedrock model for a shipping inquiry assistant. The dataset contains customer names, addresses, tracking numbers, and credit card information scattered throughout the email threads. The company's privacy compliance policy requires that all PII be removed or masked before the data is used for fine-tuning. The team wants to minimize operational overhead and ensure PII detection is performed automatically at scale.

Which TWO AWS services should the ML team use to detect and mask PII in the training data? (Select TWO)

  • A) Use Amazon Comprehend's PII detection API to identify PII entities in the email threads and mask them with entity-type placeholders (e.g., [NAME], [ADDRESS]).
  • B) Use Amazon Bedrock Guardrails sensitive information filters to detect and block/mask PII (e.g., customer names, credit card numbers) during data preparation.
  • C) Use AWS Glue Data Quality with a DQDL ruleset containing regex patterns to detect PII in the prompt and response columns.
  • D) Use Amazon SageMaker Data Wrangler's built-in anomaly detection transform to identify PII-containing rows and flag them for manual review.
  • E) Use Amazon Macie to scan the training data stored in Amazon S3 for PII and automatically encrypt any files containing sensitive information.
Click to reveal Answer & Explanation

Correct Answers: A and B

Correct Options Analysis

  • A is Correct: Amazon Comprehend's PII detection API is purpose-built to identify PII entities in unstructured text (names, addresses, phone numbers, credit card numbers) at scale. It can output entity types and positions, allowing the team to replace them with type placeholders before fine-tuning. Comprehend is fully managed and requires no custom model training.
  • B is Correct: Amazon Bedrock Guardrails sensitive information filters can detect and mask PII in both prompts and responses. Guardrails can be configured to block or mask specific sensitive information types (CVV, credit card numbers, names, addresses) before data reaches the fine-tuning job—satisfying compliance requirements with minimal operational overhead.

Distractor Analysis

  • C is Incorrect: AWS Glue Data Quality with regex patterns can only detect narrowly defined PII patterns (e.g., 16-digit numbers matching credit card format). This approach is brittle, requires manual maintenance, and misses context-dependent PII (names, addresses) that require ML-based entity detection. Comprehend and Guardrails provide more robust PII detection with minimal effort.
  • D is Incorrect: SageMaker Data Wrangler's anomaly detection is designed for structured tabular data, not unstructured text. PII detection in free-form email text requires NLP-based entity recognition, not statistical anomaly detection on numeric columns.
  • E is Incorrect: Amazon Macie is designed to discover and classify sensitive data stored in S3 buckets, providing visibility into where PII exists—not to identify and mask PII within individual text documents for training data preparation. Macie operates at the file/bucket level, not at the entity level within documents.

Key Takeaway: For PII detection in unstructured training data, use Amazon Comprehend (entity-level detection for text) or Amazon Bedrock Guardrails (sensitive information filters for fine-tuning) rather than regex-based rules or S3-level scanning tools like Macie.

Question 5 - Detecting Malformed Prompt-Response Pairs with Embeddings

A startup is fine-tuning a Amazon Bedrock foundation model for a code generation assistant using 25,000 prompt-response pairs collected from internal developer forums. The team notices that some pairs are structurally complete but semantically mismatched—the response is well-formed but does not answer the corresponding prompt (e.g., a Python prompt with a JavaScript response). The team wants to detect and quarantine these semantically misaligned pairs before fine-tuning.

Which solution should the team use to detect semantically misaligned prompt-response pairs with the least operational overhead? (Select ONE)

  • A) Use Amazon Bedrock to generate vector embeddings for each prompt and response, compute cosine similarity, and flag pairs below a similarity threshold for review.
  • B) Train a custom classification model using Amazon SageMaker AI to identify misaligned prompt-response pairs, then apply the model during the data processing pipeline.
  • C) Use Amazon Bedrock Guardrails denied topics filters to block responses that reference topics unrelated to their prompts.
  • D) Use AWS Glue Data Quality with DQDL rules comparing the prompt's programming language to the response's programming language (e.g., prompt_language == response_language).
  • E) Use Amazon SageMaker AI Ground Truth to manually review all 25,000 pairs and assign a quality score to each pair.
Click to reveal Answer & Explanation

Correct Answer: A

Correct Options Analysis

  • A is Correct: Embedding-based similarity detection is the standard technique to identify semantically misaligned prompt-response pairs. By generating embeddings with Amazon Bedrock (managed service) and computing cosine similarity between prompt and response embeddings, the team can automate detection of pairs where the response does not semantically relate to the prompt. This approach requires no custom model training and leverages managed services with minimal operational overhead.

Distractor Analysis

  • B is Incorrect: Training a custom classification model for this task introduces significant operational overhead—data labeling, model training, endpoint deployment, and ongoing model maintenance. The scenario specifies minimal operational overhead, and embedding similarity is a simpler, well-established technique that does not require custom model development.
  • C is Incorrect: Amazon Bedrock Guardrails denied topics filters are designed to block specific thematic categories (e.g., political content, violence), not to detect semantic alignment between a prompt and its corresponding response. The misalignment here is between answer language/topic and question language/topic, not a safety or compliance violation.
  • D is Incorrect: AWS Glue Data Quality DQDL rules operate on structured column values and support basic checks like regex matching and string comparisons. While a regex check could detect language markers, it cannot detect the semantic relationship between prompt and response. DQDL is not designed for NLP-based semantic analysis.
  • E is Incorrect: Manual review of 25,000 pairs via SageMaker AI Ground Truth introduces massive operational overhead and time costs. Embedding similarity provides automated detection and can prioritize only the borderline cases for human review, dramatically reducing manual effort.

Key Takeaway: Embedding-based similarity is the primary technique for detecting semantically misaligned prompt-response pairs—use Amazon Bedrock's embedding models and cosine similarity thresholds to automate this check without custom model training.

Question 6 - Structural Validation of Generative AI Training Data

A company is preparing a large-scale dataset of 200,000 prompt-response pairs for fine-tuning a Amazon Bedrock model for a customer service chatbot. The team discovers several potential structural issues during initial data profiling: missing prompt fields, empty response strings, duplicated prompt-response pairs, and inconsistent formatting across different data sources. The team wants to automate structural validation with repeatable rules that run on every data processing pipeline execution before fine-tuning.

Which AWS service should the ML team use to declare explicit rules for validating the structural integrity of the training data? (Select ONE)

  • A) Amazon SageMaker AI Ground Truth with a custom data validation workflow
  • B) AWS Glue Data Quality using Data Quality Definition Language (DQDL) rulesets
  • C) Amazon Bedrock Guardrails with sensitive information filters
  • D) Amazon SageMaker Data Wrangler with custom Python transformation scripts
  • E) Amazon Comprehend Custom Classification to detect malformed prompt-response pairs
Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

  • B is Correct: AWS Glue Data Quality with DQDL rulesets is the purpose-built service for declaring explicit, repeatable data quality rules. DQDL supports checks for completeness (non-empty fields), uniqueness (no duplicate prompt-response pairs), string length constraints, and custom conditions. These rules are evaluated on every pipeline run, integrating directly with AWS Glue jobs and the Data Catalog.

Distractor Analysis

  • A is Incorrect: Amazon SageMaker AI Ground Truth is a human labeling service, not a data validation service. Manual validation of 200,000 pairs would take weeks, be extremely costly, and violate the need for automated, repeatable rules.
  • C is Incorrect: Amazon Bedrock Guardrails screens for content safety (toxicity, PII, denied topics), not structural integrity. Guardrails cannot detect duplicate pairs, missing fields, or inconsistent formatting.
  • D is Incorrect: SageMaker Data Wrangler with custom Python scripts can perform structural validation, but it requires significant coding effort and does not provide declarative, repeatable rules that run automatically on every pipeline execution. AWS Glue Data Quality's DQDL is specifically designed for this use case.
  • E is Incorrect: Amazon Comprehend Custom Classification requires training a custom classifier from scratch to identify malformed pairs, introducing significant operational overhead. Structural validation (missing fields, duplicates, format issues) is a deterministic problem best solved by declarative rules, not a trained NLP model.

Key Takeaway: AWS Glue Data Quality with DQDL is the standard for declaring and automating structural validation rules on training data—use it for deterministic checks like completeness, uniqueness, and format validation.

Question 7 - Detecting Duplicate Prompt-Response Pairs

A company is fine-tuning a Amazon Bedrock model for an internal knowledge assistant. The training dataset consists of 150,000 prompt-response pairs extracted from multiple sources (ticket systems, chat logs, wikis). The ML team discovers that many pairs are duplicated across sources with slight variations—some are exact duplicates, while others are near-duplicates with only punctuation or whitespace differences. Duplicates can cause the model to overfit to frequently repeated patterns. The team wants to remove duplicates before fine-tuning.

Which technique should the team use to detect and remove near-duplicate prompt-response pairs with minimal operational overhead? (Select ONE)

  • A) Write a custom Python script using fuzzy matching libraries (e.g., RapidFuzz) to detect near-duplicates in AWS Lambda.
  • B) Use AWS Glue Data Quality with a Uniqueness rule on the prompt column and the response column.
  • C) Use Amazon Bedrock to generate embeddings for each pair, compute similarity scores, and deduplicate pairs with embedding distance below a threshold.
  • D) Use Amazon SageMaker Data Wrangler's built-in deduplication transform.
Click to reveal Answer & Explanation

Correct Answers: C

Correct Options Analysis

  • C is Correct: Embedding-based deduplication is the most robust technique for detecting near-duplicate prompt-response pairs that differ in surface-level features (punctuation, whitespace, word order) but are semantically identical. Amazon Bedrock generates embeddings with minimal operational overhead, and a similarity threshold (e.g., cosine similarity > 0.95) identifies near-duplicate clusters. This method captures semantic rather than just lexical duplication.

Distractor Analysis

  • A is Incorrect: A custom Lambda function with fuzzy matching libraries requires manual code development, threshold tuning, and can be computationally expensive for 150,000 pairs. Fuzzy matching (edit distance, token similarity) may also miss semantically identical pairs that use entirely different wording.
  • B is Incorrect: AWS Glue Data Quality's Uniqueness rule only detects exact duplicates within a single column. It cannot detect near-duplicates with slight variations (whitespace, punctuation, paraphrasing). The scenario explicitly mentions near-duplicates, so exact matching is insufficient.
  • D is Incorrect: SageMaker Data Wrangler does not have a built-in deduplication transform that can detect near-duplicates in unstructured text. Data Wrangler's transformations target structured tabular data; near-duplicate detection for text requires either custom fuzzy matching or embedding-based approaches.

Key Takeaway: Use embedding-based deduplication (Amazon Bedrock + cosine similarity) to detect near-duplicate prompt-response pairs that exact-match rules (Glue Data Quality Uniqueness) would miss—semantic similarity captures variations beyond surface text differences.

Question 8 - Content Safety Screening for Multimodal Training Data

A media company is building a multimodal foundation model fine-tuning pipeline on Amazon Bedrock using a dataset containing both text captions and associated images. The training data was sourced from user-generated content, and the team discovers that some images contain graphic violence, while some text captions contain hate speech. The company's responsible AI policy requires that content safety screening cover both modalities—text and images—before the data reaches the fine-tuning stage. The team wants a fully managed solution with minimal operational overhead.

Which solution should the team implement to screen the multimodal training data for content safety? (Select ONE)

  • A) Use Amazon Bedrock Guardrails content filters for text screening and Amazon Rekognition's moderation API for image screening before the data reaches the fine-tuning job.
  • B) Use Amazon Comprehend to detect toxic text and write a custom convolutional neural network in Amazon SageMaker AI for image violence detection.
  • C) Use AWS Glue Data Quality with DQDL rules containing regex patterns for text toxicity and image file name patterns to flag potentially harmful images.
  • D) Use Amazon SageMaker AI Ground Truth with a third-party workforce to manually review all text-image pairs for content safety violations.
  • E) Use Amazon OpenSearch Serverless with a pre-trained vector index of toxic content to screen both text and images using similarity search.
Click to reveal Answer & Explanation

Correct Answer: A

Correct Options Analysis

  • A is Correct: Amazon Bedrock Guardrails provides content safety filters for text (hate speech, violence, profanity) as a managed service. Amazon Rekognition's moderation API (Content Moderation) detects unsafe images (violence, nudity) with pre-trained models—no custom training required. Together, these two managed services cover both modalities with minimal operational overhead.

Distractor Analysis

  • B is Incorrect: Training a custom CNN in SageMaker AI for image violence detection requires data labeling, model development, training, and endpoint management—significant operational overhead. Amazon Rekognition provides a fully managed, pre-trained moderation model that requires no custom development.
  • C is Incorrect: AWS Glue Data Quality with regex patterns cannot detect semantic toxicity in text (it would miss variations) and cannot analyze image content at all. File name patterns are unreliable for image safety screening since the visual content cannot be predicted from file names.
  • D is Incorrect: Manual review of all text-image pairs via SageMaker AI Ground Truth is feasible only as a secondary validation step for flagged content, not as the primary screening mechanism. This approach is prohibitively expensive and slow for large-scale training data.
  • E is Incorrect: Amazon OpenSearch Serverless with a vector index of toxic content is a custom-built solution that requires significant engineering effort and may produce inconsistent results. Bedrock Guardrails and Rekognition provide pre-trained, production-ready content moderation that requires no custom vector database or similarity search development.

Key Takeaway: For multimodal content safety screening, pair Amazon Bedrock Guardrails (text) with Amazon Rekognition's moderation API (images) to cover all modalities with fully managed services—avoid building custom models or manual review for primary screening.

Question 9 - Detecting Malformed Fields in Prompt-Response Data

A startup is preparing a dataset of 30,000 prompt-response pairs for fine-tuning a Amazon Bedrock model for a technical support assistant. The team discovers that some entries are malformed: the prompt field is missing in 2% of records, the response field contains only whitespace in 1.5% of records, and 3% of records have the response as a null value. The team wants to automatically detect and quarantine all records with missing or empty fields before the data reaches the fine-tuning job.

Which AWS service should the team use to declare rules that automatically flag records with missing or empty prompt/response fields? (Select ONE)

  • A) Amazon Comprehend's custom entity recognition to detect empty fields in the dataset
  • B) AWS Glue Data Quality with DQDL rules such as Completeness "prompt_column" > 0.98 and ColumnLength "response_column" > 1
  • C) Amazon Bedrock Guardrails with word filters configured to block empty strings
  • D) Amazon SageMaker Data Wrangler's anomaly detection transform to flag rows with empty values
Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

  • B is Correct: AWS Glue Data Quality with DQDL rules is purpose-built for this type of deterministic structural validation. Completeness rules check for non-null values in a column, and ColumnLength rules can verify that strings are non-empty (length > 1 for non-whitespace strings). The rules are declarative, repeatable, and can flag or quarantine rows that fail the checks.

Distractor Analysis

  • A is Incorrect: Amazon Comprehend's custom entity recognition is designed to identify custom entities (like product names or technical terms) in text, not to detect missing or empty fields in structured datasets. Using it for structural validation is inappropriate and requires custom model training.
  • C is Incorrect: Amazon Bedrock Guardrails word filters block specific words or phrases, not empty strings or null values. Guardrails operates on the content of text, not on the structural completeness of dataset fields.
  • D is Incorrect: SageMaker Data Wrangler's anomaly detection transform is designed for statistical anomaly detection on numeric data (e.g., outliers, unusual distributions), not for detecting empty or null values in text columns. Missing value detection is a deterministic check best handled by rule-based validation, not anomaly detection.

Key Takeaway: Use AWS Glue Data Quality with Completeness and ColumnLength DQDL rules to automatically detect missing or empty fields in training data—deterministic structural checks should not be delegated to ML-based services (Comprehend, Data Wrangler anomaly detection).

Question 10 - Detecting Inconsistent Prompt-Response Formatting

A company is aggregating prompt-response pairs from five different data sources for fine-tuning a Amazon Bedrock model. The team discovers that some sources store responses in JSON format, others in plain text, and some include metadata tags. Inconsistent formatting can degrade fine-tuning quality. The team wants to automatically profile the dataset to detect inconsistent formats and flag records that deviate from a declared expected format.

Which solution should the team use to detect inconsistent formatting across the training data with minimal operational overhead? (Select ONE)

  • A) Write a custom AWS Lambda function using Python's json library to parse each record and flag format deviations.
  • B) Use AWS Glue Data Quality with DQDL rules such as ColumnMatchesRegex to validate that all prompt and response fields conform to declared format patterns.
  • C) Use Amazon SageMaker AI Ground Truth to manually inspect all records and identify format inconsistencies.
  • D) Use Amazon Bedrock Guardrails denied topics filters to block records with formatting inconsistencies.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

  • B is Correct: AWS Glue Data Quality's DQDL supports ColumnMatchesRegex rules that can validate whether all values in a column conform to a declared regular expression pattern. The team can define separate regex patterns for each acceptable format (JSON, plain text, etc.) and flag any records that do not match. This is a declarative, repeatable, automated approach with minimal operational overhead.

Distractor Analysis

  • A is Incorrect: A custom Lambda function requires code development and maintenance, and scaling it to handle large datasets introduces operational overhead. AWS Glue Data Quality's DQDL rules provide equivalent validation with no custom code required.
  • C is Incorrect: Manual inspection of all records is infeasible for large-scale datasets (the scenario implies at least 30,000+ pairs). Human review should be reserved for borderline cases flagged by automated validation, not as the primary detection mechanism.
  • D is Incorrect: Amazon Bedrock Guardrails denied topics filters are designed to block content based on thematic topics (e.g., politics, violence), not to detect formatting inconsistencies. Format validation is a deterministic structural check handled by rule-based validation services like Glue Data Quality.

Key Takeaway: Use ColumnMatchesRegex DQDL rules in AWS Glue Data Quality to validate formatting consistency across training data fields—regex-based declarative rules are more maintainable and efficient than custom code for format validation.