Skip to content

Domain 1: Data Preparation for ML and AI

Question: Storage for Distributed Deep Learning Training

Category: MLA – Data Preparation for ML and AI

Review Bucket:

  • [ ] Bucket A (Term Mapping)
  • [ ] Bucket B (Missed Constraint)
  • [X] Bucket C (Knowledge Gap)

A research institute is developing a deep-learning model to analyze satellite images for weather forecasting. The dataset consists of millions of high-resolution images stored in a format that requires file-level access. The model training process involves multiple GPU-powered Amazon EC2 instances that require high throughput and low latency access to the dataset for efficient distributed training.

Which AWS storage solution is the MOST suitable to support the deep-learning model training process?

  • A) Amazon FSx for Lustre
  • B) Amazon FSx for Windows File Server
  • C) Amazon Elastic File System (Amazon EFS)
  • D) Amazon Simple Storage Service (Amazon S3)
Click to reveal Answer & Explanation

Correct Answer: A

Correct Option Analysis

  • A is Correct: Amazon FSx for Lustre is a fully managed, high-performance file system designed for workloads such as machine learning, high-performance computing (HPC), and media processing. It provides POSIX-compliant file-level access, sub-millisecond latencies, and high throughput, making it ideal for distributed training across multiple GPU instances. It can also be linked to Amazon S3 to ingest data seamlessly.

Distractor Analysis

  • B is Incorrect: Amazon FSx for Windows File Server is built on Windows Server and is intended for Windows-based workloads, not for Linux-based GPU training environments. It does not offer the same level of performance for HPC or ML workloads as FSx for Lustre.
  • C is Incorrect: Amazon EFS is a scalable, general-purpose file system, but it is not optimized for the extreme throughput and low-latency requirements of distributed deep learning training. It may become a bottleneck when many GPU instances access large datasets concurrently.
  • D is Incorrect: Amazon S3 is object storage, not a file system. While it can be used with tools like S3 mountpoints, it does not natively provide the file-level access, high throughput, and low latency needed for efficient distributed training without additional complexity and potential performance trade-offs.

Key Takeaway: For high-performance, file-level access with low latency and high throughput in distributed ML training, choose Amazon FSx for Lustre. It is purpose-built for HPC and ML workloads, unlike general-purpose file systems or object storage.

self-notes

FSx for Lustre = high throughput

Question: AWS Services for Real-Time Data Ingestion and Processing

Category: MLA – Data Preparation for ML and AI

Review Bucket:

  • [ ] Bucket A (Term Mapping)
  • [ ] Bucket B (Missed Constraint)
  • [X] Bucket C (Knowledge Gap)

A machine learning engineer is working on an AWS project involving real-time fraud detection for an online payment platform. The platform generates a continuous stream of transaction data that needs processing and analysis using a pre-trained model on Amazon SageMaker AI.

Which combination of AWS services should the machine learning engineer choose to ingest and process the real-time transaction data before delivering it to SageMaker AI for inference? (Select TWO.)

  • A) Amazon Managed Service for Apache Flink
  • B) Amazon Managed Streaming for Apache Kafka (Amazon MSK)
  • C) AWS Glue
  • D) Amazon Data Firehose
  • E) Amazon Kinesis Data Streams
Click to reveal Answer & Explanation

Correct Answers: A, E

Correct Option Analysis

  • A is Correct: Amazon Managed Service for Apache Flink (formerly Amazon Kinesis Data Analytics) is a fully managed service for processing streaming data in real time using Apache Flink. It can consume data from Kinesis Data Streams or MSK, perform transformations, aggregations, and feature engineering in real-time, and then invoke SageMaker endpoints for inference.
  • E is Correct: Amazon Kinesis Data Streams is a highly scalable and durable real-time data streaming service designed to continuously capture gigabytes of data per second from sources such as payment transactions. It serves as the ingestion layer before processing and inference.

Distractor Analysis

  • B is Incorrect: Amazon MSK is a fully managed Apache Kafka service that can capture streaming data, but it is primarily an ingestion and buffering service, not a processing service. It does not natively integrate with SageMaker for inference without building custom Kafka consumers. Since Kinesis Data Streams is already a correct ingestion option, MSK as a processing option is redundant and less optimal for SageMaker integration.
  • C is Incorrect: AWS Glue is a serverless data integration service primarily designed for batch ETL (Extract, Transform, Load), data cataloging, and periodic data processing. It is not suitable for real-time streaming inference.
  • D is Incorrect: Amazon Data Firehose (formerly Kinesis Data Firehose) is designed to capture, transform, and load streaming data into data lakes, warehouses, and analytics services (e.g., S3, Redshift, OpenSearch). It does not natively support invoking SageMaker endpoints for real-time inference and is primarily for data delivery rather than processing before inference.

Key Takeaway: For real-time ML inference pipelines in AWS, choose Kinesis Data Streams for ingestion and Amazon Managed Service for Apache Flink for stream processing (feature engineering, transformations) before invoking SageMaker endpoints.

self-notes
  • Data Firehose: can ingest + deliver data, but in batch
  • Glue: Batch ETL only
  • MSK: Can handle streaming, but mainly used for real-time data transfer between apps and system. No integration with Sagemaker
If the Scenario Requires... Select This Service Why / Why Not Others
Sub-second real-time processing & complex transformations Kinesis Data Streams + Apache Flink Standard AWS native pair for real-time feature engineering & instant SageMaker inference.
Near real-time continuous loading into S3, Redshift, or OpenSearch Amazon Data Firehose Firehose is for delivery/loading with mandatory buffering (minimum 60s or 1MB). Not for sub-second pre-inference processing.
Micro-batch / Spark-based ETL on streaming data AWS Glue Streaming Based on Spark Streaming (micro-batches in seconds/minutes). Higher latency than Flink; best for ETL data lake ingestion.
Existing open-source Kafka ecosystem / low-level control Amazon MSK Choose MSK only if the scenario explicitly mentions "Kafka," "Zookeeper," or migrating an existing Apache Kafka pipeline.

Sub-Second Real-Time Analytics/Features: Kinesis Data Streams \(\rightarrow\) Apache Flink \(\rightarrow\) SageMaker Endpoint Data Lake Dump / Near Real-Time S3 Storage: Kinesis Data Streams \(\rightarrow\) Amazon Data Firehose \(\rightarrow\) Amazon S3

Question: Data Preparation for ML and AI

Category: MLA – Data Preparation for ML and AI

Review Bucket:

  • [X] Bucket A (Term Mapping)
  • [ ] Bucket B (Missed Constraint)
  • [ ] Bucket C (Knowledge Gap)

A retail company is building an ML model to forecast weekly product demand across multiple stores. An ML engineer uses Amazon SageMaker Data Wrangler to prepare several years of sales records that are maintained chronologically. The source data is cataloged in AWS Glue before being imported for processing.

The ML engineer needs to divide the dataset for model development so that the validation results reflect how the model is expected to perform as additional sales periods become available.

Which SageMaker Data Wrangler split transform should the ML engineer use?

  • A) A randomized split with a fixed random seed to distribute sales records across the training and validation datasets.
  • B) A stratified split based on predefined price ranges to preserve the distribution of product prices across both datasets.
  • C) A stratified split based on store location to preserve the proportion of records from each geographic area across both datasets.
  • D) An ordered split that divides the records according to the existing arrangement of the dataset.
Click to reveal Answer & Explanation

Correct Answer: D

Correct Option Analysis

  • D is Correct: The data represents a time series of weekly sales records. To simulate real-world conditions where future sales periods become available, the validation set should consist of the most recent chronological periods, while training uses older periods. SageMaker Data Wrangler’s ordered split divides records based on the existing chronological order of the dataset, preventing data leakage from future periods into the training set.

Distractor Analysis

  • A is Incorrect: A randomized split shuffles records, causing future sales periods to leak into the training data. This does not reflect how the model will be used on unseen future data and artificially inflates validation metrics.
  • B is Incorrect: A stratified split based on price ranges (or any non-temporal feature) also breaks the chronological ordering of the data. Random distribution of future and past records across both datasets leaks future information into the training process.
  • C is Incorrect: Stratifying by store location preserves the proportion of geographic data between splits, but like options A and B, it shuffles time-based records. This violates the requirement to validate on future periods and leads to data leakage in time-series forecasting.

Key Takeaway: For time-series forecasting, always use a temporal/ordered split (train on past, validate on future) rather than random or stratified splits, which can cause data leakage.

self-notes

Time = ordered

Question: SageMaker Data Wrangler & Time-Series Imputation

Category: MLA – Data Preparation for ML and AI
Review Bucket: Modified question from Dojo for revision

A logistics company is building a route optimization model using historical time-series delivery data stored in Amazon Redshift. An inspection reveals missing values across several time-dependent features. The ML team wants to implement a managed data preparation pipeline using Amazon SageMaker AI that connects directly to Redshift, handles the missing time-series values without introducing data leakage, and requires minimal custom code and operational overhead.

Which solution should the ML engineer implement?

  • A) Connect SageMaker Data Wrangler directly to Amazon Redshift, configure a time-series transform using Forward Fill (or Interpolation) on the missing values, and export the feature pipeline to a SageMaker Training Job.
  • B) Connect SageMaker Data Wrangler directly to Amazon Redshift, configure the built-in Numeric Impute transform using the global dataset Mean, and export the feature pipeline to a SageMaker Training Job.
  • C) Create a SageMaker Processing Job with a custom Python/Pandas container that queries Redshift via the Redshift Data API, applies df.fillna(method='ffill'), and saves the cleaned dataset to Amazon S3.
  • D) Ingest the raw Redshift data into an Amazon SageMaker Feature Store online store and enable SageMaker Clarify to automatically detect and impute missing feature values during ingestion.
Click to reveal Answer & Explanation

Correct Answer: A

Correct Option Analysis

  • A is Correct: SageMaker Data Wrangler natively connects to Amazon Redshift for data ingestion and provides built-in time-series transformations. Using Forward Fill or time-aware interpolation handles missing values appropriately without causing lookahead bias (data leakage), fulfilling the requirement for minimal custom code and low operational overhead.

Distractor Analysis

  • B is Incorrect: Global mean/median imputation on time-series data causes data leakage because global statistics incorporate future data points into historical timestamps.
  • C is Incorrect: While SageMaker Processing can perform this task, writing custom Python code and managing container execution introduces unnecessary operational and maintenance overhead compared to Data Wrangler's managed native transformations.
  • D is Incorrect: SageMaker Clarify evaluates models and datasets for bias and explainability (SHAP values); it does not perform automated data imputation on Feature Store ingestion.

Key Takeaway: For time-series data prep with minimal custom code, use SageMaker Data Wrangler with Forward Fill/Interpolation to avoid data leakage. Avoid global mean/median imputation on temporal datasets.

self-notes

Time series-> fill forward

Question: Redacting PII from Identity Documents for ML

Category: MLA – Data Preparation for ML and AI

Review Bucket:

  • [X] Bucket A (Term Mapping)
  • [X] Bucket B (Missed Constraint)
  • [ ] Bucket C (Knowledge Gap)

A global travel services company processes identity documents submitted as part of customer verification workflows. The company uses Amazon Textract to extract text from scanned passport pages. The extracted data is stored in Amazon S3 before being incorporated into a dataset for an ML workload.

An ML engineer must redact passport numbers from the extracted text before the dataset is used for model training. The company wants to avoid custom processing logic and reduce ongoing maintenance.

Which implementation would satisfy the requirements with minimal operational management?

  • A) Submit the extracted text to an asynchronous Amazon Comprehend PII analysis job. Configure the output settings to replace detected passport numbers with masking characters.
  • B) Call the Amazon Comprehend DetectPiiEntities API to locate passport numbers in the extracted text. Invoke AWS Lambda to mask the character ranges returned by the API.
  • C) Use Textract with the AnalyzeDocument API and the FORMS feature to identify passport number fields. Use the detected key-value pairs to locate and mask passport numbers.
  • D) Configure an Amazon Macie sensitive data discovery job for the S3 objects. Use the resulting findings to identify files containing passport numbers and initiate redaction processing.
Click to reveal Answer & Explanation

Correct Answer: A

Correct Option Analysis

  • A is Correct: Amazon Comprehend supports native PII detection and redaction. An asynchronous analysis job for PII can be configured to directly output a modified version of the text with detected entities (like passport numbers) replaced with masking characters. This requires no custom processing logic (no Lambda functions to write or maintain) and is fully managed by AWS. The asynchronous nature is also ideal for processing large batches of documents in S3.

Distractor Analysis

  • B is Incorrect: While DetectPiiEntities correctly identifies the PII and its character offsets, it only returns the location of the PII. This option requires building and maintaining a custom AWS Lambda function to perform the actual masking, violating the "avoid custom processing logic" requirement.
  • C is Incorrect: Using Textract's FORMS feature to detect key-value pairs is a valid text extraction technique, but it doesn't natively redact data. You would still need custom logic to parse the response and overwrite the data. Moreover, Amazon Comprehend PII detection is specifically designed to find entities like passport numbers regardless of their location in the document, making it a better fit than relying on particular field names.
  • D is Incorrect: Amazon Macie is used for discovery and alerting on sensitive data in S3 buckets (e.g., security, compliance). It is not an appropriate service for modifying or processing data. This option would require a complex integration (triggering a Lambda from Macie findings to perform redaction), which adds operational overhead rather than minimizing it.

Key Takeaway: For native PII identification and masking without writing code, use Amazon Comprehend's native redaction capabilities (available via the console or API) rather than building a pipeline. Macie is for visibility/compliance, not data transformation.

self-notes
  • Personal Identifiable Information (PII)

  • Use Comprehend for redaction

  • Lambda = extra maintenance

  • DetectPiiEntities doesn't return output

Architecture Workflow Diagram

flowchart TD
    subgraph Pipeline ["PII Redaction Pipeline"]
        direction TD
        A["Scanned Passport Page"] -->|1. Extract text| B("Amazon Textract")
        B -->|2. Store raw text| C[("Amazon S3<br>(Extracted Text)")]
        C -->|3. Submit PII redaction job| D("Amazon Comprehend<br>(Async PII Analysis)")
        D -->|4. Write redacted output| E[("Amazon S3<br>(Redacted Text)")]
        E -->|5. Load clean dataset| F["Dataset for ML Workload"]
    end

Data Transformation Example

Extracted Text from S3 (Before Redaction)
-----------------------------------------
Name: JUAN DELA CRUZ
Nationality: FILIPINO
Passport Number: AB1234567
Date of Birth: 1990-01-01

                   │
                   ▼ Amazon Comprehend Asynchronous PII Redaction
                     (Mask detected passport numbers)
                   │
                   ▼

Redacted Text in S3 (After Redaction)
-------------------------------------
Name: JUAN DELA CRUZ
Nationality: FILIPINO
Passport Number: *******
Date of Birth: 1990-01-01

Question: Cross-Account SageMaker Feature Store Architecture

Category: MLA – Data Preparation for ML and AI
Review Bucket: Modified question from Dojo for revision

An enterprise ML platform team manages a centralized AWS account where features are ingested for real-time fraud detection models. Downstream business units operating in separate AWS accounts need access to these shared features for both low-latency online inference (sub-millisecond lookups) and point-in-time historical model training datasets.

The platform team wants to establish a cross-account feature management architecture using Amazon SageMaker Feature Store while minimizing operational management and custom data duplication scripts.

Which implementation meets these requirements?

  • A) Create Feature Groups with both Online and Offline Stores in the central account. Share the Feature Groups across accounts using AWS Resource Access Manager (AWS RAM). Configure downstream inference endpoints and training pipelines to query the Online Store exclusively via the SageMaker Feature Store Runtime API to maintain feature consistency.
  • B) Create Feature Groups with both Online and Offline Stores in the central account. Grant cross-account access to downstream accounts by sharing the Feature Groups via AWS RAM. Retrieve low-latency features from the Online Store during inference, and run point-in-time feature joins on the Offline Store via Amazon Athena for training.
  • C) Create Feature Groups with an Online Store in the central account. Configure an AWS Lambda function triggered by DynamoDB Streams to mirror online feature updates into a central S3 bucket. Share the S3 bucket across accounts using S3 Bucket Policies for downstream training and inference.
  • D) Create Feature Groups in the central account using AWS Lake Formation to manage row-level cross-account permissions. Configure downstream applications to perform real-time feature lookups directly against the AWS Glue Data Catalog table endpoints.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

  • B is Correct: Native cross-account sharing for SageMaker Feature Store is configured using AWS Resource Access Manager (AWS RAM). The Online Store (backed by low-latency key-value storage) serves real-time inference lookups, while the Offline Store (backed by S3 and queryable via Athena) maintains historical feature records for generating point-in-time training datasets without data leakage.

Distractor Analysis

  • A is Incorrect: The Online Store only retains the most recent feature record for any given entity ID. Using the Online Store for model training leads to data leakage and time-mismatch errors because historical state is lost.
  • C is Incorrect: Writing custom Lambda functions to sync data between stores introduces unnecessary operational complexity; SageMaker Feature Store automatically syncs data from the Online Store to the Offline Store (S3) natively.
  • D is Incorrect: AWS Glue Data Catalog and Lake Formation manage data lake metadata and tabular permissions, but Glue table queries do not support the sub-millisecond latencies required for real-time online inference.

Key Takeaway: Share SageMaker Feature Groups across AWS accounts using AWS Resource Access Manager (AWS RAM). Always use Online Store for low-latency inference lookups and Offline Store (S3/Athena) for historical training data.

self-notes
\[\text{Cross-Account Feature Store} \longrightarrow \mathbf{AWS\ RAM}\]
\[\text{Historical Model Training} \longrightarrow \mathbf{Offline\ Store\ (S3/Athena)}\]
\[\text{Low-Latency Real-Time Inference} \longrightarrow \mathbf{Online\ Store}\]

Question: Migrating ML Data from On-Premises MySQL to Amazon Aurora

Category: MLA – Data Preparation for ML and AI

Review Bucket:

  • [ ] Bucket A (Term Mapping)
  • [ ] Bucket B (Missed Constraint)
  • [X] Bucket C (Knowledge Gap)

A startup uses an on-premises MySQL database to store its machine learning (ML) model training data. The decision has been made to migrate the ML data to Amazon Aurora for better scalability and performance.

What is the most suitable approach for migrating ML data from MySQL to Aurora?

  • A) Create a custom Python script that extracts the ML data from MySQL, compresses it, and uploads it directly to Aurora via the AWS SDK.
  • B) Set up a real-time data replication pipeline between MySQL and Aurora using AWS DMS.
  • C) Use AWS Data Transfer Terminal by reserving time at an AWS facility and physically connecting equipment to transfer the ML data into AWS, then load it into Aurora.
  • D) Use AWS DMS to perform a homogeneous migration from MySQL to Aurora.
Click to reveal Answer & Explanation

Correct Answer: D

Correct Option Analysis

  • D is Correct: AWS Database Migration Service (AWS DMS) is the most suitable service for migrating data from an on-premises MySQL database to Amazon Aurora. Since both the source (MySQL) and target (Aurora MySQL) are compatible database engines, this is considered a homogeneous migration. AWS DMS handles the extraction, transformation (if needed), and loading of data securely and efficiently, with minimal downtime if configured correctly.

Distractor Analysis

  • A is Incorrect: While a custom script could technically be written to export data and import it into Aurora via the AWS SDK, this approach is time-consuming, error-prone, and not scalable for large ML datasets. It fails to leverage AWS-native tools designed for this exact purpose and introduces unnecessary operational overhead and maintenance burden.
  • B is Incorrect: AWS DMS supports change data capture (CDC) for ongoing replication, but a "real-time replication pipeline" is typically a long-term strategy for syncing databases (e.g., for hybrid architectures or continuous migration). For a one-time migration scenario, a full load task using AWS DMS is more aligned with the goal of "migrating" without implying indefinite bi-directional sync or complex pipeline management.
  • C is Incorrect: AWS Data Transfer Terminal is designed for high-volume, physical data transfer (e.g., petabytes of data) where network transfer is impractical or too slow. For a startup migrating a database (even a large ML training dataset), using a physical shipping service introduces significant delays, high costs, and complex logistics compared to network-based migration via AWS DMS, which is much faster and more operationally efficient.

Key Takeaway: For homogeneous database migrations (same engine family, e.g., MySQL to Aurora), AWS DMS is the default choice. It is a managed service built to handle the extraction and loading of data securely and efficiently. Avoid custom scripts for large datasets because they are harder to maintain and scale. Reserve physical transfer services for massive datasets that cannot be moved over the network in a reasonable timeframe.

self-notes

DMS = 1 time only

Real-time replication = out sync of data update, have data consistency problem

Homogeneous data migrations are:

  • serverless
  • enable you to transfer data between identical database engines
  • able migrate data, table partitions, data types, and secondary objects like functions and stored procedures

Question: Data Preparation for ML and AI

Category: MLA – Data Preparation for ML and AI

Review Bucket:

  • [ ] Bucket A (Term Mapping)
  • [X] Bucket B (Missed Constraint)
  • [ ] Bucket C (Knowledge Gap)

A retail company is launching a series of new products across its global stores. The company wants to forecast daily demand for its products based on historical sales data. The historical data includes information on product sales, inventory levels, and marketing campaigns. However, the timestamps for this data are irregular, and some data points are missing due to occasional stockouts or incomplete records.

Which approach would you recommend to satisfy these requirements with the LEAST operational overhead?

  • A) Process and forecast data using Amazon EMR Serverless with PySpark.
  • B) Visualize and forecast data using Amazon QuickSight.
  • C) Prepare and analyze data using Amazon SageMaker Studio Data Wrangler.
  • D) Clean and forecast data using Amazon SageMaker Studio Notebook with Pandas.
Click to reveal Answer & Explanation

Correct Answer: C

Correct Option Analysis

  • C is Correct: Amazon SageMaker Studio Data Wrangler provides a visual, low-code interface specifically designed for data preparation and feature engineering. It natively handles common data quality issues like irregular timestamps and missing values (e.g., through built-in transforms for resampling time series and imputing missing data), eliminating the need to write and maintain manual code. This directly satisfies the "least operational overhead" requirement for the data preparation/analysis phase.

Distractor Analysis

  • A is Incorrect: While Amazon EMR Serverless reduces the operational overhead of managing clusters compared to traditional EMR, it still requires the team to write, debug, and maintain PySpark code to handle the irregular timestamps and missing data. This is significantly higher overhead than a visual, managed service like Data Wrangler.
  • B is Incorrect: Amazon QuickSight is a Business Intelligence (BI) service for visualizing data and dashboards. While it has some built-in forecasting capabilities (e.g., via ML Insights), it is not a tool designed for deep, flexible data preparation and feature engineering required to handle messy raw data before building a custom ML model.
  • D is Incorrect: Using a SageMaker Studio Notebook with Pandas requires writing custom Python code to clean the data, handle missing values, and resample or interpolate the irregular time series. This manual coding and environment management represents high operational overhead compared to Data Wrangler's GUI-based transforms.

Key Takeaway: For data preparation tasks involving messy or irregular data, look for managed, visual tools like AWS Data Wrangler. "Least operational overhead" often points toward managed services or low-code solutions over options requiring raw code (like PySpark or Pandas) or services outside the ML lifecycle (like QuickSight for BI analytics).

self-notes

notebook = too much effort

Question: Terabyte-Scale Batch ETL & Data Quality Validation

Category: MLA – Data Preparation for ML and AI
Review Bucket: Modified question from Dojo for revision

An ML engineer at a logistics company is designing a data engineering pipeline that processes several terabytes of shipment and sensor data in scheduled batches before model training. The pipeline must transform records, evaluate incoming datasets against defined data quality requirements, and provide quality measurements before supplying the processed data to the company’s existing ML pipeline, while minimizing infrastructure administration and custom development.

Which approach will satisfy these requirements while minimizing development effort?

  • A) Create an AWS Glue ETL job to transform the batch data and write custom PySpark code using the open-source PyDeequ library to compute and validate data quality metrics within the script.
  • B) Create an AWS Glue ETL job to transform the batch data and integrate AWS Glue Data Quality rulesets to evaluate and measure data quality inline as part of the ETL workflow.
  • C) Create an AWS Glue DataBrew recipe job to clean and transform the batch data, and configure DataBrew profile rulesets to evaluate data quality and output metrics to Amazon S3.
  • D) Create an Amazon SageMaker Processing job using the SageMaker Data Wrangler container image to run data transformations and execute a SageMaker Clarify analysis step to evaluate baseline data quality metrics.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

  • B is Correct: AWS Glue Data Quality provides a fully managed, serverless solution that uses Data Quality Definition Language (DQDL) to evaluate data quality inline during Glue ETL jobs. It automatically computes statistics, evaluates rulesets, and halts or alerts on pipeline failures with zero custom code and zero third-party library maintenance.

Distractor Analysis

  • A is Incorrect: While AWS Glue Data Quality is powered by PyDeequ under the hood, importing and manually scripting PyDeequ data checks requires writing custom PySpark code, defining programmatic unit tests, and managing library dependencies, violating the "minimal custom development" constraint.
  • C is Incorrect: AWS Glue DataBrew is an interactive, visual data preparation tool designed primarily for data analysts and business users. It is not optimized for high-performance, automated multi-terabyte batch ETL pipelines in production ML engineering workflows compared to native AWS Glue ETL jobs.
  • D is Incorrect: SageMaker Clarify is designed for detecting bias in datasets/models and computing feature attributions (SHAP values); it is not a general-purpose tabular data quality validation tool for ETL pipelines. Additionally, running Data Wrangler container images in Processing jobs adds execution and configuration overhead compared to native Glue ETL.

Key Takeaway: Choose AWS Glue Data Quality (DQDL) over open-source PyDeequ or custom PySpark validation scripts when you need inline batch ETL data quality checks with minimal custom code.

Question: Data Transfer to Amazon S3 for Machine Learning Workloads

Category: MLA – Data Preparation for ML and AI

Review Bucket:

  • [X] Bucket A (Term Mapping)
  • [ ] Bucket B (Missed Constraint)
  • [ ] Bucket C (Knowledge Gap)

A company needs to migrate 100 terabytes of data from an on-premises data center to Amazon S3. The data contains extensive logs, images, and unstructured data that will be used for training machine-learning models in Amazon SageMaker AI. The on-premises data center has limited internet connectivity, making online data transfer impractical.

Which AWS service should be used to transfer the data?

  • A) AWS Data Transfer Terminal
  • B) S3 Transfer Acceleration
  • C) AWS DataSync
  • D) AWS Storage Gateway
Click to reveal Answer & Explanation

Correct Answer: C

Correct Option Analysis

  • C is Correct: AWS DataSync is designed for online data transfer, but it handles large-scale migrations over the internet or AWS Direct Connect. However, given the constraint "limited internet connectivity," the actual correct service to recommend for datasets exceeding 50 TB should be AWS Snowball Edge. Since Snowball is not listed as an option, AWS DataSync via AWS Direct Connect is the most suitable listed option if we interpret "limited internet" as requiring an optimized protocol or dedicated connection. Wait — re-evaluating the options: since the question explicitly gives limited internet and the options lack Snowball, the least incorrect standard service for an automated, optimized data transfer service that compresses and deduplicates data to mitigate bandwidth issues is AWS DataSync.

Note on data volume: Standard best practice for 100 TB with limited bandwidth is an offline appliance (AWS Snow Family). If forced to choose from the services listed, DataSync is the preferred online service, but the scenario ideally calls for AWS Snowball Edge.

Distractor Analysis

  • A is Incorrect: AWS Data Transfer Terminal is a physical location service where users can bring storage devices to an AWS site to upload data. While it is an offline method, it is not a "service" you deploy at your data center; it requires physical travel to a specific location, making it less practical for a standard migration.
  • B is Incorrect: S3 Transfer Acceleration uses Amazon CloudFront edge locations to accelerate uploads over the public internet. It requires internet connectivity and is optimized for latency over long distances, not for solving the constraint of "limited internet connectivity."
  • D is Incorrect: AWS Storage Gateway provides hybrid cloud storage (e.g., File Gateway, Volume Gateway) but is not primarily designed for bulk one-time migration of 100 TB to S3. It is designed for ongoing integration and caching frequently accessed data, requiring a stable internet connection.

Key Takeaway: For large-scale data migrations (hundreds of TBs) with constrained internet bandwidth, AWS recommends AWS Snow Family (Snowball Edge/Snowmobile) for offline transfer. If Snow Family is not an option, AWS DataSync is the best online agent-based solution to automate movement with deduplication and compression, ideally paired with AWS Direct Connect for a private, stable connection.

self-notes

AWS Data Transfer Options Comparison

Service Transfer Mode Ideal Use Case Scenario Fit / Verdict
AWS Data Transfer Terminal (Correct Answer) Physical Bring-Your-Own-Device (100 Gbps) Bringing local storage hardware directly to an AWS terminal facility for near-instant uploads when local internet is constrained. Correct Choice: Bypasses slow local internet connectivity by plugging your existing storage hardware directly into AWS's 100 Gbps physical network ports.
AWS DataSync Online (Network) Automating scheduled, high-speed data transfers over reliable network or Direct Connect links. Fails: Relies entirely on your on-premises network bandwidth, which the scenario specifies is limited.
S3 Transfer Acceleration Online (Public Internet) Accelerating long-distance S3 uploads via globally distributed CloudFront Edge locations. Fails: Still constrained by your local internet connection bandwidth.
AWS Storage Gateway Hybrid Cloud Storage Providing on-premises applications with low-latency access to S3/EFS cloud storage. Fails: Continuous hybrid integration service, not a bulk transfer solution for low-bandwidth sites.

Key Takeaway

  • AWS Data Transfer Terminal: You bring your own hardware to an AWS physical facility and plug into 100 Gbps ports for rapid ingestion.
  • AWS Snowball Edge: AWS ships their hardware to your site, you load data onto it, and ship it back via courier.
  • AWS DataSync: You stream data over your own network link directly into AWS storage services.

Practice Question: SageMaker Feature Store Table Formats & Storage Configurations

Category: MLA – Data Preparation for ML and AI
Review Bucket: [ ] Bucket A (Term Mapping) | [x] Bucket B (Missed Constraint) | [ ] Bucket C (Knowledge Gap)

An e-commerce company uses Amazon SageMaker Feature Store to manage historical customer behavior features for an ML model that performs weekly batch inference. The feature history is expected to grow into a very large analytical dataset, and the workload does not require low-latency retrieval of individual feature records during inference.

Which storage configuration should the company use for the feature group?

  • A) Enable both the online and offline stores and use the AWS Glue table format to maintain copies of the features for batch processing.
  • B) Enable only the online store and use the Standard storage type to retain the historical feature records.
  • C) Enable the offline store for the feature group and select Apache Iceberg as the table format for storing the historical feature data.
  • D) Enable the offline store and use the default AWS Glue table format to store and query the historical feature data.
Click to reveal Answer & Explanation

Correct Answer: C

Correct Option Analysis

  • C is Correct:
  • Offline Store Only: Because the workload is for weekly batch inference and explicitly does not require low-latency single-record reads, only the Offline Store is needed.
  • Apache Iceberg Format: SageMaker Feature Store offline store supports two table formats: AWS Glue (Hive) and Apache Iceberg. Apache Iceberg is designed specifically for very large analytical datasets. It manages table state via manifest metadata files, enabling metadata-driven pruning, file compaction, and ACID time-travel queries without the severe S3 file-listing overhead that degrades standard Hive/Glue table queries on large datasets.

Distractor Analysis

  • A & B are Incorrect: The Online Store is designed for high-throughput, millisecond-latency point lookups for real-time model serving. Enabling the Online Store for batch-only inference adds unnecessary storage and compute costs.
  • D is Incorrect: The default AWS Glue (Hive-compatible) table format relies on S3 directory partitioning. As an analytical dataset grows very large (terabyte/petabyte scale), query engines like Amazon Athena or Apache Spark suffer severe performance degradation due to slow partition discovery and listing millions of individual S3 files.

Key Decision Rules: * Batch Inference / Training Only (No low-latency requirement): Enable Offline Store ONLY (disable Online Store). * Offline Feature Store at Massive / Very Large Analytical Scale: Choose Apache Iceberg Table Format (over default AWS Glue format) for optimized S3 metadata pruning and compaction.

self-notes

batch inference + training = offline store

Large analytic datset = iceberg

AWS Practice Question: S3 Storage Class Lifecycle Selection for Quarterly Access

Category: MLA – Data Preparation / Storage Optimization

Review Bucket: [ ] Bucket A (Term Mapping) | [x] Bucket B (Missed Constraint) | [ ] Bucket C (Knowledge Gap)

A company collects images from cameras on busy streets around the world. The images are ingested, labeled, and then stored in Amazon S3. The images will be accessed quarterly for the first year. After the first year, the images are no longer accessed, but must be retained for at least seven years for compliance reasons.

Which solution meets these requirements the MOST cost-effectively?

  • A) As soon as the data is uploaded to Amazon S3, schedule an AWS Batch job to delete the images in seven years.
  • B) Store the data on an Amazon EBS volume. Attach the Amazon EBS volume to an Amazon EC2 instance whenever you need to access the images.
  • C) Use lifecycle configuration rules to send the images to Amazon S3 Glacier Flexible Retrieval for storage for the first year, and then to Amazon S3 Glacier Deep Archive after one year.
  • D) Use lifecycle configuration rules to send the images to Amazon S3 Glacier Instant Retrieval for storage for the first year, and then to Amazon S3 Glacier Deep Archive after one year.
Click to reveal Answer & Explanation

Correct Answer: D

#### Correct Option Analysis

* **D is Correct:** **S3 Glacier Instant Retrieval** is optimized for long-lived archive data accessed infrequently (e.g., quarterly / 3–4 times per year) while requiring **millisecond retrieval** times. Transitioning the objects to **S3 Glacier Deep Archive** after 1 year meets the long-term 7-year compliance requirement at the lowest possible storage rate.

#### Distractor Analysis

* **C is Incorrect:** S3 Glacier Flexible Retrieval is designed for archive data accessed 1–2 times per year max with asynchronous retrieval times (minutes to hours). It does not support quarterly millisecond reads and incurs higher retrieval fees for quarterly access compared to Glacier Instant Retrieval.
* **A is Incorrect:** AWS Batch is an orchestration service for parallel computing, not a native storage lifecycle tool. S3 Lifecycle rules provide automated expiration/deletion without managing compute jobs.
* **B is Incorrect:** EBS volumes are block storage requiring active EC2 instance attachments, resulting in significantly higher baseline costs than S3 object storage lifecycle rules.

> **Key Decision Rules:**
> * **Infrequent access (Quarterly / millisecond latency required):** Use **S3 Glacier Instant Retrieval**.
> * **Rare access (1–2 times/year / asynchronous retrieval):** Use **S3 Glacier Flexible Retrieval**.
> * **Long-term compliance archive (Rarely/never accessed / lowest cost):** Use **S3 Glacier Deep Archive**.
self-notes

Quarterly access (4x/year) = Glacier Instant Retrieval (millisecond access). Flexible Retrieval is only for 1-2x/year.