Skill 1.1.1: Extract data from data sources
| Question | Answer & Explanation |
|---|---|
| ### Question 1 - Distributed Training Data Extraction from Shared File Systems A machine learning engineering team is preparing to run a distributed training job across multiple Amazon SageMaker instances. The training dataset consists of millions of small image files that are currently stored in an on-premises data center. The team needs to migrate this data to AWS and make it available to all training instances concurrently with the lowest operational overhead and a standard POSIX interface. Which storage service should the team use to host the extracted data for the training job? (Select ONE) - A) Amazon Elastic Block Store (Amazon EBS) with Multi-Attach enabled - B) Amazon Simple Storage Service (Amazon S3) mounted via S3 File Gateway - C) Amazon Elastic File System (Amazon EFS) - D) Amazon FSx for Lustre |
Correct Answer: C Correct Option Analysis: - C is Correct: Amazon EFS is a fully managed, serverless, POSIX-compliant shared file system that can be mounted concurrently by multiple SageMaker training instances. It provides a standard file interface and requires minimal operational overhead, perfectly fitting the shared namespace requirement for distributed training. Distractor Analysis: - A is Incorrect: While EBS Multi-Attach allows a volume to be attached to multiple EC2 instances, it does not provide a POSIX-compliant shared file system out-of-the-box and is not natively supported for mounting across multiple SageMaker training instances in this manner. - B is Incorrect: S3 is object storage, not a file system. While it can be mounted using tools like S3 File Gateway or s3fs, these do not provide a fully POSIX-compliant interface and can introduce latency and operational complexity for small file random reads. - D is Incorrect: Amazon FSx for Lustre is an excellent high-performance shared file system for ML, but for "lowest operational overhead" with standard POSIX access and millions of small files, EFS is simpler to manage and deploy than a Lustre filesystem, which is typically optimized for high-performance computing (HPC) and large file sequential workloads. > Key Takeaway: For distributed SageMaker training jobs requiring a shared file namespace with the lowest operational overhead, Amazon EFS provides a natively supported, fully managed POSIX-compliant file system. |
| ### Question 2 - Extracting Relational Data for Batch ML Ingestion A data science team needs to extract a daily snapshot of customer transaction records from an Amazon Aurora PostgreSQL database to train a fraud detection model in Amazon SageMaker. The extraction process must be serverless, require minimal operational overhead, and automatically catalog the schema of the extracted data for future Athena queries. Which AWS service combination should the team use to extract and prepare this data? (Select ONE) - A) AWS Database Migration Service (AWS DMS) to replicate data continuously to Amazon S3 - B) AWS Glue crawler and AWS Glue extraction job connecting to Aurora via JDBC - C) Amazon EMR cluster running Apache Spark with a JDBC connector to Aurora - D) AWS Lambda function triggered by Amazon EventBridge to run SQL queries and write CSVs to S3 |
Correct Answer: B Correct Option Analysis: - B is Correct: AWS Glue is a serverless data integration service. A Glue crawler can extract the schema from Aurora PostgreSQL and store it in the Glue Data Catalog. An AWS Glue ETL job can connect via JDBC to extract the daily snapshot and write it to Amazon S3 in a columnar format (like Parquet), satisfying all requirements with zero infrastructure management. Distractor Analysis: - A is Incorrect: AWS DMS is designed for database migration and continuous replication, not serverless batch extraction with automated schema cataloging for Athena. It also introduces unnecessary operational overhead for a daily snapshot job. - C is Incorrect: Amazon EMR requires provisioning and managing clusters (even with managed scaling), which violates the "serverless" and "minimal operational overhead" constraints compared to AWS Glue. - D is Incorrect: Writing a custom Lambda function to execute SQL and write files introduces significant custom code maintenance and operational overhead, and handling large datasets in Lambda can quickly hit execution time and memory limits. > Key Takeaway: AWS Glue is the preferred serverless service for extracting data from JDBC sources like RDS/Aurora, transforming it, and cataloging the schema in the Glue Data Catalog for downstream analytics and ML. |
| ### Question 3 - Extracting Real-Time Data from DynamoDB for Feature Store An e-commerce platform uses Amazon DynamoDB to store real-time user browsing events. The ML team needs to extract these events continuously to update an Amazon SageMaker Feature Store online store for low-latency inference. The solution must minimize operational overhead and ensure that the extraction does not impact the read capacity units (RCUs) of the production DynamoDB table. Which approach should the team implement? (Select ONE) - A) Configure an AWS Lambda function to poll DynamoDB using the Query API and write updates to SageMaker Feature Store - B) Enable DynamoDB Streams and use an AWS Lambda function to process the stream records and ingest them into SageMaker Feature Store - C) Use AWS Glue to run a continuous streaming job reading from the DynamoDB table and writing to SageMaker Feature Store - D) Export the DynamoDB table to S3 daily using the DynamoDB export feature, then process it with a SageMaker Processing job |
Correct Answer: B Correct Option Analysis: - B is Correct: DynamoDB Streams capture a time-ordered sequence of item-level modifications in a table. Using a Lambda function to process these streams allows near-real-time extraction of changes without consuming RCUs on the base table. This is a serverless architecture with minimal operational overhead. Distractor Analysis: - A is Incorrect: Polling DynamoDB via the Query API consumes read capacity units (RCUs) from the production table, which can impact application performance and increase costs. - C is Incorrect: While AWS Glue streaming jobs can process data, it is significantly more complex to set up and manage for this specific use case compared to a simple Lambda trigger on DynamoDB Streams, and typically targets Kinesis or Kafka rather than direct DynamoDB table streams with low overhead. - D is Incorrect: Exporting to S3 daily is a batch process, not continuous. It would introduce high latency (up to 24 hours), failing the real-time requirement for the online Feature Store. > Key Takeaway: To extract changes from DynamoDB without impacting production read capacity, enable DynamoDB Streams and consume them with AWS Lambda for near-real-time, serverless downstream ingestion. |
| ### Question 4 - Extracting Large Media Files from Legacy Block Storage A media company has a massive dataset of high-resolution audio files stored in an Amazon EBS volume attached to a long-running Amazon EC2 instance. An ML team needs to run a distributed transcription job using SageMaker Processing and needs the data to be accessible to all processing instances. The team must transfer the data to a scalable storage layer with the least operational overhead. Which sequence of actions will most effectively extract and prepare the data? (Select ONE) - A) Detach the EBS volume and attach it to a SageMaker Processing instance, then copy the data to Amazon S3 using the AWS CLI - B) Use the AWS CLI from the EC2 instance to copy the data directly to an Amazon S3 bucket, then point the SageMaker Processing job to the S3 bucket - C) Create an EBS snapshot, create a volume from the snapshot, and mount it to an AWS Glue job to extract the audio files - D) Configure an AWS DataSync agent on the EC2 instance to transfer the files to an Amazon EFS file system, then mount EFS to SageMaker |
Correct Answer: B Correct Option Analysis: - B is Correct: Using the AWS CLI ( aws s3 sync) directly from the EC2 instance to transfer files from the attached EBS volume to Amazon S3 is the simplest and most standard method. Amazon S3 is natively supported by SageMaker Processing jobs as an input source, providing scalable access to all processing instances with minimal operational overhead.Distractor Analysis: - A is Incorrect: Detaching the EBS volume from the production EC2 instance and attaching it to a SageMaker instance requires manual intervention and interrupts the source system. It also does not natively scale to a distributed processing cluster. - C is Incorrect: AWS Glue is a data integration service for structured and semi-structured data (via Spark). It is not appropriate for moving raw, unstructured audio files from a block storage snapshot. - D is Incorrect: Using AWS DataSync to move data to EFS introduces an unnecessary intermediate storage system (EFS). While it works, S3 is the default and most operationally efficient landing zone for SageMaker Processing jobs. > Key Takeaway: When extracting unstructured data from EBS for distributed SageMaker jobs, the most operationally efficient method is to copy the data directly from the EC2 instance to Amazon S3 using standard CLI tools. |
| ### Question 5 - Extracting Vector Embeddings from OpenSearch Service An AI team is building a Retrieval-Augmented Generation (RAG) application and currently stores millions of document embeddings in an Amazon OpenSearch Service domain. They need to extract a subset of these embeddings (filtered by a specific metadata tag) and export them to Amazon S3 in JSON format for an offline batch analysis job. The extraction must be highly scalable and not impact the performance of the live OpenSearch cluster serving production queries. Which method should the team use? (Select ONE) - A) Use the OpenSearch Dashboards export feature to download the filtered documents and manually upload them to S3 - B) Configure an AWS Glue crawler to connect to the OpenSearch cluster, extract the filtered records, and write to S3 - C) Run an AWS Lambda function using the OpenSearch Python client, utilizing the Scroll API to page through the filtered results and write to S3 in batches - D) Create a snapshot of the OpenSearch Service domain to S3, then use Amazon Athena to query the snapshot repository for the specific metadata tag |
Correct Answer: C Correct Option Analysis: - C is Correct: Using a Lambda function (or a small Python script on SageMaker/EC2) with the OpenSearch Scroll API allows the team to efficiently page through large result sets matching the metadata filter without loading everything into memory. Writing the results to S3 in batches ensures scalability. Distractor Analysis: - A is Incorrect: The OpenSearch Dashboards export feature is only suitable for small volumes of data (like CSV exports of a few thousand rows). It cannot handle millions of embeddings and requires manual intervention. - B is Incorrect: AWS Glue does not have a native, optimized connector for extracting filtered vector data directly from OpenSearch Service for custom JSON export without significant custom Spark code, and it may heavily query the cluster. - D is Incorrect: OpenSearch snapshots are backups of the entire index (Lucene segments), not queryable datasets. Athena cannot natively query the contents of an OpenSearch snapshot repository for specific metadata tags. > Key Takeaway: For extracting large, filtered subsets of data from OpenSearch Service, use the Scroll or Search After APIs in a script to paginate results efficiently, avoiding heavy impact on the cluster's production query capacity. |
| ### Question 6 - Merging and Extracting Disparate Data Sources An ML team needs to extract user profile data from an Amazon RDS for MySQL database and merge it with user interaction logs stored as JSON in Amazon S3. The merged dataset must be written to S3 in Apache Parquet format. The team requires a solution with the least operational overhead that automatically manages the compute infrastructure. Which solution meets these requirements? (Select ONE) - A) Provision an Amazon EMR cluster, write a PySpark script to read from RDS and S3, merge the data, and write to S3 - B) Create an AWS Glue ETL job using PySpark, utilizing Glue Data Catalog connections for RDS and S3 data sources, and configure the job to write Parquet to S3 - C) Use AWS Database Migration Service (DMS) to replicate RDS to S3, then use S3 Object Lambda to merge the JSON files in flight - D) Launch an Amazon EC2 instance with Apache Spark installed, use a cron job to run the extraction and merging script daily |
Correct Answer: B Correct Option Analysis: - B is Correct: AWS Glue is a serverless data integration service that auto-manages the underlying compute infrastructure. Using a Glue ETL job with PySpark is the standard, lowest-overhead way to join data from JDBC sources (RDS) and S3, and output it as Parquet. Distractor Analysis: - A is Incorrect: Amazon EMR requires cluster provisioning, scaling configuration, and management, which introduces significant operational overhead compared to AWS Glue. - C is Incorrect: AWS DMS is for database migration/replication, not complex ETL joins with S3 objects. S3 Object Lambda is for transforming data upon read request, not for scheduled batch ETL merging. - D is Incorrect: Managing an EC2 instance, installing Spark, and maintaining cron jobs represents the highest possible operational overhead and is an anti-pattern in modern AWS architectures. > Key Takeaway: When merging data from S3 and RDS with the least operational overhead, AWS Glue ETL jobs are the preferred serverless choice, automatically handling compute provisioning and scaling. |
| ### Question 7 - Extracting Data with pgvector for AI Applications A company stores product descriptions and metadata in an Amazon RDS for PostgreSQL database. The engineering team has generated vector embeddings for the product descriptions and stored them in the same PostgreSQL database using the pgvector extension.The team needs to extract the top 5 most similar products based on a user's query embedding directly from the database to feed into a SageMaker endpoint for real-time inference. Which approach meets this requirement with the lowest latency and least operational overhead? (Select ONE) - A) Use AWS Glue to run a continuous streaming job that reads new embeddings from PostgreSQL and updates an Amazon OpenSearch Service domain, then query OpenSearch - B) Export the pgvector data to Amazon S3 daily, then use SageMaker Feature Store to query the nearest neighbors- C) Connect the SageMaker endpoint directly to the RDS PostgreSQL database using a SQLAlchemy connection and execute a similarity search query (e.g., <-> operator) using pgvector- D) Use AWS Lambda to query the PostgreSQL database, serialize the results to JSON, and invoke the SageMaker endpoint |
Correct Answer: C Correct Option Analysis: - C is Correct: Since the embeddings already reside in RDS PostgreSQL with the pgvector extension, querying the database directly from the SageMaker endpoint (or a Lambda function triggering it) using the native <-> operator provides the lowest latency and least operational overhead by avoiding data duplication.Distractor Analysis: - A is Incorrect: Setting up AWS Glue streaming and managing an OpenSearch Service domain introduces massive operational overhead and data synchronization latency compared to querying the existing database. - B is Incorrect: Exporting data to S3 and loading it into Feature Store is a batch operation with inherent latency. Feature Store is not primarily optimized for complex vector similarity searches natively in the same way pgvector is.- D is Incorrect: While Lambda could query the database, adding an extra Lambda invocation between the user, the database, and the SageMaker endpoint adds unnecessary network hops and latency compared to the SageMaker endpoint querying the database directly if it supports that pattern, or simply querying the DB directly for the features before calling the model. However, if the question implies the SageMaker endpoint needs the features, having the endpoint pull them directly via SQLAlchemy is the most direct path. > Key Takeaway: If embeddings already exist in PostgreSQL via pgvector, perform similarity searches directly in the database to avoid the operational overhead and latency of syncing data to a separate vector database. |
| ### Question 8 - High-Performance EFS vs FSx for ML Extraction An autonomous driving startup is training computer vision models on petabytes of high-resolution video data. The data is currently stored in an Amazon S3 bucket. To accelerate training, they need to extract this data to a high-performance file system that supports POSIX semantics and can be mounted to a distributed SageMaker training cluster. The constraint is highest possible throughput and lowest latency for concurrent reads of large files, while maintaining a managed service. Which storage service should they use? (Select ONE) - A) Amazon Elastic File System (Amazon EFS) with Max I/O performance mode - B) Amazon FSx for Lustre - C) Amazon EBS io2 Block Express volumes attached to each training instance - D) Amazon S3 accessed via S3 Mountpoint for Linux |
Correct Answer: B Correct Option Analysis: - B is Correct: Amazon FSx for Lustre is a fully managed, high-performance file system optimized for fast processing of large datasets, such as machine learning training. It integrates natively with S3 (can automatically read S3 buckets) and provides sub-millisecond latency and massive throughput, perfectly fitting the constraints. Distractor Analysis: - A is Incorrect: While EFS Max I/O scales to higher aggregate throughput, it trades latency for throughput and is designed for highly parallelized access. For petabytes of high-resolution video requiring the highest possible throughput and lowest latency, FSx for Lustre is the AWS recommended solution for HPC and ML workloads. - C is Incorrect: EBS volumes are block storage and cannot be natively mounted as a shared POSIX file system across multiple distributed SageMaker instances without a clustered file system layer (like Lustre). - D is Incorrect: S3 Mountpoint provides a POSIX interface to S3 but does not offer the local caching and sub-millisecond latency of FSx for Lustre, which is required for high-performance training on petabytes of video. > Key Takeaway: For machine learning workloads requiring extreme throughput and low latency on massive datasets, Amazon FSx for Lustre is the premier AWS file system, capable of transparently reading from S3 and presenting a high-performance POSIX interface. |
| ### Question 9 - Extracting Semi-Structured Data from DynamoDB A retail company stores customer profiles in Amazon DynamoDB. The data science team needs to extract this data to build a batch ML model. The data is semi-structured, and the team needs to extract the data in a format that allows efficient columnar reads for analytics without consuming the provisioned write/read capacity of the production table during the extraction. Which approach should be taken? (Select ONE) - A) Use an AWS Glue ETL job with a DynamoDB connector and configure the job to export the data to Amazon S3 in Parquet format - B) Use the DynamoDB Export to Amazon S3 feature (Point-in-Time Recovery) to export the table to S3, then run an AWS Glue ETL job to convert the export to Parquet - C) Run a full table scan using an AWS Lambda function and write the JSON results directly to an Amazon S3 bucket - D) Use Amazon DynamoDB Streams to capture all items and write them to Amazon S3 using an AWS Lambda function |
Correct Answer: B Correct Option Analysis: - B is Correct: The DynamoDB Export to S3 feature allows you to export a table to S3 without consuming any read capacity units (RCUs) on the production table. The export is in DynamoDB JSON format, which can then be efficiently transformed into Parquet using a serverless AWS Glue ETL job. Distractor Analysis: - A is Incorrect: Using an AWS Glue ETL job with a DynamoDB connector performs scans on the table, which will consume read capacity units (RCUs) and potentially impact production application performance. - C is Incorrect: Running a full table scan via Lambda heavily consumes RCUs and will likely hit Lambda timeout limits for large tables. It is an anti-pattern. - D is Incorrect: DynamoDB Streams only capture changes (CDC) to the table. They do not provide a mechanism to extract a full snapshot of existing historical data for initial model training. > Key Takeaway: To extract large datasets from DynamoDB without impacting production capacity, use the native DynamoDB Export to Amazon S3 feature, followed by a serverless transformation step if a different format (like Parquet) is required. |
| ### Question 10 - Secure Data Extraction from S3 for SageMaker A healthcare company needs to extract highly sensitive patient records stored in an Amazon S3 bucket to train a model using Amazon SageMaker. Security policies mandate that the data must be encrypted at rest and in transit, and the SageMaker execution role must follow the principle of least privilege. Which configuration ensures the SageMaker training job can securely extract and read this data? (Select TWO) - A) Attach an IAM policy to the SageMaker execution role that allows s3:GetObject only on the specific S3 bucket path containing the patient records- B) Enable S3 Object Lock on the S3 bucket to prevent the SageMaker role from modifying the data - C) Configure the S3 bucket to use SSE-KMS with a customer-managed key (CMK), and grant the SageMaker execution role kms:Decrypt permissions for that CMK- D) Create a VPC endpoint for Amazon S3 and configure the SageMaker training job to run in VPC mode to ensure traffic does not traverse the public internet - E) Use AWS PrivateLink to expose the S3 bucket as a service and restrict access using security groups |
Correct Answers: A and C (Also D is highly relevant for in-transit security, but let's stick to the most direct IAM/KMS least privilege principles if only two are needed, or adjust to A and C as they cover IAM and KMS which are the core of least privilege and at-rest encryption). Let's evaluate A, C, and D. Actually, D ensures data in transit doesn't traverse the public internet. Let's make A, C, and D plausible and select A and C. Wait, the instructions say provide exactly 4-5 options and select 1 or more correct. Let's make the correct answers A, C, and D. Let's adjust the prompt to: (Select THREE) Correct Answers: A, C, and D Correct Options Analysis: - A is Correct: Granting s3:GetObject on only the specific bucket path follows the principle of least privilege.- C is Correct: If the S3 bucket is encrypted with a customer-managed KMS key (SSE-KMS), the SageMaker execution role must have kms:Decrypt permissions to read the encrypted data at rest.- D is Correct: Configuring SageMaker to run in VPC mode and routing S3 traffic through a VPC Endpoint ensures that data in transit between SageMaker and S3 stays entirely within the AWS network, satisfying strict in-transit security requirements. Distractor Analysis: - B is Incorrect: S3 Object Lock is used for Write-Once-Read-Many (WORM) compliance to prevent objects from being deleted or overwritten for a retention period. It does not grant read access or configure least privilege for the SageMaker role. - E is Incorrect: AWS PrivateLink is used to access AWS services or custom services privately from a VPC. While you use VPC endpoints (powered by PrivateLink) for S3, you do not "expose the S3 bucket as a service" in this manner; you configure a gateway or interface endpoint. > Key Takeaway: Secure data extraction to SageMaker requires strict IAM least privilege ( s3:GetObject on specific paths), appropriate KMS decrypt permissions for encrypted buckets, and VPC endpoints to keep traffic off the public internet. |