Skill 1.1.8: Ingest and store diverse data types (for example, text, images, audio) for AI and ML applications
| Question | Answer & Explanation |
|---|---|
| ### Question 1 - Ingesting Medical Images and Metadata for ML A healthcare AI company is building a diagnostic model that requires ingesting 500 GB of high-resolution X-ray images (DICOM format) alongside structured patient metadata (CSV). The data science team needs to query the metadata to filter specific patient cohorts for training, while keeping storage costs as low as possible. The ingestion pipeline must have the least operational overhead. Which storage architecture meets these requirements? (Select ONE) - A) Store images in Amazon EFS and metadata in Amazon RDS for PostgreSQL. - B) Store images in Amazon S3 Standard, convert the metadata CSV to Apache Parquet, and store it in Amazon S3. - C) Store images in Amazon S3 Glacier Instant Retrieval and metadata in Amazon DynamoDB. - D) Store both images and metadata in Amazon S3 using CSV format for metadata. |
Correct Answer: B Correct Answer Analysis: - B is Correct: Storing images in Amazon S3 Standard is cost-effective and the standard for ML datasets. Converting the CSV metadata to Apache Parquet (a columnar format) allows efficient, low-cost querying via Amazon Athena without managing infrastructure. This provides the lowest operational overhead and optimized read performance for columnar queries. Distractor Analysis: - A is Incorrect: Using Amazon EFS for 500 GB of static image data introduces higher costs and unnecessary operational overhead compared to S3. Managing an RDS instance for static metadata also adds overhead. - C is Incorrect: Amazon S3 Glacier Instant Retrieval is for rarely accessed data; ML training data is typically accessed more frequently. DynamoDB is excellent for key-value lookups but not ideal for complex cohort filtering and analytical queries compared to a columnar format on S3. - D is Incorrect: Keeping the metadata in CSV format on S3 prevents column pruning during analytical queries, resulting in higher Athena scan costs and slower performance compared to Parquet. > Exam Tip / AWS Best Practice: For analytical queries over structured metadata in a data lake, convert row-oriented formats (CSV, JSON) to columnar formats (Parquet, ORC) to reduce scanned data and lower costs. |
| ### Question 2 - Real-Time Audio Ingestion for Sentiment Analysis A call center platform streams live customer audio to an ML application that performs real-time speech-to-text and sentiment analysis. The ingestion pipeline must capture the continuous audio streams with sub-second latency and allow multiple independent consumers (the transcription service and the sentiment model) to read the streams independently. The team wants to use an AWS-native service to minimize operational complexity. Which service should the ML engineer use to ingest the audio data? (Select ONE) - A) Amazon Kinesis Data Streams - B) Amazon Managed Streaming for Apache Kafka (Amazon MSK) - C) Amazon SQS - D) Amazon S3 Event Notifications |
Correct Answer: A Correct Answer Analysis: - A is Correct: Amazon Kinesis Data Streams is an AWS-native service designed for continuous, low-latency data ingestion. It supports multiple independent consumers (e.g., using Kinesis Client Library or enhanced fan-out) allowing both the transcription and sentiment applications to read the same audio stream concurrently without impacting each other. Distractor Analysis: - B is Incorrect: While Amazon MSK supports multiple consumers, it introduces significantly more operational overhead (managing brokers, partitions, ZooKeeper historically) compared to a fully managed AWS-native service like Kinesis, violating the "minimize operational complexity" constraint. - C is Incorrect: Amazon SQS is a message queue, not a stream. It does not support multiple independent consumers reading the same message for stream processing; once a message is consumed and deleted, it's gone. - D is Incorrect: S3 Event Notifications are event-driven and not designed for continuous, low-latency streaming of audio data. > Exam Tip / AWS Best Practice: For real-time streaming ingestion with multiple independent consumers and low operational overhead, use Amazon Kinesis Data Streams with Enhanced Fan-Out. |
| ### Question 3 - High-Performance Shared Storage for Distributed Training A machine learning team is training a computer vision model across 20 SageMaker training instances. The training dataset consists of 5 TB of high-resolution images and text annotations. Every instance must read the entire dataset concurrently during each epoch. The team requires the highest possible throughput and lowest read latency to minimize training time. Which AWS storage service should be used to provide this shared file namespace? (Select ONE) - A) Amazon Elastic Block Store (Amazon EBS) - B) Amazon S3 with S3 Transfer Acceleration - C) Amazon FSx for Lustre - D) Amazon Elastic File System (Amazon EFS) |
Correct Answer: C Correct Answer Analysis: - C is Correct: Amazon FSx for Lustre is a fully managed, high-performance file system optimized for fast processing of large datasets. It can be linked to an S3 bucket and provides sub-millisecond latencies and hundreds of GBps of throughput, making it ideal for distributed ML training requiring concurrent access to massive image and text datasets. Distractor Analysis: - A is Incorrect: EBS volumes can only be attached to a single instance at a time (with rare, specific multi-attach exceptions that don't fit 20 nodes). It cannot provide a shared file namespace for distributed training. - B is Incorrect: While S3 stores the data, it is object storage and does not provide the POSIX file system interface and local-cache-like low latency required for high-throughput, concurrent file access during intensive training epochs. - D is Incorrect: EFS provides a shared file system, but its throughput scales with storage capacity. For a 5 TB dataset requiring maximum performance, FSx for Lustre is explicitly designed for HPC and ML workloads, whereas EFS may bottleneck on extreme IOPS/throughput. > Exam Tip / AWS Best Practice: Use Amazon FSx for Lustre when distributed ML training requires extremely high throughput and low-latency concurrent access to large datasets stored in S3. |
| ### Question 4 - Vector Storage for Image Similarity Search An e-commerce platform wants to implement a reverse image search feature. They process millions of product images daily, generate vector embeddings, and need to store them in a highly scalable database. The application must perform approximate nearest neighbor (ANN) searches to find visually similar products. The team requires a fully managed, serverless solution that scales automatically without infrastructure management. Which AWS service is the MOST appropriate for storing and querying these image embeddings? (Select ONE) - A) Amazon RDS for PostgreSQL with the pgvector extension - B) Amazon OpenSearch Serverless - C) Amazon DynamoDB - D) Amazon S3 with vector indexes |
Correct Answer: B Correct Answer Analysis: - B is Correct: Amazon OpenSearch Serverless is a fully managed, serverless option for search and analytics that natively supports vector storage and approximate nearest neighbor (ANN) search. It automatically scales compute and storage capacity based on workload demands, eliminating infrastructure management for millions of embeddings. Distractor Analysis: - A is Incorrect: While RDS with pgvector can handle vector similarity search, it requires instance management and capacity provisioning, violating the "serverless" and "automatically scales without infrastructure management" constraint. - C is Incorrect: Amazon DynamoDB is a key-value and document database. It does not natively support vector similarity search or ANN algorithms without complex workarounds. - D is Incorrect: While S3 can store vectors, it is object storage and requires a separate compute engine to perform similarity searches, making it unsuitable for low-latency, interactive reverse image search compared to a purpose-built vector store. > Exam Tip / AWS Best Practice: For serverless, auto-scaling vector similarity search (like image or text embeddings), use Amazon OpenSearch Serverless with its vector search collection type. |
| ### Question 5 - Low-Latency Feature Ingestion for Online Inference A fraud detection ML model requires real-time features derived from user text inputs and transaction metadata. The inference endpoint needs to retrieve these features with single-digit millisecond latency. The features are continuously updated by an upstream stream processing job. The architecture must prevent training-serving skew. How should the ML engineer store these features? (Select ONE) - A) Store features in Amazon S3 and use Amazon Athena for low-latency lookups. - B) Ingest features into Amazon SageMaker Feature Store, enabling the online store. - C) Store features in Amazon EFS and mount it to the SageMaker endpoint. - D) Ingest features into Amazon SageMaker Feature Store, enabling only the offline store. |
Correct Answer: B Correct Answer Analysis: - B is Correct: Amazon SageMaker Feature Store provides an online store backed by DynamoDB, designed for low-latency, real-time feature retrieval during inference. By using a single feature definition in Feature Store, it ensures consistency between the features used in training (offline store) and inference (online store), preventing training-serving skew. Distractor Analysis: - A is Incorrect: Athena is an interactive query service over S3 and has query latencies in the seconds, not single-digit milliseconds. It is not suitable for real-time inference lookups. - C is Incorrect: Mounting EFS to a SageMaker endpoint introduces latency and operational complexity, and EFS is not designed for ultra-low-latency, high-throughput key-value feature lookups. - D is Incorrect: The offline store is optimized for batch training jobs (historical records), not for low-latency online inference. Enabling only the offline store prevents real-time retrieval. > Exam Tip / AWS Best Practice: Use the SageMaker Feature Store Online Store for single-digit millisecond inference lookups and to maintain a single source of truth that prevents training-serving skew. |
| ### Question 6 - Merging Audio Files and Text Transcripts with Serverless ETL A media company ingests thousands of podcast audio files (WAV) and their corresponding text transcripts (JSON) daily into an Amazon S3 bucket. The data engineering team needs to merge the metadata from the JSON files with the audio files' S3 URIs by PodcastID, outputting a unified manifest file for a downstream SageMaker Processing job. The solution must be serverless and require the least operational overhead. Which approach satisfies these requirements? (Select ONE) - A) Spin up an Amazon EMR cluster to run a Spark job that joins the JSON and audio file metadata. - B) Use AWS Glue Studio to create a visual ETL job that reads the JSON metadata, joins it with the S3 audio file inventory, and writes the manifest to S3. - C) Deploy a long-running AWS Lambda function to iterate over the S3 bucket, parse the JSON, and write the manifest file. - D) Use Amazon SageMaker Pipelines to orchestrate a custom Python script running on SageMaker Processing instances. |
Correct Answer: B Correct Answer Analysis: - B is Correct: AWS Glue is a serverless data integration service. AWS Glue Studio provides a visual interface to easily create ETL jobs that can read from S3, perform joins, and write the output back to S3 without managing any underlying compute infrastructure, perfectly meeting the serverless and least operational overhead constraints. Distractor Analysis: - A is Incorrect: Amazon EMR requires provisioning and managing clusters, which introduces significant operational overhead compared to serverless AWS Glue. - C is Incorrect: Using a Lambda function to iterate over thousands of files and perform complex joins is an anti-pattern due to execution time limits (15 mins) and lack of built-in distributed processing. - D is Incorrect: SageMaker Processing requires provisioning instance types and managing the execution environment, which is more operational overhead than a serverless Glue job. > Exam Tip / AWS Best Practice: For serverless, low-operational-overhead ETL tasks like merging metadata and file inventories in S3, use AWS Glue (or AWS Glue Studio) instead of managing EMR clusters or custom compute. |
| ### Question 7 - Cost-Effective Storage for Historical Text Logs A natural language processing (NLP) team needs to store 10 TB of historical text logs for a batch ML training job that runs once a month. The data is rarely accessed outside of this monthly job. Compliance rules dictate that the data must be retained for 5 years. The primary constraint is to minimize storage costs while ensuring the data is available when needed for the monthly job. Which Amazon S3 storage class is the MOST appropriate? (Select ONE) - A) S3 Standard - B) S3 Standard-Infrequent Access (S3 Standard-IA) - C) S3 Glacier Flexible Retrieval - D) S3 Intelligent-Tiering |
Correct Answer: B Correct Answer Analysis: - B is Correct: S3 Standard-IA is designed for data accessed less frequently (like a monthly batch job) but requires rapid access when needed. It offers a lower storage price than S3 Standard while still providing millisecond access, making it ideal for monthly ML training jobs without retrieval delays. Distractor Analysis: - A is Incorrect: S3 Standard is for frequently accessed data. Storing 10 TB of rarely accessed data here would incur higher costs than necessary. - C is Incorrect: S3 Glacier Flexible Retrieval takes minutes to hours to retrieve data, which adds latency to the start of the monthly batch job. Standard-IA provides immediate access. - D is Incorrect: S3 Intelligent-Tiering has a small monthly monitoring and automation fee per object. For a flat, predictable access pattern (strictly once a month), Standard-IA is more cost-effective than Intelligent-Tiering. > Exam Tip / AWS Best Practice: Use S3 Standard-IA for data accessed monthly or quarterly where millisecond retrieval is required, avoiding Glacier retrieval delays and Standard pricing. |
| ### Question 8 - Native Ingestion of Unstructured PDFs for GenAI RAG A legal tech company is building a Retrieval-Augmented Generation (RAG) application using AWS. They have a corpus of 100,000 unstructured legal PDF documents stored in an Amazon S3 bucket. The ML engineer needs to parse the PDFs, extract text, perform semantic chunking, generate vector embeddings, and store them in a vector database. The constraint is to use a fully managed AWS native service to eliminate custom pipeline code and minimize operational overhead. Which service natively supports this entire ingestion workflow? (Select ONE) - A) Amazon Textract - B) Amazon SageMaker Processing - C) Amazon Bedrock Knowledge Bases - D) AWS Glue Studio |
Correct Answer: C Correct Answer Analysis: - C is Correct: Amazon Bedrock Knowledge Bases provides a fully managed ingestion pipeline for RAG. It natively handles reading documents from S3, parsing/chunking text, generating embeddings using Bedrock models, and storing them in a supported vector store (like OpenSearch Serverless), all without writing custom ETL code. Distractor Analysis: - A is Incorrect: Amazon Textract only extracts text and data from documents. It does not handle chunking, embedding generation, or vector database storage. - B is Incorrect: SageMaker Processing requires writing custom scripts and managing compute instances to handle the parsing, embedding, and storage logic, which violates the least operational overhead constraint. - D is Incorrect: AWS Glue Studio is for structured/semi-structured data ETL. It cannot natively generate vector embeddings or manage the RAG ingestion pipeline out-of-the-box. > Exam Tip / AWS Best Practice: When asked for the most automated, code-free way to ingest unstructured documents (PDFs, text) for RAG, choose Amazon Bedrock Knowledge Bases. |
| ### Question 9 - Secure Ingestion of Sensitive Audio Recordings A financial services firm is building an ML pipeline to analyze sensitive customer service audio recordings. Compliance policies mandate that the audio data must be encrypted at rest using a customer-managed key (CMK) and that the compute instances processing the audio must not retain any data locally after the job completes. The solution must guarantee zero data retention on the ephemeral compute resources. Which combination of AWS services meets these compliance constraints? (Select TWO) - A) Store the audio files in Amazon S3 with SSE-KMS encryption using a CMK. - B) Process the audio files using Amazon SageMaker Processing jobs configured with encrypted Amazon EBS volumes. - C) Store the audio files in Amazon S3 with SSE-S3 encryption. - D) Process the audio using an always-on Amazon EC2 instance with instance store volumes. - E) Use AWS Glue to process the audio files in memory only. |
Correct Answers: A and B Correct Answer Analysis: - A is Correct: Storing the audio in S3 with SSE-KMS fulfills the requirement for encryption at rest using a customer-managed key (CMK). - B is Correct: SageMaker Processing jobs use ephemeral compute instances. By configuring the job with encrypted EBS volumes, the data processed locally is encrypted. Once the processing job terminates, the EBS volumes and instance storage are destroyed, ensuring zero data retention on compute resources. Distractor Analysis: - C is Incorrect: SSE-S3 uses AWS managed keys, not customer-managed keys (CMK), violating the compliance policy. - D is Incorrect: An always-on EC2 instance does not guarantee zero data retention unless manually wiped, and instance store volumes persist data until the instance is terminated, introducing risk. It also violates ephemeral processing best practices. - E is Incorrect: AWS Glue is not designed for processing binary audio files in memory and does not natively support ephemeral EBS encryption for ML workloads the way SageMaker Processing does. > Exam Tip / AWS Best Practice: To meet strict compliance for ephemeral processing with encrypted local storage, use SageMaker Processing with KMS-encrypted EBS volumes. For data at rest in S3, use SSE-KMS with a CMK. |
| ### Question 10 - Scalable Ingestion of Video Frames and Metadata An autonomous vehicle startup needs to ingest a continuous stream of video feeds, extract frames at 1 frame-per-second (FPS), and store these images alongside telemetry metadata in a scalable, highly available data lake. The pipeline must handle massive spikes in data volume without dropping frames. The startup wants to use AWS-native streaming and storage services. Which architecture best meets these requirements? (Select ONE) - A) Use Amazon Kinesis Data Streams to ingest video, use a Kinesis Data Analytics application to extract frames, and store the images and metadata in Amazon S3. - B) Use Amazon Kinesis Video Streams to ingest video, use an AWS Lambda consumer to extract frames and metadata, and store the output in Amazon DynamoDB. - C) Use Amazon SQS to queue video chunks, process them with EC2 instances, and store the output in Amazon EFS. - D) Use Amazon Kinesis Video Streams to ingest video, use a consumer application to extract frames and telemetry metadata, and store the output in Amazon S3. |
Correct Answer: D Correct Answer Analysis: - D is Correct: Amazon Kinesis Video Streams is specifically designed to ingest streaming video at scale. A consumer application can process the stream to extract frames and metadata. Storing the extracted images and metadata in Amazon S3 creates a highly available, scalable data lake capable of handling spikes in volume. Distractor Analysis: - A is Incorrect: Kinesis Data Streams is optimized for text/binary data records, not continuous video ingestion. Kinesis Data Analytics is for SQL/streaming analytics, not frame extraction. - B is Incorrect: While Kinesis Video Streams is correct, storing thousands of image frames in DynamoDB is an anti-pattern. DynamoDB has a 400 KB item size limit and is not meant for storing binary image data at high volume. S3 is the appropriate store for images. - C is Incorrect: Amazon SQS is not built for real-time video streaming, and using EFS for a massive data lake of images is less scalable and cost-effective than S3. > Exam Tip / AWS Best Practice: For ingesting video streams at scale on AWS, use Amazon Kinesis Video Streams. Extracted image frames and telemetry should be stored in Amazon S3 as a scalable data lake. |