Skip to content

Skill 1.1.2: Make storage decisions and configure storage services based on cost, performance, data structure, and data compliance

Question 1 - Columnar Format Optimization for Analytics Workloads

A data engineering team at a financial services company is building a curated data lake on Amazon S3 to support quarterly risk modeling. The analytics platform uses Amazon Athena to run SQL queries that consistently scan historical transaction records across 500 million rows. Each query targets only 4 of the 40 available columns (e.g., transaction_amount, counterparty_risk_score, settlement_date, and currency_code). The team wants to reduce query time and minimize data scanned per query.

Which storage format should the team use for this curated dataset? (Select ONE)

  • A) JSON Lines (one JSON object per line)
  • B) CSV with gzip compression
  • C) Apache Parquet with Snappy compression
  • D) Plain text with server-side encryption
  • E) Uncompressed TSV
Click to reveal Answer & Explanation

Correct Answer: C

Correct Options Analysis

  • C is Correct: Apache Parquet is a columnar storage format that allows Athena (and other query engines like Spark, Glue) to perform column pruning (projection pushdown). When a query reads only 4 of 40 columns, Parquet enables reading only those column-chunks from S3, dramatically reducing data scanned, query time, and Athena scan costs. Snappy compression provides a good balance of compression ratio and decompression speed.

Distractor Analysis

  • A is Incorrect: JSON Lines stores data in row-oriented document format. Athena must scan and parse all 40 fields in each record even when only 4 are requested, causing excessive I/O and CPU overhead.
  • B is Incorrect: CSV is row-oriented, and while gzip reduces storage size, Athena will still scan every column in every row during query execution. Additionally, gzip is not splittable, limiting query parallelism.
  • D is Incorrect: Plain text has no schema or type metadata, and encryption does not change the row vs. column scan I/O behavior.
  • E is Incorrect: Uncompressed TSV is row-oriented. It scans all columns across many rows, leading to maximum Athena query time and cost.

Key Takeaway: For analytics workloads that repeatedly read few columns across many rows, use columnar formats like Parquet or ORC to enable column pruning and dramatically reduce per-query I/O and cost.

Question 2 - Vector Store Selection with pgvector for Existing PostgreSQL

A healthcare startup has patient records and clinical metadata stored in Amazon RDS for PostgreSQL. The ML team is building a semantic search application that will retrieve similar patient cases based on medical embeddings. The team wants to minimize data movement and infrastructure changes: embeddings must reside alongside the existing relational records so joins between patient metadata and vector similarity results are trivial.

Which storage configuration should the ML team use? (Select ONE)

  • A) Migrate all embeddings to Amazon OpenSearch Service and use SQL-to-Elasticsearch connectors for joins across systems.
  • B) Export embeddings to Amazon S3 and use Amazon Athena v2 vector functions to run similarity search via SQL.
  • C) Enable the pgvector extension on the Amazon RDS for PostgreSQL instance and store embeddings in a vector column.
  • D) Store embeddings as JSON documents in Amazon DynamoDB and query nearest neighbors using a DynamoDB Streams consumer.
  • E) Import embeddings into Amazon ElastiCache for Redis and run k-NN queries with RediSearch.
Click to reveal Answer & Explanation

Correct Answer: C

Correct Options Analysis

  • C is Correct: The pgvector extension adds native vector data type and similarity search operators (<-> for L2 distance, <#> for inner product, <=> for cosine distance) to PostgreSQL. Since the data already lives in RDS PostgreSQL, enabling pgvector allows embeddings to sit beside relational records, supporting native SQL joins without data movement, sync complexity, or new infrastructure.

Distractor Analysis

  • A is Incorrect: Migrating embeddings to OpenSearch Service introduces a second storage system, cross-system joins are non-trivial, and operational overhead increases significantly compared to enabling a native extension.
  • B is Incorrect: Athena does not natively support vector similarity search in the way a vector database or pgvector does. Additionally, storing embeddings in S3 with Athena would decouple them from relational records, making bio-directional joins cumbersome.
  • D is Incorrect: DynamoDB cannot natively compute k-NN vector similarity. Implementing it with DynamoDB Streams and custom consumers would be a costly, custom, high-overhead workaround.
  • E is Incorrect: ElastiCache for Redis with RediSearch is a legitimate vector store option, but it introduces a caching layer and data replication from PostgreSQL to Redis. This violates the constraint of minimizing data movement and infrastructure changes.

Key Takeaway: When vector search requirements pair with existing relational data in PostgreSQL, enabling pgvector is the lowest-friction, lowest-data-movement solution.

Question 3 - Streaming Ingestion Capacity Troubleshooting

A fraud detection platform ingests real-time payment events from Amazon Kinesis Data Streams into Amazon OpenSearch Service. The operations team notices that the Kinesis consumer (an AWS Lambda function using event source mapping) has a growing iterator age behind the latest record, indicating a backlog. The Lambda function's CPU and memory utilization are consistently below 30%, and error metrics are within normal limits. The OpenSearch cluster's WriteThrottles metric is at 90%.

What is the MOST likely bottleneck in this ingestion path? (Select ONE)

  • A) The Kinesis stream is under-provisioned in shards
  • B) The Lambda function's concurrency is insufficient
  • C) The OpenSearch cluster's write throughput is saturated
  • D) The Kinesis Data Streams retention period is set too low
  • E) The Lambda function's invocation payload size is set too large
Click to reveal Answer & Explanation

Correct Answer: C

Correct Options Analysis

  • C is Correct: The WriteThrottles metric at 90% on the OpenSearch cluster is the smoking gun： the cluster is rejecting or queueing write requests because its indexing throughput (CPU, I/O, or index shard capacity) is saturated. When downstream write targets are throttling, Kinesis consumers will be forced to retry writes, causing them to fall behind the stream and increasing iterator age. Because Lambda CPU and memory are well below limits, the bottleneck is clearly in the downstream OpenSearch cluster, not the consumer itself.

Distractor Analysis

  • A is Incorrect: A Kinesis stream under-provisioned in shards would limit total ingestion or consumption throughput. However, the stream is able to receive records (producers are keeping up), and Lambda is not maxing out its capacity. The issue manifests after Lambda attempts to write to OpenSearch.
  • B is Incorrect: Lambda concurrency is not the bound because CPU, memory, and invocation counts are comfortably under Lambda's limits. If Lambda were the bottleneck, we'd observe high concurrency utilization or throttling on the Lambda side.
  • D is Incorrect: A low retention period affects data durability/timeline for replay, not the real-time ability of consumers to keep up. If the retention is 24 hours vs. 48 hours, it doesn't cause a growing live backlog within minutes.
  • E is Incorrect: The Lambda invocation payload size affects how many records can be processed per invocation and can influence efficiency, but it is not the root cause when the target write system is clearly throttling.

Key Takeaway: In streaming backlogs, check all links in the chain—especially downstream write targets—before adding consumer capacity, because adding consumer workers to a saturated sink only amplifies write throttling.

Question 4 - Multi-Source Data Merge and Data Catalog Governance

A media company generates user engagement data in two environments: a web app reading from Amazon DynamoDB and a mobile app reading from Amazon RDS for PostgreSQL. The ML team wants to create a unified training dataset in Amazon S3 by merging daily event records from both sources. Each system assigns its own user identifiers, and some events appear in both systems with conflicting field values (e.g., last_seen timestamps). The team wants to use AWS Glue to orchestrate this pipeline and ensure downstream ML jobs can discover the merged table.

What are the TWO actions the ML team should take to build a reliable, queryable merged dataset? (Select TWO)

  • A) Use an AWS Glue crawler to catalog both source tables into the AWS Glue Data Catalog before the merge.
  • B) Use AWS Glue Studio to visually configure an ETL job that resolves source conflicts using a deterministic rule (e.g., "latest timestamp wins" for overlapping fields).
  • C) Write a custom SQL query in Amazon QuickSight to merge the two tables at runtime without storing the merged output.
  • D) Configure an AWS Direct Connect link between the DynamoDB and PostgreSQL endpoints to allow direct database-to-database replication.
  • E) Move all DynamoDB records into Amazon S3 Glacier to reduce data transfer costs during the merge.
Click to reveal Answer & Explanation

Correct Answer: A and B

Correct Options Analysis

  • A is Correct: Using a Glue crawler to populate the AWS Glue Data Catalog with table definitions gives the Glue ETL job and downstream consumers (e.g., Athena, SageMaker) a consistent metadata reference. The merge job can then reference both sources by catalog table names instead of ad-hoc connection strings.
  • B is Correct: Merging two heterogeneous data sources with overlapping and conflicting fields requires a deterministic conflict resolution strategy. AWS Glue ETL (written in Python/PySpark or built visGlue Studio) can implement this logic—e.g., a rule that the more recent timestamp wins for overlapping fields—and write the unified dataset to S3.

Distractor Analysis

  • C is Incorrect: QuickSight is a BI visualization tool, not a reusable ETL or catalog layer. It doesn't build the durable, cataloged training dataset that downstream ML jobs require.
  • D is Incorrect: AWS Direct Connect is a network connectivity service that links on-premises data centers to AWS. It does not enable database-to-database replication between two managed AWS services and is irrelevant to the actual data merge logic.
  • E is Incorrect: S3 Glacier is an archival storage class optimized for infrequent access at low cost. It is not appropriate for online data merge workloads involving daily events with frequent access, and it does not assist with joining DynamoDB and PostgreSQL records.

Key Takeaway: Cataloging sources and defining explicit conflict-resolution rules are mandatory design decisions for trustworthy multi-source data merges—your merged dataset is only as good as your source-of-truth policy.

Question 5 - Shared File Storage for Distributed Training

A natural language processing (NLP) team at a biotechnology firm is fine-tuning a large transformer model using Amazon SageMaker distributed training (data parallelism). The team runs 8 training instances, each requiring simultaneous and consistent read access to a 200 GB preprocessed dataset stored as TFRecord files. The dataset updates hourly during active retraining cycles, and the job requires low, consistent read latency. Amazon EBS provisioned IOPS volumes are considered but rejected because they cannot be attached to multiple instances simultaneously.

Which AWS storage service BEST meets the team's latency and shared access requirements? (Select ONE)

  • A) Amazon S3 (Standard) with S3 Transfer Acceleration
  • B) Amazon ElastiCache for Redis with RDB persistence
  • C) Amazon EFS (Elastic File System) mounted to all training instances
  • D) Amazon RDS for MySQL with General-Purpose SSD storage
  • E) Amazon DynamoDB with on-demand capacity mode
Click to reveal Answer & Explanation

Correct Answer: C

Correct Options Analysis

  • C is Correct: Amazon EFS provides a shared, POSIX-compliant file system that can be mounted concurrently on multiple EC2 and SageMaker training instances. It delivers low-latency file access over a single, unified namespace and can scale throughput automatically as the dataset or number of readers grows. TFRecord files are files on disk/datastore, and EFS matches this file-system access pattern.

Distractor Analysis

  • A is Incorrect: While S3 is a common ML dataset source, S3 is object storage with higher read latency than a mounted file system. Repeated high-frequency reads across hundreds of GB of data during training will incur S3 GET request overhead and may throttle or become cost-ineffective. S3 Transfer Acceleration only optimizes upload speed over long distances, not read access for distributed training.
  • B is Incorrect: ElastiCache for Redis is an in-memory key-value store, not a general-purpose file system or shared namespace for TFRecord files. Loading 200 GB into Redis memory is expensive and architecturally mismatched.
  • D is Incorrect: RDS MySQL is a relational database for structured tables, not a file storage layer. It cannot serve TFRecord file reads to distributed training instances efficiently.
  • E is Incorrect: DynamoDB is a NoSQL key-value/document database; it doesn't serve a shared file namespace or large binary files like TFRecords.

Key Takeaway: For distributed training jobs needing a single shared file namespace with concurrent low-latency reads across multiple instances, use Amazon EFS or Amazon FSx rather than object storage (S3) or block volumes (EBS).

Question 6 - Compliance-Driven Data Retention and Cost Optimization

A regulated insurance company is building an Amazon S3-based data lake for unstructured claim documents and historical policy data. Compliance rules mandate that all raw claim documents be retained immutably for at least 7 years, after which they must be automatically transitioned to a cheaper archival tier. The data will be accessed rarely after the first 90 days; access during the subsequent 7 years is unpredictable (likely a few times per year). The company's security team also requires that no object can be deleted or modified for the full 7-year retention window.

Which TWO S3 lifecycle and compliance actions should the company configure? (Select TWO)

  • A) Create an S3 Lifecycle rule to transition objects to S3 Glacier Flexible Retrieval after 90 days, then to S3 Glacier Deep Archive after 7 years.
  • B) Use Amazon S3 Intelligent-Tiering to automatically move objects between access tiers based on access patterns.
  • C) Enable S3 Object Lock in Compliance mode with a 7-year retention period on the bucket.
  • D) Use AWS Backint Agent for SAP HANA to manage automated backups that expire after 7 years.
  • E) Set a S3 Lifecycle delete rule to permanently remove objects exactly on day 2,555 (7 years).
Click to reveal Answer & Explanation

Correct Answer: A and C

Correct Options Analysis

  • A is Correct: A lifecycle rule transitioning objects to S3 Glacier Flexible Retrieval after 90 days (low-activity period) and eventually to S3 Glacier Deep Archive (if needed) optimizes cost while maintaining compliance. In this scenario, Glacier Flexible Retrieval or Deep Archive meets the "rare access after 90 days" pattern.
  • C is Correct: S3 Object Lock in Compliance Mode guarantees that no user—including root or administrators—can delete or overwrite objects until the retention period (7 years) expires. This directly satisfies the immutability and non-deletion requirement.

Distractor Analysis

  • B is Incorrect: S3 Intelligent-Tiering optimizes costs by moving data between frequent and infrequent access tiers, but it does not enforce immutability or prevent deletion. It cannot meet the compliance requirement for non-deletion/non-modification.
  • D is Incorrect: AWS Backint Agent is a backup integration for SAP HANA databases, not relevant to S3 object lifecycle or document compliance retention. This option uses an unrelated, non-applicable service.
  • E is Incorrect: A lifecycle delete rule will permanently remove objects on day 2,555, which directly violates the compliance mandate to retain records immutably for 7 years. The correct approach is retention lock or legal hold, not automatic deletion.

Key Takeaway: Always map compliance retention requirements to S3 Object Lock (data immutability) and use S3 Lifecycle rules for cost-optimization of storage class transitions—never the other way around.

Question 7 - Real-Time Feature Lookup Latency and Consistency

A machine learning team at a ride-hailing company uses Amazon SageMaker Feature Store to serve features for both training and online inference. The inference system (a set of Amazon SageMaker real-time endpoints) requires p95 feature lookup latency under 20 ms for a single entity (a driver or rider). Training jobs run nightly and read the full historical feature set for millions of entities. The feature store is enabled with both online and offline stores.

Which TWO statements accurately describe how SageMaker Feature Store meets these requirements? (Select TWO)

  • A) The online store is optimized for low-latency (milliseconds) reads by primary key during inference.
  • B) The offline store is served by a high-throughput analytics query engine (Athena/Glue) for batch training reads.
  • C) The online store stores features in columnar format on S3 with no latency guarantees for inference.
  • D) Training jobs must read only from the online store to ensure feature consistency.
  • E) The offline store automatically expires features after 24 hours to reduce storage costs.
Click to reveal Answer & Explanation

Correct Answer: A and B

Correct Options Analysis

  • A is Correct: The SageMaker Feature Store online store is a NoSQL (Cassandra-based) store engineered for single-digit millisecond lookups by record_id (primary key) at inference time. It meets p95 < 20 ms latency for entity lookups.
  • B is Correct: The offline store is stored in Amazon S3 as Parquet files and can be queried at scale using Amazon Athena, AWS Glue, or Spark. Training jobs read this historical store in batch, which is the expected design pattern.

Distractor Analysis

  • C is Incorrect: The online store does not keep features in columnar S3 format for inference. It uses a NoSQL online store for fast key-based reads; the offline store uses S3 Parquet.
  • D is Incorrect: Training jobs typically read from the offline store for historical data and consistency across the full entity set. Reading only from the online store would be inefficient and miss historical snapshots.
  • E is Incorrect: There is no automatic 24-hour expiration in SageMaker Feature Store. Retention and deletion policies are user-defined (e.g., via TTL or manual deletion), not a default forced expiry.

Key Takeaway: SageMaker Feature Store separates low-latency online reads (inference) from high-throughput offline reads (training) using two distinct storage layers, each optimized for its access pattern.

Question 8 - Choosing Apache Kafka vs. Kinesis for On-Premises Hybrid Streaming

A global retail enterprise is migrating its streaming analytics pipeline to AWS. The pipeline already ingests 500,000 events/second from on-premises Apache Kafka clusters. The company wants to maintain Kafka compatibility, use the same client libraries, and gradually migrate to AWS without re-architecting its producers or consumers in the next 12 months. The workload requires exactly-once processing semantics for critical financial transactions, and the team needs to scale to 1,000,000 events/sec during Black Friday peaks.

Which AWS service and architecture best satisfies these hybrid and compatibility requirements? (Select ONE)

  • A) Use Amazon Kinesis Data Streams with the Kafka Client Library adapter (kinesis-client-library) to emulate Kafka compatibility.
  • B) Run Amazon MSK (Managed Streaming for Apache Kafka) and use MSK Connect with Kafka exactly-once semantics.
  • C) Use Amazon Managed Service for Apache Flink to read from Kinesis Data Streams and write to an on-premises Kafka via a custom connector.
  • D) Replace Kafka with Amazon SQS FIFO queues to guarantee exactly-once processing for all events.
  • E) Replicate Kafka data into Amazon DynamoDB Streams and consume via Kinesis Client Library (KCL).
Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

  • B is Correct: Amazon MSK is a fully managed Apache Kafka service that preserves Kafka's native protocol, client libraries, and topic/partition model. The existing on-premises Kafka applications can connect to MSK with the same Kafka APIs, minimizing migration effort. MSK Connect enables integration with Kafka connectors, and the Kafka ecosystem supports exactly-once semantics (EOS) for critical transactions.

Distractor Analysis

  • A is Incorrect: Kinesis Data Streams is a proprietary AWS streaming service with its own API. While a Kafka-compatible adapter exists, it is not a drop-in replacement for Kafka client libraries, topic semantics, or exactly-once guarantees. Re-architecting producers and consumers would be necessary.
  • C is Incorrect: Amazon Managed Service for Apache Flink (formerly Kinesis Data Analytics) is a stream processing engine (Apache Flink), not a Kafka replacement. It can read from both Kinesis and Kafka, but it doesn't solve the on-premises Kafka compatibility or migration requirement on its own.
  • D is Incorrect: Amazon SQS FIFO queues offer exactly-once processing for messages, but they do not implement Kafka's publish-subscribe, partitioning, or consumer group model. Migrating from Kafka to SQS would require a complete consumer/producer rewrite.
  • E is Incorrect: DynamoDB Streams is a change data capture (CDC) mechanism for tables, not a Kafka compatibility layer. Replicating Kafka events into DynamoDB and consuming via KCL is architecturally backwards and introduces unnecessary complexity.

Key Takeaway: For hybrid cloud streaming with Kafka client compatibility, exactly-once semantics, and minimal producer/consumer code changes, Amazon MSK is the default choice over Amazon Kinesis.

Question 9 - Image and Audio Data Cataloging for Large-Scale ML

A computer vision team at an autonomous vehicle startup must ingest and store millions of camera images and LiDAR point cloud files for model training. The data is produced in batches every hour by vehicle fleets, and each batch contains roughly 50,000 files. The team needs to find specific training subsets by metadata such as vehicle_id, camera_position, timestamp, and weather_condition—and then efficiently feed these files into Amazon SageMaker training jobs.

Which TWO design choices will enable efficient metadata search and data ingestion at scale? (Select TWO)

  • A) Store all raw files in Amazon S3 and maintain a metadata index in Amazon DynamoDB (with composite keys like vehicle_id + timestamp) for fast lookups during batch selection.
  • B) Store all raw files in Amazon EBS volumes attached to a single EC2 instance; use the operating system's file search to filter by file name patterns.
  • C) Use AWS Glue Data Catalog with a crawler to automatically discover and store table definitions for file metadata on S3, enabling serverless SQL queries via Amazon Athena for subset selection.
  • D) Use Amazon Rekognition to perform vector-based similarity search across all image files to locate the correct phone for a given vehicle.
  • E) Store all images in Amazon OpenSearch Service as base64-encoded binaries to perform full-text search on metadata.
Click to reveal Answer & Explanation

Correct Answer: A and C

Correct Options Analysis

  • A is Correct: S3 is the optimal landing zone for massive volumes of unstructured data like images and LiDAR files. A DynamoDB metadata index with composite keys enables fast, pre-computed lookups by attributes (e.g., vehicle_id, timestamp, weather_condition) without scanning all S3 objects. This decouples metadata queries from large object reads.
  • C is Correct: AWS Glue crawlers can scan S3 buckets, infer object schemas (including file paths, partition keys, and metadata via custom classifiers), and register table definitions in the Data Catalog. Athena can then run serverless SQL queries to select specific subsets of training data based on these catalog table columns or S3 partition keys.

Distractor Analysis

  • B is Incorrect: Storing millions of files on a single EC2-instance-attached EBS volume is severely capacity-limited, has no redundancy guarantees, and cannot be read concurrently by distributed SageMaker training jobs. It's not scalable.
  • D is Incorrect: Amazon Rekognition is an image and video analysis service for detecting objects, faces, and content moderation labels. It does not perform vector similarity search or file subset selection based on arbitrary metadata fields like weather_condition.
  • E is Incorrect: Amazon OpenSearch Service is a search engine, not a binary object storage layer for large media files. Storing base64-encoded LiDAR files in OpenSearch would be extremely costly, inflate index size, and degrade search performance.

Key Takeaway: For large unstructured media datasets, use S3 as the system of record (raw files) and maintain a separate metadata index (DynamoDB or Glue Catalog) to enable fast, queryable training subset selection without scanning S3.

Question 10 - Feature Store vs. Vector Store Confusion

A startup's ML platform team is debating where to store two distinct types of data for a recommendation system:

  1. User historical engagement features (e.g., last_purchase_date, total_orders_30d, preferred_category).
  2. Product embedding vectors (512-dimensional float arrays) used for similarity search (e.g., "customers who viewed this product also viewed").

The inference system requires sub-20 ms individual lookup for both the user's current features and the top-10 similar products. The team already uses Amazon SageMaker for training and inference and Amazon OpenSearch Service for log analytics.

Which TWO storage services should the team use to best serve these two different data types? (Select TWO)

  • A) Store user engagement features in SageMaker Feature Store for high-throughput, low-latency online feature lookups.
  • B) Store product embeddings in SageMaker Feature Store online store for k-NN similarity search.
  • C) Store product embeddings in Amazon OpenSearch Service with a knn_vector field type for fast approximate nearest neighbor search.
  • D) Store both user features and product embeddings in Amazon DynamoDB auto-generated composite sort keys to support similarity search.
  • E) Store product embeddings as Parquet files in Amazon S3 and query them directly at inference time using Amazon Athena.
Click to reveal Answer & Explanation

Correct Answer: A and C

Correct Options Analysis

  • A is Correct: SageMaker Feature Store is purpose-built for ML feature management. The online store guarantees single-digit millisecond lookups of recent user features by primary key, and the offline store provides historical data for training. This perfectly matches the user engagement feature use case.
  • C is Correct: Amazon OpenSearch Service supports k-NN vector search via the knn_vector data type and can efficiently return approximate nearest neighbors with low latency (sub-20 ms) at scale. This is the correct choice for product embedding similarity search.

Distractor Analysis

  • B is Incorrect: SageMaker Feature Store does not provide native vector similarity search (k-NN) capabilities. It is a key-value feature store, not a vector search engine.
  • D is Incorrect: DynamoDB supports key-value and document data models, not built-in k-NN vector search. Implementing approximate nearest neighbor search on DynamoDB would require a custom, complex, high-latency external solution.
  • E is Incorrect: Using Athena (a serverless SQL query service) at inference time introduces seconds-to-minutes query latency, completely incompatible with a sub-20 ms retrieval requirement for product similarity.

Key Takeaway: A feature store answers identity questions ("what are this entity's current features?") while a vector store answers similarity questions ("what is closest to this embedding?"). They serve different jobs and are not interchangeable.

Question 11 - Cost Optimization for Infrequent Data Access in Training Pipeline

An ML engineering team at a logistics company has built a training pipeline on Amazon S3 using AWS Glue ETL jobs. The raw data ingested from IoT sensors is accessed heavily for the first 30 days for feature engineering and model retraining, then accessed only once or twice per year for audit purposes. The company's finance team wants to reduce S3 storage costs while retaining the data for 5 years for compliance.

Which storage configuration should the team implement? (Select ONE)

  • A) Store the data in S3 Standard for the full 5 years; use S3 server access logs to monitor cost.
  • B) Use S3 Lifecycle rules to transition data from S3 Standard to S3 Glacier Flexible Retrieval after 30 days, and then to S3 Glacier Deep Archive after 180 days for the remaining retention period.
  • C) Move the data to Amazon ElastiCache for Redis after 30 days to reduce storage cost.
  • D) Store the data in Amazon EBS SC1 (Cold HDD) volumes and attach them to an EC2 instance for periodic audit queries.
  • E) Use S3 One Zone-Infrequent Access (S3 One Zone-IA) for all data after 30 days and delete the data manually after 5 years.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

  • B is Correct: S3 lifecycle rules provide automated storage class transitions based on time. S3 Glacier Flexible Retrieval offers low storage cost with retrieval times in minutes to hours (appropriate for occasional audits). S3 Glacier Deep Archive further reduces cost for data accessed once or twice per year, meeting the 5-year retention at minimal cost.

Distractor Analysis

  • A is Incorrect: Keeping all data in S3 Standard for 5 years incurs the highest storage cost. S3 Standard is optimized for frequent access, not for data accessed once or twice per year.
  • C is Incorrect: ElastiCache for Redis is an in-memory data store, meaning it's substantially more expensive per GB than any S3 tier and is not designed for long-term archival of raw IoT sensor data.
  • D is Incorrect: EBS SC1 volumes are cold HDD block storage for EC2 instances. They cannot be queried directly by serverless services like Athena or Glue and are not a cost-effective archival solution when compared to S3 Glacier tiers.
  • E is Incorrect: S3 One Zone-IA stores data in a single Availability Zone and provides no resilience for multi-AZ failures. Additionally, manual deletion after 5 years is error-prone and does not align with automated lifecycle best practices. It is also more expensive than Glacier tiers for rare-access data.

Key Takeaway: S3 Lifecycle policies automate the cost curve from hot (frequent access) to cold (rare access) storage—use Glacier tiers for audit-only retained data and avoid costly hot tiers for long-term storage.

A social media platform processes high-frequency user clickstream events in real time. The platform currently uses Apache Kafka clusters on EC2, but the operations team wants to reduce operational overhead and align with AWS-native services. The stream processing logic requires windowed aggregates (e.g., counting clicks per user in 5-minute tumbling windows) and must emit results to Amazon DynamoDB for downstream alerting.

The team prefers a fully managed, serverless solution and wants to minimize code changes for the stream processor.

Which combination of AWS services should the team use? (Select ONE)

  • A) Amazon Kinesis Data Streams + AWS Lambda function consuming events via Kinesis event source mapping, with windowed aggregation implemented in Lambda code.
  • B) Amazon MSK + Amazon Managed Service for Apache Flink (formerly Kinesis Data Analytics for Apache Flink) to perform windowed aggregations and write to DynamoDB.
  • C) Amazon Managed Service for Apache Flink + Amazon Kinesis Data Firehose for windowed aggregation and direct writes to DynamoDB.
  • D) Amazon Kinesis Data Streams + Amazon Redshift for streaming windowed aggregation, and Redshift COPY to DynamoDB via Data Pipeline.
  • E) Amazon MSK + AWS Glue Streaming to run continuously scheduled PySpark jobs that emit aggregates to DynamoDB.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

  • B is Correct: Amazon Managed Service for Apache Flink is designed for stream processing with exactly-once processing, event-time windowing, and native connectors for Kafka, Kinesis, and DynamoDB. Since the team is already using Kafka (Amazon MSK), using Managed Flink to read Kafka topics, compute 5-minute tumbling window aggregations, and write to DynamoDB satisfies all requirements with minimal custom code changes (using Flink SQL or DataStream API). MSK reduces operational overhead for Kafka cluster management.

Distractor Analysis

  • A is Incorrect: Lambda's event source mapping with Kinesis provides per-record or batch processing, but windowed aggregation (5-minute tumbling windows) is not natively supported. You'd need to implement custom windowing logic using DynamoDB or Elasticache or rely on Kinesis Data Analytics to do true stream windowing.
  • C is Incorrect: Kinesis Data Firehose is a near real-time delivery stream for batch loading into destinations (e.g., S3, Redshift, OpenSearch). It does not support arbitrary SQL windowing or direct DynamoDB writers for stream processing.
  • D is Incorrect: Redshift is a data warehouse, not a real-time streaming processor. It cannot perform low-latency SQL windowed aggregation with sub-second latency. Also, Redshift COPY to DynamoDB is not a native integration.
  • E is Incorrect: AWS Glue Streaming is intended for ETL tasks (transform and load to S3 or data warehouses) using continuous Spark jobs. It is not optimized for fast windowed streaming aggregations and does not serve as a low-latency stream processor for DynamoDB alerts.

Key Takeaway: For managed Kafka plus real-time windowed stream processing, Amazon MSK + Managed Service for Apache Flink is the canonical serverless-ish AWS architecture; use Lambda primarily for simple record-by-record transformations, not complex windowed event-time analytics.

Question 13 - Data Format Decision for Real-Time vs. Batch Ingestion

A marketing analytics team ingests campaign event data from two systems: (1) a real-time clickstream pipeline that writes continuous events from web/mobile apps into Amazon Kinesis Data Streams, and (2) a nightly batch ETL job from AWS Glue that writes a large set of customer profiles from Amazon RDS into Amazon S3. The team later queries customer profile data with Amazon Athena and Spark SQL several times per day.

What is the BEST file format choice for the nightly batch output in S3 to optimize Athena/Spark queries, and what is the appropriate data structure for the real-time Kinesis events to be consumed by downstream stream processors? (Select ONE)

  • A) Real-time: Apache Parquet; Batch: JSON Lines
  • B) Real-time: JSON; Batch: Apache Parquet
  • C) Real-time: CSV; Batch: ORC with zlib compression
  • D) Real-time: Avro with schema registry; Batch: JSON
  • E) Real-time: XML; Batch: Apache Avro
Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

  • B is Correct: For real-time Kinesis events, JSON is the most common and simple data interchange format because it's human-readable, widely supported by Flink/Kinesis Analytics/Lambda consumers, and can be easily serialized at high throughput (with lower compute cost than Parquet in streaming). For the nightly batch Parquet to S3, Amazon Athena and Spark SQL support column pruning and predicate pushdown, dramatically reducing scanned data and query cost for columnar analytics.

Distractor Analysis

  • A is Incorrect: Parquet in real-time streaming introduces write complexity, file batching latency, and higher CPU overhead. JSON Lines is row-oriented and would be a poor choice for Athena/Spark queries on large batch datasets.
  • C is Incorrect: CSV is row-oriented and inefficient for analytic queries. ORC is a columnar format that would work for batch, but choosing CSV for real-time adds unnecessary parsing complexity when JSON is the de facto standard.
  • D is Incorrect: Avro with schema registry adds a high operational burden for real-time and is typically used for Kafka/Schema Registry pipelines. For batch, JSON is again row-oriented and inefficient.
  • E is Incorrect: XML is verbose, slow to parse, and not supported by most stream processors in a native way. Avro for batch is okay, but not ideal compared to Parquet for analytical queries.

Key Takeaway: Use lightweight row-oriented formats (JSON, Avro) for high-throughput streaming writes, and columnar formats (Parquet, ORC) for large batch analytics that supports column pruning and predicate pushdown.

Question 14 - Encrypted Vector Database Configuration in OpenSearch Serverless

A security-conscious generative AI startup is building a retrieval-augmented generation (RAG) application using Amazon Bedrock Knowledge Bases integrated with Amazon OpenSearch Serverless. The startup's compliance team requires that all vector embeddings stored in OpenSearch Serverless be encrypted at rest using a customer-managed key (CMK) in AWS KMS, and that an S3 bucket logging all access to the vector collection be configured for audit purposes.

What TWO steps must be completed BEFORE creating the OpenSearch Serverless vector collection to meet the encryption and audit requirements? (Select TWO)

  • A) Define an encryption policy in OpenSearch Serverless referencing the KMS CMK ARN.
  • B) Enable S3 server access logging on the source S3 bucket holding documents.
  • C) Define an access policy in OpenSearch Serverless that grants the Bedrock Knowledge Base role permissions to write to the vector collection.
  • D) Create a KMS grant allowing OpenSearch Serverless to use the CMK for all crypto operations.
  • E) Register the S3 bucket with AWS CloudTrail to record all GetObject and PutObject requests.
Click to reveal Answer & Explanation

Correct Answer: A and C

Correct Options Analysis

  • A is Correct: In OpenSearch Serverless, encryption settings are configured via an encryption policy that must be attached to a vector collection at creation time. The policy references the KMS CMK ARN for at-rest encryption, satisfying the compliance requirement.
  • C is Correct: An access policy (in OpenSearch Serverless's security model) defines which IAM roles or users can access the collection. It must be created and attached before or at creation so the Bedrock Knowledge Base ingestion job has necessary write permissions. Without it, Bedrock will fail to index embeddings.

Distractor Analysis

  • B is Incorrect: S3 server access logging records requests to the S3 bucket itself (source documents), not to the vector collection. The question's audit requirement is about logging "all access to the vector collection," which is handled by OpenSearch Serverless audit logs, not S3 access logging.
  • D is Incorrect: While KMS grants are helpful for specific cryptographic operations, they are not an OpenSearch Serverless configuration artifact. The required step is the OpenSearch Serverless encryption policy with the CMK ARN.
  • E is Incorrect: CloudTrail logs API calls to AWS services (e.g., S3, OpenSearch) at the management and data plane level, but it is not a prerequisite to OpenSearch Serverless collection creation for encryption. Also, it does not provide the specific OpenSearch Serverless audit logs required by the scenario.

Key Takeaway: In OpenSearch Serverless, encryption policies and access policies must be defined before creating a vector collection; post-creation changes are limited and cannot enable encryption on an already-unencrypted collection.

Question 15 - Troubleshooting Capacity and Scaling in AWS Glue Large Dataset Processing

An ML data pipeline uses AWS Glue ETL jobs to transform raw JSON logs from Amazon S3 into Parquet for downstream model training. The Glue job has been running successfully for 3 months but recently the job execution time has doubled. The data volume has grown by only 20%, and the Glue Data Catalog table definitions remain unchanged. The ops team increased Glue worker nodes from 10 to 20, but execution time stayed roughly the same.

What is the MOST likely cause of the performance degradation? (Select ONE)

  • A) The Glue job is running out of executor memory due to an increased number of small JSON files.
  • B) The S3 bucket is using S3 Glacier Flexible Retrieval as the default storage class for new data.
  • C) AWS Glue has a built-in maximum execution time of 2 hours and is auto-terminating.
  • D) The JSON data now contains a much larger number of columns, causing schema drift.
  • E) The data is now being written to a DynamoDB table that has exceeded its provisioned write throughput.
Click to reveal Answer & Explanation

Correct Answer: A

Correct Options Analysis

  • A is Correct: A classic cause of Glue performance degradation is the "small files problem." When many small JSON files are ingested (e.g., thousands of small files from Kinesis or IoT), the Spark executors spend excessive I/O opening and reading each file, leading to CPU saturation on the driver/executors and longer job times. Adding more worker nodes doesn't solve this overhead because the bottleneck is file I/O and task scheduling.

Distractor Analysis

  • B is Incorrect: If data were in S3 Glacier Flexible Retrieval, Glue jobs would fail to read objects without a restore operation. The scenario says the job runs successfully but slower, indicating data is accessible.
  • C is Incorrect: AWS Glue jobs can run longer than 2 hours (many run for hours depending on the Spark job). There is no built-in auto-termination at exactly 2 hours; this is a distractor that misrepresents a known AWS data transfer limit.
  • D is Incorrect: Schema drift (unexpected new columns or type changes) could cause job failures or changes in transformation logic, but the scenario does not mention any inability to parse records or catalog inconsistencies. The job still runs.
  • E is Incorrect: The scenario says output goes to S3 Parquet, not DynamoDB. A DynamoDB write throughput issue would be a downstream bottleneck, but here the pipeline writes to S3.

Key Takeaway: When Glue job performance degrades disproportionately to data growth, inspect file size distribution (small files cause task overhead), not just total volume; adding workers may not help if the bottleneck is I/O or metadata scheduling.

Question 16 - Choosing Object Lock Mode for Different Compliance Scenarios

A financial data processor must comply with two distinct regulations for its Amazon S3 data lake:

  1. Regulation A: Records must be immutably retainable for 7 years. No one, including administrators or AWS root users, may delete records before the retention period expires. After expiration, the company may choose to keep or delete at its discretion.
  2. Regulation B: Records must be retainable for 3 years, but a compliance officer needs the ability to extend or remove the retention lock for specific legal holds placed on individual objects.

Which S3 Object Lock modes should be applied for regulations A and B respectively? (Select ONE)

  • A) Regulation A: Governance mode with 7-year retention; Regulation B: Compliance mode with 3-year retention.
  • B) Regulation A: Compliance mode with 7-year retention; Regulation B: Governance mode with 3-year retention.
  • C) Regulation A: Governance mode with 7-year retention; Regulation B: Legal Hold with no retention period.
  • D) Regulation A: Compliance mode with 7-year retention; Regulation B: Compliance mode with 3-year retention plus legal holds.
  • E) Regulation A: No Object Lock (S3 Standard); Regulation B: Compliance mode with 3-year retention.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

  • B is Correct: S3 Object Lock Compliance Mode ensures that no user, including the root account, can delete or overwrite an object until the retention period expires, perfectly matching Regulation A (7-year immutability with no compromise). Governance Mode allows most users to be prevented from deleting objects, but grants special IAM permissions (s3:BypassGovernanceRetention) to allow compliance officers to extend, shorten, or remove retention locks, matching Regulation B.

Distractor Analysis

  • A is Incorrect: This option flips Compliance and Governance. Regulation A demands non-deletion by any entity including root—that's Compliance mode, not Governance.
  • C is Incorrect: Governance mode for Regulation A would allow a privileged user to bypass retention, violating the "no one can delete" mandate. Legal Hold without a retention period for Regulation B is incomplete because Legal Hold is not the same as retention (it's a separate on/off flag).
  • D is Incorrect: Applying Compliance mode to Regulation B would prevent compliance officers from removing or extending retention as needed. Compliance mode cannot be reversed until expiration, contradicting the requirement for flexibility.
  • E is Incorrect: "No Object Lock" for Regulation A completely fails the immutability requirement. S3 Standard provides no deletion protection.

Key Takeaway: Compliance Mode enforces total immutability that no identity can bypass; Governance Mode allows designated privileged users to manage retention based on business needs—select the right mode per regulation, not per service-level convenience.

Question 17 - Selecting EBS vs. EFS vs. FSx for High-Performance Training

A deep learning training team at a semiconductor firm runs a PyTorch job on 4 Amazon EC2 P4d instances using distributed data parallelism. Each instance must read the same 2 TB dataset consisting of many large binary files. The training loop performs random access reads from different positions in the dataset per batch. The team needs the highest achievable I/O throughput for this shared dataset across all 4 instances.

Which AWS storage option BEST meets the requirement for high-performance distributed file access with random reads? (Select ONE)

  • A) Amazon EFS (General Purpose performance mode)
  • B) Amazon S3 (Standard) with Amazon S3 Transfer Acceleration
  • C) Amazon FSx for Lustre
  • D) Amazon EBS Multi-Attach (io2 Block Express volumes)
  • E) Amazon ElastiCache for Redis with Cluster Mode Enabled
Click to reveal Answer & Explanation

Correct Answer: C

Correct Options Analysis

  • C is Correct: Amazon FSx for Lustre is a fully managed file system optimized for high-performance computing (HPC) and machine learning workloads. It delivers hundreds of GB/s throughput, sub-millisecond latencies, and supports concurrent random-access reads from multiple instances. Lustre is the de facto HPC parallel file system and integrates natively with Amazon S3 for data ingestion, making it ideal for distributed deep learning training.

Distractor Analysis

  • A is Incorrect: Amazon EFS General Purpose mode is designed for general-purpose file sharing with moderate throughput. It is not optimized for high-performance random-access reads across multiple instances with multi-GB/s demands; it's better for low-to-moderate I/O workloads.
  • B is Incorrect: While S3 provides high aggregate throughput when many instances read concurrently, its per-request latency and lack of native POSIX file locking or random-access semantics make it less ideal for high-performance random reads into a training loop at very high IOPS. S3 Transfer Acceleration only accelerates uploads, not training reads.
  • D is Incorrect: EBS Multi-Attach volumes (io2 Block Express) can be attached to up to 16 Nitro-based instances, but they do not scale like a parallel file system. Random reads across instances to a single volume still face block-level contention and do not provide a shared namespace or cache tiering like FSx for Lustre.
  • E is Incorrect: ElastiCache for Redis is an in-memory key-value store, appropriate for caching or lookup latency, not for serving 2 TB binary datasets to training jobs. In-memory storage would be extremely expensive and is not designed for large file random access.

Key Takeaway: For high-throughput shared file workloads with random read patterns across many training nodes, use Amazon FSx for Lustre; EFS is for general purpose shared files, S3 for object-scale batch reads, and EBS for single-instance low-latency block storage.

Question 18 - Optimizing S3 for Frequent Small File Reads in Model Serving

A model serving team at an e-commerce recommendation platform stores 2 million small product metadata JSON files (average 10 KB each) in Amazon S3. The inference service (a set of Amazon SageMaker real-time endpoints) fetches these files frequently—an average of 3 GET requests per second per endpoint, across 50 endpoints. The team notices S3 GET costs are high and some endpoints experience throttling.

What TWO changes can the team make to reduce S3 costs and improve access performance? (Select TWO)

  • A) Aggregate the small JSON files into larger batched files (e.g., using Parquet or a consolidated JSONL file) and serve them using Amazon Athena queries at inference time.
  • B) Use S3 Transfer Acceleration to speed up GET requests.
  • C) Implement an Amazon ElastiCache for Redis cache layer in front of S3 to store frequently-accessed product metadata, reducing repeated S3 GETs.
  • D) Prefix the S3 object keys with random hash prefixes to improve S3 GET throughput across multiple prefixes.
  • E) Use Amazon CloudFront as a CDN in front of the S3 bucket to cache small JSON files and reduce S3 request load.
Click to reveal Answer & Explanation

Correct Answer: C and D

Correct Options Analysis

  • C is Correct: ElastiCache for Redis provides sub-millisecond reads and is a classic solution for caching hot metadata or small file payloads. By caching product metadata, 50 endpoints can read from Redis instead of S3, drastically reducing S3 GET costs and throttling.
  • D is Correct: S3 automatically scales to support thousands of requests per second, but extremely high request rates against a single prefix can lead to 503 throttling. Random hashing of object key prefixes distributes requests across multiple partitions, improving throughput. While this is often less critical with recent S3 improvements, it is still a recognized best practice for very hot workloads.

Distractor Analysis

  • A is Incorrect: Athena at inference time introduces seconds-level query latency, which is completely incompatible with real-time model serving. Additionally, aggregation still requires a query layer; it doesn't solve endpoint throttling.
  • B is Incorrect: S3 Transfer Acceleration optimizes uploads over long distances using CloudFront edge locations. It does not reduce GET costs for high-frequency reads from within the same AWS Region.
  • E is Incorrect: While CloudFront can cache objects and reduce S3 request load, it introduces additional latency for cache misses and is primarily designed for static web content delivery, not for internal high-frequency inference metadata lookups. ElastiCache is the better fit for low-latency caching.

Key Takeaway: For high-frequency small-object reads from a cluster of inference endpoints, use an in-memory cache (ElastiCache) and optimize S3 key naming for request distribution—not CDNs or query engines that add latency.

Question 19 - Data Compliance and Location Restrictions

A European healthcare data processor must store patient records in Amazon S3 in the eu-west-1 (Ireland) region to comply with data residency laws. The data must be encrypted at rest using AWS KMS customer-managed keys (CMKs), and access must be restricted by IAM policies to only a specific set of principals. Additionally, the S3 bucket must be configured to deny any access from outside the EU region and prevent accidental public exposure.

Which THREE S3 configuration actions are necessary to meet these security and compliance requirements? (Select THREE)

  • A) Enable S3 Block Public Access on the bucket.
  • B) Apply a bucket IAM policy with aws:SecureTransport: true and aws:PrincipalOrgID conditions for fine-grained access.
  • C) Use S3 Transfer Acceleration to enforce encryption in transit.
  • D) Attach a bucket policy denying requests where aws:RequestedRegion is not eu-west-1.
  • E) Enable Amazon Macie to automatically discover and classify all data stored in the bucket.
Click to reveal Answer & Explanation

Correct Answer: A, B, and D

Correct Options Analysis

  • A is Correct: S3 Block Public Access is an account- or bucket-level setting that prevents objects from ever being exposed publicly, even if an individual bucket policy or ACL allows public access. This satisfies the "prevent accidental public exposure" requirement.
  • B is Correct: Bucket policies can enforce secure transport (HTTPS via aws:SecureTransport) and restrict access to specific principals using conditions such as OUs or user IDs. This is a necessary IAM-level control.
  • D is Correct: A bucket policy with a condition aws:RequestedRegion is a geographic access control mechanism that denies API requests originating from outside a specified AWS region (even if a malicious user has IAM permissions). This satisfies the "deny any access from outside the EU" requirement.

Distractor Analysis

  • C is Incorrect: S3 Transfer Acceleration speeds up data uploads over long distances using CloudFront edge locations. It does not enforce encryption in transit for S3 requests in general; encryption in transit is enforced via bucket policies (HTTPS), not Transfer Acceleration.
  • E is Incorrect: Amazon Macie helps discover and classify sensitive data (e.g., PII, PHI) using machine learning, but it does not itself enforce access controls, encryption, or geographic restrictions. It's a monitoring and visibility tool, not an IAM enforcement mechanism.

Key Takeaway: Combine S3 Block Public Access, IAM conditional bucket policies (SecureTransport, aws:RequestedRegion), and KMS encryption to meet data residency and security mandates; use Macie for visibility, not enforcement.

Question 20 - Ingesting Diverse Data Types for Generative AI

A generative AI team at a media production company is building a multimodal model that trains on three types of data: text transcripts, audio clips, and video frames. The sources are stored in separate S3 buckets with distinct partition structures. The training job runs on a SageMaker distributed training cluster, and the team needs an efficient way to load batches with different modalities without downloading entire datasets.

Which design pattern best supports efficient access to diverse data types for distributed training? (Select ONE)

  • A) Create an S3 Select query for each modality and stream only the required byte ranges at training time.
  • B) Store all modalities in a single consolidated S3 bucket with prefix-based partitioning (e.g., data/text/, data/audio/, data/video/) and use SageMaker Input channels with ShardedByS3Key for distributed data loading.
  • C) Load all modalities into a SageMaker Feature Store and read from the online store during training for fast retrieval.
  • D) Convert all raw files into Parquet format with embedded base64 binary columns and query using Amazon Athena.
  • E) Use Amazon OpenSearch Service to index all data types and return custom keyword-search results during training batches.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

  • B is Correct: SageMaker training jobs can use input channels to distribute S3 data across instances using ShardedByS3Key mode. By organizing modalities under separate prefixes (e.g., text/, audio/, video/), each training instance fetches only the shard assigned to it—enabling efficient parallel reads without downloading full datasets. This is the canonical SageMaker approach for large-scale multimodal S3 data.

Distractor Analysis

  • A is Incorrect: S3 Select is designed for selecting subsets of CSV, JSON, or Parquet objects using SQL-like expressions. It does not support binary audio or video files and cannot handle complex multimodal training data at scale.
  • C is Incorrect: SageMaker Feature Store is for low-latency lookup of structured features, not for storing large binary media (audio/video) or serving them efficiently to distributed training jobs. It also does not support fetching large binary objects.
  • D is Incorrect: Converting audio and video files to base64 inside Parquet adds significant overhead (processing time, storage size) and is not a standard distributed training pattern. Base64-encoded binary in columnar format is inefficient for model training I/O.
  • E is Incorrect: OpenSearch Service is a search engine, not a training data delivery system. It does not provide high-throughput batch loading of large media files for distributed training. Also, keyword search is not the right tool for training batch retrieval.

Key Takeaway: For multimodal distributed training from S3, use SageMaker Input channels with ShardedByS3Key and prefix-segregated data to enable parallel, efficient reads per instance; never query training batches from search engines or online feature stores.