Skill 1.1.4: Use AWS streaming data sources to ingest data (for example, by using Amazon Kinesis, Apache Flink, Apache Kafka)

Question 1 - Kinesis Data Streams Capacity Scaling

A media analytics company ingests real-time clickstream events from a fleet of IoT sensors into Amazon Kinesis Data Streams for a machine learning inference pipeline. The stream has 20 shards, and each shard receives approximately 1,000 records per second, with each record averaging 2 KB in size. A downstream application intermittently receives ProvisionedThroughputExceededException errors from the Kinesis Data Streams API when reading records.

The company needs to resolve this issue with the lowest operational overhead and without losing any data.

Which solution should a machine learning engineer implement? (Select ONE)

A) Switch the consumer from using the Kinesis Data Streams API to reading directly from the underlying S3 bucket where Kinesis automatically persists all stream data.
B) Enable Enhanced Fan-Out (EFO) on the stream and register the consumer with the SubscribeToShard API.
C) Increase the stream's shard count to 40 and use the SplitShard API to redistribute the hot shards.
D) Configure an AWS Lambda consumer with an event source mapping and set the batch size to 10,000 records.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

B is Correct: ProvisionedThroughputExceededException from Kinesis Data Streams indicates the consumer is hitting the standard shared throughput limit (5 reads per second per shard, or 2 MBPS per shard aggregate read throughput). Enhanced Fan-Out (EFO) assigns a dedicated read throughput of 2 MBPS per shard to each registered consumer, eliminating contention between multiple consumers and resolving the throttling errors. EFO requires no change to the producer side or the shard topology.

Distractor Analysis

A is Incorrect: This statement is factually wrong. Amazon Kinesis Data Streams does not automatically persist stream data to an S3 bucket. There is no underlying S3 bucket that can be directly read, and retrieve records through the GetRecords API or an EFO consumer is the only supported access mechanism.
C is Incorrect: The error is related to read throughput limitations, not necessarily an aggregate stream capacity problem. While splitting hot shards can address uneven data distribution at the producer side, it does not solve the consumer throttling issues. Additionally, increasing shard count adds operational cost and complexity when a simpler EFO consumer configuration resolves the issue.
D is Incorrect: Configuring a Lambda event source mapping for Kinesis does not increase the read throughput limit for the consumer. Lambda still relies on the standard Kinesis Data Streams API to poll each shard, and the batch size parameter does not override the underlying throughput limits. Lambda can still be throttled under high incoming record rates.

Key Takeaway: Use Enhanced Fan-Out (EFO) with SubscribeToShard when a consumer requires dedicated, high-throughput read access or when multipl consumers contention causes ProvisionedThroughputExceededException errors on Kinesis Data Streams.

Question 2 - MSK Consumer Group Configuration

A financial services firm uses Amazon Managed Streaming for Apache Kafka (Amazon MSK) to ingest real-time market transactions that feed a fraud detection model. The Kafka topic receives 5,000 messages per second, and a single consumer group with two consumers processes the messages. The processing latency in the consumer group increases over time, and the consumer group lag steadily grows.

The operations team verifies that the brokers are healthy and the topic is not under-replicated. Each consumer instance is an Amazon EC2 c5.xlarge that is well below CPU and memory thresholds.

What is the MOST likely cause of the growing consumer lag, and what action should be taken? (Select ONE)

A) The Kafka topic is under-partitioned; increase the number of partitions and restart the broker cluster.
B) The consumer group does not have enough consumers; add more consumer instances up to the topic partition count.
C) The producers are using an incompatible serialization format; reconfigure them to use Avro with a schema registry.
D) The MSK cluster has run out of broker disk space; increase the storage capacity via the MSK console.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

B is Correct: In Kafka, the parallelism of a consumer group is bounded by the number of partitions in the topic. With 5,000 messages per second, two consumers cannot keep pace if the topic has more than two partitions, or if the partition processing requirement exceeds their capacity. Adding more consumer instances up to the number of topic partitions allows the group to process partitions in parallel, reducing lag.

Distractor Analysis

A is Incorrect: The scenario does not indicate that the topic is under-partitioned; the problem is that the existing consumer group has fewer consumers than partitions. Increasing partitions on a Kafka topic is non-trivial and does not solve the issue; if the problem were under-partitioning (fewer partitions than consumers), it would be a different symptom — but here the constraint is clearly the consumer count.
C is Incorrect: Serialization format mismatches would cause immediate consumer failures or deserialization exceptions, not a gradual increase in lag. The topic is being consumed successfully, just too slowly.
D is Incorrect: The operations team already verified that the brokers are healthy, and no disk capacity issues were reported. Consumer lag is a consumer-side issue, not a broker storage capacity problem in this scenario.

Key Takeaway: In Kafka, consumer group parallelism is limited by topic partition count; adding more consumers than partitions does not help, but scaling consumers up to the partition count resolves lag when consumers are under-provisioned.

Question 3 - Real-Time Stream Processing with Apache Flink

A retail company needs to build a real-time anomaly detection system that processes streaming transaction data from Amazon Kinesis Data Streams. The system must correlate incoming transaction events with historical customer profiles stored in Amazon DynamoDB, perform time-windowed aggregations, and emit alerts to an Amazon SNS topic when suspicious patterns are detected.

The solution must support exactly-once processing semantics and minimize operational overhead.

Which AWS service should the company use to implement this streaming analytics pipeline? (Select ONE)

A) AWS Lambda functions with Kinesis Data Streams event source mappings, with DynamoDB Streams to trigger anomaly detection logic.
B) Amazon Managed Service for Apache Flink with a Flink application that uses Kinesis Data Streams as the source, DynamoDB as a side-input for enrichment, and SNS as the sink.
C) Amazon EMR cluster running Apache Spark Structured Streaming with Kinesis Data Streams as a source.
D) AWS Glue streaming ETL job reading from Kinesis Data Streams and writing to SNS.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

B is Correct: Amazon Managed Service for Apache Flink is purpose-built for stateful stream processing with exactly-once semantics, event-time processing, time-windowed aggregations, and support for side inputs (e.g., DynamoDB enrichment). It natively integrates with Kinesis Data Streams as a source and SNS as a sink, supports checkpointing and state management, and is fully managed, minimizing operational overhead.

Distractor Analysis

A is Incorrect: AWS Lambda functions do not support exactly-once processing semantics with Kinesis Data Streams. Lambda consumers have at-least-once delivery guarantees and are limited to 5 minutes of execution time, making time-windowed aggregations (especially sliding or session windows longer than 5 minutes) impractical. Lambda's poll-based consumption also limits throughput control.
C is Incorrect: Running Amazon EMR with Spark Structured Streaming introduces significant operational overhead (cluster provisioning, scaling, and management) compared to Amazon Managed Service for Apache Flink. While Spark Structured Streaming does support stateful operations, the scenario explicitly requires minimal operational overhead.
D is Incorrect: AWS Glue streaming ETL is designed for batch-oriented micro-batch processing and transformation before writing to a data store; it does not support complex time-windowed event-time aggregations with exactly-once guarantees as robustly as Apache Flink, nor does it have a native SNS sink connector.

Key Takeaway: For stateful, low-latency stream processing with exactly-once semantics and time-windowed aggregations over Kinesis Data Streams, Amazon Managed Service for Apache Flink is the AWS-native managed choice; avoid Lambda for stateful windowing due to duration limits and at-least-once delivery.

Question 4 - Kinesis Data Firehose vs. Data Streams Selection

A healthcare company needs to collect patient monitoring data from medical devices and deliver it to Amazon S3 in near-real time for downstream batch model training. The data arrives at a variable rate, with bursts up to 5,000 records per second. The company wants a zero-administration solution that automatically scales to the incoming data rate and integrates natively with S3 without requiring the team to manage shard capacity or consumer applications.

Which AWS service best meets these requirements? (Select ONE)

A) Amazon Kinesis Data Streams with a custom AWS Lambda consumer that reads records and writes them to S3.
B) Amazon Kinesis Data Firehose configured with an S3 destination.
C) Amazon Managed Service for Apache Flink with a Kinesis Data Streams source and an S3 sink.
D) Amazon MSK with a Kafka Connect S3 sink connector.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

B is Correct: Amazon Kinesis Data Firehose is a fully managed, serverless, zero-administration data delivery service that automatically scales to match incoming data volume, supports native S3 destinations with configurable buffering intervals, and does not require shard management or custom consumer code. It is the correct choice for delivering streaming data to S3 with minimal operational overhead.

Distractor Analysis

A is Incorrect: Amazon Kinesis Data Streams requires manual provisioning and scaling of shards, plus development and maintenance of a consumer application (the Lambda function) to write data to S3. This introduces significantly more operational complexity than needed for a zero-administration requirement.
C is Incorrect: Using Amazon Managed Service for Apache Flink to simply write data to S3 is overkill. Flink applications require development, deployment, job configuration (parallelism, checkpointing), and ongoing monitoring. Firehose handles the same task with no code and zero managed infrastructure.
D is Incorrect: Amazon MSK requires provisioning a Kafka cluster with broker nodes, managing partition counts and replication factors, and configuring a Kafka Connect S3 sink. Compared to Firehose, this is a high-operational-overhead solution for the stated requirement of zero administration.

Key Takeaway: Choose Kinesis Data Firehose when the only requirement is to reliably deliver streaming data to a destination (e.g., S3, Redshift, OpenSearch) with automatic scaling and zero administration; choose Kinesis Data Streams when custom real-time processing or multiple consumers are required.

Question 5 - Stream Processing Exactly-Once with Kinesis Data Streams

An e-commerce platform ingests order events into Amazon Kinesis Data Streams and uses a consumer application to process events and update inventory levels in Amazon DynamoDB. The business requires that each order event be processed exactly once to prevent double-decrementing inventory.

The team is uncertain whether the existing consumer architecture, using the standard GetRecords API, can guarantee exactly-once semantics.

What is the correct understanding of the delivery semantics of Kinesis Data Streams and the most appropriate solution? (Select ONE)

A) Kinesis Data Streams guarantees exactly-once delivery; the current GetRecords consumer is sufficient, and the team only needs to enable idempotency on DynamoDB writes.
B) Kinesis Data Streams guarantees at-least-once delivery; the team must implement idempotency in the consumer using a unique order ID and a conditional DynamoDB write to ensure no duplicate processing.
C) Kinesis Data Streams guarantees at-most-once delivery; the team should switch to Amazon Kinesis Data Firehose for exactly-once semantics.
D) Kinesis Data Streams does not provide any delivery guarantees; the team must redesign the system using Amazon MSK to achieve exactly-once processing.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

B is Correct: Amazon Kinesis Data Streams provides at-least-once delivery semantics for consumers using the standard GetRecords API. Records may be delivered multiple times due to retries following network failures or consumer restarts. To achieve exactly-once processing, the consumer must be idempotent — for example, by using the unique order ID as the DynamoDB key with a conditional write (e.g., attribute_not_exists) to ensure each event only decrements inventory once.

Distractor Analysis

A is Incorrect: Kinesis Data Streams does not guarantee exactly-once delivery with the standard GetRecords API. At-least-once delivery means duplicates can and will occur, and relying solely on DynamoDB idempotency without explaining the delivery semantics is misleading.
C is Incorrect: Kinesis Data Streams does not provide at-most-once delivery. Kinesis Data Firehose also provides at-least-once delivery to destinations, although with certain configurations (e.g., S3 destination) it can achieve near-exactly-once through built-in deduplication for some sinks, but not universally. Switching to Firehose does not inherently solve the inventory application's processing logic problem.
D is Incorrect: This statement is incorrect. Kinesis Data Streams does provide delivery guarantees (at-least-once). Amazon MSK also provides at-least-once semantics for standard consumers and is not the only way to achieve exactly-once; both services require idempotent consumers or transactional support for exact-once processing.

Key Takeaway: Streaming ingestion services (Kinesis Data Streams, MSK) provide at-least-once delivery; exactly-once processing is achieved at the application layer through idempotent writes or transactional consumers, not by the transport layer alone.

Question 6 - Kinesis Data Streams Shard Count Calculation

A logistics company needs to ingest vehicle telemetry data into Amazon Kinesis Data Streams. The telemetry data is generated at a constant rate of 500 records per second, with each record averaging 4 KB in size. The company wants to design the stream with enough capacity to handle this load, plus a 50% buffer to account for unexpected bursts.

Using the standard Kinesis Data Streams capacity limits, what is the minimum number of shards required for this workload? (Select ONE)

A) 1 shard
B) 2 shards
C) 3 shards
D) 4 shards

Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

B is Correct: Each Kinesis Data Streams shard supports up to 1,000 records per second and 1 MB/second ingress. The telemetry workload requires:
- Records per second: 500 × 1.5 (50% buffer) = 750 records/second → within 1 shard's 1,000 records/second limit
- Data volume: 500 × 4 KB = 2,000 KB/second = ~1.953 MB/second → with 50% buffer: ~2.93 MB/second → requires 3 shards at 1 MB/s per shard
- Calculate based on the restrictive dimension (throughput volume): 2.93 MB/s ÷ 1 MB/s per shard = ~3 shards = 3 shards. However, at 4 KB average record size, the effective data rate is 1.95 MB/s, which requires 2 shards at minimum without the buffer. With the 50% buffer, the requirement is 2.93 MB/s, requiring 3 shards. Rounding up: 3 shards.

Wait — let me re-examine this. The correct answer should be C (3 shards), not B. Let me recompute.

Given: 500 records/sec × 4 KB = 2,000 KB/sec = 1.95 MB/sec. With 50% buffer: 1.95 × 1.5 = 2.93 MB/sec. Each shard supports 1 MB/sec ingress. Therefore: 2.93 ÷ 1 = 2.93 → round up to 3 shards. The number of records per second after buffer is 750, well within the 1,000 records/sec/shard limit.

Actually, the final answer depends on which constraint is limiting. With 4 KB records: 1 MB/sec ÷ 4 KB per record = 256 records/sec per shard (due to volume throughput, not record count). So 750 records/sec × 4 KB = 3,000 KB/sec = 2.93 MB/sec → 3 shards.

The correct answer is C (3 shards).

Distractor Analysis

A is Incorrect: 1 shard can handle 1,000 records/sec but only 1 MB/s ingress. At 500 records/sec × 4 KB = 1.95 MB/sec, even without the buffer, a single shard would exceed the 1 MB/s ingress capacity.
B is Incorrect: 2 shards provide 2 MB/s ingress, which is just enough for the base 1.95 MB/s but does not accommodate the 50% buffer requirement (2.93 MB/s).
C is Correct: 3 shards provide 3 MB/s ingress capacity, which accommodates the 2.93 MB/s requirement with the 50% buffer.
D is Incorrect: 4 shards would provide 4 MB/s ingress, which is more than the required 2.93 MB/s. This is over-provisioned and would incur unnecessary cost.

Key Takeaway: Kinesis shard capacity is constrained by both record count (1,000 records/sec) and throughput volume (1 MB/sec ingress); calculate based on the more restrictive limit and always round up.

Question 7 - MSK vs. Kinesis for Enterprise Streaming

A large enterprise is migrating its on-premises Apache Kafka infrastructure to AWS. The enterprise's streaming architecture processes multiple real-time data pipelines, with existing Kafka consumer groups, custom Kafka Connect connectors, and a schema registry for Avro serialized events. The enterprise wants to minimize rewrite effort during migration, while maintaining full compatibility with the Kafka ecosystem (including Kafka Streams applications and third-party Kafka tools).

Which AWS service should the enterprise use to host its streaming platform? (Select ONE)

A) Amazon Kinesis Data Streams with the Kinesis Client Library (KCL) to replicate Kafka consumer group semantics.
B) Amazon Managed Streaming for Apache Kafka (Amazon MSK) with an MSK cluster configured to match the existing Kafka version.
C) Amazon Managed Service for Apache Flink with Kafka connector plugins to simulate the Kafka ecosystem.
D) AWS Glue streaming ETL jobs that read from Kafka topics via custom connectors.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

B is Correct: Amazon MSK is a fully managed Apache Kafka service that preserves the complete Kafka ecosystem, including Kafka consumer groups, Kafka Streams, Kafka Connect connectors, schema registries (via AWS Glue Schema Registry or Confluent Schema Registry), and third-party Kafka tools. Migrating from on-premises Kafka to MSK requires minimal code changes because MSK is protocol-compatible with Apache Kafka.

Distractor Analysis

A is Incorrect: Amazon Kinesis Data Streams is not a Kafka-compatible service. Migrating to Kinesis would require rewriting custom Kafka consumers, Kafka Streams applications, and Kafka Connect connectors to use the Kinesis Client Library (KCL), which is a significant engineering effort and breaks third-party compatibility.
C is Incorrect: Amazon Managed Service for Apache Flink is a stream processing framework, not a Kafka hosting service. While Flink can consume from Kafka topics, it does not provide Kafka broker infrastructure, topic management, or the Kafka protocol. This option would still require hosting Kafka separately and adds unnecessary complexity.
D is Incorrect: AWS Glue streaming ETL is a batch/stream transformation service, not a Kafka-compatible broker. It cannot replace the Kafka infrastructure itself; it would still require an external Kafka cluster to read from, making this an invalid architectural choice for the stated migration goal.

Key Takeaway: For migrations from on-premises Apache Kafka, always choose Amazon MSK to preserve protocol compatibility, consumer groups, and the Kafka ecosystem without requiring application rewrites.

Question 8 - Real-Time ML Feature Pipeline for Fraud Detection

A credit card company needs to compute real-time features for a fraud detection model that scores every transaction within 100 milliseconds. The features include the customer's average transaction amount over the last 24 hours, the number of transactions in the last hour, and the customer's current account status. The transaction data flows through Amazon Kinesis Data Streams, and account status is stored in Amazon DynamoDB.

The solution must minimize end-to-end latency and be able to handle thousands of transactions per second.

Which TWO AWS services or features should the company use to build this real-time feature computation pipeline? (Select TWO)

A) Amazon Managed Service for Apache Flink with a Kinesis Data Streams source, DynamoDB side-input enrichment, and tumbling/sliding window aggregations for feature computation.
B) AWS Lambda functions triggered by Kinesis Data Streams that compute features and write them to DynamoDB.
C) Amazon SageMaker Feature Store online store to serve computed features to the fraud detection model in real time.
D) Amazon Kinesis Data Firehose to batch transactions and deliver them to S3 for offline feature computation.
E) Amazon EMR cluster running Apache Spark on a 15-minute micro-batch schedule.

Click to reveal Answer & Explanation

Correct Answers: A and C

Correct Options Analysis

A is Correct: Amazon Managed Service for Apache Flink is the right choice for stateful stream processing with sliding/tumbling windows over a continuous stream. It can enrich transaction events with DynamoDB side-inputs (customer account status), compute real-time aggregations (e.g., average transaction amount over 24 hours), and meet sub-100ms processing latency when configured correctly.
C is Correct: Once features are computed, they must be served to the fraud detection model with very low latency. Amazon SageMaker Feature Store's online store provides a low-latency, high-throughput API for retrieving the most recent feature values, ensuring the model's inference request has the necessary inputs within the 100ms latency budget.

Distractor Analysis

B is Incorrect: Lambda functions can process Kinesis data, but building stateful windowing aggregations (24h average, 1h count) in Lambda requires external state management (e.g., DynamoDB) and is not designed for complex event-time windowing. Lambda also introduces execution overhead that can be less predictable at thousands of TPS.
D is Incorrect: Kinesis Data Firehose delivers data in batches to destinations like S3; it is intended for near-real-time data ingestion, not for low-latency feature computation. Batching and writing to S3 adds significant latency (seconds to minutes) and cannot meet the 100ms requirement.
E is Incorrect: Spark Structured Streaming on EMR runs in micro-batches (typically 100ms–seconds), which is borderline for a 100ms requirement and introduces significant operational overhead for cluster management. Apache Flink (with continuous streaming, not micro-batching) is better suited for sub-100ms latency.

Key Takeaway: For real-time ML feature pipelines requiring sub-second latency, combine Amazon Managed Service for Apache Flink for stateful stream processing/windowing with SageMaker Feature Store or a low-latency key-value store for online serving.

Question 9 - Troubleshooting Kinesis Data Stream Throttling

A machine learning team has a Kinesis Data Streams consumer application using the GetRecords API. The consumer frequently logs ProvisionedThroughputExceededException errors. The stream has 10 shards and receives 5,000 records per second with an average record size of 1 KB.

What is the MOST likely root cause of the throttling, and what is the optimal solution? (Select ONE)

A) The stream is under-provisioned for ingest; increase the shard count to 20.
B) The consumer is reading too frequently; implement exponential backoff and retry logic.
C) The consumer is hitting the shard-level 5 reads per second limit; use Enhanced Fan-Out (EFO) for dedicated read throughput.
D) The consumer is hitting the 2 MB/s per shard read throughput limit; reduce the number of consumers.

Click to reveal Answer & Explanation

Correct Answer: C

Correct Answer Analysis

C is Correct: With the standard GetRecords API, each Kinesis shard has a limit of 5 read transactions per second (shared across all consumers). With 10 shards and multiple consumers polling each shard, hitting this limit is common. The ProvisionedThroughputExceededException is thrown when the consumer exceeds these read limits. Enhanced Fan-Out (EFO) provides dedicated read throughput (2 MB/s per consumer per shard) and is the AWS-recommended solution for high-throughput consuming applications.

Distractor Analysis

A is Incorrect: The scenario describes a consumer-side read throttling issue, not an ingest capacity issue. At 5,000 records/sec with 1 KB records, the data volume is 5,000 × 1 KB = ~4.88 MB/s. With 10 shards, the ingest capacity is 10 × 1 MB/s = 10 MB/s, which is sufficient. The stream is not under-provisioned for ingest.
B is Incorrect: Implementing exponential backoff will reduce the frequency of throttling errors but will not solve the root cause. The consumer will still be limited to 5 reads/sec per shard and will fall behind. EFO is a much better solution.
D is Incorrect: The standard read throughput limit is 2 MB/s per shard (aggregated across all consumers). At 1 KB records and 5,000 records/sec across 10 shards, each shard receives 500 records/sec, consuming approximately 500 KB/s per shard, which is well below the 2 MB/s read limit. The throttling is due to the 5 reads/sec request rate limit, not the data volume limit. Reducing consumers would not solve the read-rate issue.

Key Takeaway: ProvisionedThroughputExceededException on Kinesis Data Streams often indicates a read request rate issue (5 reads/sec per shard) rather than a volume throughput issue; use Enhanced Fan-Out for dedicated read throughput.

Question 10 - Choosing a Streaming Ingestion Architecture for Multi-Consumer ML Pipeline

A manufacturing company needs to build a real-time predictive maintenance pipeline. The pipeline requires that:

A real-time anomaly detection model consumes sensor data from factory machines.
A real-time data quality monitoring job validates sensor data for anomalies (null values, out-of-range readings).
A data lake ingestion job archives all raw sensor data to Amazon S3.

The sensor data arrives at 3,000 records per second from multiple factory locations. Each record is approximately 500 bytes.

Which AWS services should the company use for the streaming ingestion layer to support all three consumers simultaneously? (Select ONE)

A) Amazon Kinesis Data Streams with three separate consumer applications using Enhanced Fan-Out (EFO) to provide each consumer with dedicated read throughput.
B) Amazon Kinesis Data Firehose with an AWS Lambda transformation function to route data to the three destinations.
C) Amazon MSK with three consumer groups, one for each pipeline (anomaly detection, data quality, S3 archiving).
D) Amazon Managed Service for Apache Flink with a single application that processes the stream and performs all three tasks within one pipeline.

Click to reveal Answer & Explanation

Correct Answers: A and C

Correct Options Analysis

A is Correct: Amazon Kinesis Data Streams supports multiple independent consumers. Enhanced Fan-Out (EFO) enables each consumer to have dedicated read throughput (2 MB/s per shard per consumer), eliminating contention between the three pipelines. The anomaly model, data quality monitor, and S3 archiver can each read the stream independently without affecting one another.
C is Correct: Amazon MSK (Kafka) natively supports multiple consumer groups reading from the same topic independently. Each consumer group tracks its own offsets, enabling the anomaly detection model, data quality check, and S3 archiver to consume the same data stream without interfering with one another. This is a standard Kafka pattern for multi-consumer scenarios.

Distractor Analysis

B is Incorrect: Kinesis Data Firehose is designed for a single destination. While it can invoke a Lambda function for transformation, routing to three different destinations (real-time anomaly detection, real-time data quality, and S3 archiving) is not supported without additional custom infrastructure. Firehose does not support multiple consumers reading the same stream.
D is Incorrect: Combining all three tasks into a single Flink application reduces flexibility and creates a single point of failure. If the anomaly detection logic has a bug or fails, the entire pipeline (including S3 archiving and data quality checks) is affected. Separate consumers are more robust and allow independent scaling and deployment.

Key Takeaway: For streaming architectures with multiple independent consumers of the same data, use Kinesis Data Streams with Enhanced Fan-Out or Amazon MSK with multiple consumer groups; Firehose is not designed for multiple consumers.