Skill 3.3.11: Configure AI-specific pipeline orchestration for RAG system updates and knowledge base refresh cycles
| Question | Answer & Explanation |
|---|---|
| ### Question 1 - Event-Driven Sync for Bedrock Knowledge Bases An enterprise uses an Amazon Bedrock Knowledge Base connected to an S3 bucket containing internal HR policies. HR uploads new or updated PDF documents irregularly throughout the week. The engineering team needs to automate the ingestion process to keep the vector index up-to-date with the least operational overhead and the lowest latency from the time a document is uploaded to its availability in the RAG application. Which approach meets these requirements? (Select ONE) - A) Configure an S3 Event Notification to trigger an AWS Lambda function that invokes the Bedrock StartIngestionJob API.- B) Configure an Amazon EventBridge cron rule to run the StartIngestionJob API every 15 minutes.- C) Create an AWS Glue crawler that detects changes and triggers an AWS Step Functions state machine to update OpenSearch Serverless. - D) Enable S3 Versioning and configure the Bedrock Knowledge Base to automatically listen to S3 event notifications natively without custom code. |
Correct Answer: A Correct Option Analysis: - A is Correct: Configuring an S3 Event Notification to trigger a Lambda function that calls the StartIngestionJob API provides an event-driven architecture. This ensures the ingestion job runs immediately when documents are uploaded, satisfying the lowest latency requirement with minimal operational overhead.Distractor Analysis: - B is Incorrect: While a scheduled EventBridge rule automates the process, a 15-minute cron interval introduces unnecessary latency compared to an event-driven trigger. - C is Incorrect: Using an AWS Glue crawler and Step Functions introduces unnecessary operational complexity and overhead. Bedrock Knowledge Bases natively handle parsing and chunking, so custom orchestration is redundant. - D is Incorrect: Amazon Bedrock Knowledge Bases do not currently support natively listening to S3 Event Notifications out-of-the-box to trigger ingestion jobs automatically without an intermediary compute service like Lambda. > Key Takeaway: For irregular, latency-sensitive document uploads to a Bedrock Knowledge Base, use an S3 Event Notification to Lambda to trigger the StartIngestionJob API. |
| ### Question 2 - Updating Embedding Models in RAG Pipelines A financial services company is using a RAG pipeline with Amazon OpenSearch Serverless as its vector store, populated via an Amazon Bedrock Knowledge Base. The data science team decides to upgrade the embedding model from Amazon Titan Embeddings G1 to Titan Embeddings G2 to improve retrieval accuracy. The team needs to update the pipeline while ensuring the application experiences no stale vectors and no downtime during the transition. Which strategy should the team implement? (Select ONE) - A) Update the embedding model configuration in the Bedrock Knowledge Base and trigger an incremental ingestion job to re-embed only the documents accessed most recently. - B) Update the embedding model in the Knowledge Base configuration and perform a full re-sync of the data source to regenerate all vectors. - C) Create a new OpenSearch Serverless collection and a new Knowledge Base with the G2 model, run a full ingestion job, then repoint the application to the new Knowledge Base. - D) Export the vectors from OpenSearch, apply the new model via a SageMaker Processing job, and import them back into the existing collection. |
Correct Answer: C Correct Option Analysis: - C is Correct: Creating a new collection and Knowledge Base alongside the existing one allows the new G2 model to ingest the full corpus independently. Once the new index is fully populated and verified, repointing the application to the new endpoint ensures zero downtime and guarantees no stale G1 vectors are served. Distractor Analysis: - A is Incorrect: An incremental sync will NOT re-embed unchanged documents. Because the embedding model changed, ALL existing vectors are stale, and an incremental sync will leave the majority of the corpus with G1 vectors. - B is Incorrect: Updating the existing Knowledge Base and performing a full re-sync will cause downtime or a period where the index is a mix of G1 and G2 vectors, leading to inconsistent retrieval performance. - D is Incorrect: Manually exporting, processing, and importing vectors bypasses the Bedrock Knowledge Base managed ingestion, introducing significant operational overhead and potential for error. > Key Takeaway: When upgrading the embedding model in a RAG pipeline, use a blue/green deployment pattern by creating a new vector index and Knowledge Base to avoid serving mixed or stale vectors and to ensure zero downtime. |
| ### Question 3 - Orchestrating Amazon Kendra Index Updates An AI team uses Amazon Kendra as the retriever for their RAG application. The source data is stored in Amazon S3 and is updated daily via an automated batch process that finishes at unpredictable times. The team needs to orchestrate the Kendra index refresh with the least operational overhead, ensuring the index reflects the daily updates without performing a full re-index every time. Which solution meets these requirements? (Select ONE) - A) Configure a native S3 data source in Kendra and use an Amazon EventBridge schedule to trigger the StartDataSourceSyncJob API daily.- B) Configure an AWS Glue ETL job to push documents to Kendra using the BatchPutDocument API daily.- C) Use an S3 Event Notification to trigger an AWS Lambda function that calls the BatchPutDocument API for each file.- D) Re-create the Kendra index daily using an AWS CloudFormation template to ensure a clean state. |
Correct Answer: A Correct Option Analysis: - A is Correct: Configuring a native S3 data source connector in Kendra and triggering the StartDataSourceSyncJob API via EventBridge is the most operationally efficient method. Kendra's native sync performs incremental updates, only processing changed or deleted documents, avoiding a full re-index.Distractor Analysis: - B is Incorrect: Using a Glue ETL job to push documents via BatchPutDocument introduces unnecessary operational overhead and bypasses Kendra's built-in data source synchronization and change detection logic.- C is Incorrect: Triggering a Lambda function for every S3 object upload to call BatchPutDocument is inefficient for batch updates and can lead to throttling limits, adding unnecessary operational complexity.- D is Incorrect: Re-creating the Kendra index daily is highly inefficient, destroys access control configurations, and causes significant downtime, violating the least operational overhead requirement. > Key Takeaway: For Amazon Kendra updates from S3, use the native S3 data source connector and trigger the StartDataSourceSyncJob API via EventBridge to perform incremental syncs efficiently. |
| ### Question 4 - CI/CD for Chunking Strategy Updates A healthcare company has a Bedrock Knowledge Base with a large corpus of medical journals. They want to implement a CI/CD pipeline using AWS CodePipeline that automatically updates the Knowledge Base when a new chunking strategy is committed to their repository. The update must ensure that the entire corpus is re-processed to reflect the new chunking boundaries without manual intervention. Which approach should the pipeline take? (Select ONE) - A) Configure CodePipeline to trigger a full ingestion job on the Bedrock Knowledge Base using the StartIngestionJob API.- B) Configure CodePipeline to trigger an incremental ingestion job, as the chunking strategy change will automatically flag all documents as changed. - C) Configure CodePipeline to invoke a SageMaker Processing job that manually chunks the documents and writes the output directly to the existing OpenSearch index. - D) Configure CodePipeline to deploy an AWS Lambda function that copies the existing OpenSearch index to a new collection, preserving the old vectors. |
Correct Answer: A Correct Option Analysis: - A is Correct: Changing the chunking strategy means all existing chunks and their corresponding vectors are invalid. A full ingestion job is required to re-chunk and re-embed the entire corpus according to the new configuration. Distractor Analysis: - B is Incorrect: An incremental ingestion job only detects document-level changes (added, modified, or deleted files based on metadata). It does NOT detect changes to the Knowledge Base's global chunking configuration and will skip re-processing unchanged files. - C is Incorrect: Invoking a SageMaker Processing job bypasses the managed capabilities of Bedrock Knowledge Bases, introducing unnecessary custom code and operational overhead. - D is Incorrect: Copying the existing OpenSearch index preserves the old chunking strategy and vectors, defeating the purpose of the update. > Key Takeaway: Changes to a Bedrock Knowledge Base's global settings (like embedding model or chunking strategy) require a full ingestion job (re-sync), as incremental syncs only track source data changes, not configuration changes. |
| ### Question 5 - OpenSearch Ingestion Pipeline Scaling An e-commerce company uses an OpenSearch Ingestion pipeline to ingest real-time product catalog changes into an OpenSearch Service domain for their RAG application. The pipeline must automatically scale to handle sudden spikes in catalog updates during flash sales while maintaining the lowest possible operational overhead. Which configuration achieves this? (Select ONE) - A) Manually adjust the compute units for the OpenSearch Ingestion pipeline during flash sales based on CloudWatch alarms. - B) Enable auto-scaling for the OpenSearch Ingestion pipeline and configure minimum and maximum compute unit boundaries. - C) Use an AWS Lambda function to read from an SQS queue and write to OpenSearch, scaling concurrency based on queue depth. - D) Deploy a SageMaker Processing job to batch update OpenSearch when an EventBridge rule detects a flash sale. |
Correct Answer: B Correct Option Analysis: - B is Correct: OpenSearch Ingestion natively supports auto-scaling. By setting minimum and maximum compute unit boundaries, the pipeline will automatically scale up during high traffic (flash sales) and scale down during idle periods, minimizing operational overhead and cost. Distractor Analysis: - A is Incorrect: Manually adjusting compute units requires human intervention, violating the lowest operational overhead requirement and making it prone to delayed responses during sudden spikes. - C is Incorrect: Writing custom Lambda functions to process SQS queues and write to OpenSearch introduces significant development and operational overhead compared to the native OpenSearch Ingestion service. - D is Incorrect: Using a SageMaker Processing job for real-time or near-real-time updates introduces latency and is an anti-pattern for streaming ingestion pipelines. > Key Takeaway: OpenSearch Ingestion pipelines support native auto-scaling, which should be configured with min/max compute boundaries to handle variable RAG ingestion workloads with minimal overhead. |
| ### Question 6 - Dynamic Prompt Updates via Bedrock Prompt Management A software company is developing a generative AI assistant that relies on a Bedrock Knowledge Base. The team is using Amazon Bedrock Prompt Management to version and test their RAG prompts. They want to implement an automated deployment pipeline that updates the application's prompt when a new version is approved, with the least amount of code changes to the production application. Which approach should they take? (Select ONE) - A) Update the prompt text directly in the application code repository and deploy the application via AWS CodeDeploy. - B) Use the GetPrompt API in the application to fetch the latest version of the prompt dynamically by specifying an alias.- C) Hardcode the prompt ARN in the application configuration and update the configuration file via AWS Systems Manager Parameter Store on each new release. - D) Use an S3 bucket to store the prompt text and have the application read the file at runtime. |
Correct Answer: B Correct Option Analysis: - B is Correct: Bedrock Prompt Management allows you to create version aliases (e.g., "PRODUCTION"). By configuring the application to fetch the prompt dynamically using the GetPrompt API with the alias, the team can update the prompt in the console, test it, and point the alias to the new version without redeploying or modifying the application code.Distractor Analysis: - A is Incorrect: Embedding prompt text in the application code requires a full CI/CD deployment cycle for every prompt tweak, increasing operational overhead and slowing down iterations. - C is Incorrect: Hardcoding the ARN means you still have to update the configuration and restart the application to point to a new prompt version, which is less efficient than using a dynamic alias. - D is Incorrect: Storing prompts in S3 bypasses the native versioning, testing, and management capabilities of Bedrock Prompt Management, requiring custom logic to handle versions. > Key Takeaway: Use Amazon Bedrock Prompt Management with version aliases (e.g., "PRODUCTION") to allow applications to dynamically fetch the latest approved prompt via the GetPrompt API without redeploying code. |
| ### Question 7 - Cost-Effective Stale Vector Removal A legal firm uses a RAG system with a Bedrock Knowledge Base connected to an S3 bucket containing case files. Compliance regulations require that when a case file is deleted from S3, its corresponding vectors are removed from the vector store within 24 hours. The team needs to implement an automated solution with the most cost-effective approach and least operational overhead, as file deletions happen frequently throughout the day. Which solution meets these requirements? (Select ONE) - A) Configure an S3 Event Notification for object deletion to trigger an AWS Lambda function that calls the StartIngestionJob API for every deletion event.- B) Configure an S3 Event Notification for object deletion to trigger an AWS Lambda function that manually queries and deletes vectors from OpenSearch Serverless. - C) Set up a daily scheduled EventBridge rule to run the Bedrock Knowledge Base ingestion job, which detects and removes deleted documents from the vector store. - D) Enable S3 Object Lock on the bucket to prevent files from being deleted, thereby avoiding stale vectors. |
Correct Answer: C Correct Option Analysis: - C is Correct: A daily scheduled sync via EventBridge is the most cost-effective approach for frequent deletions. The Bedrock ingestion job natively detects deleted files in the S3 data source and removes their corresponding vectors from the OpenSearch index, satisfying the 24-hour compliance window without incurring costs for running a job on every single deletion. Distractor Analysis: - A is Incorrect: Triggering an ingestion job for every single deletion event via Lambda is not cost-effective, as ingestion jobs have associated compute and embedding API costs, and running them hundreds of times a day is wasteful. - B is Incorrect: Writing custom Lambda code to query and delete vectors directly from OpenSearch Serverless introduces significant operational overhead and bypasses the managed sync capabilities of Bedrock. - D is Incorrect: Enabling S3 Object Lock prevents deletion, which violates the business process of deleting case files and fails to meet the technical requirement of removing stale vectors. > Key Takeaway: When dealing with frequent deletions that have a flexible compliance window (e.g., 24 hours), schedule a single daily Bedrock Knowledge Base sync to detect and remove stale vectors cost-effectively rather than triggering a sync per deletion. |
| ### Question 8 - Orchestrating Custom SageMaker Pipelines for S3 Updates An AI team has implemented a custom RAG ingestion process using SageMaker Pipelines. The pipeline reads documents from an S3 bucket, chunks them, generates embeddings using a SageMaker endpoint, and writes the vectors to an Amazon OpenSearch Service domain. The team wants to automate the triggering of this pipeline whenever a new batch of documents is uploaded to S3, ensuring it only runs when a specific prefix (e.g., pending_updates/) is populated, with the least operational overhead.Which approach should they use? (Select ONE) - A) Configure an S3 Event Notification on the pending_updates/ prefix to invoke an AWS Lambda function that calls the StartPipelineExecution API.- B) Use an Amazon EventBridge schedule to trigger the SageMaker Pipeline every 15 minutes, and add a condition step to check if the prefix is populated. - C) Configure an S3 Event Notification to directly trigger the SageMaker Pipeline execution without an intermediary service. - D) Use AWS Glue to monitor the S3 bucket and trigger the SageMaker Pipeline via an EventBridge rule. |
Correct Answer: A Correct Option Analysis: - A is Correct: S3 Event Notifications can be configured for specific prefixes. Routing this notification to a lightweight Lambda function that calls the StartPipelineExecution API provides an event-driven trigger with minimal operational overhead, ensuring the pipeline only runs when new data is present.Distractor Analysis: - B is Incorrect: Polling the S3 bucket every 15 minutes via EventBridge and adding a condition step wastes compute resources and introduces latency compared to an event-driven approach. - C is Incorrect: S3 Event Notifications cannot directly trigger a SageMaker Pipeline execution natively; an intermediary compute service like Lambda or EventBridge Scheduler is required to make the API call. - D is Incorrect: Using AWS Glue to monitor an S3 bucket for file arrivals is an anti-pattern and introduces unnecessary operational overhead compared to native S3 Event Notifications. > Key Takeaway: To trigger a SageMaker Pipeline based on S3 uploads, use S3 Event Notifications filtered by prefix to invoke Lambda, which then calls the StartPipelineExecution API. |
| ### Question 9 - Robust Retry Mechanisms for RAG Ingestion Jobs A global retailer is using Amazon Bedrock Knowledge Bases with an Amazon OpenSearch Serverless vector store. The ingestion job runs hourly via an EventBridge schedule. Recently, the ingestion jobs have been failing intermittently due to transient OpenSearch Serverless capacity limits during peak hours. The team needs to implement a robust orchestration pattern to monitor the job status and automatically retry the execution if it fails, adhering to MLOps best practices and minimizing manual intervention. Which solution meets these requirements? (Select ONE) - A) Configure the EventBridge schedule to retry the StartIngestionJob API call automatically by setting the maximum retry attempts in the EventBridge target settings.- B) Create an AWS Step Functions state machine to orchestrate the ingestion job, using the Bedrock StartIngestionJob API, and configure a Retry policy with an exponential backoff interval on the polling step.- C) Increase the OpenSearch Serverless capacity units manually before the scheduled run. - D) Enable auto-tuning for the OpenSearch Serverless collection to prevent capacity issues. |
Correct Answer: B Correct Option Analysis: - B is Correct: Using AWS Step Functions to orchestrate the asynchronous StartIngestionJob API is the MLOps best practice. Step Functions can start the job, poll its status, and if the status returns FAILED, a Retry policy with exponential backoff can automatically re-trigger the job, handling transient capacity issues gracefully.Distractor Analysis: - A is Incorrect: EventBridge retries only apply if the API invocation itself fails (e.g., throttling of the StartIngestionJob call). If the API call succeeds but the job execution fails later, EventBridge will not retry the job.- C is Incorrect: Manually adjusting capacity units does not automate the process and violates the requirement to minimize manual intervention. - D is Incorrect: While increasing capacity via auto-scaling might prevent the issue, OpenSearch Serverless auto-scaling takes time to react and does not provide a retry mechanism for jobs that have already failed, making it an incomplete solution for robust orchestration. > Key Takeaway: For asynchronous ML jobs (like Bedrock ingestion) that may fail during execution, use AWS Step Functions to orchestrate the API call and poll for job status, utilizing Retry policies for robust error handling. |
| ### Question 10 - Bulk and Incremental RAG Data Migration An organization is migrating a RAG system to AWS. They have millions of PDF documents stored in an on-premises database. They need to perform an initial bulk ingestion into an Amazon OpenSearch Service domain, followed by ongoing daily incremental updates. The migration must use native AWS services for orchestration and minimize the load on the on-premises database during the ongoing updates. Which combination of steps fulfills these requirements? (Select TWO) - A) Use AWS DataSync to transfer the initial batch of documents to Amazon S3. - B) Configure an Amazon Bedrock Knowledge Base pointing to the S3 bucket and trigger a full ingestion job. - C) Set up an AWS Glue crawler to run against the on-premises database daily, triggering a Lambda function to update OpenSearch. - D) Use AWS Database Migration Service (AWS DMS) to continuously replicate document changes to Amazon S3, followed by EventBridge-scheduled Bedrock syncs. - E) Use an AWS Direct Connect gateway to stream documents directly into the OpenSearch Service domain using a custom Python script. |
Correct Answers: A and B Correct Options Analysis: - A is Correct: AWS DataSync is the optimal service for transferring large batches of files from on-premises to Amazon S3 efficiently and securely, minimizing on-premises load during the initial migration. - B is Correct: Once the documents are in S3, a Bedrock Knowledge Base can be configured to point to the S3 bucket. Triggering a full ingestion job processes the initial bulk load, and subsequent EventBridge-scheduled syncs will handle incremental updates by reading the new files DataSync transfers daily. Distractor Analysis: - C is Incorrect: Running an AWS Glue crawler against an on-premises database daily places a heavy query load on the source database and is not an efficient method for extracting document files compared to DataSync. - D is Incorrect: AWS DMS is designed for relational database replication, not for migrating and syncing unstructured PDF files to S3. - E is Incorrect: Writing a custom Python script over Direct Connect to push directly to OpenSearch bypasses managed ingestion pipelines and introduces significant custom operational overhead. > Key Takeaway: For migrating on-premises unstructured documents to a RAG system on AWS, use AWS DataSync to move files to S3, then use Amazon Bedrock Knowledge Bases to manage both the bulk ingestion and ongoing incremental syncs. |