Skip to content

Skill 2.1.5: Select Retrieval Augmented Generation (RAG) architecture patterns based on use case requirements

Question 1 - RAG vs Fine-Tuning for Dynamic Content

A financial services firm is building an investment advisory assistant that must answer client questions using the firm's proprietary research reports, which are updated daily by analysts. The assistant must provide source citations for every answer to meet regulatory compliance requirements. The firm also wants to minimize operational overhead and avoid retraining models when research content changes.

Which architecture pattern BEST satisfies these requirements?

  • A) Fine-tune an Amazon Bedrock foundation model on the firm's proprietary research reports using a monthly training job.
  • B) Use Amazon Bedrock Knowledge Bases with an S3 data source to create a RAG pipeline that retrieves current research at query time.
  • C) Embed research reports into prompts manually using a custom Python application that passes documents to the model.
  • D) Train a custom language model from scratch on Amazon SageMaker using the firm's research corpus.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

B is Correct: Amazon Bedrock Knowledge Bases provides a fully managed RAG architecture that automatically ingests documents from S3, creates embeddings, stores them in a vector database, and retrieves relevant passages at query time. Since source research is updated daily, the Knowledge Base can be re-synced to pick up changes quickly. Source citations are automatically provided with retrieved chunks, satisfying regulatory compliance. Operational overhead is minimal because Bedrock manages the entire retrieval pipeline.

Distractor Analysis

  • A is Incorrect: Fine-tuning writes knowledge into model weights, which requires retraining whenever research content changes. A monthly training job would leave the assistant outdated for up to 30 days, violating the "updated daily" requirement. Fine-tuning also cannot provide source citations since knowledge is embedded in weights.
  • C is Incorrect: This describes a manual RAG approach, but building custom Python code to parse, chunk, embed, and inject documents creates significant operational overhead compared to Bedrock Knowledge Bases. It also requires manually managing the vector store.
  • D is Incorrect: Training a custom language model from scratch is excessive and impractical. This would require massive computational resources, extensive training data, and would not provide the citation or freshness requirements any better than standard RAG.

Key Takeaway: RAG is the correct architecture when knowledge changes frequently and citations are required, because content lives outside the model and can be updated without retraining.

Question 2 - Choosing Between Managed and Custom RAG

A healthcare startup is building a patient-facing chatbot that answers questions from medical policy documents. The engineering team has deep expertise in vector databases and retrieval algorithms. They need granular control over the chunking strategy (custom semantic chunking), the embedding model (fine-tuned domain-specific embeddings), and the reranking logic. They are willing to accept higher operational overhead for this flexibility.

Which approach should the team select?

  • A) Use Amazon Bedrock Knowledge Bases for fully managed RAG with default chunking options.
  • B) Build a custom RAG pipeline using Amazon SageMaker for embeddings, Amazon OpenSearch Serverless for vector storage, and custom Lambda/Step Functions for orchestration.
  • C) Fine-tune a Bedrock foundation model on all policy documents and skip retrieval entirely.
  • D) Use Amazon Kendra for enterprise search and manually pass search results to a foundation model.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

B is Correct: A custom RAG pipeline provides the required granular control over every component. Amazon SageMaker can host a fine-tuned embedding model, OpenSearch Serverless provides scalable vector storage, and Lambda/Step Functions can orchestrate custom chunking and reranking logic. This approach directly addresses the team's need for custom semantic chunking, domain-specific embeddings, and custom reranking.

Distractor Analysis

  • A is Incorrect: Bedrock Knowledge Bases is a managed service with limited chunking strategy options (fixed-size, semantic, hierarchical). While it reduces operational overhead, it does not provide the granular control over chunking, embedding models, and reranking that the team explicitly requires.
  • C is Incorrect: Fine-tuning alone cannot replace retrieval for frequently updated policy documents. Knowledge embedded in weights becomes stale when policies change, and fine-tuning on documents does not provide source citations.
  • D is Incorrect: Amazon Kendra is a managed search service that can work with RAG, but it does not address the team's requirements for custom semantic chunking, fine-tuned embedding models, or custom reranking logic. Kendra uses its own semantic understanding and relevance tuning, which limits control.

Key Takeaway: Choose custom RAG when granular control over components (chunking, embeddings, reranking) is critical; choose managed Knowledge Bases when reducing operational overhead is the priority.

Question 3 - RAG Component Identification

A large media company is designing a document Q&A system. The system must process thousands of PDF files from an S3 bucket, semantically chunk them, generate vector embeddings using Amazon Titan Embeddings, store vectors in OpenSearch Serverless, retrieve relevant passages, and generate answers with Claude v2 on Bedrock. The data science team wants minimal code maintenance.

Which TWO components can be delegated entirely to AWS managed services? (Select TWO)

  • A) Document parsing and chunking
  • B) Embedding generation
  • C) Vector storage and retrieval
  • D) Answer generation with the foundation model
  • E) Chunk-level source metadata tracking and citation formatting
Click to reveal Answer & Explanation

Correct Answers: B and D

Correct Options Analysis

  • B is Correct: Embedding generation can be delegated to Amazon Bedrock's Titan Embeddings model, which is a fully managed API call. No infrastructure or code maintenance is required.
  • D is Correct: Answer generation uses a foundation model (Claude v2 on Bedrock), which is managed by AWS. The team only needs to make an API call with the retrieved context.

Distractor Analysis

  • A is Incorrect: Document parsing and semantic chunking are not fully managed AWS services. While Bedrock Knowledge Bases manages these internally, the standalone processes of parsing custom PDFs and configuring semantic chunking require custom code (Lambda, Glue, or self-managed pipelines).
  • C is Incorrect: OpenSearch Serverless is a managed service, but the team still needs to configure, monitor, and manage the vector index—including encryption policies, data access policies, and scaling. It is not as "hands-off" as Bedrock API calls.
  • E is Incorrect: Source metadata tracking and citation formatting are application-level responsibilities that require custom code. No AWS service automatically formats citations for the end user.

Key Takeaway: In a RAG pipeline, model-based components (embeddings, generation) are API-level managed services, while document processing and citation formatting require application code.

Question 4 - RAG for Frequently Updated Knowledge Base

An e-commerce company is building a product recommendation assistant. Product details (prices, descriptions, availability) are stored in an Amazon Aurora PostgreSQL database and change frequently throughout the day. The assistant must provide accurate, up-to-the-minute product information to customers.

Which architecture is MOST appropriate for this use case?

  • A) Use Amazon Bedrock Knowledge Bases with Aurora as the data source, syncing the knowledge base hourly.
  • B) Implement a real-time RAG system using Aurora PostgreSQL with pgvector extension for live vector search, coupled with a Bedrock embedding model.
  • C) Fine-tune a foundation model on daily product data extracts.
  • D) Store product data in an S3 bucket and use Athena queries at request time.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

B is Correct: For frequently changing data (multiple times per day), a real-time RAG system with pgvector on Aurora PostgreSQL allows immediate vector search over live data. When product details update in Aurora, the corresponding embeddings can be updated in the same transaction, ensuring the assistant retrieves current information without batch synchronization delays.

Distractor Analysis

  • A is Incorrect: Bedrock Knowledge Bases supports data sources like S3, but hourly syncing introduces up to 60 minutes of staleness. For products changing "frequently throughout the day," this latency could result in incorrect pricing or availability information being presented to customers.
  • C is Incorrect: Fine-tuning cannot keep pace with data that changes multiple times daily. Training jobs require significant time and compute resources, making daily or intra-day retraining impractical and cost-prohibitive.
  • D is Incorrect: Athena queries against S3 data do not provide vector similarity search capabilities. Athena is a SQL query engine, not a vector database, so it cannot perform the semantic retrieval required for RAG.

Key Takeaway: When data changes continuously, use a database with native vector support (like pgvector in Aurora) for real-time semantic search rather than batch-synced managed knowledge bases.

Question 5 - RAG Failure Mode Identification

A company implemented a RAG system for their legal document analysis platform. The system retrieves individual passages from contracts and generates summaries based on those passages. However, users report that the system often misses important cross-references between sections of the same contract and fails to understand the document's overall structure.

What is the MOST likely architectural limitation causing this issue?

  • A) The embedding model is not powerful enough to capture semantic meaning.
  • B) RAG inherently struggles with document-level synthesis tasks that require understanding relationships across the entire document.
  • C) The vector database is not properly indexed for the size of the documents.
  • D) The foundation model used for generation lacks sufficient parameters.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

B is Correct: RAG is designed to retrieve relevant passages and generate answers from those isolated chunks. It is fundamentally weaker when the task requires whole-document synthesis—understanding cross-references, overall structure, and relationships between sections that span the entire document. This is a known limitation of the retrieval pattern, which works best for Q&A from discrete passages rather than holistic document analysis.

Distractor Analysis

  • A is Incorrect: A standard embedding model can effectively capture semantic similarity between queries and passages. The issue described is not about semantic matching but about synthesizing relationships across the entire document structure.
  • C is Incorrect: Vector database indexing problems would manifest as poor retrieval quality (irrelevant or missing passages), not as failures to understand cross-document relationships. The issue described is architectural, not infrastructural.
  • D is Incorrect: While model capacity affects reasoning quality, even powerful models struggle with document-level synthesis when they only receive isolated chunks. The problem is the retrieval architecture, not the generation model.

Key Takeaway: RAG is best for passage-level Q&A; for document-level synthesis tasks that require understanding global structure, consider alternative approaches like long-context models or fine-tuning.

Question 6 - Production RAG Security and Compliance

A government agency is building a RAG system for processing classified documents. The system must ensure that retrieved documents are never logged, stored, or retained by the foundation model provider. All data must remain within the agency's AWS account boundaries. The agency requires complete auditability of every retrieval action.

Which configuration BEST satisfies these requirements?

  • A) Use Amazon Bedrock with default settings and enable CloudTrail logging.
  • B) Use a custom RAG pipeline with Amazon SageMaker hosting an open-source embedding model and OpenSearch Serverless for vector storage, processing all requests within the agency's VPC.
  • C) Use Amazon Bedrock Knowledge Bases with a custom KMS key for encryption.
  • D) Use Amazon Kendra with an IAM role that restricts access to the agency's S3 bucket.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

B is Correct: A custom RAG pipeline with SageMaker-hosted open-source models and OpenSearch Serverless ensures that all processing occurs within the agency's AWS account. No data is sent to external foundation model providers. VPC configuration adds network isolation. OpenSearch Serverless provides comprehensive audit logging via CloudTrail and AWS CloudFormation auditing extensions. This design provides complete data sovereignty and auditability.

Distractor Analysis

  • A is Incorrect: Amazon Bedrock may route inference requests outside the customer's account depending on the model provider. However, the key concern is that some Bedrock models (like Claude) are hosted by third-party providers (Anthropic), which may not meet strict government data residency or sovereignty requirements for classified documents.
  • C is Incorrect: Bedrock Knowledge Bases with KMS encryption provides at-rest encryption but does not guarantee that model inference occurs within the agency's account. The foundation model provider may still process the data in their own infrastructure.
  • D is Incorrect: Amazon Kendra is a managed search service, but it does not address the core issue of data sovereignty for classified documents. Kendra processes data in AWS-managed infrastructure, which may not meet the requirement for data to remain within the agency's account.

Key Takeaway: When data sovereignty requires everything to remain within your AWS account, use self-hosted open-source models on SageMaker rather than Bedrock's third-party foundation models.

Question 7 - RAG Latency Optimization

A customer support application uses RAG to answer user queries. The current pipeline has a latency of 8 seconds, which is unacceptable. The breakdown is: embedding generation (300ms), vector search (500ms), retrieval post-processing (1.2s), prompt assembly (200ms), and foundation model generation (6s). The team must reduce total latency to under 3 seconds.

Which TWO changes would MOST effectively reduce latency? (Select TWO)

  • A) Switch from a foundation model to a smaller, faster model on Bedrock for generation.
  • B) Add caching for embedding generation using ElastiCache.
  • C) Increase the number of retrieved passages to improve answer quality.
  • D) Use semantic caching to serve common queries from a response cache.
  • E) Migrate the vector store from OpenSearch Serverless to a self-managed OpenSearch cluster for faster search.
Click to reveal Answer & Explanation

Correct Answers: A and D

Correct Options Analysis

  • A is Correct: The foundation model generation accounts for 6 seconds of the 8-second total latency. Switching to a smaller, faster model could reduce this to under 1 second, making it the single most impactful change.
  • D is Correct: Semantic caching stores responses to previously seen queries and their semantically similar variants. For customer support applications with repetitive questions, this eliminates both the embedding generation and generation time for common queries, reducing latency to milliseconds.

Distractor Analysis

  • B is Incorrect: Embedding generation only accounts for 300ms. Caching embeddings would provide marginal improvements and does not address the dominant latency contributor (foundation model generation).
  • C is Incorrect: Increasing the number of retrieved passages would likely increase latency (longer prompt assembly, more tokens for generation) and is the opposite of what is needed to reduce overall response time.
  • E is Incorrect: Vector search only takes 500ms. Migrating to a self-managed cluster introduces significant operational overhead for minimal latency improvement (perhaps 100-200ms). The foundation model generation is the bottleneck, not the vector search.

Key Takeaway: Always profile latency before optimizing. Target the bottleneck (often foundation model generation) and use semantic caching for repetitive queries.

Question 8 - RAG Data Source Integration Strategy

A logistics company wants to build a RAG system that answers questions from three data sources: (1) PDF shipping manifests in S3, (2) real-time package tracking data in DynamoDB, and (3) historical incident reports in a data warehouse. The system should provide near-real-time answers for tracking questions while still retrieving from static documents.

Which architecture BEST accommodates these heterogeneous data sources?

  • A) Use Amazon Bedrock Knowledge Bases with S3 as the single data source, exporting DynamoDB and warehouse data to S3 daily.
  • B) Rule out RAG entirely and fine-tune a model on all three data sources.
  • C) Implement a hybrid RAG architecture: Bedrock Knowledge Bases for PDF documents, plus a custom retrieval path with pgvector in Aurora for DynamoDB data, and batch embeddings for warehouse data.
  • D) Upload all data to a single OpenSearch Serverless collection and query it with a custom retriever.
Click to reveal Answer & Explanation

Correct Answer: C

Correct Answer Analysis

C is Correct: A hybrid RAG architecture is the optimal approach for heterogeneous data sources with different freshness requirements. Bedrock Knowledge Bases handles the static PDF documents (managed, low overhead). A separate real-time path with pgvector on Aurora provides instant access to DynamoDB tracking data. Warehouse data can be batch-embedded and stored in a vector store since it changes less frequently. This federated retrieval approach gives each data type the architecture it needs.

Distractor Analysis

  • A is Incorrect: Exporting DynamoDB and warehouse data to S3 daily introduces significant staleness. Tracking data requires near-real-time answers, and a daily batch export could provide outdated information.
  • B is Incorrect: Fine-tuning cannot handle real-time data changes. It is impossible to fine-tune a model on data that changes in DynamoDB every second. This approach fundamentally cannot satisfy the real-time requirement.
  • D is Incorrect: Uploading all data to a single OpenSearch Serverless collection creates a uniform treatment of data with very different update cadences. Real-time DynamoDB data would require constant re-indexing, and the static PDFs would be unnecessarily mixed with highly volatile data.

Key Takeaway: Use hybrid RAG architectures when data sources have different freshness requirements—managed Knowledge Bases for static content and custom real-time paths for dynamic data.

Question 9 - Distributed RAG with Multiple Data sources

A global enterprise is building a centralized RAG-based knowledge assistant that must query documents across multiple departments (HR, legal, finance, IT). Each department's documents have different access permissions. The system must enforce document-level access control so users only receive answers from documents they are authorized to view.

Which approach BEST enforces document-level access control in a RAG system?

  • A) Use a single Bedrock Knowledge Base and filter results in the application layer after retrieval.
  • B) Create separate Bedrock Knowledge Bases for each department and query only the relevant one based on user permissions.
  • C) Store all documents in a single OpenSearch Serverless collection with per-document metadata containing access roles, and apply metadata filtering during retrieval.
  • D) Fine-tune a separate model for each department.
Click to reveal Answer & Explanation

Correct Answer: C

Correct Answer Analysis

C is Correct: Storing all documents in OpenSearch Serverless with per-document metadata (access roles, department tags) and using metadata filtering during retrieval is the cleanest approach. The retriever can construct queries that only search documents where the metadata matches the user's permissions. This provides fine-grained, document-level access control in a single, manageable system.

Distractor Analysis

  • A is Incorrect: Filtering after retrieval is a security vulnerability. Documents can leak into the foundation model's context, and even if the final answer is filtered, the model may have generated content based on unauthorized documents. Also, post-retrieval filtering wastes tokens and reduces answer quality.
  • B is Incorrect: While separate Knowledge Bases provide basic department-level access control, this approach does not scale well for fine-grained (document-level) permissions within departments. It also creates operational overhead managing multiple Knowledge Bases.
  • D is Incorrect: Fine-tuning separate models for each department is extremely impractical and expensive. It also does not directly enforce access control—a fine-tuned model might still generate answers from knowledge learned from another department's data.

Key Takeaway: Enforce access control at the retrieval stage (before the foundation model sees the data) using metadata filtering in the vector store.

Question 10 - RAG Orchestration with Step Functions

A company is building a document processing and Q&A system. The pipeline must: download PDFs from S3, parse them with Amazon Textract, chunk the text, generate embeddings with Amazon Bedrock, store embeddings in OpenSearch Serverless, and expose a query API that retrieves relevant chunks and generates answers. The operations team needs visibility into each stage, automatic retries on failure, and the ability to handle fluctuating document volumes.

Which architecture BEST orchestrates the ingestion pipeline?

  • A) Use a single Lambda function that performs all steps sequentially.
  • B) Use AWS Step Functions to orchestrate Lambda functions for each pipeline stage, with automatic retry policies and CloudWatch monitoring.
  • C) Use Amazon SageMaker Pipelines to create and manage the ingestion workflow.
  • D) Use Amazon Bedrock Knowledge Bases and let AWS handle the entire ingestion pipeline.
Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

B is Correct: AWS Step Functions is specifically designed for orchestrating multi-step workflows with built-in retry policies, error handling, and integration with CloudWatch for monitoring. Each pipeline stage (download, parse, chunk, embed, store) can be a Lambda function or direct service integration. Step Functions provides automatic retries, parallel processing for multiple documents, and detailed execution history for observability.

Distractor Analysis

  • A is Incorrect: A single Lambda function with sequential processing has a 15-minute timeout limitation, making it unsuitable for processing large document batches. It also lacks built-in retry mechanisms and makes it difficult to monitor and debug individual pipeline stages.
  • C is Incorrect: SageMaker Pipelines is designed for ML training and deployment workflows, not document ingestion and RAG pipelines. Using it for this purpose would be an architectural mismatch and introduce unnecessary complexity.
  • D is Incorrect: While Bedrock Knowledge Bases handles ingestion automatically, the team explicitly specified custom processing requirements (Textract parsing, custom chunking, monitoring, retries). Bedrock Knowledge Bases does not provide visibility into each ingestion stage or custom retry policies.

Key Takeaway: Use Step Functions for multi-stage data processing pipelines that require visibility, retry logic, and operational observability; Bedrock Knowledge Bases is preferred when no custom processing is needed.

Summary of Key Patterns for Skill 2.1.5

Requirement Recommended Pattern
Frequently updated content RAG with real-time sync or pgvector
Source citations required RAG (not fine-tuning)
Minimal operational overhead Bedrock Knowledge Bases
Granular control over components Custom RAG pipeline
Document-level access control Metadata filtering in vector store
Real-time data sources pgvector on Aurora / custom retrieval
High latency in generation Smaller model or semantic caching
Document synthesis (not Q&A) Long-context models or fine-tuning
Data sovereignty concerns Self-hosted models on SageMaker
Complex ingestion pipeline Step Functions orchestration