Skill 2.1.5: Select Retrieval Augmented Generation (RAG) architecture patterns based on use case requirements
Task 2.1: ML Model and Foundation Model (FM) Development - Choose appropriate modeling approaches for ML and AI solutions
RAG objective: Ground FM answers in external, updatable, access-controlled knowledge without retraining. Use when facts change, citations are required, or proprietary data exceeds prompt capacity. Do not use RAG to teach style/format/behavior—that is prompting or fine-tuning.
Core Decision: RAG vs Fine-Tuning vs Prompting vs Rules
| Requirement | Best Fit | Why | AWS Path |
|---|---|---|---|
| Fresh/volatile facts, citations, ACLs | RAG | Knowledge stays outside weights; re-index in minutes | Bedrock Knowledge Bases, OpenSearch, Kendra |
| Style, tone, format, domain jargon | Fine-tuning / SFT | Behavior is written into weights | Bedrock custom models |
| Simple instructions, formatting, few-shot | Prompt engineering | Zero training, immediate change | Bedrock prompts |
| Fixed business logic | Rules / code | Cheaper, faster, auditable | Lambda, Step Functions |
| General language capability | FM only | Capability already in model | Bedrock |
| Standard pre-solved task | Managed AI service | No model build | Textract, Rekognition, Comprehend, Transcribe |
RAG Architecture Anatomy
| Component | Purpose | AWS Options | Exam Notes |
|---|---|---|---|
| Ingestion / connectors | Pull source docs | S3, SharePoint, Confluence, Salesforce, web crawler, custom | Sync schedule must match update frequency |
| Parsing / OCR | Extract text, tables, forms | Textract, Comprehend, Transcribe | Scanned PDFs need Textract; audio needs Transcribe |
| Chunking | Split into retrievable passages | Bedrock KB chunking, Lambda custom | Poor chunking is a top RAG failure |
| Embedding | Convert text to vectors | Titan Embeddings, Cohere Embed | Same model for ingestion and query |
| Vector store | Store/search embeddings | OpenSearch Serverless, Aurora pgvector, Neptune Analytics, Kendra GenAI Index | Managed vs custom control trade-off |
| Retriever | Fetch top-k relevant passages | Bedrock KB, OpenSearch, Kendra | Vector, keyword, hybrid, rerank |
| Generation | Produce grounded answer | Bedrock FMs | Prompt must enforce “answer only from context” |
| Orchestration | Chain retrieve → generate → cite | Bedrock Agents, Step Functions, Lambda | Agentic RAG adds multi-step tool use |
| Guardrails | Safety, PII, grounding | Bedrock Guardrails, Comprehend PII | Block prompt injection and ungrounded answers |
RAG Pattern Selection Matrix
| Pattern | When to Use | AWS Implementation | Trade-offs |
|---|---|---|---|
| Fully Managed RAG | Fastest deployment, least ops, common Q&A, policy libraries | Bedrock Knowledge Bases | Limited control over retriever, chunking, vector store |
| Custom RAG | Need control over embeddings, hybrid search, metadata filters, reranking | OpenSearch Serverless, Aurora pgvector + Bedrock | More ops, tuning, security responsibility |
| Enterprise Search RAG | Many sources, ACL-aware, high accuracy | Amazon Kendra + Bedrock | Higher cost; less ML tuning |
| Hybrid RAG | Exact terms, IDs, acronyms, technical docs | OpenSearch hybrid, Kendra | Best default for enterprise; needs fusion/rerank |
| GraphRAG | Multi-hop relationships, supply chain, fraud | Neptune Analytics + Bedrock | Complex, costly, relationship-heavy |
| Multimodal RAG | Images, PDFs, audio, video | Textract, Rekognition, Transcribe + Bedrock multimodal | Higher cost, complex embedding/storage |
| Agentic RAG | Query decomposition, multi-step retrieval, tools | Bedrock Agents, Step Functions | Latency and cost increase; harder to debug |
| Parent-Child RAG | Precise retrieval + broad context | Custom chunking + vector store | Better context; more storage and logic |
Retrieval Strategy Selection
| Strategy | Best For | Weakness |
|---|---|---|
| Dense vector | Semantic similarity, natural language, multilingual | Misses exact IDs, codes, rare terms |
| Keyword / BM25 | Exact terms, names, codes, acronyms | Misses paraphrase and synonyms |
| Hybrid | Enterprise search, technical docs, mixed queries | Needs score fusion / reranking |
| Reranking | Improve top-k precision | Adds latency and cost |
| Metadata filtering | Tenant isolation, date, doc type, ACLs | Requires clean metadata at ingestion |
| Query transformation | Recall improvement, complex questions | Adds latency; can drift from intent |
Exam Gotcha: Vector search alone is rarely enough for enterprise RAG. If requirements mention exact IDs, acronyms, or access control, choose hybrid + metadata filters + reranking.
Chunking & Embedding Decisions
- Fixed-size chunking: Simple, fast, baseline. Risk: splits meaning across boundaries.
- Hierarchical / parent-child: Retrieve small child chunks, send larger parent context. Better answer quality.
- Semantic chunking: Embedding-based boundaries. Higher cost, better coherence.
- Document-aware chunking: Split by headings, sections, tables. Best for structured docs.
- Overlap: Preserve context across boundaries. Common 10–20% overlap.
- Metadata: Always capture source, page, section, tenant, ACL. Required for citations and security.
- Embedding model consistency: Query and document embeddings must use the same model. Changing models requires re-embedding.
- Dimension trade-off: Higher dimensions can improve accuracy but increase storage, latency, and cost.
Managed vs Custom RAG Trade-offs
| Dimension | Managed RAG (Bedrock KB) | Custom RAG |
|---|---|---|
| Speed to production | Fastest | Slowest |
| Operational overhead | Lowest | Highest |
| Control over retriever | Limited | Full |
| Control over vector store | Limited to supported stores | Any vector DB |
| Chunking control | Built-in options | Fully custom |
| Security responsibility | Shared | You manage more |
| Best for | Common Q&A, rapid rollout | Complex retrieval, strict tuning, existing vector DB |
Security, Compliance, and Isolation
- IAM roles: Bedrock KB needs roles to access S3, OpenSearch, KMS, and data sources.
- Encryption at rest: KMS for S3, vector store, Bedrock custom models.
- Encryption in transit: TLS for all service calls.
- VPC endpoints: Use PrivateLink for Bedrock, S3, OpenSearch, Kendra, Textract, Comprehend, Transcribe.
- Guardrails: Bedrock Guardrails for PII redaction, denied topics, contextual grounding, prompt injection.
- Document ACLs: Kendra is ACL-aware; custom RAG needs metadata filters for tenant/user isolation.
- Audit: CloudTrail, CloudWatch, S3 access logs.
- Data residency: Keep models, vector stores, and data in approved Regions.
Anti-Patterns & Exam Gotchas
RAG is not fine-tuning. Do not fine-tune to add fresh facts. Do not use RAG to enforce strict style/format if prompting/fine-tuning is the better fit.
- No citations: If source attribution is required, RAG must return document metadata.
- No reranking: Top-k vector search alone can miss exact terms and lower precision.
- Poor chunking: Too large dilutes relevance; too small loses context.
- No metadata filters: Risks cross-tenant data leakage.
- Ignoring latency: Retrieve + rerank + generate adds hundreds of ms to seconds.
- Context stuffing: More chunks can reduce accuracy and increase cost.
- Stale index: Sync schedule must match update frequency.
- Embedding mismatch: Query and document embeddings must use the same model.
- No evaluation: Measure retrieval recall/precision, answer faithfulness, citation accuracy.
- Over-reliance on FM knowledge: RAG should ground answers in retrieved context, not blend unverified model memory.
High-Yield Recall: Volatile facts + citations → RAG. Style/format → fine-tuning. Fixed logic → rules. Standard task → managed AI service. For enterprise RAG: hybrid retrieval + metadata filters + reranking + guardrails + citations.