Skill 3.1.8: Apply Retrieval Augmented Generation (RAG) System Configurations
Task 3.1: Deployment and Orchestration of ML and AI Workflows – Manage deployment infrastructure for ML and AI model types
Core Concept & Business Objective
RAG = Retrieve → Augment → Generate. At inference time, relevant passages are pulled from a private knowledge store and injected into the FM prompt so the answer is grounded in your data.
- Primary objective: inject current, private, or proprietary knowledge into a frozen FM without retraining, with citations and per-user access control.
- Deployment shape: retrieval pipeline (ingest → chunk → embed → index) + query path (retrieve → rerank → generate). In Amazon Bedrock, Knowledge Bases manages both; the generation step is any Bedrock FM via
RetrieveAndGenerate. - Independent levers:
numberOfResults(top_k), search type, metadata filters, reranking, query transformation. Each fixes a different complaint — diagnosing the complaint is the exam skill.
Hard rule: RAG changes what the model knows at inference time, not how the model behaves. Style, format, tone, or deep domain-reasoning gaps → fine-tuning, not RAG.
RAG vs. Fine-Tuning vs. Long Context
| Approach | Best For | Fails When |
|---|---|---|
| RAG | Fast-changing data, citations required, per-tenant/per-user ACLs, traceable sources | Question needs new reasoning style or output format the FM lacks |
| Fine-Tuning | Fixed behavior, tone, structured output, domain jargon, latency/cost reduction via smaller model | Data changes daily; source attribution required |
| Long Context / Large Prompt | Small, static corpora that fit the window | Cost and latency scale with tokens; recall degrades on very long inputs |
Bedrock Knowledge Base Pipeline — Config Levers
| Stage | Levers & Notes |
|---|---|
| Data source | Amazon S3, Confluence, SharePoint, Salesforce, Web Crawler, custom. Sync is manual/scheduled — index is stale until re-synced. |
| Parsing | Default text parser; Amazon Bedrock Data Automation parser; FM-as-a-parser (better for tables/figures/complex PDFs); custom parser Lambda. |
| Chunking | Default = fixed-size 300 tokens, ~20% overlap. Hierarchical (parent ~1500 / child ~300) for precision + context; semantic; custom Lambda; no chunking for structured/tabular data. |
| Embedding | Titan Text Embeddings V2 (1024/512/256 dims) or Cohere Embed. Changing the embedding model requires full re-index. |
| Vector store | OpenSearch Serverless (default), Aurora PostgreSQL/pgvector, Neptune Analytics, OpenSearch managed, Pinecone, Redis Enterprise Cloud, MongoDB Atlas, S3 Vectors. |
| Retrieval API | Retrieve (passages + citations only) vs RetrieveAndGenerate (retrieval + FM invocation + citations + optional session for multi-turn). |
Hard rule: The vector store must be in the same Region and account as the knowledge base, and reachable by the KB service role.
Retrieval Strategy Matrix — Symptom → Lever
| Symptom / Requirement | Lever | Why | Trade-off |
|---|---|---|---|
| Right answer exists but isn't retrieved at all | Raise top_k, improve chunking, or hybrid search | Recall problem — widen the candidate set | More unrelated text in prompt; higher token cost |
| Exact terms, IDs, SKUs, or acronyms missed | Hybrid search (keyword + vector) | BM25 recovers literal tokens embeddings blur | Requires a filterable text field in the vector store — not available on every store |
| Stale / out-of-scope / wrong-tenant docs surfacing | Metadata filters | Pre-filters the candidate set before similarity | Requires metadata sidecar files + maintenance |
| Right passage retrieved but ranked too low | Reranking | Re-scores candidates the first pass already found | Adds latency/cost; cannot recover un-retrieved passages |
| Compound question answered half-wrong | Query decomposition | Splits into sub-questions, each retrieving its own topic | Multiple retrieval calls, more latency |
| Vague/short user query | Query expansion / HyDE | Improves the embedding query's match quality | Risk of drift from user intent |
Exam Gotcha: Reranking improves ordering, not recall. If the correct passage was never retrieved, a reranker cannot save it — the fix is top_k, chunking, or hybrid search. If it is retrieved but buried, lowering top_k would drop it entirely.
Reranking — Two-Stage Retrieval
- Stage 1 (bi-encoder / vector search): fast ANN over embeddings, returns a broad candidate list (e.g., top 20–50).
- Stage 2 (cross-encoder reranker): scores query + each candidate jointly for deep semantic relevance, then returns the top-N.
- AWS options: Amazon Rerank (
amazon.rerank-v1:0) and Cohere Rerank 3.5, invoked inside the Bedrock KB retrieval flow. - Net effect: higher precision in the context window, allowing a smaller final context than raw top_k.
Vector Store Selection
| Store | Choose When |
|---|---|
| OpenSearch Serverless | Default; vector + hybrid + metadata filtering; serverless ops |
| Aurora PostgreSQL (pgvector) | Already running relational workloads; SQL joins with metadata; hybrid |
| Neptune Analytics | Graph relationships matter alongside vectors |
| Pinecone / Redis / MongoDB Atlas | Existing third-party platform investment or existing data gravity |
| S3 Vectors | Lowest-cost, infrequent-query, very large corpora |
Security & Compliance
- Encryption at rest: KMS CMK on the knowledge base, the vector store, and the S3 data source; TLS in transit.
- Service role: the KB assumes an IAM role with
s3:GetObject/ListBucket(and vector-store permissions). Broken sync is usually a role or bucket-policy issue. - Isolation: metadata filters are the row-level-security mechanism — tag documents per tenant/user and filter at query time. Never rely on the FM to withhold retrieved content.
- Guardrails: apply Amazon Bedrock Guardrails on
RetrieveAndGenerateto block PII, denied topics, and prompt injection from retrieved text. - VPC: private S3 access and private vector store endpoints via VPC endpoints keep traffic off the public internet.
Evaluation
- Amazon Bedrock RAG Evaluation (LLM-as-a-judge) scores: Correctness, Completeness, Faithfulness, Helpfulness, Context Relevance, Context Coverage.
- Faithfulness / groundedness = answer is supported by retrieved context (hallucination detector). Context relevance = retrieved passages actually address the query — diagnose retrieval vs generation separately before tuning.
Exam Gotchas
Cold-start of a KB sync ≠ real-time freshness. Newly uploaded S3 objects are invisible until a sync/ingestion job completes.
Increasing top_k is not a universal fix. It raises recall but floods the prompt with noise and raises cost/latency — pair it with a reranker.
Hybrid search is store-dependent. It needs a vector store with a filterable text field; a requirement naming a store without it rules hybrid out.
Custom model + KB ≠ automatic. A model imported via Bedrock Custom Model Import is invoked on demand; a model fine-tuned inside Bedrock requires Provisioned Throughput to serve.
Anti-Patterns
- Using RAG to teach a new output format or reasoning style — that is fine-tuning's job.
- Re-embedding the entire corpus when the actual complaint is ordering — add a reranker first.
- Filtering after generation ("don't mention other tenants") instead of pre-filtering retrieval with metadata.
- Shipping an unsynced index and blaming the FM for stale answers.
- Skipping reranking on a hybrid-search pipeline where top_k is large — precision collapses.
- Evaluating only the final answer — you cannot distinguish retrieval failure from generation failure without context-relevance metrics.