Skill 1.2.7: Prepare documents for Retrieval Augmented Generation (RAG) applications (chunking strategies, metadata extraction)
Task 1.2: Data Preparation for ML and AI - Perform data transformation, feature engineering, and pre-processing
Core Objective: Convert unstructured documents into retrievable, attributable, filterable chunks + embeddings + metadata for a vector store.
RAG = dynamic knowledge retrieval; fine-tuning = behavior; continued pre-training = domain vocabulary; distillation = model compression.
RAG Ingestion Pipeline:
S3 source → Extract (Textract, Comprehend, Lambda) → Clean/Normalize → Chunk → Enrich Metadata → Embed (Bedrock embedding models) → Index (OpenSearch/Aurora pgvector/Neptune/Pinecone) → Retrieve + Rerank → Generate.
Chunking Strategies
| Strategy | Mechanism | Best For | Trade-offs / Exam Notes |
|---|---|---|---|
| Default / Fixed-size | Token/character count; Bedrock KB default ≈ 300 tokens. Fixed-size allows max tokens (100–1500) and overlap (0–25%). | Homogeneous text, logs, simple docs. | May split mid-sentence; overlap preserves context. Too much overlap = duplicate retrieval. |
| Hierarchical | Parent + child chunks. Retrieve child, return parent for context. | Long structured docs: manuals, policies, contracts. | More storage; needs parent/child metadata. Good for precision + context. |
| Semantic | Splits on meaning/topic shifts, not length. | Narrative, varied topics, transcripts. | Higher cost/latency; variable size; not default. |
| No chunking | Whole file as one chunk. | Small, atomic docs (FAQ, single policy). | Large files exceed context/token limits; poor retrieval. |
| Custom (Lambda) | Domain-specific splitting (clauses, code, tables). | Legal, code, specialized formats. | Must manage size, overlap, metadata, idempotency. |
Chunk Size & Overlap Tuning
- Too large: diluted embedding, irrelevant context, “lost in the middle”, token cost.
- Too small: fragmented meaning, missing context, low recall.
- Overlap: 10–20% typical; preserves cross-boundary context; too much duplicates index and retrieval.
- Respect structure: split on headings, sections, paragraphs, tables, code blocks. Do not split tables mid-row.
Metadata Extraction
| Metadata Category | Purpose | Examples |
|---|---|---|
| Citation/attribution | Tell user where answer came from | source URI, title, page, section, chunk ID |
| Filtering | Narrow retrieval before similarity | product, version, language, doc type, department, date |
| Access control | Enforce permissions at retrieval | ACL, tenant ID, role, sensitivity, classification |
| Freshness | Avoid stale answers | effective date, version, last modified, expiry |
| Lineage/dedup | Trace and deduplicate | doc ID, parent ID, hash, ingestion timestamp |
Extraction methods: parser/regex, Amazon Textract (forms/tables), Amazon Comprehend (entities/PII), LLM-based extraction, Lambda custom logic. Store metadata alongside vectors; filter at query time.
Security & Compliance
- IAM: least-privilege role for Bedrock KB to read S3, write vector store, invoke embedding model.
- Encryption: KMS at rest for S3, vector DB, Bedrock; TLS in transit.
- Network: VPC endpoints (PrivateLink) for S3, Bedrock, OpenSearch, Secrets Manager.
- Sensitive data: Amazon Macie discovers sensitive data in S3; Amazon Comprehend detects PII; redact/mask/anonymize before embedding. Never embed secrets/PII.
- Access control: preserve ACLs as metadata; enforce filters at retrieval. Avoid separate index per user unless required.
- Audit: CloudTrail, CloudWatch, Bedrock invocation logging.
Decision Matrix: RAG vs Customization
| Goal | Artifact | AWS Path | When to Use |
|---|---|---|---|
| Dynamic knowledge, citations | Chunks + embeddings + metadata | Bedrock KB + vector store | Docs change frequently; need attribution; least rework. |
| Behavior/tone | Labeled prompt-response JSONL | Bedrock fine-tuning | Prompting fails; stable behavior. |
| Domain vocabulary | Unlabeled domain text | Bedrock continued pre-training | Large domain corpus; absorb jargon/style. |
| Smaller/cheaper model | Prompt set | Bedrock distillation | Compress teacher model; cost/latency reduction. |
Managed vs Custom RAG Pipeline
| Aspect | Bedrock Knowledge Bases | Custom Pipeline |
|---|---|---|
| Best for | Standard docs, S3, quick RAG | Non-standard formats, strict compliance, custom vector DB |
| Chunking | Default, fixed-size, hierarchical, semantic, none | Lambda/Glue/EMR custom chunking |
| Metadata | Metadata files + filtering | Custom extraction + store |
| Sync | Managed sync from S3 | Custom orchestration |
| Trade-off | Less control, faster | More control, more ops |
Anti-Patterns & Exam Gotchas
Default Bedrock KB chunking = fixed-size ≈ 300 tokens. If no strategy is chosen, this is used.
Semantic chunking is not default. It costs more; use when topic shifts matter.
No chunking only for small, atomic docs. Large files fail retrieval/token limits.
Metadata is not embedded by default. It is stored with vectors for filtering/citations. Without metadata, no access control or attribution.
RAG updates knowledge without retraining. Fine-tuning bakes knowledge into weights; frequent changes cause rework.
Redact PII before ingestion. Anything embedded can surface in model output.
Overlap too high = duplicate retrieval; too low = lost context.
Chunking on fixed characters can split tables/code. Respect document structure.
Hierarchical chunking returns parent for context; child for retrieval.
For access control, use metadata filters, not separate indexes per user.
Embeddings capture semantic meaning; categorical encoding is for tabular labels. Do not use encoding for RAG.