Skip to content

Skill 3.2.8: Implement retrieval pipelines to meet business needs

Task 3.2: Deployment and Orchestration of ML and AI Workflows - Provision and configure resources for ML and AI workloads based on existing architecture and requirements

Core Concept & Business Objective

  • Retrieval pipeline = ingestion → chunking → embedding → indexing → query → retrieval → reranking → generation.
  • RAG objective: ground LLM responses in enterprise data to improve accuracy, reduce hallucination, and meet SLAs for latency, cost, freshness, and security.
  • Primary AWS service: Amazon Bedrock Knowledge Bases (managed RAG). Alternatives: custom pipeline with OpenSearch, Aurora pgvector, Neptune Analytics, or object vector storage.
  • Key trade-off: managed KB = faster, less ops, less control; custom pipeline = full control over chunking, embedding, retrieval, reranking, orchestration.

Retrieval Pipeline Stages

Stage Purpose Key Configurations AWS Services
Ingestion Load and parse documents Data source connectors, chunking, parsing S3, Bedrock KB, custom
Embedding Convert chunks to vectors Embedding model, dimensions, normalization Bedrock, SageMaker
Indexing Store vectors + metadata Vector store, index field mapping, dimensions OpenSearch, Aurora pgvector, Neptune Analytics
Retrieval Find relevant chunks Search type, top-k, metadata filters, reranking Bedrock KB, OpenSearch
Generation Produce grounded answer Prompt template, context window, citations Bedrock, retrieve-and-generate
Monitoring Track quality/cost Recall@k, precision@k, latency, cost CloudWatch, CloudTrail

Vector Store Decision Matrix

Store Best For Retrieval Capabilities Watch-outs
OpenSearch Serverless Managed, auto-scaling, hybrid search Vector, keyword, hybrid, filters, reranking Cost at scale, OCU limits
OpenSearch Managed Control, VPC, existing clusters Same as serverless, more tuning Ops overhead, sizing
Aurora PostgreSQL pgvector Existing relational data, SQL joins Vector + SQL, filters Scaling, index tuning, not native hybrid
Neptune Analytics Graph relationships, entity traversal Graph + vector Specialized, cost
Object storage / S3 Vectors Large, infrequent, low cost Vector only, high latency No hybrid, no low-latency

Search Type Decision Matrix

Search Type Use When Avoid When
Semantic (vector) Conceptual, natural language, synonyms Exact IDs, codes, rare terms
Keyword (BM25) Exact terms, IDs, legal citations Synonyms, paraphrases
Hybrid Both exact and semantic needed Simple, low-latency, cost-sensitive
Graph Relationships between entities Unstructured text without graph

Chunking & Embedding

  • Chunk size: smaller = precise, less context; larger = more context, diluted embedding. Tune empirically.
  • Overlap: preserves context across boundaries; increases storage/cost.
  • Hierarchical: parent-child chunks for context preservation.
  • Semantic: split by meaning; higher compute.
  • Embedding model: must be same for ingestion and query. Dimensions must match vector index.
  • Metadata: attach source, tenant, classification, date, author. Enables filtering and access control.

Retrieval Optimization by Business Need

Business Need Primary Levers Trade-offs
Accuracy / relevance Hybrid search, reranking, metadata filters, better chunking Latency, cost
Low latency Smaller top-k, skip rerank, serverless vector store, cache Lower recall
Cost efficiency Incremental sync, lower dimensions, object vector store, smaller top-k Freshness, latency
Freshness Event-driven ingestion, incremental resync, near-real-time Complexity, cost
Security / isolation Pre-filter metadata, IAM, KMS, VPC endpoints Operational overhead
Exact match Hybrid search (BM25 + vector) Needs filterable text field
Multi-tenant Tenant metadata filter at query time Must enforce pre-filter

Security & Compliance

  • IAM: KB execution role needs least-privilege to data source, vector store, embedding model.
  • KMS: encrypt S3, OpenSearch, Aurora, Bedrock. Use CMK for control.
  • VPC: use VPC endpoints for S3, OpenSearch, Bedrock, KMS. Private subnets.
  • Data governance: metadata filters for tenant isolation, row-level security. Audit with CloudTrail, CloudWatch.
  • Guardrails: Bedrock Guardrails for PII, toxicity, grounding, hallucination.

Provisioning & Configuration Checklist

  • Vector store: provision OpenSearch Serverless collection or Aurora cluster; size for expected vectors and QPS.
  • Embedding model: choose and note dimensions; ensure vector index matches.
  • Index mapping: define vector, text, metadata fields before ingestion.
  • Data source: configure S3 bucket, connectors, access role.
  • Ingestion: schedule or event-driven; use incremental resync.
  • Retrieval: configure search type, top-k, filters, reranking.
  • Network: private subnets, security groups, VPC endpoints for S3, OpenSearch, Bedrock, KMS.
  • IAM: execution role with least privilege.
  • Encryption: KMS CMK for S3, OpenSearch, Aurora, Bedrock.
  • Monitoring: CloudWatch metrics, CloudTrail logs, retrieval quality metrics.

Key Operations

  • Bedrock Knowledge Bases: create/update knowledge base, start ingestion job, retrieve, retrieve-and-generate.
  • Retrieve-and-generate: end-to-end RAG with citations.
  • Retrieve: custom orchestration; you control generation.
  • Ingestion job: full sync or incremental resync.
  • Update knowledge base: change config, but some changes require reindexing.

Common Anti-Patterns & Distractor Traps

  • Mismatched embedding models between ingestion and query.
  • Changing embedding model or dimensions without reindexing.
  • Metadata field mapping after ingestion → cannot change without reindexing.
  • Semantic search alone for exact IDs → use hybrid.
  • Reranking as recall fix → reranking improves precision, not recall.
  • Full reindex for every update → use incremental resync.
  • Post-filtering for security → pre-filter to avoid leakage.
  • Oversized chunks → diluted embeddings.
  • No evaluation → track recall@k, precision@k, MRR, nDCG.

Embedding model and vector index dimensions must match. Changing either requires reindexing. Metadata field mapping must be declared before ingestion; cannot change without reindexing. Hybrid search requires a filterable text field alongside vectors. Resync processes only changed documents since last ingestion, not full reindex. Reranking improves precision, not recall. If relevant docs are not in top-k, reranking cannot recover them. For multi-tenant security, apply metadata filters at query time; post-filtering can leak data if not carefully implemented.