Skip to content

Skill 1.1.7: Configure scalable vector databases for AI applications

Task 1.1: Data Preparation for ML and AI - Collect and store data

Core Concept & Spec Inputs

  • Vector DB role: Store embeddings and run similarity search (k-NN/ANN) for RAG, semantic search, recommendations, deduplication, anomaly detection.
  • Primary AWS objective: Match non-functional specs—latency, QPS, recall, scale, cost, compliance—to a managed vector search service. Avoid moving data when an existing store already fits.
  • Spec-to-design inputs:
Spec input Design impact
Vector count Sharding, storage tier, index type
Dimensions Memory, index limits, dimensionality reduction
QPS / p95 latency Replicas, Serverless OCUs, HNSW vs IVF
Recall target HNSW/IVF params, exact vs approximate
Filter selectivity Pre-filter vs post-filter, partitioning
Update/delete rate Index maintenance, rebuild strategy
Compliance VPC, KMS, IAM, audit logging, residency

Vector Store Selection Matrix

Store Best-fit spec Scaling model Latency Ops burden Anti-pattern
OpenSearch Service (provisioned) Semantic + keyword hybrid, filters, aggregations, dashboards Shards, replicas, node memory Low Medium Under-sized hot nodes for HNSW
OpenSearch Serverless Spiky/unpredictable QPS, no cluster ops, vector search collection Auto-scale via OCUs Low Low Steady low traffic where OCU minimums cost more
RDS/Aurora PostgreSQL + pgvector Existing PostgreSQL, ACID, joins, moderate vector count Vertical, read replicas, Aurora Serverless v2 Low-medium Medium Billions of vectors or very high QPS
Amazon S3 Vectors Massive scale, cost-sensitive, infrequent queries, data lake integration Object storage, vector buckets/indexes Higher Low Real-time high-QPS RAG
Raw Amazon S3 Batch/offline similarity, archival embeddings S3 + Athena/EMR/Spark High Low Using raw S3 as a real-time vector DB

OpenSearch Service (Provisioned & Serverless)

  • Use when: Need managed k-NN, hybrid search, metadata filtering, aggregations, or existing OpenSearch stack.
  • Algorithms: Approximate k-NN via HNSW for low latency; IVF for cost-sensitive large sets; exact k-NN for small sets/validation. Engines: Lucene, FAISS, nmslib.
  • Distance metrics: Cosine for normalized text embeddings; inner product often equivalent for normalized vectors; Euclidean/L2 for some image/audio models.
  • Tuning: HNSW m, ef_construction, ef_search; IVF nlist, nprobes. Higher = better recall, more memory/latency.
  • Filtering: Pre-filtering preserves recall; post-filtering can return fewer than k results. High-selectivity filters may need partitioning.
  • Scaling: Shards distribute data; replicas increase QPS and HA. Hot nodes need memory for HNSW graphs. Serverless auto-scales.
  • Security: VPC, IAM, fine-grained access control, KMS at rest, TLS in transit, node-to-node encryption, audit logs.

Amazon RDS/Aurora PostgreSQL with pgvector

  • Use when: Embeddings belong beside relational records; need ACID, joins, transactions, moderate scale.
  • Index choice:
Index Recall/Latency Memory Build time Key gotcha
HNSW Best High Slow Tune m, ef_construction, ef_search
IVFFlat Good Medium Fast Build after data load; tune lists, probes
Exact 100% Low None Too slow for large sets
  • Scaling: Vertical instance size, read replicas, Aurora Serverless v2, partitioning. Aurora scales better than RDS for high read QPS.
  • Security: KMS at rest, TLS in transit, IAM database auth, security groups, Secrets Manager.
  • Anti-pattern: Billions of vectors, very high QPS, or high dimensions without dimensionality reduction. Index support is dimension-bounded—validate engine/version; consider halfvec or reduction.

Amazon S3 Vectors / S3

  • Use when: Cost-optimized, massive vector sets, batch/offline retrieval, infrequent queries, data lake integration.
  • Model: S3 vector buckets and vector indexes provide native vector storage and similarity search in object storage.
  • Trade-offs: Excellent durability and scale; higher latency than in-memory/search clusters. Not for low-latency high-QPS.
  • Security: IAM, bucket policies, KMS/SSE-KMS, VPC endpoints, Block Public Access, access points.
  • Anti-pattern: Storing embeddings as objects and scanning with Athena for real-time RAG.

Similarity & Indexing Trade-offs

Method Recall Latency Memory Build Use case
Exact k-NN 100% High Low None Small sets, validation
HNSW High, tunable Low High Slow Low-latency RAG
IVF Tunable Low-medium Medium Fast Large sets, cost-sensitive

Normalize embeddings when using cosine. For normalized vectors, cosine and inner product are equivalent.

Scaling & Capacity Debugging

  • Consumer falling behind: Separate throughput limit from downstream slowdown. Adding consumers to a blocked write target only moves the queue.
  • High latency: Check memory for HNSW, CPU for distance compute, disk for IVFFlat, network for distributed shards.
  • Low recall: Increase ef_search/nprobes, reduce post-filtering, or use exact search for small sets.
  • High churn: HNSW/IVFFlat indexes fragment; plan rebuilds or use stores with incremental updates.
  • Read scaling: Replicas increase QPS; shards distribute data. Replicas do not reduce index build time.

Security & Compliance Checklist

  • IAM: Least privilege, resource policies, fine-grained access in OpenSearch, IAM database auth for RDS.
  • Encryption: KMS at rest; TLS in transit; OpenSearch node-to-node encryption; RDS SSL/TLS; S3 SSE-KMS.
  • Network: VPC, security groups, PrivateLink/VPC endpoints; no public access for sensitive data.
  • Audit: CloudTrail, OpenSearch audit logs, RDS logs, S3 access logs.
  • Secrets: Secrets Manager for database credentials.

Exam Gotchas

Vector store ≠ feature store. Vector store answers similarity; Feature Store answers identity + training/serving consistency.

OpenSearch Serverless is not always cheaper. OCU minimums can cost more for steady low traffic; provisioned may win.

S3 Vectors is cost-optimized, not latency-optimized. Do not choose it for high-QPS real-time RAG.

IVFFlat requires data present before index creation for meaningful lists. Build after load.

Filters can hurt recall. Post-filtering may return fewer than k results; use efficient filtering or pre-filtering.

Scale reads with replicas, not shards alone. Shards distribute data; replicas serve more queries.

Anti-Patterns & Distractors

  • Raw S3 for real-time vector search instead of S3 Vectors or OpenSearch.
  • pgvector for billions of vectors/high QPS without Aurora, replicas, or sharding.
  • OpenSearch Serverless for steady low traffic without cost analysis.
  • Distance metric mismatch or forgetting to normalize embeddings.
  • Building ANN index before data load for IVFFlat.
  • No metadata for filtering/traceability alongside embeddings.
  • Not versioning embeddings when the embedding model changes.
  • Ignoring recall/latency trade-offs when tuning HNSW/IVF parameters.