Skill 2.2.9: Optimize retrieval components and embedding models

Question 1 - RAG Retrieval Relevance in SageMaker

A financial services company has built a Retrieval-Augmented Generation (RAG) application using Amazon Bedrock Knowledge Bases to answer customer inquiries about investment products. The application retrieves passages from policy documents stored in Amazon S3 and generates responses using an LLM.

Users report that the generated answers often contain information that is only tangentially related to their questions. The team needs to improve retrieval precision with minimal operational overhead and without changing the foundation model.

Which TWO actions should the ML engineer take to directly improve retrieval relevance? (Select TWO)

A) Fine-tune the foundation model on the policy documents using supervised fine-tuning in Amazon Bedrock.
B) Re-chunk the source documents using semantic chunking boundaries (e.g., by section, heading, or paragraph structure) instead of fixed-size character splits.
C) Switch to a domain-specialized embedding model available in the Bedrock Knowledge Bases configuration.
D) Increase the max_tokens parameter on the foundation model inference call.
E) Deploy a SageMaker multi-model endpoint to serve multiple embedding models behind a shared container.

Click to reveal Answer & Explanation

Correct Answers: B and C

Correct Options Analysis

B is Correct: Chunking strategy directly determines what semantic units are indexed and retrieved. If fixed-size chunks split sentences across key context, embeddings fail to capture complete meaning. Semantic chunking preserves document structure and produces more coherent retrieval units.
C is Correct: The embedding model controls how text is mapped to vector space. Switching to a domain-aligned embedding model (e.g., FinBERT-based embeddings) directly improves query-document similarity accuracy without touching the generator.

Distractor Analysis

A is Incorrect: Fine-tuning the generator model does not affect which passages are retrieved from the vector store. It changes generation style and domain anchoring, not retrieval precision. Additionally, fine-tuning introduces significant operational overhead.
D is Incorrect: max_tokens only limits response length during generation. It has no effect on vector search, chunking, or embedding quality.
E is Incorrect: Multi-model endpoints optimize hosting cost for multiple models, not retrieval precision. This would also add unnecessary operational complexity when Bedrock Knowledge Bases already manages the embedding pipeline natively.

Key Takeaway: Fix retrieval problems at the retrieval layer—chunking and embedding model choice—before modifying the generator model.

Question 2 - Vector Search Latency Optimization

An e-commerce platform uses Amazon OpenSearch Serverless as a vector store with an Amazon Titan Embeddings model for semantic product search. The system serves 50,000 queries per hour during peak traffic.

The ML team has observed that vector search queries are taking 400–600ms, which is pushing total response latency beyond the 800ms product requirement. The team must reduce search latency while minimizing cost and avoiding any changes to the application code.

Which action achieves the latency reduction MOST cost-effectively?

A) Migrate from OpenSearch Serverless to a self-managed OpenSearch cluster on Amazon EC2 with 10 r6g.large instances.
B) Enable Approximate Nearest Neighbor (ANN) search with HNSW indexing and tune the m, ef_construction, and ef_search parameters to trade a small amount of recall for significant speedup.
C) Replace the Titan Embeddings model with a significantly larger embedding model to increase vector dimensionality.
D) Add a SageMaker inference endpoint in front of the OpenSearch Serverless collection to cache query results.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

B is Correct: OpenSearch supports ANN algorithms like HNSW (Hierarchical Navigable Small World) that dramatically speed up vector search compared to exact k-NN. Tuning ef_search, m, and ef_construction allows the team to trade a small amount of recall (e.g., 1–2%) for 10–20x faster query times—all within OpenSearch Serverless with no application changes.

Distractor Analysis

A is Incorrect: Self-managed clusters require significant operational effort (patching, scaling, backup management) and likely exceed the cost of the serverless option. It also doesn't fundamentally solve the latency issue if the search algorithm remains exact k-NN.
C is Incorrect: Larger embedding models with higher-dimensional vectors typically increase query latency because dot product computations scale with dimensionality. This worsens the problem.
D is Incorrect: A SageMaker inference endpoint and result cache adds an extra network hop and operational overhead while doing nothing to speed up the underlying vector search for novel queries.

Key Takeaway: For vector search latency issues, enable ANN algorithms (HNSW, IVF) and tune the recall-latency tradeoff rather than adding infrastructure.

Question 3 - Embedding Model Selection for Multilingual Use Case

A global customer support team is designing a RAG system using Amazon SageMaker and Amazon OpenSearch Serverless to retrieve content from knowledge base articles written in 12 languages. The system must answer queries in any of the supported languages and retrieve relevant articles regardless of the query language.

The team has budget constraints and wants lowest operational overhead while ensuring language-agnostic retrieval.

Which embedding approach should the ML engineer recommend?

A) Deploy 12 separate embedding models, one per language, using SageMaker multi-model endpoints, and route each query to the appropriate language model.
B) Use a multilingual embedding model (e.g., Amazon Titan Text Embeddings V2 or a sentence-transformer model like paraphrase-multilingual-MiniLM) that maps text from different languages into a shared vector space, and embed both documents and queries with this single model.
C) Translate all documents and queries into English using Amazon Translate before embedding with a monolingual English embedding model.
D) Use a single-language embedding model and rely on the vector database to handle cross-language matching through cosine similarity.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

B is Correct: Multilingual embedding models are specifically trained to map text from many languages into a unified vector space where semantically similar content from different languages clusters together. Using one such model for both documents and queries provides language-agnostic retrieval with minimal operational overhead and no translation pipeline.

Distractor Analysis

A is Incorrect: Managing 12 separate embedding models (with 12 containers, 12 deployments, language detection and routing logic) introduces enormous operational complexity and cost, violating the "lowest operational overhead" constraint.
C is Incorrect: Adding an Amazon Translate step before every embed operation doubles the pipeline latency, can introduce translation errors that degrade retrieval quality, and adds an extra billed service to the architecture.
D is Incorrect: A monolingual embedding model encodes words in different languages into unrelated vector spaces. Cosine similarity cannot bridge language barriers without multilingual training.

Key Takeaway: For multilingual RAG retrieval, select a multilingual embedding model that aligns cross-language text into a shared vector space.

Question 4 - Hybrid Search Retrieval Quality

A legal research platform uses Amazon OpenSearch Service to build a document search system. The current implementation uses BM25 lexical search (keyword matching) and returns poor results when users describe legal concepts using synonyms or semantically similar terms. The team wants to improve search relevance by capturing semantic similarity without losing the ability to match exact legal terminology and statute numbers.

They need a solution that integrates with their existing SageMaker-based ML pipeline and provides high retrieval quality in production.

Which architecture should the ML engineer implement?

A) Completely replace BM25 with dense vector search using embeddings generated by Amazon Titan.
B) Implement hybrid search by combining BM25 scores with vector similarity scores (using reciprocal rank fusion or weighted sum) across both lexical and semantic indexes in OpenSearch.
C) Create two separate search endpoints—one for BM25 and one for vector search—and have users manually choose which one to use.
D) Remove all search functionality from OpenSearch and send every document to the foundation model in the prompt for direct answer generation.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

B is Correct: Hybrid search combines the strengths of both approaches: BM25 excels at matching exact terms (statute numbers, legal citations, proper names) while vector search captures semantic similarity (synonyms, conceptual matches). OpenSearch natively supports hybrid queries with score normalization and fusion techniques for production use.

Distractor Analysis

A is Incorrect: Pure vector search struggles with rare words, exact codes, and precise identifiers—critical for legal research. Replacing BM25 entirely would degrade exact-match quality for statute citations.
C is Incorrect: Manual selection creates poor user experience and eliminates the ability to rank results across both search types in a single unified view. Users shouldn't need to know which search method suits their query.
D is Incorrect: Sending all documents to the LLM in the prompt is computationally infeasible for a large legal corpus, introduces high token costs, and violates typical confidence and provenance requirements for legal content.

Key Takeaway: Hybrid search (BM25 + dense vector) is the recommended pattern when both exact lexical matching and semantic relevance are required in production search systems.

Question 5 - Chunking Strategy Selection for RAG

An engineering team is building a RAG application to answer questions about technical API documentation. The documents contain code examples, tables, parameter definitions, and multi-paragraph conceptual sections. The current implementation uses fixed-size chunks of 300 characters with no overlap.

Users report that answers frequently miss important context because code examples get separated from their explanations, and multi-step instructions get split across chunks. The team must improve retrieval quality while keeping ingestion time and storage costs reasonable.

Which chunking strategy should the ML engineer implement FIRST?

A) Reduce chunk size to 100 characters to create more precise semantic units.
B) Implement structure-aware chunking that uses document hierarchy (headings, subheadings, code blocks, tables) to define chunk boundaries, with different strategies for code vs. prose sections.
C) Increase chunk size to 5,000 characters to ensure all related content stays in a single chunk.
D) Use a random chunking approach with 50% overlap to create diverse retrieval units.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

B is Correct: Structure-aware chunking respects the natural boundaries of technical documentation. It keeps code blocks intact with their surrounding context, preserves table structures, and aligns chunks with heading hierarchy. This produces meaningful retrieval units without unnecessarily inflating chunk size or storage.

Distractor Analysis

A is Incorrect: Smaller chunks increase the likelihood of splitting context across boundaries and increase the number of indexed chunks (higher storage and compute costs). Tiny chunks also lose the surrounding context needed for effective embedding.
C is Incorrect: Extremely large chunks dilute the semantic signal in the embedding vector with noise from irrelevant content, reducing retrieval precision. Large chunks also increase the tokens read into the generator model, raising per-query cost.
D is Incorrect: Random chunking ignores document semantics and produces unpredictable retrieval quality. High overlap (50%) also duplicates content across chunks, inflating index size without proportional retrieval benefit.

Key Takeaway: Structure-aware chunking that respects document hierarchy is the best first step for technical documents with code, tables, and multi-section prose.

Question 6 - Embedding Model Evaluation and Selection

A healthcare company is selecting an embedding model for a retrieval system that searches medical research papers. They have collected a curated evaluation set of 500 query-document pairs with relevance labels. They want to objectively compare candidate embedding models before deployment.

The team needs a methodology that uses their evaluation set correctly and aligns with AWS best practices for ML model evaluation.

Which approach should the ML engineer recommend?

A) Deploy each candidate model to production and A/B test with real users for 30 days before making a decision.
B) Compute retrieval metrics (e.g., recall@k, MRR, nDCG) by running each candidate model against the evaluation set and comparing the ranked retrieval results against the relevance labels.
C) Choose the model with the highest embedding dimensionality, as it will capture more information.
D) Choose the model with the largest parameter count, as larger models are always more accurate.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

B is Correct: Retrieval metrics like recall@k (what fraction of relevant documents appear in the top-k results), MRR (Mean Reciprocal Rank, measuring position of the first relevant result), and nDCG (Normalized Discounted Cumulative Gain, measuring overall ranking quality) are the standard evaluation metrics for retrieval systems. They directly answer "does this model retrieve the right documents?"

Distractor Analysis

A is Incorrect: Production A/B testing without prior offline evaluation risks poor user experience and requires lengthy experimentation to produce statistically meaningful results. Offline evaluation on a curated set is the standard first step.
C is Incorrect: Higher dimensionality increases information capacity but also increases memory/storage requirements and may capture noise. Embedding quality depends on training data and architecture, not just vector size.
D is Incorrect: Parameter count alone does not guarantee better retrieval quality. Many domain-specialized models (e.g., MiniLM) outperform larger general-purpose models on specific tasks.

Key Takeaway: Evaluate embedding models with offline retrieval metrics (recall@k, MRR, nDCG) on a labeled evaluation set before deploying to production.

Question 7 - Cost-Efficient Indexing with Large Document Volume

A legal tech company ingests 10 million case-law documents monthly into a semantic search system. Each document averages 2 pages. The current pipeline uses Amazon Textract for OCR, then embeds each page individually before indexing into OpenSearch Serverless.

The team faces high ingestion costs from embedding API calls on every page. They want to reduce embedding and storage costs while maintaining retrieval quality for case-law research.

Which optimization approach should the ML engineer implement?

A) Skip embedding and use only BM25 lexical search for all case-law documents.
B) Embed shorter, semantically meaningful chunks (e.g., paragraphs or sections) after consolidating OCR output, rather than embedding every full page indiscriminately.
C) Use a smaller embedding dimension (e.g., 128) for all documents by applying PCA to reduce Titan's 1536-dimensional vectors.
D) Reduce OpenSearch Serverless OCU capacity to minimum and rely on event-driven scaling to manage peak loads.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

B is Correct: Embedding meaningful chunks (paragraphs, legal sections, or arguments) rather than full pages eliminates redundant noise and reduces the number of index entries. This directly lowers embedding API call costs, reduces storage requirements, and typically improves retrieval precision because fewer irrelevant tokens dilute each vector's semantic representation.

Distractor Analysis

A is Incorrect: BM25-only search misses semantic relationships in legal text (e.g., similar legal arguments phrased differently). For case-law research, semantic retrieval provides significant value in finding related precedents.
C is Incorrect: Applying PCA as a separate post-processing step to reduce vector dimensions loses information and adds complexity. It is not a standard AWS pattern for OpenSearch indexing. Dimension reduction should be done by choosing an embedding model that natively outputs lower dimensions.
D is Incorrect: Reducing OpenSearch Serverless capacity below required levels will degrade query performance. OCU scaling affects search capacity and throughput, not the primary driver of embedding cost (API call count and vector storage).

Key Takeaway: Right-size chunk granularity to balance embedding cost, storage cost, and retrieval quality. Embedding every page indiscriminately often inflates cost without improving quality.

Question 8 - Query Rewriting for Retrieval

A RAG application serves a knowledge management platform for a software company. Users frequently ask follow-up questions with pronouns and implicit context (e.g., seeing "How do I configure it?" after previous queries about "S3 bucket lifecycle policies").

The current system embeds each user query as-is for vector search, causing poor retrieval for follow-up questions. The ML team wants to improve retrieval without fine-tuning the embedding model or changing the vector store architecture.

Which technique should the engineer implement to address this directly?

A) Add a query reformulation step that uses conversational history and an LLM (e.g., Bedrock Claude) to rewrite the user's question into a self-contained query before computing embeddings.
B) Deploy a larger embedding model with higher dimensionality.
C) Add more documents to the vector store to improve coverage.
D) Increase the top_k parameter to retrieve more documents for every query.

Click to reveal Answer & Explanation

Correct Answer: A

Correct Options Analysis

A is Correct: Query reformulation (also called query rewriting or query expansion) is the recommended technique for handling context-dependent queries in RAG systems. Using a lightweight LLM call to expand "How do I configure it?" into "How do I configure Amazon S3 lifecycle policies?" transforms the query into a self-contained embedding that matches the document index effectively.

Distractor Analysis

B is Incorrect: A larger embedding model improves representation power but does not resolve anaphora or missing context in the query text itself. "How do I configure it?" will still lack the referent "it" regardless of model size.
C is Incorrect: Adding more documents increases coverage but doesn't address the core issue of ambiguous or context-dependent query text.
D is Incorrect: Increasing top_k returns more documents regardless of relevance, adding noise and cost without solving the query context problem. It may actually decrease precision by including irrelevant results.

Key Takeaway: Query reformulation with conversational context is the standard technique for handling follow-up questions in RAG systems without modifying the embedding model or vector store.

Question 9 - Cross-Encoder Reranking for Precision

A retail company's semantic search system retrieves 100 candidate documents using an embedding model and OpenSearch vector search. Users report that some irrelevant products appear in their results, particularly when queries are short and ambiguous (e.g., "warm jacket"). The current system returns the top-20 retrieved products directly.

The team wants to significantly improve ranking precision without modifying the embedding index or changing the retrieval pipeline excessively. Budget is available for additional compute within reasonable limits.

Which approach should the ML engineer recommend?

A) Increase the embedding model's vector dimensions to capture more detail.
B) Rebuild the entire index using a large language model to generate embeddings.
C) Add a cross-encoder reranking model (e.g., BERT-based) after the initial retrieval step that scores the query against each of the top-100 candidates and returns the top-20 ranked by cross-encoder scores.
D) Manually review and filter the product catalog entries before indexing.

Click to reveal Answer & Explanation

Correct Answer: C

Correct Options Analysis

C is Correct: Cross-encoder reranking is the recommended two-stage retrieval pattern. The bi-encoder (embedding model) efficiently retrieves a broad set of candidates, then a cross-encoder scores each candidate against the query at a finer granularity, dramatically improving ranking precision. This is implemented as an additional step without needing to modify the dense index.

Distractor Analysis

A is Incorrect: Increasing vector dimensions improves representation granularity but does not fundamentally address ranking issues caused by short and ambiguous queries. It also requires re-indexing all documents and increases storage cost.
B is Incorrect: LLM-generated embeddings may or may not improve quality and require complete re-indexing, which is costly and unnecessary when the issue is ranking precision, not retrieval coverage.
D is Incorrect: Manual review is infeasible at scale and does not scale to new product additions or changing user queries. This is not a sustainable ML solution.

Key Takeaway: For improved ranking precision, use a two-stage retrieval pipeline: fast bi-encoder retrieval followed by cross-encoder reranking.

Question 10 - Embedding Model Compression for Edge Inference

An IoT security company deploys a face recognition system on edge devices (small embedded systems with limited memory and compute). The system currently uses a 250MB embedding model from Amazon SageMaker, which exceeds the device's memory constraints.

The team needs to deploy an embedding model that fits within the device's 50MB memory budget with minimal impact on embedding quality and no changes to the device hardware.

Which approach should the ML engineer recommend?

A) Use SageMaker Neo to compile and optimize the embedding model for the specific edge hardware, then deploy the optimized model.
B) Use the original 250MB model but move all inference to a SageMaker endpoint, sending each embedding request over network calls (with offline fallback when the network is unavailable).
C) Use Amazon Bedrock to perform embedding generation remotely and install an edge device manager.
D) Switch to a much smaller model (e.g., 5MB parameter-count model) without any optimization or evaluation on the edge device.

Click to reveal Answer & Explanation

Correct Answer: A

Correct Options Analysis

A is Correct: SageMaker Neo compiles models for target hardware (ARM, NVIDIA, Intel, etc.) and applies hardware-specific optimizations (quantization, pruning, kernel fusion) to reduce model size and improve inference speed. This approach preserves as much of the original model's quality as possible while meeting the memory constraint. Quantization (e.g., from FP32 to INT8) can reduce model size by 75% without significant accuracy loss.

Distractor Analysis

B is Incorrect: Moving inference to a SageMaker endpoint violates edge deployment constraints (low-latency, offline-capable) and requiring network connectivity for every embedding call is operationally fragile for an IoT security system.
C is Incorrect: Bedrock is primarily a cloud service for foundation models and not designed for edge embedding generation. This also introduces network dependency, defeating the purpose of edge inference.
D is Incorrect: Arbitrarily switching to a very small model without evaluating embedding quality could cause significant quality degradation in the face recognition system, compromising security and accuracy.

Key Takeaway: SageMaker Neo with quantization is the primary AWS mechanism for reducing model size for edge deployment while maintaining acceptable accuracy.

Exam Summary Table

Concept	Key Takeaway
Chunking strategy	Match chunk boundaries to document structure for coherent semantic units
Embedding model choice	Select domain/multilingual alignment, not just model size
Search architecture	Hybrid search (BM25 + vector) when both exact and semantic match needed
Query handling	Rewrite context-dependent queries before embedding
Ranking precision	Two-stage retrieval: bi-encoder → cross-encoder reranking
Evaluation methodology	Use recall@k, MRR, nDCG on labeled evaluation sets
Edge deployment	SageMaker Neo + quantization for hardware-specific optimization
Latency optimization	ANN algorithms (HNSW/IVF) with tuned recall-latency tradeoffs