Skill 2.2.9: Optimize retrieval components and embedding models

Question	Answer & Explanation
### Question 1 - RAG Chunking and Context Preservation A legal tech company built a RAG application using Amazon Bedrock Knowledge Bases and Amazon S3. The LLM answers are often incomplete because key legal clauses are split across two separate chunks during ingestion, losing the context of the full paragraph. The team must optimize the retrieval component to preserve cross-chunk context with the least operational overhead. Which approach satisfies this requirement? - A) Write a custom AWS Lambda function to pre-merge S3 documents before ingestion. - B) Increase the chunk overlap parameter in the Bedrock Knowledge Base data source configuration. - C) Fine-tune the generating LLM on a dataset of legal documents. - D) Increase the LLM's `max_tokens_to_sample` parameter.	Correct Answer: B Correct Answer Analysis: - B is Correct: Adjusting the chunk overlap in the Bedrock Knowledge Base ensures that sentences or clauses spanning chunk boundaries are preserved in adjacent chunks, maintaining context without custom code. Distractor Analysis: - A is Incorrect: Writing a custom Lambda pipeline introduces unnecessary operational overhead compared to native configuration options. - C is Incorrect: Fine-tuning the LLM cannot fix missing context if the relevant data was never retrieved due to poor chunking. - D is Incorrect: Increasing generation length does not fix the underlying retrieval issue of missing context. > Key Takeaway: Always fix chunking boundaries (e.g., using overlap) before modifying the LLM to solve context loss in RAG pipelines.
### Question 2 - Selecting a Multilingual Embedding Model An international news agency stores articles in English and Spanish in an Amazon S3 bucket. They use Amazon Bedrock Knowledge Bases with an English-only embedding model. Spanish queries consistently fail to retrieve relevant Spanish articles. The team must optimize the embedding model for multilingual retrieval using AWS native services and the lowest operational overhead. Which action fulfills these requirements? - A) Switch the embedding model in the Bedrock Knowledge Base to Cohere Embed Multilingual. - B) Use Amazon Translate to convert all Spanish documents to English before ingestion. - C) Fine-tune the generating LLM to handle cross-lingual retrieval. - D) Increase the chunk size to capture more surrounding context for the English embedding model.	Correct Answer: A Correct Answer Analysis: - A is Correct: Cohere Embed Multilingual is supported natively in Amazon Bedrock Knowledge Bases and creates a shared vector space for multiple languages, allowing Spanish queries to match Spanish documents directly. Distractor Analysis: - B is Incorrect: Translating all documents adds high operational overhead, pipeline complexity, and may lose linguistic nuance. - C is Incorrect: LLM fine-tuning does not fix the vector mismatch occurring at the embedding/retrieval stage. - D is Incorrect: Larger chunks do not bridge the language gap in an English-only embedding model's vector space. > Key Takeaway: Use natively supported multilingual embedding models for cross-lingual RAG instead of building custom translation pipelines.
### Question 3 - Hybrid Search for Keyword and Semantic Matching An e-commerce platform uses Amazon Bedrock Knowledge Bases for product search. Semantic search retrieves conceptually relevant products, but users searching for specific alphanumeric product SKUs (e.g., "XJ-9000") get irrelevant results because the embedding model fails to capture exact string matches. The team must optimize the retriever with the least operational overhead. Which solution is the MOST appropriate? - A) Fine-tune the embedding model on alphanumeric strings. - B) Implement a custom OpenSearch query pipeline outside of Bedrock. - C) Enable hybrid search in the Bedrock Knowledge Base configuration. - D) Increase the number of returned search results (`top_k`).	Correct Answer: C Correct Answer Analysis: - C is Correct: Amazon Bedrock Knowledge Bases supports hybrid search, which combines vector embeddings with lexical text search (like BM25), effectively solving exact keyword/SKU matching issues natively. Distractor Analysis: - A is Incorrect: Fine-tuning embeddings to memorize exact alphanumeric strings is highly complex, computationally expensive, and often ineffective. - B is Incorrect: Bypassing Bedrock Knowledge Bases to write custom OpenSearch queries increases operational overhead and maintenance. - D is Incorrect: Retrieving more irrelevant results does not solve the lack of exact match precision. > Key Takeaway: Enable hybrid search in Bedrock Knowledge Bases to combine semantic understanding with exact keyword matching for optimal retrieval.
### Question 4 - Targeting the Correct Component for Optimization A financial services firm built a RAG application. When querying the system, the retrieved passages from the vector store contain the correct financial data, but the generated response is poorly structured and misses key points from the provided context. The team must target the correct component for optimization based on this specific failure mode. Which action should they take? - A) Change the chunking strategy to smaller fixed-size chunks. - B) Update the embedding model to a larger dimension size. - C) Apply prompt engineering to the generating LLM, explicitly instructing it to format the output and use the provided context. - D) Implement a custom reranker model on the retrieved passages.	Correct Answer: C Correct Answer Analysis: - C is Correct: If the retrieved context contains the answer, the retrieval component is working correctly, and the issue lies in the generator (LLM). Prompt engineering is the first, lowest-overhead step to fix generation formatting. Distractor Analysis: - A is Incorrect: Chunking affects the retrieval stage, which is already succeeding since the correct data is present in the context. - B is Incorrect: Increasing embedding dimensions improves vector granularity but does not fix the LLM's formatting issues. - D is Incorrect: A reranker improves the precision of retrieved documents, but the correct documents are already being retrieved. > Key Takeaway: If retrieved passages contain the answer but the response is poor, optimize the generator (prompt/customization), not the retriever.
### Question 5 - Implementing a Reranking Model An enterprise RAG application retrieves 20 chunks from an Amazon OpenSearch Serverless vector index. However, the most relevant chunk is often ranked 15^th out of 20, causing the LLM to miss it due to context window truncation. The team must optimize the retrieval ranking component without replacing the embedding model. Which solution meets these requirements? - A) Increase the chunk overlap. - B) Enable a reranking model (e.g., Cohere Rerank) in the Bedrock Knowledge Base retrieval pipeline. - C) Fine-tune the generating LLM on domain-specific data. - D) Use a smaller embedding model to generate denser vectors.	Correct Answer: B Correct Answer Analysis: - B is Correct: A reranker (like Cohere Rerank available in Bedrock) evaluates the retrieved passages against the query and re-orders them based on true semantic relevance, pushing the best chunks to the top so they aren't truncated. Distractor Analysis: - A is Incorrect: Chunk overlap fixes boundary context loss, not the ranking order of retrieved chunks. - C is Incorrect: Fine-tuning the LLM does not change the vector index ranking or which chunks are truncated. - D is Incorrect: Smaller embedding models generally reduce semantic precision rather than improving ranking order. > Key Takeaway: Use a reranker in the retrieval pipeline to re-order retrieved chunks by relevance, ensuring the best context fits within the LLM's context window.
### Question 6 - Metadata Filtering for Vector Search A healthcare company stores millions of patient forms in Amazon S3. They are building a RAG application and need to ensure users querying "cardiology" only retrieve cardiology forms, not general hospital forms that merely mention the word. The team must optimize the retrieval component to filter results natively and cost-effectively. Which approach should they use? - A) Use metadata filtering in the Amazon Bedrock Knowledge Base ingestion and query pipeline. - B) Train a custom text classification model to filter chunks before passing them to the LLM. - C) Increase the `top_k` parameter to 100 and rely on the LLM to find the cardiology forms. - D) Fine-tune the embedding model to distinguish between hospital departments.	Correct Answer: A Correct Answer Analysis: - A is Correct: Bedrock Knowledge Bases supports metadata filtering, allowing you to attach metadata (e.g., `department=cardiology`) to chunks during ingestion and filter vector searches by this metadata during retrieval. Distractor Analysis: - B is Incorrect: Adding a custom ML classification model introduces high operational overhead and latency. - C is Incorrect: Passing 100 chunks to the LLM increases token costs and latency without guaranteeing the correct forms are included. - D is Incorrect: Fine-tuning an embedding model for strict categorical classification is inefficient and less reliable than exact metadata filtering. > Key Takeaway: Use metadata filtering in vector stores to restrict searches to specific categories natively, reducing cost and improving retrieval accuracy.
### Question 7 - Optimizing Embedding Dimensions for Latency A tech company is migrating their RAG application's vector store to Amazon OpenSearch Serverless. They currently use an embedding model with 4096 dimensions. The ML team notices high memory consumption and slow query latency during k-NN search. The team must optimize the retrieval component for the lowest latency and cost without significantly sacrificing accuracy. Which action fulfills this requirement? - A) Switch to an embedding model with fewer dimensions (e.g., 768 or 1024). - B) Increase the number of OpenSearch Serverless replicas. - C) Switch to an exact k-NN search algorithm. - D) Fine-tune the LLM to tolerate fewer retrieved chunks.	Correct Answer: A Correct Answer Analysis: - A is Correct: Vector dimension size directly impacts memory usage and search latency. Reducing dimensions from 4096 to 1024 drastically reduces the index size and speeds up similarity searches in OpenSearch. Distractor Analysis: - B is Incorrect: Increasing replicas improves read throughput but does not reduce the per-query latency caused by processing high-dimensional vectors. - C is Incorrect: Exact k-NN is computationally heavier and slower for high-dimensional data compared to approximate k-NN (like HNSW). - D is Incorrect: Fine-tuning the LLM does not fix the infrastructure latency of the vector store. > Key Takeaway: Lower embedding dimensions reduce memory and search latency in vector databases; always choose the smallest embedding model that maintains accuracy.
### Question 8 - Semantic and Structural Chunking An engineering team is using Amazon Bedrock Knowledge Bases to ingest HTML product manuals. They notice that standard fixed-size chunking splits HTML tables, rendering them useless when retrieved by the LLM. The team must optimize the chunking strategy natively within Bedrock Knowledge Bases with the least operational overhead. Which solution is the MOST appropriate? - A) Write a custom AWS Glue job to convert HTML tables to plain text before ingestion. - B) Configure the Bedrock Knowledge Base parsing to use a semantic or structural chunking strategy that respects document boundaries. - C) Increase the chunk size to 10,000 tokens so tables are never split. - D) Disable chunking and pass the entire HTML file directly to the LLM.	Correct Answer: B Correct Answer Analysis: - B is Correct: Bedrock Knowledge Bases supports advanced parsing and chunking strategies (like semantic chunking or using AWS Lambda parsers) that respect structural boundaries like HTML tables, keeping them intact. Distractor Analysis: - A is Incorrect: Building a custom AWS Glue ETL job adds unnecessary operational overhead compared to native parsing features. - C is Incorrect: Massive chunks dilute semantic meaning, increase embedding costs, and may exceed the embedding model's token limit. - D is Incorrect: Passing entire files bypasses vector search, exceeds LLM context limits, and drastically increases inference costs. > Key Takeaway: Use structural or semantic chunking in Bedrock Knowledge Bases to preserve the integrity of tables and lists during ingestion.
### Question 9 - Evaluating Retrieval Quality Natively An ML engineer is evaluating two different embedding models for a Retrieval-Augmented Generation (RAG) pipeline. They want to quantitatively compare which model retrieves the most relevant chunks based on a predefined set of test queries. The team must measure retrieval quality natively in Amazon Bedrock Knowledge Bases. Which approach should they use? - A) Monitor Amazon CloudWatch metrics for API latency. - B) Use the Bedrock Knowledge Bases evaluation feature to calculate retrieval metrics like Recall@K and NDCG. - C) Fine-tune the LLM to output a self-evaluation confidence score. - D) Count the number of tokens in the generated response.	Correct Answer: B Correct Answer Analysis: - B is Correct: Bedrock Knowledge Bases provides an evaluation feature that uses an LLM as a judge to score retrieval quality using standard metrics like Recall@K and Normalized Discounted Cumulative Gain (NDCG). Distractor Analysis: - A is Incorrect: API latency measures infrastructure performance, not the semantic relevance of retrieved chunks. - C is Incorrect: LLM self-evaluation confidence scores are subjective and do not directly measure vector retrieval precision. - D is Incorrect: Token count measures the response generation length, not the accuracy of the retrieval pipeline. > Key Takeaway: Use Bedrock Knowledge Bases evaluation to compute retrieval metrics (e.g., NDCG, Recall@K) to objectively compare embedding models and chunking strategies.
### Question 10 - Fine-tuning Domain-Specific Embedding Models A global retail company uses a RAG application with an OpenSearch vector store. For specific retail jargon (e.g., "shrinkflation", "omnichannel"), their generic embedding model returns irrelevant chunks because it lacks domain-specific semantic understanding. The team must optimize the embedding model for domain-specific vocabulary while minimizing infrastructure management. Which solution meets these requirements? - A) Increase the chunk size to capture more surrounding context. - B) Train a generative LLM from scratch on retail data. - C) Fine-tune a pre-trained embedding model using Amazon SageMaker JumpStart on the company's retail corpus. - D) Switch to an exact keyword match search engine.	Correct Answer: C Correct Answer Analysis: - C is Correct: Fine-tuning an embedding model on domain-specific text (retail jargon) adjusts the vector space so similar concepts are closer together, improving retrieval. SageMaker JumpStart provides managed fine-tuning for embedding models, minimizing infrastructure overhead. Distractor Analysis: - A is Incorrect: Larger chunks do not solve the lack of domain vocabulary understanding in the vector space. - B is Incorrect: Training an LLM from scratch is massively expensive, complex, and unnecessary for fixing retrieval embeddings. - D is Incorrect: Switching entirely to keyword search loses the semantic matching capabilities of vector search that RAG relies on. > Key Takeaway: Fine-tune embedding models on domain-specific corpora using SageMaker JumpStart to improve vector representations for specialized vocabulary.