Skill 2.1.4: Evaluate tradeoffs between custom solutions, managed services, pre-trained models, and FMs to meet business needs

Question	Answer & Explanation
### Question 1 - Demand Forecasting & Interpretability A large retail chain needs to generate weekly store-level demand forecasts based on years of historical transaction data. The patterns involve complex seasonality and local trends. The finance team requires the ability to understand and audit how each prediction is made, and inference costs must remain low. Which approach meets these requirements with the LEAST operational overhead while satisfying the interpretability constraint? - A) Select a large foundation model in Amazon Bedrock and supply the historical sales data as context in the prompt. - B) Train a supervised machine learning model using Amazon SageMaker AI, favoring an interpretable algorithm like XGBoost. - C) Write fixed business rules in application code to trigger reorders when stock drops below a static threshold. - D) Call a managed AWS AI service like Amazon Comprehend to extract insights from the transaction text.	Correct Answer: B Correct Answer Analysis: - B is Correct: When complex patterns exist solely within an organization's own historical data, training a traditional ML model is the correct approach. Using an interpretable algorithm like XGBoost via SageMaker AI satisfies the finance team's requirement to audit predictions, while keeping inference costs low. Distractor Analysis: - A is Incorrect: Foundation models (LLMs) are not designed for precise, large-scale tabular time-series forecasting. Passing years of tabular data into a prompt is inefficient, costly, and acts as a "black box" that fails the interpretability constraint. - C is Incorrect: Fixed rules cannot model dynamic demand patterns, seasonality, or regional trends. - D is Incorrect: Amazon Comprehend is a natural language processing (NLP) service for text analysis, not a numerical time-series forecasting tool. Exam Tip / AWS Best Practice: When the answer lives in proprietary tabular data and interpretability is required, train a traditional ML model rather than using a foundation model.
### Question 2 - Document Extraction & Managed Services An accounting department receives thousands of scanned, unstructured invoices from various vendors every day. The team needs to automatically extract raw text, key-value pairs, and table data from these documents to populate their internal database. The company has no in-house computer vision expertise and wants the fastest time-to-market. Which solution meets these requirements with the LEAST development effort? - A) Train a custom computer vision model on labeled invoice images using Amazon SageMaker AI. - B) Fine-tune a foundation model in Amazon Bedrock on a corpus of past invoices. - C) Use Amazon Textract to automatically extract text and structured data from the scanned invoices. - D) Write fixed rules to read text at specific coordinate positions on each invoice layout.	Correct Answer: C Correct Answer Analysis: - C is Correct: Amazon Textract is a managed AWS AI service pre-trained on millions of documents to extract text, forms, and tables. It requires zero custom model training, zero labeling, and no infrastructure management, perfectly meeting the "least development effort" constraint. Distractor Analysis: - A is Incorrect: Training a custom computer vision model requires building a bounding-box annotation workflow, selecting architectures, and managing hosting infrastructure, resulting in high overhead. - B is Incorrect: Fine-tuning a foundation model on invoices is resource-intensive, overly complex for standard key-value extraction, and foundation models are not optimized for dense OCR tasks. - D is Incorrect: Position-based rules are extremely brittle and will fail whenever vendor invoice layouts, margins, or scan scales change. Exam Tip / AWS Best Practice: For already-solved, standard tasks (like OCR, image recognition, or sentiment analysis), always use a managed AWS AI service (Textract, Rekognition, Comprehend) before building or training a custom model.
### Question 3 - Prompt Engineering vs. Fine-Tuning An enterprise is deploying a foundation model via Amazon Bedrock to summarize customer support tickets. The engineering team has noticed the model's summaries do not consistently use the company's internal product terminology and specific formatting guidelines. The team wants to enforce this behavior with the fastest turnaround time and lowest overhead. Which approach should the engineering team take FIRST? - A) Initiate a supervised fine-tuning job using labeled examples of ideal summaries. - B) Increase the `max_tokens` parameter to allow the model more room to generate text. - C) Apply task-specific prompt engineering with clear instructions and few-shot examples. - D) Switch to a larger, more capable foundation model in Amazon Bedrock.	Correct Answer: C Correct Answer Analysis: - C is Correct: Task-specific prompt engineering (using instructions and few-shot examples) is the fastest and lowest-overhead method to adjust a foundation model's tone, terminology, and formatting. It applies immediately without requiring data preparation or training jobs. Distractor Analysis: - A is Incorrect: Fine-tuning incurs significant overhead (dataset prep, GPU compute, versioning) and should only be considered when evaluations prove that prompt engineering cannot close the gap. - B is Incorrect: Increasing `max_tokens` only allows for longer outputs; it does not improve the semantic precision or force the model to adopt specific terminology. - D is Incorrect: Simply switching to a larger model increases cost and latency, and does not guarantee adherence to specific internal terminology without explicit prompt guidance. Exam Tip / AWS Best Practice: Always follow the "Prompt First, Fine-Tune Second" rule. Fine-tune only when a gap persists that no prompt can close.
### Question 4 - RAG vs. Fine-Tuning for Dynamic Knowledge A legal technology company is building an automated Q&A assistant for its employees. The assistant must answer questions based on the company's HR policies, which are edited frequently. The compliance team requires that the assistant must cite the exact document and section used for each answer, and policy edits must be reflected in the bot within minutes. Which modeling approach best satisfies these requirements? - A) Supervised fine-tuning of a foundation model on the company's HR policy documents. - B) Retrieval-Augmented Generation (RAG) using Amazon Bedrock Knowledge Bases. - C) Training a custom NLP model from scratch using Amazon SageMaker AI. - D) Using Amazon Comprehend to detect key phrases in the HR policies.	Correct Answer: B Correct Answer Analysis: - B is Correct: RAG fetches relevant passages from a vector database at query time. This allows the model to explicitly cite source documents and enables near real-time updates (within minutes) simply by re-ingesting the edited policy, without modifying the model's weights. Distractor Analysis: - A is Incorrect: Customizing model weights via fine-tuning takes hours, cannot easily cite exact source documents, and requires continuous costly re-training whenever policies are edited. - C is Incorrect: Training a custom model from scratch for Q&A is extremely complex, expensive, and fails the citation and dynamic update constraints. - D is Incorrect: Amazon Comprehend is for extracting entities and sentiment from text, not for generating contextual answers or functioning as a conversational Q&A assistant. Exam Tip / AWS Best Practice: When a use case requires source citations and near real-time knowledge updates, use RAG (Amazon Bedrock Knowledge Bases) instead of fine-tuning.
### Question 5 - Fixed Rules vs. Machine Learning A global e-commerce platform processes millions of transactions daily. The security team wants to automatically block any transaction over $10,000 that originates from IP addresses located in two specific high-risk countries. The logic for this block will not change frequently. What is the MOST cost-effective and operationally efficient approach to implement this? - A) Train a supervised anomaly detection model using Amazon SageMaker AI. - B) Use Amazon Bedrock to evaluate each transaction with a foundation model prompt. - C) Use Amazon Comprehend to analyze the transaction metadata for malicious intent. - D) Implement fixed business logic rules in the application code to block the transactions.	Correct Answer: D Correct Answer Analysis: - D is Correct: When the business logic is fixed and easily expressed with conditionals (e.g., `if amount > 10000 and country in [X, Y]`), a plain rule is the cheapest, fastest, and most easily auditable solution. No machine learning model is required. Distractor Analysis: - A is Incorrect: Training an ML model for static, easily defined rules introduces unnecessary operational complexity, cost, and potential false positives. - B is Incorrect: Using a foundation model to evaluate deterministic rules adds massive latency and cost for zero added benefit. - C is Incorrect: Amazon Comprehend is an NLP service for text analysis, not for evaluating numerical transaction thresholds and geographic IP rules. Exam Tip / AWS Best Practice: If the logic is fixed, skip the model entirely and write a rule. Don't use ML when a simple `if/else` statement solves the problem.
### Question 6 - SageMaker JumpStart vs. Built-in Algorithms A medical imaging startup needs to build a custom solution to classify X-ray images for specific anomalies. They have a proprietary, labeled dataset of 5,000 images, but they want to avoid the massive computational cost and time of training a complex computer vision model from scratch. Which approach provides the BEST starting point to meet their needs while minimizing training time? - A) Use a built-in SageMaker AI algorithm (like Image Classification) and initialize it with random weights. - B) Select a pre-trained computer vision model from SageMaker JumpStart and fine-tune it on the 5,000 images. - C) Use Amazon Rekognition to detect custom anomalies in the X-rays. - D) Pass the images and a description of the anomaly to a multimodal foundation model in Amazon Bedrock.	Correct Answer: B Correct Answer Analysis: - B is Correct: SageMaker JumpStart provides access to pre-trained models. Fine-tuning a pre-trained model on a small, proprietary dataset (transfer learning) drastically reduces training time and compute costs compared to training from scratch, while still allowing the model to learn domain-specific patterns. Distractor Analysis: - A is Incorrect: Training a built-in algorithm from scratch with random weights on only 5,000 images will take much longer, cost more compute, and likely underperform compared to transfer learning. - C is Incorrect: Amazon Rekognition is a managed service for standard object/scene detection, but it does not natively support training on custom medical anomalies without custom labels, and it lacks the specialized domain performance for X-rays. - D is Incorrect: Multimodal FMs in Bedrock are generally not specialized for high-accuracy medical anomaly classification on X-rays compared to fine-tuned computer vision models. Exam Tip / AWS Best Practice: For domain-specific image/audio/text tasks where you have some labeled data but want to avoid training from scratch, use SageMaker JumpStart to fine-tune a pre-trained model.
### Question 7 - Latency vs. Accuracy Tradeoff in Foundation Models A high-frequency trading platform uses a generative AI assistant to analyze breaking news sentiment and alert traders. Traders require the sentiment analysis to be returned in under 500 milliseconds to be actionable. The engineering team is currently using the largest available foundation model in Amazon Bedrock, which achieves 99% accuracy but takes 2 seconds to respond. Which tradeoff should the engineering team evaluate to meet the latency constraint? - A) Switch to a smaller, faster foundation model that may slightly reduce accuracy but meets the 500ms latency target. - B) Keep the large model but increase the batch size of news articles processed simultaneously. - C) Switch to Amazon Textract to process the news articles faster. - D) Fine-tune the largest foundation model to make it process tokens faster.	Correct Answer: A Correct Answer Analysis: - A is Correct: In AI model selection, there is a direct tradeoff between output quality/accuracy, latency, and cost. To meet a strict sub-second latency SLA, the team must select a smaller, faster foundation model, accepting a minor reduction in accuracy to satisfy the operational constraint. Distractor Analysis: - B is Incorrect: Batching increases throughput (total volume processed over time) but does not decrease the per-request latency; a single request will still take 2 seconds. - C is Incorrect: Amazon Textract extracts text from images/documents; it does not perform sentiment analysis or generate text. - D is Incorrect: Fine-tuning a model does not fundamentally change its underlying inference latency, as latency is primarily bound by the model's parameter size and infrastructure. Exam Tip / AWS Best Practice: Latency, cost, and accuracy are a zero-sum tradeoff. For real-time, latency-sensitive applications, choose a smaller, faster model over the largest, most accurate model.
### Question 8 - Mitigating Catastrophic Forgetting An AI engineering team is performing supervised fine-tuning on a foundation model in Amazon Bedrock using a highly specialized dataset of internal IT support tickets. After the fine-tuning job completes, they notice the model performs exceptionally well on IT tickets but has lost its general conversational capabilities and struggles with basic reasoning tasks. Which strategy should the team employ to mitigate this issue? - A) Increase the learning rate during the fine-tuning job. - B) Remove all general conversational data from the fine-tuning dataset. - C) Blend broader, general data into the fine-tuning dataset to preserve general capabilities. - D) Decrease the `max_tokens` parameter during inference to force shorter responses.	Correct Answer: C Correct Answer Analysis: - C is Correct: This issue is known as "catastrophic forgetting," where a model trained hard on a narrow domain loses its general capabilities. The standard defense is to blend broader, general data into the training mixture so the model retains its foundational abilities while learning the new domain. Distractor Analysis: - A is Incorrect: Increasing the learning rate will make the model adapt faster to the new data, which would likely worsen catastrophic forgetting, not mitigate it. - B is Incorrect: Removing general data is the exact opposite of the correct strategy and will exacerbate the loss of general capabilities. - D is Incorrect: Adjusting `max_tokens` affects the length of the output at inference time but does not restore the model's reasoning capabilities lost during training. Exam Tip / AWS Best Practice: To prevent catastrophic forgetting during fine-tuning, mix a portion of general-domain data into your specialized training dataset.
### Question 9 - Managed NLP vs. Foundation Models A marketing agency processes thousands of customer reviews daily. They need to extract key phrases, detect entities (like product names and locations), and run sentiment analysis to populate a daily analytics dashboard. The data is completely generic, and the agency wants the absolute lowest cost per transaction. Which AWS service is the MOST cost-effective and appropriate solution for this standard NLP task? - A) Amazon Bedrock hosting a large foundation model with a custom NLP prompt. - B) Amazon Comprehend using the standard `DetectSentiment` and `DetectEntities` APIs. - C) Training a custom NLP model from scratch using Amazon SageMaker AI. - D) Amazon Transcribe to convert the text reviews into speech for analysis.	Correct Answer: B Correct Answer Analysis: - B is Correct: Amazon Comprehend is a managed AWS AI service specifically built for standard NLP tasks like key phrase extraction, entity recognition, and sentiment analysis. Because it is a pre-trained, purpose-built managed service, it is significantly cheaper and more efficient per transaction than a generative foundation model. Distractor Analysis: - A is Incorrect: While an FM in Bedrock could perform this task via prompting, it would be vastly more expensive and slower than a purpose-built managed service for standard NLP tasks. - C is Incorrect: Training a custom model introduces massive unnecessary overhead and cost for a generic task that Amazon Comprehend already solves out-of-the-box. - D is Incorrect: Amazon Transcribe converts speech to text; the inputs here are already text reviews, making this the wrong tool. Exam Tip / AWS Best Practice: For standard, solved NLP tasks (sentiment, entities, key phrases), use Amazon Comprehend. Reserve FMs for generative, creative, or complex reasoning tasks.
### Question 10 - Custom ML for Proprietary Sensor Data A manufacturing company wants to predict equipment failures on its factory floor. They have collected 5 years of proprietary, high-frequency time-series sensor data (vibration, temperature, pressure) from their machines. The data contains highly specific patterns indicating imminent failure that only exist within their factory environment. Which modeling approach is the MOST appropriate to capture these proprietary patterns? - A) Use Amazon Bedrock to prompt a foundation model with the sensor data. - B) Use Amazon Rekognition to analyze the sensor logs for visual anomalies. - C) Train a custom time-series machine learning model using Amazon SageMaker AI. - D) Use Amazon Textract to extract failure indicators from printed log reports.	Correct Answer: C Correct Answer Analysis: - C is Correct: The patterns indicating equipment failure exist only inside the company's own proprietary historical data. Because this is a proprietary data pattern problem, the correct approach is to train a custom ML model (like an XGBoost or DeepAR model) using Amazon SageMaker AI on the historical sensor data. Distractor Analysis: - A is Incorrect: Foundation models are not designed to ingest massive amounts of raw, high-frequency numerical time-series sensor data for predictive maintenance. They are built for text, code, and image generation. - B is Incorrect: Amazon Rekognition is for analyzing images and video, not numerical time-series sensor logs. - D is Incorrect: Amazon Textract extracts text from scanned documents; it does not perform predictive analytics on sensor data. Exam Tip / AWS Best Practice: When the predictive pattern lives strictly within your organization's proprietary tabular/time-series data, train a custom ML model. FMs are for general capabilities, not proprietary numerical pattern recognition.