Skill 2.1.3: Compare and select appropriate ML models, generative AI (GenAI) models, algorithms, and solution templates to meet business needs (for example, interpretability, domain-specific performance, latency)
| Question | Answer & Explanation |
|---|---|
| ### Question 1 - Interpretability and Model Selection A financial institution needs to predict loan defaults based on historical applicant data, including credit score, income, and loan amount. The compliance department mandates that the model's decisions must be fully interpretable to explain exactly why an applicant was denied. The solution must minimize operational overhead while maintaining strict adherence to regulatory explainability requirements. Which approach BEST meets these requirements? (Select ONE) - A) Train a deep neural network using Amazon SageMaker AI and use SageMaker Clarify for post-hoc feature attribution. - B) Train an XGBoost model using a SageMaker AI built-in algorithm and utilize its native feature importance metrics for explainability. - C) Fine-tune a large language model in Amazon Bedrock on historical loan data to generate natural language explanations for denials. - D) Deploy Amazon Comprehend to analyze the sentiment of the loan application text fields to determine default risk. |
Correct Answer: B Correct Answer Analysis: - B is Correct: XGBoost is a tree-based algorithm that provides native feature importance scores, making it inherently more interpretable than deep neural networks. Using a SageMaker AI built-in algorithm minimizes operational overhead and satisfies the compliance department's requirement for explaining how features contributed to a denial. Distractor Analysis: - A is Incorrect: Deep neural networks are "black box" models. While SageMaker Clarify can provide post-hoc feature attribution (SHAP values), it does not provide the transparent, inherent interpretability required by strict compliance departments for credit decisions. - C is Incorrect: Foundation models like LLMs are not designed for precise tabular numerical predictions (like default probability) and lack the deterministic interpretability required for regulatory compliance. - D is Incorrect: Amazon Comprehend performs NLP tasks like sentiment analysis and entity extraction; it is not a predictive modeling tool for structured numerical credit risk assessment. > Exam Tip: When a scenario requires strict regulatory interpretability for tabular data, favor simpler, inherently interpretable models (like XGBoost or Logistic Regression) over black-box neural networks, even if post-hoc explainability tools are available. |
| ### Question 2 - General Capability vs. Custom Training A customer support team wants to automatically summarize long, multi-turn chat transcripts into concise 3-bullet point overviews. The engineering team has thousands of historical chat transcripts but wants to ensure the rollout is fast and incurs the least operational overhead. The summarized capabilities must reflect standard conversational understanding. Which solution is the MOST appropriate? (Select ONE) - A) Fine-tune an LLM in Amazon Bedrock using the historical transcripts to teach it how to summarize. - B) Train a custom sequence-to-sequence model in Amazon SageMaker AI from scratch using the chat transcripts. - C) Use a pre-trained foundation model in Amazon Bedrock with a prompt containing instructions and a few-shot example of a summary. - D) Use Amazon Transcribe to convert the chat transcripts into speech and analyze the audio tone. |
Correct Answer: C Correct Answer Analysis: - C is Correct: Text summarization is a general capability that already exists inside foundation models. The "prompt first, fine-tune second" rule dictates that starting with prompt engineering (instructions + few-shot examples) in Amazon Bedrock provides the fastest rollout with zero training overhead and minimal operational cost. Distractor Analysis: - A is Incorrect: Fine-tuning introduces unnecessary operational overhead (data preparation, training jobs, versioning) for a general capability that prompt engineering can likely solve immediately. - B is Incorrect: Training a custom sequence-to-sequence model from scratch requires massive amounts of data, extensive ML expertise, and significant compute time, violating the "fastest rollout" and "least overhead" constraints. - D is Incorrect: Amazon Transcribe is for converting speech to text, not for analyzing text transcripts or generating summaries. Furthermore, analyzing audio tone does not fulfill the requirement for 3-bullet point text summaries. > Exam Tip: If the problem requires a general language capability (summarization, translation), start with a Foundation Model + Prompt Engineering. Only fine-tune if evaluations show a persistent gap that prompting cannot close. |
| ### Question 3 - Fixed Logic vs. Machine Learning A logistics company operates a fleet of trucks equipped with IoT sensors. They want to trigger an immediate alert to the dispatch center whenever a truck's tire pressure drops below 32 PSI or its engine temperature exceeds 240°F. The company has a streaming data pipeline and needs the most cost-effective and lowest-latency solution to implement this monitoring logic. Which architectural approach BEST meets these requirements? (Select ONE) - A) Deploy a SageMaker AI real-time endpoint to evaluate incoming sensor data and predict if an alert should be triggered. - B) Use Amazon Bedrock to analyze the streaming sensor data and generate a natural language alert for dispatchers. - C) Implement simple threshold-based rules in an AWS Lambda function triggered by incoming IoT data streams. - D) Train a custom anomaly detection model in SageMaker AI using historical sensor data to predict tire failures. |
Correct Answer: C Correct Answer Analysis: - C is Correct: The logic for this alert is fixed and explicitly defined (pressure < 32 PSI, temperature > 240°F). When business logic is fixed, a plain rule or code condition is the cheapest, fastest (lowest latency), and easiest to audit approach. Using AWS Lambda provides a highly cost-effective, serverless execution. Distractor Analysis: - A is Incorrect: Using a SageMaker AI endpoint for fixed threshold logic introduces unnecessary latency, high operational overhead, and cost. ML should be used for patterns, not explicit thresholds. - B is Incorrect: Amazon Bedrock is for generative AI tasks. Generating natural language alerts for simple threshold breaches adds latency and cost without adding value to a dispatch system that just needs a flag. - D is Incorrect: Anomaly detection is useful for finding unknown patterns, but the exact failure conditions are already known and fixed. Training a model here is massive over-engineering. > Exam Tip: If the decision rules are explicitly known and fixed, skip AI entirely and use deterministic code (e.g., Lambda, simple database queries). |
| ### Question 4 - Managed AI Service vs. Custom Build An enterprise accounts payable department needs to extract text, tables, and form key-value pairs from thousands of scanned PDF invoices daily. The engineering team has no experience in computer vision or Optical Character Recognition (OCR). The solution must be deployed with the least development effort and fastest time-to-market. Which AWS service is the BEST fit for this scenario? (Select ONE) - A) Train a custom OCR model using Amazon SageMaker Ground Truth and a custom computer vision algorithm. - B) Fine-tune a multimodal foundation model in Amazon Bedrock on a dataset of scanned invoices. - C) Use Amazon Textract to automatically extract text, tables, and form data from the invoices. - D) Deploy an open-source OCR model from SageMaker JumpStart and fine-tune it on the invoices. |
Correct Answer: C Correct Answer Analysis: - C is Correct: Amazon Textract is a managed AWS AI service specifically pre-trained to extract text, tables, and form key-value pairs from scanned documents. It requires zero machine learning expertise, zero custom model training, and provides immediate functionality via an API call, satisfying the least development effort constraint. Distractor Analysis: - A is Incorrect: Training a custom OCR model from scratch requires massive datasets, specialized computer vision expertise, and significant compute time, completely violating the "least development effort" constraint. - B is Incorrect: While multimodal models can process images, fine-tuning them for structured table and key-value extraction is complex and not the intended use case compared to a specialized service like Textract. - D is Incorrect: While SageMaker JumpStart reduces the burden of starting from scratch, deploying and fine-tuning an open-source OCR model still requires ML operational overhead compared to the fully managed Textract API. > Exam Tip: When a scenario involves a "solved task" (like OCR, image recognition, speech-to-text) and the team lacks ML expertise, always choose a managed AWS AI service (Textract, Rekognition, Transcribe, Comprehend). |
| ### Question 5 - RAG vs. Fine-Tuning for Dynamic Content A legal tech startup is building an AI assistant to answer questions based on a dynamic library of corporate contracts. The contracts are updated frequently, and the assistant must cite the exact clause and contract version used to formulate the answer. The solution must reflect the most recent contract edits within minutes. Which architecture BEST meets these requirements? (Select ONE) - A) Fine-tune an LLM in Amazon Bedrock on the corporate contracts so it memorizes the clauses. - B) Implement a Retrieval-Augmented Generation (RAG) architecture using Amazon Bedrock Knowledge Bases and a vector store. - C) Train a custom NLP model using SageMaker AI to classify questions and retrieve hardcoded answers. - D) Use Amazon Comprehend to extract entities from the contracts and inject them into a static prompt. |
Correct Answer: B Correct Answer Analysis: - B is Correct: RAG architecture (like Amazon Bedrock Knowledge Bases) keeps content outside the model weights. When a contract changes, re-ingesting the document into the vector store takes only minutes, and the system can automatically cite the source document. This perfectly satisfies the dynamic content and citation requirements. Distractor Analysis: - A is Incorrect: Fine-tuning writes knowledge into the model weights. It cannot easily cite sources, and updating the model for frequent contract edits requires hours of re-training, failing the "minutes" requirement. - C is Incorrect: A custom NLP classification model cannot handle the dynamic generation of nuanced legal answers and lacks the flexibility of an LLM. Hardcoding answers also fails the dynamic content requirement. - D is Incorrect: Amazon Comprehend extracts entities but does not provide a managed retrieval path for dynamic Q&A or cite specific document clauses for generative answers. > Exam Tip: Use RAG when you need dynamic content updates (without retraining) and explicit source citations. Use Fine-Tuning for teaching the model a specific tone, style, or format. |
| ### Question 6 - Latency and Cost Tradeoffs in GenAI An e-commerce platform is implementing a real-time chatbot for product recommendations using a foundation model in Amazon Bedrock. The business requires that the chatbot responds to user queries in under 500 milliseconds to maintain a smooth user experience. Cost is a secondary concern, but the model must support fast inference. Which approach is the MOST appropriate? (Select ONE) - A) Choose the largest available LLM on Bedrock to ensure maximum accuracy, and implement response streaming. - B) Choose a smaller, optimized foundation model (e.g., Amazon Titan Text Lite) to minimize inference latency. - C) Batch the user queries and process them hourly using an asynchronous SageMaker AI training job. - D) Fine-tune a large LLM on product data to improve accuracy, ignoring the inference latency limit. |
Correct Answer: B Correct Answer Analysis: - B is Correct: Smaller foundation models (like Titan Text Lite) have significantly lower inference latency than larger models. Given the strict 500ms latency requirement, prioritizing a smaller, optimized model is the correct architectural choice, even if it means a minor tradeoff in reasoning capability. Distractor Analysis: - A is Incorrect: The largest LLMs have high inference latency (often several seconds) and will fail the strict 500ms requirement, regardless of response streaming (streaming helps time-to-first-token but not overall generation time). - C is Incorrect: Batching queries and processing them hourly violates the real-time, interactive nature of a chatbot and the 500ms response requirement. - D is Incorrect: Fine-tuning a large LLM does not solve the fundamental latency bottleneck of running a massive model for real-time inference. > Exam Tip: In GenAI, latency constraints often dictate model size. If a scenario demands sub-second real-time responses, choose a smaller, optimized model over a large, highly capable one. |
| ### Question 7 - Domain-Specific Performance (Supervised Fine-Tuning) A healthcare startup has developed a prompt-based application using an Amazon Bedrock foundation model to classify medical transcripts into specialized categories. Despite using clear instructions and few-shot examples, the model consistently misclassifies rare, domain-specific diseases because it lacks the nuanced medical vocabulary. They have a large dataset of 50,000 labeled transcripts. What is the MOST appropriate next step? (Select ONE) - A) Continue adding more few-shot examples to the prompt until the context window is full. - B) Switch to a larger foundation model with a bigger context window to include more examples. - C) Perform supervised fine-tuning on the Bedrock foundation model using the labeled transcripts. - D) Use Amazon Comprehend Medical to extract medical entities and pass them to the foundation model via prompt. |
Correct Answer: C Correct Answer Analysis: - C is Correct: The scenario explicitly states that prompt engineering (instructions + few-shot examples) has failed to close the performance gap for domain-specific terminology. When evaluations show a persistent gap that prompting cannot close, supervised fine-tuning on labeled data is the standard progression to bake the domain knowledge and vocabulary into the model weights. Distractor Analysis: - A is Incorrect: The scenario states few-shot examples have already failed. Continually stuffing the prompt is inefficient, consumes context limits, and does not update the model's underlying weights or vocabulary understanding. - B is Incorrect: A larger model might have slightly better general knowledge, but it still lacks the specific company/medical domain logic, and few-shot examples have already proven ineffective for this specific classification task. - D is Incorrect: While Comprehend Medical is great for entity extraction, the task is complex transcript classification. If the LLM is failing at the classification logic itself, just passing entities might not solve the core problem, whereas fine-tuning directly addresses the classification behavior. > Exam Tip: The order of operations for adapting FMs is: Prompt Engineering -> Fine-Tuning. Move to fine-tuning ONLY when you have a large labeled dataset AND prompting has definitively failed. |
| ### Question 8 - Tradeoffs in Custom ML (Training Time vs Performance) A data science team is training a computer vision model to detect manufacturing defects. They are using a SageMaker AI managed training cluster with a custom PyTorch script. They need to balance training time and model performance. The current algorithm takes 48 hours to train, which heavily delays their iteration cycle. They are willing to sacrifice a small amount of accuracy (1-2%) to speed up the experimentation phase. Which strategy BEST balances these requirements? (Select ONE) - A) Increase the instance count for distributed training to reduce total training time, while accepting minor communication overhead. - B) Switch to a pre-trained model from SageMaker JumpStart and only retrain the final classification layer (transfer learning). - C) Train the model entirely on local GPUs and bypass SageMaker to avoid cluster provisioning delays. - D) Fine-tune a foundation model on Amazon Bedrock using images of the defects instead of using SageMaker. |
Correct Answer: B Correct Answer Analysis: - B is Correct: Using a pre-trained model from SageMaker JumpStart and applying transfer learning (retraining only the final layers) dramatically reduces training time compared to training a deep network from scratch. This approach aligns with the willingness to sacrifice a small amount of accuracy for a much faster iteration cycle. Distractor Analysis: - A is Incorrect: While distributed training reduces wall-clock time, it often doesn't reduce the total compute cost and can introduce communication overhead. It doesn't fundamentally change the 48-hour per-epoch complexity like transfer learning does. - C is Incorrect: Bypassing SageMaker removes managed infrastructure, EFA networking, and distributed training capabilities, usually resulting in slower training times and higher operational overhead for the team. - D is Incorrect: Amazon Bedrock foundation models are primarily for text (and increasingly multimodal, but not for specialized industrial defect detection in the same way). Training a custom vision model in SageMaker is the correct approach; the issue is the training strategy, not the platform. > Exam Tip: Transfer learning (using pre-trained models from JumpStart) is the go-to strategy when you need to significantly reduce training time and compute while accepting a slight accuracy tradeoff. |
| ### Question 9 - AWS AI Services for Specific NLP Tasks A marketing agency wants to analyze thousands of daily social media posts to determine the overall sentiment (positive, negative, neutral) regarding their client's new product launch. They need a solution that requires no machine learning expertise, can be integrated into an existing AWS Step Functions workflow via API, and has minimal operational overhead. Which AWS service is the MOST appropriate choice? (Select ONE) - A) Train a custom text classification model in SageMaker AI using the built-in BlazingText algorithm. - B) Use Amazon Comprehend to perform sentiment analysis on the social media posts. - C) Prompt an LLM in Amazon Bedrock to classify the sentiment of each post one by one. - D) Use Amazon Rekognition to detect the client's brand logos in images accompanying the posts. |
Correct Answer: B Correct Answer Analysis: - B is Correct: Amazon Comprehend is a managed AWS AI service that natively performs sentiment analysis out-of-the-box. It requires zero ML expertise, is accessible via a simple API call (perfect for Step Functions), and operates with minimal operational overhead. Distractor Analysis: - A is Incorrect: Training a custom BlazingText model requires gathering labeled datasets, managing training jobs, and hosting an endpoint, violating the "no ML expertise" and "minimal operational overhead" constraints. - C is Incorrect: While an LLM could do this, calling it post-by-post for standard sentiment analysis is more expensive, slower, and over-engineered compared to a dedicated managed service like Comprehend. - D is Incorrect: Amazon Rekognition is for image and video analysis (like detecting logos or faces), not for performing text sentiment analysis on social media posts. > Exam Tip: For standard, out-of-the-box NLP tasks (sentiment, entity extraction, key phrase extraction), always prefer Amazon Comprehend over custom training or LLM prompting to minimize cost and overhead. |
| ### Question 10 - Custom ML for Predictive Analytics An agricultural company wants to predict the optimal harvest time for its crops based on soil moisture, daily weather forecasts, and historical satellite imagery. They have 10 years of historical data correlating these factors to optimal harvest yields. The solution requires integrating with existing IoT sensors and needs low operational overhead for batch predictions. Which modeling approach is the MOST appropriate? (Select ONE) - A) Use Amazon Bedrock to generate a weekly narrative report on crop conditions based on the data. - B) Train a regression model on the historical tabular and time-series data using a SageMaker AI built-in algorithm. - C) Implement a RAG architecture to retrieve past harvest data and feed it to a foundation model. - D) Use Amazon Transcribe to convert voice notes from farmers into text for sentiment analysis on crop health. |
Correct Answer: B Correct Answer Analysis: - B is Correct: The answer is "already in the data." Predicting a numerical outcome (optimal harvest time/yield) based on historical tabular and time-series data is a classic supervised learning problem. Using a SageMaker AI built-in algorithm (like XGBoost or Linear Learner) is the optimal, low-overhead approach for batch predictive analytics on proprietary historical data. Distractor Analysis: - A is Incorrect: Generating a narrative report via Bedrock does not perform the precise numerical/time-series predictions required to determine an exact optimal harvest window. - C is Incorrect: RAG is for retrieving textual knowledge to ground an LLM's generative answers, not for predictive modeling on structured numerical time-series data. - D is Incorrect: Amazon Transcribe converts speech to text; it has no predictive capability for crop yields based on soil and weather data. > Exam Tip: When the problem requires predicting a numerical value based on proprietary historical tabular/time-series data, use a Traditional ML model (e.g., SageMaker XGBoost) rather than a Foundation Model. |