Task 2.1 - Choose appropriate modeling approaches for ML and AI solutions

Overview

Task 2.1, Choose appropriate modeling approaches for ML and AI solutions, is the decision that constrains every decision after it. It asks whether a problem needs a trained model at all, and if it does, whether the answer lives in an organization's own data or in a capability a foundation model already carries.

Interpretability, latency, and cost belong inside that decision rather than after it.

As you review these lessons for Task 2.1, check that you understand how to do the following.

Evaluate and select appropriate FMs from Amazon Bedrock based on task requirements and performance criteria.
Identify fine-tuning strategies for pre-trained FMs to meet business needs.
Compare and select appropriate ML models, generative AI (GenAI) models, algorithms, and solution templates to meet business needs (for example, interpretability, domain-specific performance, latency).
Evaluate tradeoffs between custom solutions, managed services, pre-trained models, and FMs to meet business needs.
Select Retrieval Augmented Generation (RAG) architecture patterns based on use case requirements.
Assess tradeoffs between ML model performance, training time, and cost.
Assess tradeoffs between AI model performance, latency, and cost.
Apply AWS AI services to solve specific business problems (for example, Amazon Textract, Amazon Rekognition, Amazon Comprehend, Amazon Transcribe).

AWS services overview

The services in this task divide by where the capability comes from: a foundation model reached through one API, a managed service that already performs a specific task, or a training service that fits a model to an organization's own data. Recognize what each one removes from the build. Review each service to learn more.

AWS Service	Description
Amazon Bedrock	Managed access to foundation models through a single API, alongside the customization and evaluation paths those models need.
Amazon Bedrock Knowledge Bases	The managed retrieval path. Documents are ingested, converted to embeddings, and queried so a model can answer from an organization's own content.
Amazon SageMaker AI	The managed service for the custom machine learning lifecycle, including training with a built-in algorithm or with a training script of your own.
Amazon SageMaker JumpStart	Pre-trained models and solution templates, so a project starts from something that already works rather than from an empty notebook.
Amazon Textract	Extracts text and structured content from scanned documents.
Amazon Rekognition	Analyzes images and video for objects, scenes, and other visual content.
Amazon Comprehend	Finds entities, key phrases, sentiment, and other structure inside text.
Amazon Transcribe	Converts speech in an audio recording into text.

Knowledge Check

An application must answer questions from a policy library that is revised often, and the answers must reflect the newest wording. Which service provides the managed retrieval path for that content? (Choose 1)

A. Amazon Bedrock Knowledge Bases
B. Amazon Transcribe
C. Amazon Textract
D. Amazon Comprehend

Answer & Explanation

Correct Answer: A. Amazon Bedrock Knowledge Bases

Explanation:
Amazon Bedrock Knowledge Bases provides a fully managed Retrieval-Augmented Generation (RAG) workflow. It automates document parsing, chunking, embedding generation, vector store syncing, and contextual retrieval. For policy libraries that change frequently, re-syncing Knowledge Bases updates the vector index, ensuring foundation models always retrieve and cite the newest wording without needing model retraining or fine-tuning.

Why the other options are incorrect: * B. Amazon Transcribe: Converts spoken speech in audio recordings into written text; it does not provide document storage, vector embedding, or RAG retrieval features. * C. Amazon Textract: Extracts raw text, form fields, and structured tables from scanned documents and PDFs, but does not serve as a vector database or managed retrieval engine for Q&A applications. * D. Amazon Comprehend: Performs natural language processing (NLP) tasks like detecting entities, key phrases, and sentiment, but does not manage dynamic retrieval or vector indexing for foundation models.

AWS skills overview

In this lesson, you review the skills behind Task 2.1 as a light refresher. The eight skill statements group into three topics: selecting a foundation model and a fine-tuning strategy, weighing custom builds against managed options, and reading the requirements that decide a retrieval pattern. Read through them and notice which ones already feel familiar, and which ones feel shaky enough that you want to come back to them before exam day.

Selecting foundation models and fine-tuning strategies

In practice, this group of skills lets the machine learning engineer pick a foundation model on evidence rather than reputation, and then decide whether that model needs customizing at all.

Be comfortable doing the following:

Comparing available foundation models against what the task actually requires, including output quality, response latency, and cost per request, and treating the comparison as a measurement rather than an opinion.
Deciding when a fine-tuning strategy is warranted, and which one: supervised fine-tuning from labeled examples, reinforcement fine-tuning from reward functions that score responses, or distillation of a larger model's behavior into a smaller one.
Recognizing what each customization path asks you to prepare, because distillation is prepared as a set of prompts and the managed path generates the teacher responses from them, while supervised fine-tuning is prepared as labeled input and output pairs.

The order of operations is where teams lose time. Task-specific prompt engineering, with clear instructions and a few worked examples, usually closes a wording or format gap on its own, and it changes immediately. Customization asks for prepared data, a training job, and a version to manage, so it earns its place only when evaluations show a gap that prompting cannot close. Customization also carries a real risk worth naming: a model trained hard on a narrow domain can lose general capability it previously had, which is catastrophic forgetting, and blending broader data into the training mixture is the usual defense.

Knowledge Check

Evaluations show a foundation model still misses an organization's internal terminology after prompt instructions and worked examples were added. Which step is the MOST appropriate next one? (Choose 1)

A. Fine-tune the model on labeled examples that demonstrate the expected wording
B. Add more retrieved passages to the prompt at request time
C. Raise the limit on how many tokens the model may generate
D. Select a smaller model to reduce inference cost

Answer & Explanation

Correct Answer: A. Fine-tune the model on labeled examples that demonstrate the expected wording

Explanation:
The standard progression for adapting foundation model behavior is: Prompt Engineering (Instructions + Few-Shot Examples) \(\longrightarrow\) Supervised Fine-Tuning (SFT). When prompt engineering and in-context learning fail to consistently enforce domain-specific terminology, style, or formatting guidelines, fine-tuning the model on a dataset of labeled prompt-response pairs (JSONL) updates the model's internal weights to consistently adopt the target vocabulary.

Why the other options are incorrect: * B. Add more retrieved passages to the prompt at request time: RAG supplies external factual knowledge at query time, but simply stuffing more context into the prompt window does not teach the model how to adhere to specific terminology or stylistic conventions if few-shot examples have already failed. * C. Raise the limit on how many tokens the model may generate: Increasing max_tokens allows the model to produce longer outputs, but does not improve the semantic precision or terminology of the generated text. * D. Select a smaller model to reduce inference cost: Switching to a smaller foundation model generally reduces reasoning capabilities and instruction-following quality, making it even less likely to follow specialized internal terminology.

Weighing custom builds, managed services, and pre-trained models

In practice, this group of skills lets the machine learning engineer say no to building something, which is often the highest-value decision available.

Be comfortable doing the following:

Comparing candidate models, algorithms, and solution templates against the requirements that are easy to forget: interpretability for the people who must defend a prediction, performance on this specific domain, and the latency the application can absorb.
Weighing a custom build against a managed service, a pre-trained model, and a foundation model, and being explicit about what each option costs in engineering time as well as in compute.
Reaching for the AWS AI service that already performs a solved task, such as extracting content from documents, analyzing images and video, finding structure in text, or transcribing speech.

Two failure modes bracket this decision. One is building a model for a capability a managed service already provides, which buys nothing and creates something to maintain. The other is reaching for a model when the logic is fixed, where a plain rule is cheaper, faster, and easier to audit. Between those, the question is where the answer lives. Patterns that exist only inside an organization's own history call for training on that history. A general capability such as summarizing language already exists inside a foundation model, and no amount of local history teaches a regression model to write a sentence.

Knowledge Check

A team must pull text out of scanned invoices, and no comparable model exists in house. Which approach reaches a working result with the LEAST development effort? (Choose 1)

A. Call a managed AWS AI service that already extracts text from documents
B. Train a custom computer vision model on labeled invoice images
C. Fine-tune a foundation model on a corpus of past invoices
D. Write fixed rules that read text at set positions on each layout

Answer & Explanation

Correct Answer: A. Call a managed AWS AI service that already extracts text from documents

Explanation:
AWS managed AI services like Amazon Textract (specifically the AnalyzeExpense API) are pre-trained on millions of documents, forms, and invoices. Calling a managed API requires zero custom model training, zero manual labeling, and no dedicated compute infrastructure to manage—making it the option with the lowest development effort and fastest time-to-market.

Why the other options are incorrect: * B. Train a custom computer vision model on labeled invoice images: Requires building a bounding-box annotation workflow, selecting vision/OCR architectures, training, and managing hosting infrastructure, incurring high development and operational overhead. * C. Fine-tune a foundation model on a corpus of past invoices: Preparing specialized multimodal datasets and running fine-tuning jobs is resource-intensive and unnecessary for standard document key-value extraction. * D. Write fixed rules that read text at set positions on each layout: Position-based bounding box rules are extremely brittle. Invoices vary widely across vendors, causing coordinate-based rules to fail whenever layouts, margins, or scan scales change.

Retrieval patterns and the performance, latency, and cost tradeoffs

In practice, this group of skills lets the machine learning engineer read a set of requirements and derive the architecture from them, instead of choosing an architecture and hoping the requirements fit.

Be comfortable doing the following:

Selecting a retrieval pattern from the use case, and knowing the parts a production retrieval system needs whichever pattern you choose: connectors to the sources, document processing, an embedding model, a vector store, a retriever, the generating model, guardrails, and an orchestrator.
Choosing between a fully managed retrieval service and a custom retrieval architecture, where the managed path removes work and the custom path returns control over components such as the retriever and the vector store.
Assessing tradeoffs between model performance, training time, and cost for a trained model, and between output quality, latency, and cost for a foundation model, and naming which one a requirement is asking you to give up.

Retrieval and customization answer different questions, so the requirements pick between them. Retrieval keeps content outside the model, which means an edited document is reflected as soon as it is re-ingested, and the response can point at the source it used. Customization writes behavior into the weights, so a content change waits for another training job, and there is no source to cite. Retrieval is a weaker fit when a request needs whole-document synthesis rather than the retrieval of relevant passages, and the two approaches can be combined when the requirements genuinely call for both.

Knowledge Check

A support assistant must cite the document behind each answer and must reflect a policy edit within minutes. Which property of a retrieval approach satisfies both requirements? (Choose 1)

A. The answer is generated from passages retrieved at request time
B. The policy text sits in the model weights after a customization job
C. A larger model is selected to raise answer quality
D. Requests are batched to lower the cost of each response

Answer & Explanation

Correct Answer: A. The answer is generated from passages retrieved at request time

Explanation:
Retrieval-Augmented Generation (RAG) fetches fresh, relevant passages from a vector database at query time and injects them directly into the foundation model's prompt context. Because retrieval relies on an external vector index rather than model weights: 1. Document Citations: Each retrieved chunk carries source metadata (such as document title, section, or page number) that the model can explicitly cite in its final answer. 2. Near Real-Time Updates: When a policy changes, updating or re-embedding the affected document chunk in the vector index takes only minutes, immediately providing up-to-date knowledge without requiring model fine-tuning or re-training.

Why the other options are incorrect: * B. The policy text sits in the model weights after a customization job: Customizing model weights via fine-tuning takes hours or days, cannot easily cite exact source documents, and requires continuous, costly re-training whenever policies are edited. * C. A larger model is selected to raise answer quality: Model parameters affect reasoning capability, but a larger model still cannot retrieve newly updated document edits or provide verified source citations unless paired with a retrieval mechanism. * D. Requests are batched to lower the cost of each response: Batching requests optimizes compute throughput and inference costs, but has no effect on knowledge freshness or document attribution.

Choosing a Modeling Approach

Original video transcript

Choosing a modeling approach sounds like the easy part. Most people can list the options train a model, use a pre-train one, or call a foundation model. Right? But listing them isn't the skill. The skill is looking at a business problem and knowing which one fits. And sometimes the honest answer is none of them. Exactly. So let's reason through it with a scenario. I'll set the situation, Joe. You talk me through the approach, and we'll stop wherever the choice gets uncomfortable. You're an ML engineer at a grocery chain. Leadership wants weekly demand forecasts for every store. You have years of historical sales data. The finance team needs to understand the predictions, and inference costs has to stay low. So where do you start? Honestly, my first question is whether this needs AI at all. If the business logic were fixed, like reorder when stock drops below a set number, a plain rule would beat any model. Good instinct. So why does forecasting pass the test? Well, because demand moves with seasonality, promotions, and local patterns that no fixed rule can express. The patterns exist, but only inside the company's own history. The answer is already in the data, and training pulls it out. So let's hold on to that idea. The answer is already in the data because the second scenario is going to flip it. So what does training look like here? Well, this is supervised learning on the company's own history, and with the seasonality captured as features, it becomes a regression problem. That's a classic case for a built-in algorithm in Amazon Sagemaker AI. The data is already labeled by history because every past week has its actual sales, and the finance team's requirement, they won't accept a prediction nobody can explain. Yeah, that requirement shapes the model choice more than accuracy does. A simpler, interpretable model that finance trusts beats a complex one they reject. Interpretability, latency, and cost are selection criteria, not afterthoughts. Nice. So that's the traditional path, and most people are comfortable there. Let's change up the problem. Same grocery chain, and now customer support wants automatic summaries of long support tickets. The summaries need company terminology. The rollout needs to be fast, and the cost needs to stay predictable. So walk me through it. Well, here's where your phrase flips. The answer is not in your data this time, because summarizing language is a general capability. No amount of ticket history teaches a regression model to write a sentence. That capability already exists inside a foundation model. So that's why I'd select one of those through Amazon Bedrock. OK. And this is the point where many teams move straight to fine tuning. The terminology requirement sounds like a training job, right? And that's the trap because prompting hasn't been tried yet. Task-specific prompt engineering with instructions and a few example summaries usually covers terminology on its own. Fine tuning's the answer when a gap persists that no prompt can close. So the rule is prompt first and fine tune only when your evaluations show a gap that stays. Exactly. And the order matters because of cost and speed. A prompt changes immediately, while a fine-tuned model needs prepared data, training, and versioning. You accept that overhead only when the problem demands it. And what happened to your training skills in the second scenario? Did they just disappear? No, they shifted to different decisions. Model selection replaced algorithm selection, so I compared foundation models on quality, latency, and cost, and the data changed roles too, from training input to a few examples inside a prompt. So let's review the two scenarios together. Well, the skills stayed the same, which is matching the approach to the problem, but the forecast trained the model because the answer was in the data. The summaries selected a foundation model because the capability was in the model. Same skill, different source. And the third option sat underneath both. When the logic is fixed, skip the model entirely and write the rule. Yeah, that's the full decision. Fixed logic gets a rule, and an already solved task gets a managed AI service. Patterns inside your own data get a trained model, and general language ability gets a foundation model with prompting first. So here's the self check. If the reasoning at every decision point felt obvious, you're well prepared for these decisions. And if any of it surprised you, especially the prompt before fine tuning order, that's your gap. Go back, understand why the approach changed, and then keep studying.

Core Principle: Model selection is not just about listing options—it is about matching business constraints (interpretability, latency, cost) with where the underlying knowledge or capability resides.

1. The 4-Tier Selection Hierarchy

Problem Profile	Recommended Approach	Example AWS Tool / Implementation
Fixed Business Logic	Simple Rules / Code	Plain code conditionals (e.g., reorder stock if \(qty < 10\))
Pre-Defined / Standard Task	Managed AWS AI Service	Amazon Textract, Rekognition, Transcribe, Comprehend
Patterns in Proprietary Data	Train Traditional ML Model	Amazon SageMaker AI built-in algorithms (e.g., XGBoost)
General Language / Creative Task	Foundation Model (Bedrock)	Amazon Bedrock + Prompting \(\longrightarrow\) Fine-Tuning (if needed)

2. Comparative Scenario Analysis

Scenario A: Weekly Demand Forecasting (Traditional ML)

Goal: Predict store-level sales from historical transactions for finance stakeholders.
Core Insight: "The answer is already in your data."
Approach: Supervised Regression using historical sales, seasonality, and promotional features via Amazon SageMaker AI.
Key Constraint: Interpretability & Stakeholder Trust. A simpler, explainable model that finance trusts and understands always beats a complex "black box" model that gets rejected.

Scenario B: Support Ticket Summarization (Generative AI)

Goal: Generate concise summaries of long support tickets using company terminology.
Core Insight: "The capability is already inside the model."
Approach: Managed Foundation Model via Amazon Bedrock.
Key Constraint: Deployment Speed & Cost. Summarizing language is a general capability. No amount of tabular history teaches a model to write sentences.

3. Golden Architectural Rules for the Exam

Rule 1: The "Prompt First, Fine-Tune Second" Order

          ┌─────────────────────────────────────────┐
          │   Task-Specific Prompt Engineering     │
          │   (Instructions + Few-Shot Examples)    │
          └────────────────────┬────────────────────┘
                               │
                               ▼
                    Run Performance Evaluation
                               │
            ┌──────────────────┴──────────────────┐
            ▼                                     ▼
     Meets Quality Bar?                    Fails Consistently?
            │                                     │
            ▼                                     ▼
     Deploy to Prod                     Consider Fine-Tuning
     (Zero Training Overhead)           (Prepped Data, GPU Cost,
                                         Version Control Overhead)

Always start with Prompt Engineering: Task-specific instructions and few-shot examples inside a prompt usually cover specialized terminology and formatting.
Fine-tune only as a last resort: Fine-tuning incurs significant overhead (dataset preparation, continuous training costs, model versioning). Only accept this overhead when evaluations prove prompting alone cannot bridge the gap.

Rule 2: Non-Functional Constraints Shape Model Choice

Model selection is rarely driven by raw accuracy alone. The primary evaluation criteria are:

Interpretability: Can business stakeholders understand and audit how predictions are made?
Inference Latency: Does the model meet real-time or batch response SLAs?
Cost & Overhead: What is the long-term operational cost of hosting, maintaining, and updating the model?

Quick Recall

Without rewatching, write the test: when the answer is already in the data, train a model. When the capability is already in the model, select a foundation model and prompt it first. Then name the two cases that sit underneath both: fixed logic getting a rule and an already-solved task getting a managed AI service.

Find your gap

The video ends on a self-check. If the reasoning at every decision point felt obvious as it went, you are well prepared for these decisions. If any of it surprised you, especially the prompt-before-fine-tuning order, that swap is your gap. If necessary, return to the skills lesson above and work out why the approach changed.

Knowledge Check

A retail company needs weekly demand forecasts drawn from years of its own sales history, and the finance team must be able to explain each prediction. Which approach fits those requirements BEST? (Choose 1)

A. Call a managed AWS AI service that generates written commentary about demand
B. Train a supervised model on the company's history and favor an interpretable algorithm
C. Select a large foundation model and supply the sales history in the prompt
D. Write fixed reorder rules that trigger at a stock threshold

Answer & Explanation

Correct Answer: B. Train a supervised model on the company's history and favor an interpretable algorithm

Explanation:
When historical transactional data exists within an organization and contains complex trends/seasonality, training a supervised ML model (e.g., a regression algorithm) on proprietary data is the optimal path. Furthermore, because finance stakeholders require explainability, choosing an interpretable algorithm (or pairing a tree-based model with SHAP/SageMaker Clarify feature attribution) satisfies the core business constraint without using a "black box" model.

Why the other options are incorrect: * A. Call a managed AWS AI service that generates written commentary about demand: Higher-level AI services or written commentary do not perform store-level numerical time-series forecasting tailored to proprietary historical sales tables. * C. Select a large foundation model and supply the sales history in the prompt: LLMs and foundation models are not designed for precise, large-scale tabular time-series demand forecasting across thousands of SKUs and store locations, and context window limits make passing years of raw tabular data inefficient and costly. * D. Write fixed reorder rules that trigger at a stock threshold: Static inventory rules cannot model dynamic demand patterns, seasonality, promotional impacts, or regional trends.