AWS AI Practitioner Exam Notes

Task Statement 3.3: Describe the Training and Fine-Tuning Process for Foundation Models

This task statement focuses on:

How foundation models are trained
How foundation models are fine-tuned for specific tasks or domains
How data is prepared for fine-tuning
How AWS services support data preparation and model customization

1. Foundation Model Training Lifecycle

A foundation model (FM) is generally developed through several stages:

Pre-training
Fine-tuning
Continuous pre-training
Optional model compression or distillation
Evaluation and alignment

These stages do not all have to be performed by every customer. A customer typically starts with an existing foundation model and performs additional customization.

1.1 Pre-training

Definition

Pre-training is the process of training a model on extremely large amounts of generally unlabeled data using self-supervised learning.

The data can include:

Text documents
Books and web pages
Images
Videos
Audio
Code
Other files and multimodal content

During pre-training, the model learns general patterns such as:

Language structure
Grammar
Vocabulary
Facts and associations
Context
Patterns in code, images, or audio
General reasoning capabilities
Relationships between words, tokens, and concepts

Characteristics of pre-training

Pre-training typically requires:

Very large datasets
Millions or billions of training examples
Trillions of tokens for large language models
Large numbers of GPUs or other accelerators
Significant compute time
Large amounts of storage
Extensive experimentation and tuning

Self-supervised learning

In self-supervised learning, the training data does not need a manually assigned label for every example. Instead, the training process creates a learning task from the data itself.

For example:

Given part of a sentence, predict the next token
Given a masked token, predict the missing token
Given an image, predict associated content or relationships

Exam point

Pre-training gives a foundation model broad, general-purpose capabilities. It is not normally performed by an individual customer because of the scale and cost involved.

1.2 Fine-Tuning

Definition

Fine-tuning extends the training of a pre-trained foundation model using a smaller, task-specific or domain-specific dataset.

Fine-tuning is normally a form of supervised learning. The model is trained using examples that contain inputs and expected outputs.

Examples:

Input	Expected output
Customer complaint	Complaint category
Legal document	Summary
Product description	Marketing text
Question	Correct answer
Code prompt	Code completion

During fine-tuning:

The model receives an input or prompt.
The model generates a completion.
The completion is compared with the expected label or output.
A loss is calculated.
Model parameters are updated to reduce the loss.
The process is repeated over many batches.

Why fine-tune a foundation model?

Fine-tuning can improve performance for:

A specific business task
A particular writing style
A specific industry
Specialized terminology
A specific language
A particular response format
Domain-specific classification or generation

Examples of domain-specific language include:

Medical terminology
Legal terminology
Financial terminology
Engineering terms
Internal company acronyms
Industry-specific processes

Pre-training versus fine-tuning

Characteristic	Pre-training	Fine-tuning
Main purpose	Learn general capabilities	Adapt the model to a task or domain
Data size	Extremely large	Usually much smaller
Data type	Often unlabeled	Usually labeled input-output examples
Learning approach	Self-supervised learning	Supervised learning or preference-based learning
Cost	Very high	Lower than pre-training, but can still be substantial
Model result	General foundation model	Customized model
Typical user	FM developer or provider	Organization using an existing FM

Exam trap

Do not confuse fine-tuning with providing documents to a model at inference time.

Fine-tuning changes model parameters or representations through additional training.
Retrieval-Augmented Generation (RAG) retrieves external information and provides it to the model at inference time. RAG does not necessarily modify the model weights.

2. Continuous Pre-Training

Definition

Continuous pre-training is additional pre-training of an existing foundation model using new, usually unlabeled, data.

It is used to expand or refresh the model’s general knowledge without necessarily training it for one narrowly defined task.

Common use cases

Continuous pre-training can help a model:

Learn newly emerging information
Understand company-specific or industry-specific language
Improve knowledge of specialized topics
Adapt to new genres, topics, or contexts
Expand its vocabulary
Improve its ability to handle out-of-domain data

Fine-tuning versus continuous pre-training

Characteristic	Fine-tuning	Continuous pre-training
Primary goal	Improve a particular task or behavior	Expand or refresh general knowledge
Typical data	Labeled prompt-completion examples	Usually unlabeled domain or topic data
Scope	Narrower task or domain	Broader knowledge and adaptability
Example	Train a model to classify insurance claims	Train a model on additional insurance documents
Learning approach	Supervised or preference-based	Self-supervised

AWS example

Amazon Bedrock supports customization of certain models, including:

Amazon Titan Text Express
Amazon Titan Text Lite

These models can be continuously pre-trained using the organization’s own unlabeled data in a managed environment.

Exam tip

If the question emphasizes:

Unlabeled data
Expanding knowledge
New topics
New domain language
Continued model training

the answer is likely continuous pre-training.

If it emphasizes:

Labeled examples
Prompt and completion pairs
A specific task
Updating the model for a business use case

the answer is likely fine-tuning.

3. Distillation

Definition

Knowledge distillation is a method of transferring knowledge from a larger, more capable teacher model to a smaller student model.

The student model is trained to reproduce some of the teacher model’s behavior.

Why use distillation?

Distillation can produce a model that is:

Smaller
Faster
Less expensive to operate
More suitable for low-latency applications
More suitable for edge or resource-constrained environments

Teacher and student models

Model	Role
Teacher model	Larger or more capable model that provides outputs or knowledge
Student model	Smaller model trained to imitate the teacher

Important distinction

Distillation is not the same as:

Fine-tuning a model on labeled business data
Prompt engineering
RAG
Quantization

Distillation transfers behavior or knowledge from one model to another. It is often used after or alongside training and fine-tuning.

Exam tip

If a question describes a large model teaching a smaller model to provide similar results, the answer is knowledge distillation.

4. Full Fine-Tuning

During full fine-tuning, all or nearly all model parameters are updated using the fine-tuning dataset.

Advantages

Maximum flexibility
Can significantly adapt model behavior
Useful when the target task is substantially different from the original model behavior

Disadvantages

High GPU memory requirements
Higher compute cost
Longer training time
Requires more storage for model checkpoints
Greater risk of overfitting
Greater risk of catastrophic forgetting

When training or tuning a model, memory is needed for more than the model’s original parameters. Additional memory may be required for:

Model parameters
Gradients
Optimizer state
Forward activations
Temporary training memory

Therefore, even a model that technically fits in GPU memory for inference might not fit during full fine-tuning.

5. Parameter-Efficient Fine-Tuning

Definition

Parameter-efficient fine-tuning (PEFT) uses techniques that preserve or freeze most of the original model and train only a small number of additional parameters.

Benefits

PEFT can reduce:

GPU memory requirements
Training time
Compute cost
Storage requirements
Risk of changing the general behavior of the base model

PEFT is useful when an organization wants to adapt an FM without updating every parameter.

5.1 LoRA

Low-Rank Adaptation (LoRA) is a popular PEFT technique.

Instead of updating the original model weights directly, LoRA:

Freezes the original foundation model weights.
Adds trainable low-rank matrices to model layers.
Trains the low-rank matrices on the task-specific data.
Uses the original model plus the learned adaptation during inference.

Advantages of LoRA

Requires fewer trainable parameters
Uses less memory
Reduces training cost
Makes it easier to maintain multiple task-specific adapters
Preserves the original base model

Exam trap

LoRA is a parameter-efficient fine-tuning method. It is not the same as:

Pre-training
Continuous pre-training
RAG
Prompt engineering
Knowledge distillation

6. Representation Fine-Tuning

Definition

Representation fine-tuning (ReFT) freezes the base model and learns task-specific interventions on the model’s hidden representations.

A model’s hidden representations encode semantic information about the input. These representations are related conceptually to the information captured by embeddings.

PEFT/LoRA versus ReFT

Technique	What is adapted?
Full fine-tuning	Most or all model weights
PEFT	A small number of task-specific parameters
LoRA	Additional low-rank weight matrices
ReFT	Hidden representations or activations

Key exam distinction

LoRA and PEFT modify or add parameters associated with model weights.
ReFT modifies the model’s internal representations through task-specific interventions.

7. Instruction Tuning

Definition

Instruction tuning is fine-tuning a model using examples that show how to follow instructions and produce desired responses.

Training examples often contain:

An instruction or prompt
An expected response or completion

Examples:

“Summarize this document.” → A high-quality summary
“Translate this sentence into French.” → The translated sentence
“Classify this review.” → The correct sentiment category
“Generate Python code.” → Correct Python code

Benefits

Instruction tuning can improve a model’s ability to:

Follow natural-language instructions
Perform multiple tasks
Produce responses in a desired format
Understand task-specific prompts
Respond more consistently

7.1 Multitask Fine-Tuning

Multitask fine-tuning trains a model using examples for multiple tasks within one dataset.

The dataset might include examples for:

Summarization
Translation
Classification
Rating or review analysis
Code generation
Question answering

The result is an instruction-tuned model that can perform several tasks.

Benefits

Multitask fine-tuning can:

Improve generalization
Allow one model to support multiple use cases
Reduce the risk of specializing too narrowly
Help mitigate catastrophic forgetting

Requirement

Multitask fine-tuning generally requires a sufficiently large and diverse dataset containing examples for all target tasks.

8. Catastrophic Forgetting

Definition

Catastrophic forgetting occurs when fine-tuning on a narrow task causes the model to lose performance on capabilities it previously had.

This can happen because full fine-tuning modifies the original model weights.

Example

A general-purpose model is fine-tuned only on legal contract classification. It becomes better at contract classification but may become worse at:

Translation
General question answering
Summarization
Code generation
Other unrelated tasks

When it matters

Catastrophic forgetting is especially important when:

The model must remain general-purpose
Multiple tasks are supported by one model
Existing capabilities must be preserved
The fine-tuning dataset is narrow or unbalanced

It may matter less when:

The model is intended to perform only one task
General-purpose capabilities are not required
A dedicated task-specific model is preferred

Ways to reduce the risk

Use multitask fine-tuning
Include representative examples from other desired capabilities
Use PEFT techniques
Use a balanced and diverse dataset
Evaluate both the target task and general capabilities
Avoid excessively narrow or aggressive fine-tuning

Exam trap

Fine-tuning can improve a target task while making the model worse at other tasks. This is the definition of catastrophic forgetting.

9. Domain Adaptation Fine-Tuning

Definition

Domain adaptation fine-tuning adapts a pre-trained model to a specialized domain using domain-specific data.

Examples

Healthcare records
Legal contracts
Financial reports
Scientific research
Technical manuals
Internal company documentation
Customer support conversations

What it improves

Domain adaptation can help the model understand:

Industry jargon
Specialized terminology
Domain-specific writing styles
Organization-specific language
Specialized workflows
Technical concepts

AWS example

Amazon SageMaker JumpStart provides capabilities for fine-tuning certain foundation models, including text-generation models, with custom domain-specific datasets.

Exam tip

If the question asks which type of fine-tuning adapts a model to industry jargon or specialized technical terms, the answer is domain adaptation fine-tuning.

10. Reinforcement Learning from Human Feedback

Definition

Reinforcement learning from human feedback (RLHF) uses human preferences to align a model’s behavior with what people consider helpful, safe, accurate, or appropriate.

Human reviewers provide feedback about model responses. This feedback is used to fine-tune or optimize the model.

Typical goals

RLHF can help improve:

Helpfulness
Relevance
Safety
Instruction following
Tone
Response quality
Alignment with human preferences

Important distinction

RLHF is not simply ordinary supervised fine-tuning.

Ordinary fine-tuning uses expected labels or target outputs.
RLHF uses human preference or ranking feedback to guide optimization.

Example

Given two responses to the same prompt, human reviewers select the better response. These preference signals help the model learn which types of responses are preferred.

Exam trap

RLHF is used primarily for alignment with human preferences, not merely for adding factual knowledge to the model.

11. Preparing Data for Fine-Tuning

High-quality training data is one of the most important factors in fine-tuning performance.

A useful process is:

Collect data
Curate and clean the data
Remove or protect sensitive information
Label or structure the examples
Check quality and representativeness
Split into training, validation, and test datasets
Fine-tune the model
Evaluate results
Monitor for bias, harmful output, and degradation

11.1 Data Curation

Data curation involves selecting and organizing data suitable for model training.

Activities may include:

Removing duplicate examples
Removing irrelevant examples
Correcting errors
Removing malformed records
Standardizing formats
Removing low-quality responses
Detecting toxic or unsafe content
Removing data leakage
Removing personally identifiable information when appropriate
Ensuring the data is relevant to the target use case

Why it matters

Poor-quality data can result in:

Poor model responses
Incorrect facts
Biased predictions
Unsafe outputs
Inconsistent formatting
Overfitting
Reduced generalization

Key principle

The quality, relevance, and representativeness of the fine-tuning data are often more important than simply increasing the number of examples.

11.2 Data Governance

Data governance ensures that training data is used legally, securely, and responsibly.

Important considerations include:

Data ownership
Licensing and copyright
Privacy
Personally identifiable information
Confidential or proprietary data
Access controls
Encryption
Data retention
Auditability
Regulatory requirements
Data lineage
Human review and approval

Exam tip

Before fine-tuning with organizational data, consider whether the organization is permitted to use that data and whether sensitive information must be removed or protected.

11.3 Data Size

Fine-tuning generally requires much less data than pre-training, but the required amount depends on:

Task complexity
Number of classes
Domain complexity
Desired quality
Model size
Diversity of inputs
Quality of labels
Number of tasks
Degree of behavior change required

A small, high-quality dataset may be more useful than a large dataset containing noise, duplicates, or incorrect labels.

Multitask data

Multitask fine-tuning generally requires more data because the dataset must contain sufficient examples for each task.

11.4 Data Labeling

For supervised fine-tuning, data commonly consists of labeled input-output pairs.

Examples:

Prompt and ideal completion
Question and answer
Text and classification label
Document and summary
User request and desired assistant response

Labels should be:

Accurate
Consistent
Relevant
Complete
Unambiguous
Created according to documented guidelines

AWS service

Amazon SageMaker Ground Truth helps manage data-labeling workflows for machine learning datasets.

11.5 Representativeness

The data should represent the real-world inputs the model is expected to handle.

A representative dataset should cover:

Different user groups
Different writing styles
Different accents or languages, where applicable
Normal and unusual inputs
Relevant edge cases
Different document lengths
Different levels of difficulty
Different demographic groups
Expected variations in the production environment

Why representativeness matters

If the fine-tuning data does not represent production data, the model may perform well during testing but poorly in actual use.

11.6 Bias Detection

Training data can contain:

Imbalanced representation
Stereotypes
Labeling bias
Historical bias
Missing groups
Unequal data quality across groups

Amazon SageMaker Clarify can help analyze datasets and identify potential bias across facets such as:

Gender
Race
Age
Other protected or relevant groups

Exam tip

For detecting bias or imbalanced representation in training data, consider SageMaker Clarify.

12. Dataset Splits

After preparing the instruction dataset, divide it into separate subsets.

Training dataset

Used to update the model’s parameters or adapters.

Validation dataset

Used during training to measure performance and help with:

Hyperparameter selection
Model selection
Detecting overfitting
Comparing training configurations

Test dataset

Held out until the end and used for final evaluation.

Dataset	Purpose
Training	Train or update the model
Validation	Evaluate during development
Test	Final unbiased evaluation

Exam trap

Do not use the test dataset repeatedly during training. Doing so can cause information leakage and make the final evaluation unreliable.

13. Fine-Tuning Training Process

A simplified supervised fine-tuning process is:

Select a batch of prompt-completion pairs from the training dataset.
Pass the prompts to the model.
Generate model completions.
Compare generated completions with the expected labels.
Calculate the loss between the predicted and target token distributions.
Use the loss to calculate gradients.
Update the model weights or trainable adapter parameters.
Repeat across many batches and training iterations.
Evaluate on the validation dataset.
Perform final evaluation on the test dataset.

Token distributions

For language-model fine-tuning, the model predicts probabilities over possible next tokens. The training process compares the predicted token distribution with the target distribution and uses the difference to calculate loss.

14. AWS Services for Data Preparation

Amazon SageMaker Canvas

Use when:

Low-code or no-code preparation is desired
Users want to create data flows
Feature engineering and preprocessing are needed without extensive coding

Amazon EMR

Use when:

Data preparation needs to scale
Distributed processing is required
Frameworks such as Apache Spark or Apache Hive are appropriate

SageMaker Studio Classic provides integration with Amazon EMR.

AWS Glue interactive sessions

Use when:

Serverless data preparation is desired
Apache Spark-based processing is appropriate
Data must be aggregated, transformed, or prepared from multiple sources

JupyterLab in SageMaker Studio

Use when:

SQL or code-based preparation is needed
Data scientists want interactive notebooks
Custom data processing workflows are required

Amazon SageMaker Feature Store

Use when:

Features must be discovered and retrieved
Feature data needs centralized storage
Features should be stored in a standardized format
The organization needs a reusable feature repository

Amazon SageMaker Clarify

Use when:

Detecting bias
Analyzing imbalanced representation
Investigating labeling bias
Evaluating fairness across groups

Amazon SageMaker Ground Truth

Use when:

Human labeling is required
Labeling workflows need to be managed
Training data needs annotations

Amazon SageMaker JumpStart

Use when:

Starting with pre-trained models
Fine-tuning supported foundation models
Adapting models to custom datasets or domains

Amazon Bedrock model customization

Amazon Bedrock supports customization of certain foundation models, including continuous pre-training for selected Amazon Titan models using an organization’s own unlabeled data in a managed environment.

15. Evaluation and Validation

Generative AI models are non-deterministic. This means that the same or similar input may not always produce exactly the same output.

Therefore, evaluation should use:

Appropriate metrics
Benchmarks
Representative datasets
Holdout validation data
Holdout test data
Human evaluation where appropriate
Safety and harmful-output checks

Evaluation should measure more than accuracy

Depending on the use case, evaluate:

Correctness
Relevance
Completeness
Fluency
Factuality
Safety
Bias
Toxicity
Instruction following
Formatting
Latency and cost

Important exam point

A model can achieve good performance on a target metric and still produce harmful, biased, or unreliable outputs. Evaluation should include safety and responsible AI considerations.

16. Common Exam Comparisons

Pre-training versus continuous pre-training

Pre-training: Initial large-scale training of a foundation model.
Continuous pre-training: Additional self-supervised training of an existing model using new or domain-specific unlabeled data.

Continuous pre-training versus domain fine-tuning

Continuous pre-training: Usually uses unlabeled data and broadens knowledge.
Domain fine-tuning: Usually uses labeled examples and adapts the model to domain-specific tasks or language.

Fine-tuning versus RAG

Fine-tuning: Additional training that changes model parameters or representations.
RAG: Retrieves external information at inference time and supplies it to the model.

Full fine-tuning versus PEFT

Full fine-tuning: Updates all or most model parameters.
PEFT: Freezes most parameters and updates a small number of additional parameters.

LoRA versus ReFT

LoRA: Adds trainable low-rank matrices associated with model layers.
ReFT: Learns interventions on hidden representations.

Fine-tuning versus RLHF

Fine-tuning: Uses labeled examples to improve task performance.
RLHF: Uses human preference feedback to align model responses with human preferences.

Fine-tuning versus distillation

Fine-tuning: Adapts a model to data, a task, or a domain.
Distillation: Trains a smaller student model to imitate a larger teacher model.

17. Exam Tips and Traps

Tip 1: Look for the type of data

Data described in question	Likely technique
Huge amounts of unlabeled data	Pre-training
New unlabeled domain data	Continuous pre-training
Labeled prompt-completion pairs	Supervised fine-tuning
Human preferences or rankings	RLHF
Large teacher model and small student model	Distillation

Tip 2: Look for the objective

Objective	Likely answer
Learn broad language capabilities	Pre-training
Learn specialized domain language	Domain adaptation
Follow natural-language instructions	Instruction tuning
Support multiple tasks	Multitask fine-tuning
Reduce training memory and cost	PEFT or LoRA
Preserve most base-model parameters	PEFT or LoRA
Align responses with people’s preferences	RLHF
Make a smaller and faster model	Distillation

Tip 3: Remember catastrophic forgetting

Fine-tuning is not always an improvement across every capability. Narrow fine-tuning can improve one task while degrading unrelated capabilities.

This effect is called catastrophic forgetting.

Tip 4: Do not confuse training and inference

Training and fine-tuning update weights, adapters, or representations.
Prompt engineering changes the input prompt but does not train the model.
RAG provides retrieved information at inference time but does not necessarily change model weights.

Tip 5: Match AWS services to their purpose

Canvas: Low-code data preparation
EMR/Spark/Hive: Large-scale data processing
Glue interactive sessions: Serverless Spark-based processing
Feature Store: Centralized feature storage and discovery
Clarify: Bias detection
Ground Truth: Data labeling
JumpStart: Pre-trained models and fine-tuning
Bedrock customization: Managed model customization, including selected continuous pre-training capabilities

18. Quick Revision Summary

Pre-training uses massive amounts of generally unlabeled data and self-supervised learning to create broad foundation-model capabilities.
Fine-tuning uses task-specific or domain-specific examples to improve performance for a particular purpose.
Continuous pre-training continues self-supervised training using new, usually unlabeled data to broaden or refresh model knowledge.
Full fine-tuning updates most or all model parameters and requires substantial compute and memory.
PEFT freezes most of the base model and trains a small number of task-specific parameters.
LoRA is a popular PEFT technique that uses trainable low-rank matrices.
ReFT learns interventions on hidden representations rather than directly updating all model weights.
Instruction tuning trains a model to follow instructions.
Multitask fine-tuning trains one model on examples from multiple tasks.
Domain adaptation teaches the model industry-specific terminology and behavior.
RLHF aligns model behavior with human preferences.
Distillation transfers knowledge from a larger teacher model to a smaller student model.
Catastrophic forgetting occurs when specialization causes the model to lose previous capabilities.
Fine-tuning data should be high quality, representative, properly labeled, governed, and divided into training, validation, and test splits.
SageMaker Clarify helps detect bias.
SageMaker Ground Truth supports labeling workflows.
SageMaker Canvas, EMR, Glue, JupyterLab, and Feature Store support different data preparation requirements.