Skill 2.1.2: Identify fine-tuning strategies for pre-trained FMs to meet business needs
Task 2.1: ML Model and Foundation Model (FM) Development - Choose appropriate modeling approaches for ML and AI solutions
Core Decision: Where Does the Need Live?
| Business Need | Best-Fit Approach | Why / AWS Path |
|---|---|---|
| Fixed logic (thresholds, reorder rules) | Rules / code | No model; cheaper, faster, auditable |
| Standard pre-defined task (OCR, speech, vision, NLP) | Managed AWS AI service | Textract, Rekognition, Transcribe, Comprehend |
| Dynamic facts, citations, fresh docs | RAG | Bedrock Knowledge Bases; external vector index; no weight change |
| Consistent behavior, tone, format, domain jargon | Fine-tuning | Writes behavior into weights; custom model |
| Both fresh knowledge and behavior | RAG + fine-tuning | RAG supplies facts; fine-tuning supplies style/format |
Exam Gotcha: Fine-tuning is for behavior, format, tone, and domain adaptation — not for reliably adding new or frequently changing factual knowledge. Use RAG for knowledge freshness and citations.
Fine-Tuning Strategy Matrix
| Strategy | Best For | Data Required | AWS Path | Key Trade-offs |
|---|---|---|---|---|
| Supervised Fine-Tuning (SFT) | Domain terminology, output format, instruction following, task specialization | Labeled prompt-response pairs (JSONL, S3), validation holdout | Bedrock fine-tuning, SageMaker JumpStart | Needs quality data, training cost, versioning; risk of catastrophic forgetting |
| Continued Pre-Training (CPT) | Domain adaptation on large unlabeled corpus (legal, medical, financial) | Unlabeled domain text | Bedrock continued pre-training, SageMaker | Expensive; can degrade general ability; usually followed by SFT |
| Reinforcement Fine-Tuning / RLHF | Aligning to human preferences, safety, complex reasoning, reward optimization | Preference pairs (chosen/rejected) or reward function | Bedrock RFT, SageMaker RLHF | Complex reward design; costly; needs careful evaluation |
| Distillation | Reducing cost/latency of a large teacher model while preserving quality | Prompts; teacher generates responses | Bedrock model distillation | Student may lose quality; no human labels needed |
| PEFT / LoRA / QLoRA | Limited compute, faster iteration, lower forgetting risk | Labeled data | SageMaker JumpStart, custom training | Adapter weights; may not match full fine-tuning quality |
| Full Fine-Tuning | Maximum adaptation when PEFT insufficient | Large labeled dataset | SageMaker AI | Highest cost; highest catastrophic forgetting risk |
Order of Operations: Prompt engineering → RAG → Fine-tuning. Fine-tune only when evaluations prove prompting + retrieval cannot close a persistent gap.
AWS Implementation Paths
| Service | Use When | Customization Options | Inference / Cost Notes |
|---|---|---|---|
| Amazon Bedrock | Managed FMs, fastest path, no infrastructure | Fine-tuning, continued pre-training, reinforcement fine-tuning, distillation | Custom models require Provisioned Throughput; ongoing cost, not per-token |
| SageMaker JumpStart | Open-source FMs, PEFT/LoRA/QLoRA, solution templates | Fine-tuning, PEFT, full fine-tuning | Deploy to SageMaker endpoints; more control |
| SageMaker AI | Full control, custom containers, distributed training, HPO | Any fine-tuning method, custom scripts, pipelines | Manage endpoints, scaling, spot training, model registry |
Data & Configuration Essentials
- SFT data: labeled prompt-response pairs, JSONL, S3; high quality > large quantity; consistent formatting.
- CPT data: unlabeled domain corpus; mix general data to reduce catastrophic forgetting.
- RFT/RLHF data: preference pairs or reward function; define reward carefully.
- Distillation data: prompts only; teacher model generates responses for student training.
- Hyperparameters: epochs, batch size, learning rate, warmup, early stopping; monitor validation loss.
- Evaluation: compare baseline vs fine-tuned using automatic/human evaluation; task metrics (accuracy, ROUGE, BERTScore, human preference).
Catastrophic Forgetting: Narrow fine-tuning can erase general capabilities. Mitigate with mixed data, PEFT, or lower learning rate/epochs.
Security, Compliance & Cost
- IAM roles: Bedrock/SageMaker roles need S3, KMS, and model access permissions.
- KMS: encrypt training data, model artifacts, and custom models at rest; TLS in transit.
- VPC endpoints (PrivateLink): private access to Bedrock/SageMaker without public internet.
- Data privacy: Bedrock does not train base models on customer data; custom models are private to your account.
- Cost drivers: training GPU hours + inference. Bedrock custom models = Provisioned Throughput. SageMaker = real-time, serverless, async, or batch endpoints.
- Latency: distillation and smaller models reduce latency; PEFT does not reduce inference latency unless deployed as a smaller model.
Anti-Patterns & Exam Gotchas
Prompt first: Fine-tuning before prompt engineering is a costly anti-pattern. RAG vs fine-tuning: RAG = external knowledge, citations, freshness. Fine-tuning = internal behavior, style, format. Continued pre-training uses unlabeled data; SFT uses labeled prompt-response pairs. Distillation uses teacher-generated responses, not human labels. Bedrock custom models require Provisioned Throughput for inference — plan for ongoing cost. Fine-tuning for fixed logic is wrong — use rules. Always evaluate baseline vs fine-tuned before production rollout.