Domain 2: ML Model and Foundation Model (FM) Development
Question: Amazon Bedrock Benefits for Generative AI Model Development
Category: MLA – ML Model and Foundation Model (FM) Development
Review Bucket:
- [ ] Bucket A (Term Mapping)
- [X] Bucket B (Missed Constraint)
- [ ] Bucket C (Knowledge Gap)
A large retail company is exploring generative AI to enhance its customer service chatbot. The company aims to fine-tune models from third-party services to enhance the chatbot’s comprehension of business-specific customer inquiries.
Which benefits of Amazon Bedrock will help the retail company meet its requirements? (Select TWO.)
- A) Offers distinct APIs for accessing different foundation models (FMs).
- B) Amazon Bedrock enables private customization of foundation models (FMs).
- C) Ensures that prompts and responses remain completely private and secure.
- D) Offers a wide range of pre-trained foundation models (FMs) from multiple providers.
- E) Amazon Bedrock requires customers to handle infrastructure provisioning.
Click to reveal Answer & Explanation
Correct Answer: B and D
Correct Option Analysis
- B is Correct: Amazon Bedrock enables private customization of foundation models, allowing the retail company to fine-tune third-party models using its own business-specific data. This directly supports improving the chatbot’s comprehension of customer inquiries.
- D is Correct: Amazon Bedrock provides access to a wide range of pre-trained foundation models from multiple providers. This helps the company evaluate and use third-party generative AI models for its chatbot.
Distractor Analysis
- A is Incorrect: Amazon Bedrock provides a unified, serverless experience for accessing multiple foundation models. It does not require learning distinct APIs for each model provider.
- C is Incorrect: While Amazon Bedrock offers strong privacy and security controls, the statement that it ensures prompts and responses remain completely private and secure is overly absolute. More importantly, the primary requirements here are multi-provider access and private fine-tuning, not security alone.
- E is Incorrect: Amazon Bedrock is a managed service and does not require customers to provision or manage underlying infrastructure.
Key Takeaway: Amazon Bedrock helps teams adopt generative AI by providing access to multiple third-party FMs and by supporting private customization/fine-tuning of those models using business-specific data.
self-notes
privacy not mentioned in question -> C is wrong
Question: Topic Modeling of Unstructured Text Feedback
Category: MLA – ML Model and Foundation Model (FM) Development
Review Bucket:
- [ ] Bucket A (Term Mapping)
- [ ] Bucket B (Missed Constraint)
- [X] Bucket C (Knowledge Gap)
A financial company has accumulated a large volume of client feedback, reviews, and comments over the years. The company wants to analyze these data to identify common topics that frequently appear in the feedback to understand client sentiment better and improve their offerings. The specific topics are not predetermined, and the goal is to discover them automatically from the text data.
Which built-in Amazon SageMaker AI algorithm is the most appropriate for this task?
- A) Sequence-to-Sequence Algorithm
- B) Latent Dirichlet Allocation (LDA) Algorithm
- C) BlazingText algorithm in Text Classification Mode
- D) Text Classification – TensorFlow algorithm
Click to reveal Answer & Explanation
Correct Answer: B
Correct Option Analysis
- B is Correct: Latent Dirichlet Allocation (LDA) is an unsupervised topic modeling algorithm. It automatically discovers abstract topics within a corpus of text documents without requiring predefined labels or categories. This matches the company’s need to identify common topics from client feedback when the specific topics are not known in advance.
Distractor Analysis
- A is Incorrect: Sequence-to-sequence is a supervised learning algorithm designed for tasks where the input and output are both sequences, such as machine translation, text summarization, or question answering. It is not used for unsupervised topic discovery.
- C is Incorrect: BlazingText in Text Classification Mode is a supervised learning algorithm that requires labeled training data to classify text into predefined categories. It cannot discover unknown topics automatically without prior labels.
- D is Incorrect: Text Classification – TensorFlow is also a supervised algorithm that categorizes text into known classes. It does not perform unsupervised topic modeling and would not be suitable for discovering topics that were not predetermined.
Key Takeaway: For automatically discovering unknown topics or themes from unlabeled text, use an unsupervised topic modeling algorithm such as Latent Dirichlet Allocation (LDA). Supervised text classification algorithms require predefined labeled categories.
self-notes
Unknown topic to be discovered -> Unsupervised -> LDA
Question: IPv4 Anomaly Detection with SageMaker Built-In Algorithms
Category: MLA – ML Model and Foundation Model (FM) Development
Review Bucket: [ ] Bucket A (Term Mapping) | [x] Bucket B (Missed Constraint) | [ ] Bucket C (Knowledge Gap)
A cybersecurity company monitors network traffic for potential security threats. The team has access to a large dataset containing historical network traffic logs. The goal is to identify IPv4 addresses that deviate significantly from normal patterns.
Which of the following Amazon SageMaker AI built-in algorithms would be the most suitable for the given requirement?
- A) Random Cut Forest (RCF) Algorithm
- B) Principal Component Analysis (PCA) Algorithm
- C) Amazon SageMaker IP Insights
- D) SageMaker AI K-Means Cluster Algorithm
Click to reveal Answer & Explanation
Correct Answer: C
Correct Option Analysis
- C is Correct: Amazon SageMaker IP Insights is an unsupervised learning algorithm specifically designed to learn usage patterns for IPv4 addresses. It uses statistical modeling and neural networks to capture associations between IPv4 addresses and entities (such as user accounts or devices) to calculate an anomaly score for suspicious network behavior.
Distractor Analysis
- A is Incorrect (Distractor Trap): While Random Cut Forest (RCF) is an unsupervised algorithm for anomaly detection, it is designed for continuous scalar metric streams (e.g., CPU spikes, numerical sensor data). It does not natively encode IPv4 subnetwork semantics or entity associations.
- B is Incorrect: Principal Component Analysis (PCA) is an unsupervised dimensionality reduction algorithm used to project high-dimensional data onto fewer principal component axes, not an anomaly detector.
- D is Incorrect: K-Means is an unsupervised clustering algorithm used to group data points into \(k\) distinct clusters based on feature similarity.
Key Takeaway: Choose Amazon SageMaker IP Insights for IPv4 network traffic and access log anomaly detection. Reserve Random Cut Forest (RCF) for continuous numerical scalar metrics and time-series spikes.
self-notes
IP address is special
Question: Selecting an Inference Instance Type with Minimal Benchmarking
Category: MLA – Operating, Monitoring, and Securing ML and AI Solutions
Review Bucket:
- [ ] Bucket A (Term Mapping)
- [ ] Bucket B (Missed Constraint)
- [X] Bucket C (Knowledge Gap)
A company has trained a new recommendation model and plans to deploy it to an existing application that experiences predictable traffic spikes during peak viewing hours. The model must run on a persistent Amazon SageMaker endpoint with low response latency, and the ML engineering team needs to determine the most appropriate instance type by testing the model against a workload pattern that reflects production traffic while minimizing manual benchmarking effort.
Which approach should the engineering team use to meet these requirements with minimal operational complexity?
- A) Select an instance type by analyzing CPU, memory, and GPU utilization from a SageMaker Profiler report.
- B) Deploy the model to a SageMaker AI real-time endpoint, configure target tracking scaling based on
SageMakerVariantInvocationsPerInstance, and load test the endpoint to determine its latency characteristics. - C) Configure a SageMaker AI shadow test to evaluate a candidate instance type with replicated production requests and compare its latency against the current endpoint.
- D) Run a SageMaker Inference Recommender endpoint recommendation job with a custom traffic pattern and production latency requirements.
Click to reveal Answer & Explanation
Correct Answer: D
Correct Option Analysis
- D is Correct: SageMaker Inference Recommender can run an endpoint recommendation job using a custom traffic pattern and specified production latency requirements. It automates instance type benchmarking and selection without requiring the team to build manual load tests or manage additional infrastructure.
Distractor Analysis
- A is Incorrect: A SageMaker Profiler report shows resource utilization from an existing endpoint or training job. It does not test a candidate instance type against a production-like traffic pattern, and it does not automatically recommend an instance type.
- B is Incorrect: Deploying to a real-time endpoint and configuring target tracking scaling measures scaling behavior, but it requires manually choosing an instance type and building a load test. This approach introduces more manual benchmarking effort than required.
- C is Incorrect: SageMaker shadow tests replicate production traffic to compare a new variant against a current endpoint. This requires an existing production endpoint, adds operational setup, and focuses on validating a model variant rather than selecting an instance type from a custom traffic pattern with minimal effort.
Key Takeaway: Use SageMaker Inference Recommender to automate instance type selection and load testing based on custom traffic patterns and latency requirements.
self-notes
extra configue based on target tracking scaling = manual effort
AWS Practice Question: Mitigating Overfitting in Deep Neural Networks
Category: MLA – ML Model and Foundation Model Development
Review Bucket: [ ] Bucket A (Term Mapping) | [x] Bucket B (Missed Constraint) | [ ] Bucket C (Knowledge Gap)
During the training process of a deep neural network, a data scientist observes that the training loss decreases with each epoch. After many epochs, the data scientist observes that the validation loss increases.
What actions can the data scientist implement to mitigate this issue? (Select TWO.)
- A) Add more layers to the network.
- B) Increase the number of epochs.
- C) Implement early stopping.
- D) Use the dropout technique.
- E) Add more neurons to each layer.
Click to reveal Answer & Explanation
Correct Answers: C and D
#### Correct Option Analysis
* **C is Correct:** **Early stopping** monitors validation loss during training and halts execution as soon as validation loss begins to increase, preventing the model from over-optimizing on the training set.
* **D is Correct:** **Dropout** is a regularization technique that randomly deactivates a fraction of neurons during training, preventing co-adaptation of features and forcing the network to learn robust, generalized representations.
#### Distractor Analysis
* **A & E are Incorrect:** Adding more layers or neurons increases model capacity/complexity, which exacerbates overfitting rather than mitigating it.
* **B is Incorrect:** Increasing epochs allows the model to continue training further past the point of divergence, leading to worse validation performance.
> **Key Decision Rules:**
> * **Validation Loss Increases while Training Loss Decreases $\rightarrow$ Overfitting.**
> * **Overfitting Remedies:** **Early Stopping**, **Dropout**, **L1/L2 Regularization**, **Data Augmentation**, or **Reducing Model Capacity**.
self-notes
Validation loss going up = Overfitting. Do NOT add layers/capacity (makes it worse!). Use Early Stopping & Dropout.
Practice Question: Fine-Tuning vs. RAG for Domain Terminology & Abbreviations
Category: MLA – ML Model and Foundation Model Development
Review Bucket: [x] Bucket A (Term Mapping) | [ ] Bucket B (Missed Constraint) | [ ] Bucket C (Knowledge Gap)
An industrial company is exploring the use of generative AI to retrieve information from maintenance notes. The company currently uses a text-to-text foundation model (FM) on Amazon Bedrock. The company wants to optimize the model. The company wants to help the model better understand the terminology and abbreviations that maintenance technicians use.
What is the MOST cost-effective solution to help the model understand the company's data?
- A) Replace the model with a new model that has more parameters.
- B) Pre-train a new FM by using the Amazon SageMaker AI model parallelism library.
- C) Create an Amazon Bedrock Knowledge Base.
- D) Customize the model by using fine-tuning.
Click to reveal Answer & Explanation
Correct Answer: D
#### Correct Option Analysis
* **D is Correct:** **Fine-tuning** adapts a foundation model's internal weights using domain-specific labeled examples. It is the most effective and cost-efficient method to teach a model **specialized jargon, industry terminology, internal acronyms, and abbreviations**.
#### Distractor Analysis
* **C is Incorrect:** **Amazon Bedrock Knowledge Bases (RAG)** provides external document retrieval at query time. While RAG helps models access new or dynamic facts, it does not adapt the base model's internal language representation to understand specialized technical shorthand or jargon natively.
* **A is Incorrect:** Selecting a larger general-purpose model does not guarantee understanding of proprietary or company-specific abbreviations.
* **B is Incorrect:** Full pre-training from scratch is extremely expensive and resource-intensive compared to fine-tuning an existing base model on Bedrock.
> **Key Decision Rules:**
> * **Teach Domain Jargon, Terminology, Abbreviations, Tone, or Style:** Choose **Fine-Tuning**.
> * **Provide Access to Dynamic, Changing Facts & Internal Documents:** Choose **Bedrock Knowledge Bases (RAG)**.
self-notes
Understanding jargon/acronyms/style = Fine-Tuning (modifies weights). RAG = Retrieving external documents.