Skill 1.1.1: Extract data from data sources

Question 1 - SageMaker Deployment Guardrails Canary Rollout with Auto-Rollback

A financial services company is deploying an updated fraud detection model to their production Amazon SageMaker endpoint. The compliance team requires that the new model version is exposed to only 10% of live traffic initially, and if the model's error rate exceeds a threshold, traffic must automatically revert to the previous version without any manual intervention. The MLOps team wants to implement this using SageMaker's native deployment capabilities to minimize operational overhead.

Which configuration should the MLOps team implement to meet these requirements? (Select ONE)

A) Configure an AWS CodeDeploy deployment group with a canary deployment strategy, linking a CloudWatch alarm on the endpoint's Invocation5XXErrors metric to trigger automatic rollback.
B) Create a SageMaker endpoint with Deployment Guardrails using a canary traffic shift of 10%, specify a baking period, and attach a CloudWatch alarm on Invocation5XXErrors to the AutoRollbackConfig.
C) Set up Amazon EventBridge to monitor the endpoint's error metrics, and invoke an AWS Lambda function that manually updates the production variant weights to 0% if errors rise.
D) Use SageMaker Pipelines to register the new model, then create a second endpoint for the new version and update the application to randomly route 10% of requests to it.
E) Enable SageMaker Model Monitor on the endpoint with an error threshold, which automatically shifts traffic back to the previous model variant when violations occur.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

B is Correct: Amazon SageMaker Deployment Guardrails natively support blue/green deployments with canary traffic shifting. The AutoRollbackConfig with a linked CloudWatch alarm on Invocation5XXErrors watches for error rate violations during the baking period and automatically reverts traffic to the blue (old) fleet if the alarm trips, requiring no manual intervention.

Distractor Analysis

A is Incorrect: AWS CodeDeploy is designed for deploying applications to EC2, Lambda, or ECS, not for managing SageMaker endpoint traffic shifting or model version rollbacks. SageMaker endpoints use their own native deployment guardrails.
C is Incorrect: While EventBridge and Lambda could technically implement a custom rollback mechanism, this introduces unnecessary operational overhead (custom code, monitoring, and error handling) compared to SageMaker's native guardrails feature. It also violates the "minimize operational overhead" constraint.
D is Incorrect: Creating a second endpoint and implementing client-side routing logic shifts the burden of traffic management to the application layer, increasing complexity and introducing potential single points of failure. SageMaker's native blue/green deployment handles this at the infrastructure level.
E is Incorrect: SageMaker Model Monitor detects data and model quality drift (data quality, model quality, bias, and feature attribution), but it does not automatically rollback traffic to a previous model version. It generates alerts and reports that require external automation to act upon.

Key Takeaway: For automatic rollback on SageMaker endpoints, always use Deployment Guardrails with canary/linear traffic shifting and a CloudWatch alarm configured in AutoRollbackConfig—without an alarm, the rollout completes even if the model is failing.

Question 2 - EventBridge-Triggered SageMaker Pipeline for Irregular Data Arrival

An e-commerce company's recommendation team has a SageMaker Pipeline that preprocesses clickstream data, trains a recommendation model, evaluates it, and registers it in the SageMaker Model Registry. New labeled clickstream data arrives in an S3 bucket at irregular intervals—sometimes multiple times a day and sometimes once a week. The team wants the pipeline to run automatically whenever new data lands, without wasting computational resources during idle periods.

Which retraining trigger mechanism should the team configure? (Select ONE)

A) A fixed daily Amazon EventBridge scheduled rule (cron expression) that invokes StartPipelineExecution at midnight.
B) An S3 Event Notification on s3:ObjectCreated:* that sends the event to Amazon EventBridge, which triggers the SageMaker Pipeline execution via a rule.
C) A CloudWatch Alarm on the S3 bucket's NumberOfObjects metric that invokes the pipeline when the count exceeds a threshold.
D) A SageMaker Model Monitor schedule that detects new data and automatically triggers model retraining in response.
E) A manual approval step in the pipeline's model registry stage that requires a human to start each retraining run.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

B is Correct: For irregular data arrival, an event-driven trigger is the most cost-effective solution. S3 Event Notifications emit a ObjectCreated event when new data lands, and Amazon EventBridge captures this event and routes it to a rule that invokes StartPipelineExecution, ensuring the pipeline runs precisely when needed without idle compute costs.

Distractor Analysis

A is Incorrect: A fixed daily schedule wastes computational resources when no new data has arrived (running an empty pipeline) or introduces delays (waiting until midnight when data arrived at 9 AM). Schedule-based triggers are best for predictable, periodic workloads.
C is Incorrect: CloudWatch Alarms on NumberOfObjects are threshold-based and would require careful tuning (e.g., "run when 100 new objects appear"). They don't provide a clean one-to-one mapping between data arrival and pipeline execution, and multiple data arrivals between alarm checks could be missed.
D is Incorrect: SageMaker Model Monitor is designed for monitoring deployed endpoints for data drift, model quality drift, and bias detection. It does not provide a mechanism to trigger retraining pipelines based on new data arrival in S3.
E is Incorrect: Manual approval gates in the model registry control whether a model proceeds to deployment, not when training starts. Requiring manual startup defeats automation and introduces operational overhead and potential delays.

Key Takeaway: For irregular data arrival, use event-driven triggers (S3 Event Notifications → EventBridge → SageMaker Pipeline) rather than schedules; for predictable cadence, scheduled rules are simpler and appropriate.

Question 3 - Generative AI Pipeline Testing with Bedrock Prompt Management

A media company is building a customer support chatbot using Amazon Bedrock with a Claude 3 foundation model. The team is changing the system prompt to improve response tone and accuracy. They want to ensure in their CI/CD pipeline that the new prompt version doesn't degrade response quality before deploying to production. Responses from the generative model vary between runs even for identical inputs, so exact string matching is not feasible.

Which TWO testing strategies should the team implement in their pipeline? (Select TWO)

A) Use Amazon Bedrock Prompt Management to create and compare prompt variants as versioned resources, then use Amazon Bedrock Model Evaluation to score responses against criteria like helpfulness, faithfulness, and toxicity.
B) Assert that each response from the new prompt exactly matches a stored expected string in the unit test suite.
C) Run a fixed "golden prompt" dataset through both the current and candidate prompt versions, score responses using an LLM-as-judge evaluator, and block deployment if the aggregate score drops below the current version's baseline.
D) Run only the existing unit tests for the application code that wraps the Bedrock API calls, since prompt changes don't affect code behavior.
E) Deploy the prompt change with a canary shift and monitor endpoint latency using CloudWatch to detect quality regression.

Click to reveal Answer & Explanation

Correct Answers: A and C

Correct Options Analysis

A is Correct: Amazon Bedrock Prompt Management natively supports versioning prompts with variables and comparing variants. Amazon Bedrock Model Evaluation can assess responses programmatically using built-in metrics (helpfulness, correctness, faithfulness, toxicity) or custom metrics, making it ideal for automated generative AI testing in CI/CD pipelines.
C is Correct: For non-deterministic generative models, automated testing requires running a representative set of prompts (golden dataset) through both versions and comparing aggregate quality scores. An LLM-as-judge evaluator assesses semantic quality rather than exact matches, and a quality gate blocks deployment if the new version scores below baseline.

Distractor Analysis

B is Incorrect: Generative AI responses naturally vary between runs due to sampling and model stochasticity. Exact string matching will cause false negatives and flaky pipeline builds, making it unsuitable for evaluating semantic quality.
D is Incorrect: Application unit tests verify code correctness (API routing, data structures, error handling) but cannot detect quality degradation introduced by prompt changes. Prompt changes alter model behavior as significantly as weight changes and require dedicated evaluation.
E is Incorrect: Endpoint latency monitoring checks infrastructure responsiveness and request processing speed, not the quality, accuracy, or tone of generated responses. A prompt could produce terrible responses quickly, and latency alarms would never catch that.

Key Takeaway: Generative AI testing requires evaluation-based gates (fixed prompt set + scored comparison against baseline), not exact-match assertions; version prompts through Bedrock Prompt Management so changes are testable and auditable.

Question 4 - Model Registry Lineage and Auditability

During a security incident investigation, a healthcare analytics company must determine exactly which ML model version served predictions to a specific group of patients last month, and trace which training run, dataset, and hyperparameters produced that model. The team currently stores model artifacts in S3 and manually tracks versions in a spreadsheet.

Which combination of AWS services should the team adopt to make this audit traceable and repeatable? (Select ONE)

A) Use SageMaker Model Registry to catalog model versions within a model group, recording lineage (training job, dataset URI, hyperparameters, metrics), and use SageMaker Pipelines to automatically register trained models as new versions.
B) Store the model artifact S3 URI in a DynamoDB table along with a manually entered version number and the date it was deployed.
C) Use AWS CodeCommit to store model artifacts as Git LFS objects, using Git commit history to track which version was deployed.
D) Tag the endpoint with a manually updated description indicating the current model version and training date.
E) Use MLflow on SageMaker for experiment tracking only, without registering models into a model registry.

Click to reveal Answer & Explanation

Correct Answer: A

Correct Option Analysis

A is Correct: SageMaker Model Registry provides immutable model version records with automatic lineage tracking—each registered model version links back to its exact training job, input data, hyperparameters, and evaluation metrics. SageMaker Pipelines can automate the entire workflow from training through registration, ensuring every production candidate has complete, machine-readable provenance.

Distractor Analysis

B is Incorrect: Manual version tracking in DynamoDB is error-prone, lacks automatic lineage to training jobs and datasets, and can't scale as model iterations increase. It violates the "repeatable and audit-proof" requirement because humans can forget to update entries or enter incorrect data.
C is Incorrect: Git and Git LFS are designed for source code and binary files, not for model artifact lineage. Git history shows when files changed but cannot link a model to its training data, hyperparameters, or metric scores automatically.
D is Incorrect: Endpoint tags with manually updated descriptions are not a version control system. They only reflect the current state at the endpoint level and provide no information about training lineage, previous versions, or approval workflows.
E is Incorrect: While MLflow on SageMaker tracks experiments (parameters, metrics, runs), using it without the model registry component means there's no formal catalog of approved model versions with approval status that a deployment pipeline can read.

Key Takeaway: Model Registry with automatic lineage tracking (via Pipelines or MLflow) is the single source of truth for model versioning—every artifact that changes behavior (weights, prompts, agents) needs a version and its full lineage.

Question 5 - RAG Knowledge Base Incremental Ingestion with Bedrock

A legal research platform uses Amazon Bedrock Knowledge Bases with Amazon OpenSearch Serverless to power a retrieval-augmented generation (RAG) system over a corpus of 50,000 legal documents. The content team frequently updates a handful of case files each week. The platform's operations team wants these updates reflected in the vector index within minutes, but wants to avoid re-embedding unchanged documents to minimize embedding model costs and processing time.

Which ingestion approach should the operations team configure? (Select ONE)

A) Schedule a full re-index of all 50,000 documents through the Bedrock Knowledge Base ingestion pipeline every hour.
B) Configure an S3 Event Notification on document changes to trigger an incremental Bedrock Knowledge Base ingestion job (StartIngestionJob with INGESTION_STRATEGY: RESYNC or incremental sync).
C) Copy the existing OpenSearch Serverless index to a new one, and route queries to the new index after each document update.
D) Disable embeddings for unchanged documents and manually delete changed documents from the vector index using OpenSearch Serverless API calls.
E) Replace the OpenSearch Serverless vector store with Amazon Aurora PostgreSQL with pgvector to enable faster incremental updates.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

B is Correct: An event-driven trigger on S3 document changes (adds, modifications, deletions) starts a Bedrock Knowledge Base incremental ingestion (RESYNC) job. This job identifies only new, changed, or deleted documents using metadata and content hashing, then processes only those for parsing, chunking, embedding, and indexing—minimizing costs and time while keeping the index current.

Distractor Analysis

A is Incorrect: Full re-indexing every hour reprocesses all 50,000 documents even when only a few changed, wasting massive embedding model API costs and processing time. Incremental syncs are designed specifically to avoid this inefficiency.
C is Incorrect: Copying an index creates a snapshot but does not process document updates. The copied index would have stale vectors for modified documents and still contain deleted documents. Index replication is a backup strategy, not an update strategy.
D is Incorrect: Manually deleting documents via OpenSearch API bypasses Bedrock Knowledge Base's managed ingestion pipeline, requiring custom orchestration code and risking inconsistencies between the S3 data source and the vector index. It also doesn't handle changed documents—only deletions.
E is Incorrect: Switching vector stores (infrastructure migration) does not solve the ingestion efficiency problem—any vector store still requires processing changed documents. Aurora with pgvector also introduces additional operational overhead compared to serverless OpenSearch.

Key Takeaway: For RAG systems with frequent minor updates, use Bedrock Knowledge Base incremental ingestion (RESYNC) triggered by S3 events—it processes only changed documents; full re-indexing is only needed when changing embedding models, chunking strategies, or index dimensions.

Question 6 - CodeConnections for External Git Repository in CodePipeline

A startup's MLOps team uses AWS CodePipeline for their SageMaker model deployment pipeline. The source code repository is hosted on GitHub (outside AWS), and the security team mandates that no long-lived credentials (Personal Access Tokens) can be stored in the pipeline definition, CloudFormation templates, or CodeBuild environment variables. The pipeline must automatically trigger when a new commit is pushed to the main branch.

Which service should the team use to connect the external GitHub repository to CodePipeline? (Select ONE)

A) Store a GitHub Personal Access Token (PAT) in AWS Secrets Manager and configure CodePipeline to retrieve it during the source stage.
B) Create an AWS CodeConnections resource using an OAuth or GitHub App-based handshake, and reference it in the CodePipeline source stage.
C) Migrate the Git repository to AWS CodeCommit and configure CodePipeline to use it as the source.
D) Create a scheduled AWS Lambda function that periodically polls the GitHub repository for changes and uploads new code to S3.
E) Use the AWS CodeBuild GitHub webhook integration to trigger builds directly, bypassing CodePipeline for the source stage.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

B is Correct: AWS CodeConnections (formerly CodeStar Connections) provides a managed, secure connection between AWS services and third-party Git repositories (GitHub, GitLab, Bitbucket). It uses OAuth or GitHub App-based authentication with a one-time handshake, eliminating the need to store long-lived credentials. CodePipeline can reference this connection directly in the source stage and detect commits automatically.

Distractor Analysis

A is Incorrect: Storing a PAT in Secrets Manager still involves managing and potentially rotating a long-lived credential. CodeConnections eliminates the need for PATs entirely, providing a more secure, managed OAuth-based connection.
C is Incorrect: While CodeCommit is a valid AWS-native Git repository, migrating the repository changes the team's hosting environment and may not be feasible or desirable. The requirement is to connect to an external repository, not migrate it.
D is Incorrect: Polling-based approaches introduce latency, require custom Lambda code and scheduling, and still require storing credentials in the Lambda function. CodeConnections provides native webhook-based triggers.
E is Incorrect: CodeBuild's webhook integration can trigger builds directly but bypasses the orchestration and stage management that CodePipeline provides, and it still requires credential configuration. The requirement explicitly mentions using CodePipeline.

Key Takeaway: Use AWS CodeConnections for secure, managed integration between CodePipeline and third-party Git repositories—it eliminates the need to store long-lived credentials in pipeline configurations.

Question 7 - Model Registry Approval Gate with Step Functions Integration

A pharmaceutical company's ML pipeline automatically retrains a drug efficacy prediction model weekly. The compliance team requires that no model version can be deployed to production without explicit human approval from both the ML team lead and the compliance officer. The deployment must be fully auditable, with the approval decision and approver identity recorded.

Which architecture satisfies these requirements using AWS services? (Select ONE)

A) Configure the SageMaker Pipeline's RegisterModel step to automatically transition approved models to the production stage, and send an email notification after deployment.
B) Use SageMaker Model Registry with model version statuses (PendingManualApproval → Approved), and use SageMaker Pipelines or Step Functions to gate execution on registration, where approvers update status via the AWS console or SDK.
C) Store model approval status in a DynamoDB table, have approvers toggle a boolean field manually, and have the deployment script check this field before deploying.
D) Write a custom approval workflow using SageMaker Ground Truth with a human review task, and use the review result to trigger deployment via SNS.
E) Deploy the model to a staging endpoint and have approvers manually test it, then manually update the production endpoint configuration if satisfied.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

B is Correct: SageMaker Model Registry natively supports model version statuses including PendingManualApproval and Approved. The pipeline can register the model and stop at this stage. Approval actions (performed via AWS console, SDK, or CLI) are recorded in CloudTrail for auditability, and downstream deployment steps in SageMaker Pipelines or Step Functions can check the approval status before proceeding.

Distractor Analysis

A is Incorrect: Automatically approving models violates the requirement for explicit human approval from both approvers. Automated transition to production bypasses the compliance gate entirely.
C is Incorrect: Using DynamoDB for approval workflows requires building custom orchestration, lacks native integration with SageMaker Pipelines (requiring custom code to check status), and doesn't automatically integrate with CloudTrail for approver identity auditing unless additional logging is configured.
D is Incorrect: SageMaker Ground Truth is designed for data labeling workflows, not for model approval governance. Using it for approvals introduces unnecessary complexity and cost, and it doesn't provide the native model versioning and lineage features of the Model Registry.
E is Incorrect: Manual staging testing and endpoint updates introduce operational overhead, lack an automated audit trail of approvals, and scale poorly as model versions multiply. This approach also cannot easily enforce dual-approver requirements.

Key Takeaway: Model Registry approval statuses (PendingManualApproval, Approved, Rejected) are the native AWS way to insert human gates into automated pipelines, and all approval actions are automatically auditable via CloudTrail.

Question 8 - Prompt Versioning with Bedrock Prompt Management for Production Release

A SaaS company's product team is responsible for the system prompt that controls their customer-facing AI assistant. The prompt text currently lives in application code as a hardcoded string. When the team wants to change the prompt, they must edit the code, go through a full code deployment cycle, and cannot easily A/B test different prompt variants. They want to manage prompts as independently versioned resources that can be released, compared, and rolled back without code changes.

Which approach should the team adopt? (Select ONE)

A) Store prompt text in an S3 bucket as JSON files and have the application fetch the latest file at runtime.
B) Use Amazon Bedrock Prompt Management to create versioned prompt resources with configurable variables, and have the application reference a specific prompt version ARN at runtime.
C) Move the prompt into a DynamoDB table and update the prompt text dynamically without code deployments.
D) Continue storing prompts in code, but use feature flags in AWS AppConfig to toggle between prompt versions.
E) Create multiple copies of the application, each with a different hardcoded prompt, and use Route 53 weighted routing to split traffic between them.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

B is Correct: Amazon Bedrock Prompt Management natively supports prompts as versioned resources. Prompts can have variables, be tested as comparable variants, and be released by version identifier. The application references the prompt by its ARN (including version), so updating the prompt becomes an infrastructure change rather than a code deployment, enabling A/B testing and quick rollbacks.

Distractor Analysis

A is Incorrect: Storing prompt text in S3 lacks versioning controls (beyond S3 object versioning), provides no built-in testing or comparison capabilities, and requires custom code to fetch, cache, and parse prompts. S3 versioning also doesn't provide semantic prompt management features like variable insertion.
C is Incorrect: DynamoDB provides storage but no native prompt versioning, testing, or comparison features. The team would need to build all versioning, rollback, and A/B testing infrastructure from scratch.
D is Incorrect: AppConfig feature flags can toggle between prompt variants but don't solve the fundamental problem of managing prompts as versioned artifacts with testing and comparison capabilities. It also requires maintaining both prompt versions in code, which is what the team wants to move away from.
E is Incorrect: Deploying multiple application copies to test prompts is extremely inefficient—it doubles infrastructure costs, requires complex routing, and doesn't provide a clean versioning history or rollback mechanism for prompts specifically.

Key Takeaway: Prompt changes affect model behavior as much as weight changes, so version them like models—Amazon Bedrock Prompt Management makes prompts independently releasable, comparable, and auditable artifacts.

Question 9 - CodeDeploy Blue/Green for SageMaker Endpoint Update with Validation

A retail company's ML engineering team uses AWS CodePipeline to orchestrate their model deployment process. The pipeline trains the model, registers it in the SageMaker Model Registry, and then deploys it to a production endpoint. The team wants the deployment stage to use a blue/green strategy with a canary shift, validate the new model with automated smoke tests (invoking the endpoint with known inputs and checking responses), and automatically roll back if the tests fail.

Which TWO configurations are required in the CodePipeline deployment stage? (Select TWO)

A) Use AWS CodeDeploy with a deployment group configured for ECS blue/green deployments, linking the SageMaker endpoint as the compute platform.
B) Use SageMaker Deployment Guardrails on the endpoint with a canary traffic shift, baking period, and CloudWatch alarms monitoring smoke test results or error metrics.
C) Add a CodeBuild action in the pipeline that runs smoke tests against the new model variant after traffic is shifted and fails the pipeline if tests don't pass.
D) Use AWS Lambda functions in the pipeline to manually update the endpoint variant weights based on smoke test results, without using built-in deployment strategies.
E) Configure SageMaker Model Monitor to run pre-deployment validation and automatically trigger rollback via Step Functions if validation fails.

Click to reveal Answer & Explanation

Correct Answers: B and C

Correct Options Analysis

B is Correct: SageMaker Deployment Guardrails handle blue/green traffic shifting with canary percentages, baking periods, and automatic rollback triggered by CloudWatch alarms. This is the native SageMaker mechanism for safe endpoint updates.
C is Correct: After the canary shift begins (or during the baking period), a CodeBuild action can run automated smoke tests—sending test inputs to the endpoint and validating response quality. If tests fail, the CodeBuild action fails, which can trigger the CodePipeline to halt and initiate rollback (or the CloudWatch alarm on the test results can trigger rollback).

Distractor Analysis

A is Incorrect: AWS CodeDeploy supports ECS, Lambda, and EC2 compute platforms, not SageMaker endpoints. SageMaker endpoints use their own native deployment guardrails for blue/green strategies, not CodeDeploy.
D is Incorrect: Custom Lambda functions for traffic shifting replicate what SageMaker Deployment Guardrails already provide natively, introducing unnecessary code to maintain and potential failure modes. It violates the "minimize operational overhead" principle.
E is Incorrect: SageMaker Model Monitor is designed for post-deployment monitoring (data drift, model quality drift, bias), not for pre-deployment validation or automated rollback. It detects issues but doesn't execute rollbacks—external automation is required.

Key Takeaway: SageMaker endpoints aren't compatible with CodeDeploy—use native Deployment Guardrails for blue/green canary rollouts and add CodeBuild smoke test actions in your pipeline to validate model behavior before completing the rollout.

Question 10 - Step Functions Retraining Orchestration with Model Registry Checkpoint

An insurance company's underwriting model must be retrained every month using newly collected claims data. The retraining process involves: (1) data preprocessing with SageMaker Processing, (2) model training with SageMaker Training, (3) model evaluation with SageMaker Processing, (4) registering the model in SageMaker Model Registry only if the evaluation F1 score exceeds 0.85, and (5) rejecting the model and sending an SNS notification to the team if the score is below threshold. The entire workflow must be serverless and automatically triggered on a schedule.

Which AWS service or feature should orchestrate this workflow? (Select ONE)

A) Amazon SageMaker Pipelines with a ConditionStep that evaluates the F1 score and routes to either RegisterModel or a rejection notification step, triggered by an EventBridge scheduled rule.
B) An AWS Lambda function that calls the SageMaker APIs sequentially for each step, with the evaluation threshold checked in code, and a CloudWatch scheduled event to trigger the Lambda.
C) AWS Step Functions with a state machine that includes SageMaker Processing, Training, and a Choice state for evaluation, but no integration with SageMaker Model Registry.
D) AWS Batch with multiple job definitions that execute sequentially, triggered by a CloudWatch scheduled event, with the registration decision made by a Python script.
E) A self-managed Apache Airflow cluster on EC2 that runs the workflow DAG with SageMaker operators, triggered by a cron schedule.

Click to reveal Answer & Explanation

Correct Answer: A

Correct Option Analysis

A is Correct: SageMaker Pipelines natively orchestrates ML workflow steps (Processing, Training, Evaluation, ConditionStep, RegisterModel) in a serverless manner. The ConditionStep evaluates the evaluation metric and routes the execution path accordingly. An EventBridge scheduled rule triggers the pipeline (StartPipelineExecution) on the monthly cadence, keeping the entire workflow serverless and automated.

Distractor Analysis

B is Incorrect: Orchestrating the workflow in a Lambda function with sequential API calls creates a large, complex Lambda that must manage state, handle timeouts (SageMaker jobs can run for hours), and implement its own error handling. SageMaker Pipelines handles this natively with proper state management and built-in step orchestration.
C is Incorrect: Step Functions can orchestrate SageMaker jobs and have Choice states, but it lacks native integration with SageMaker Model Registry (requires custom code in Lambda or SDK calls). SageMaker Pipelines has a direct RegisterModel step that simplifies model versioning workflow.
D is Incorrect: AWS Batch requires managing Docker container images and job definitions, introducing unnecessary operational overhead compared to the fully managed SageMaker Pipelines. Python scripts for registration decisions are custom code that must be maintained.
E is Incorrect: Self-managed Airflow on EC2 introduces significant operational overhead—maintaining servers, managing Airflow upgrades, ensuring high availability, and incurring EC2 costs even when no workflows are running. SageMaker Pipelines is serverless and eliminates all this overhead.

Key Takeaway: SageMaker Pipelines is the native, serverless orchestration service for ML workflows with built-in conditional logic (ConditionStep) and registry integration (RegisterModel)—prefer it over Lambda, Step Functions, or Airflow for ML-specific pipelines involving training and evaluation gates.

Question 11 - Bedrock Agent Version Management with Automated Deployment

A travel company has built an AI-powered concierge agent using Amazon Bedrock Agents that integrates with multiple tools (flight search, hotel booking, itinerary management). The agent's behavior depends on its instruction configuration, the foundation model it uses, and the tools it connects to. The team wants to manage agent versions in a CI/CD pipeline, ensuring that new agent configurations are tested before promotion to production and can be rolled back if issues occur.

Which practices should the team implement for agent version management? (Select TWO)

A) Use AWS CloudFormation or CDK to define Bedrock Agent configurations as infrastructure-as-code, with different versions stored in Git branches and deployed via CodePipeline stages.
B) Store only the agent's system prompt in a versioned S3 bucket, ignoring other configuration aspects like tools and model selection.
C) Use Amazon Bedrock Agent versioning to create immutable agent versions, test them in a staging environment, and deploy the tested version to the production alias.
D) Manually edit the agent configuration in the Bedrock console for each change, documenting the changes in a shared wiki page.
E) Forgo versioning agents entirely and rely on the foundation model's built-in safety features to ensure consistent behavior.

Click to reveal Answer & Explanation

Correct Answers: A and C

Correct Options Analysis

A is Correct: Treating agent configurations (instructions, model selection, tool integration, knowledge bases) as Infrastructure-as-Code (IaC) via CloudFormation or CDK enables version control, code review, testing in CI/CD pipelines, and automated deployment through CodePipeline stages.
C is Correct: Amazon Bedrock Agents support explicit versioning—each change creates a new immutable agent version that can be tested and validated. Deploying via aliases (e.g., production alias pointing to a specific version) enables publishers to test a new version before promoting it and allows quick rollback by updating the alias.

Distractor Analysis

B is Incorrect: Agent behavior is determined by more than just the system prompt—model selection, tool configurations, knowledge bases, and action group settings all matter. Versioning only the prompt misses critical configuration changes that affect agent behavior.
D is Incorrect: Manual console edits lack version history (beyond Bedrock's internal versioning), don't support code review or automated testing, and are prone to human error. This approach also doesn't integrate with CI/CD best practices.
E is Incorrect: Foundation model safety features do not ensure consistent agent behavior across configuration changes. Any change to instructions, tools, or model selection can alter agent responses significantly, requiring proper versioning and testing.

Key Takeaway: Agent behavior is as version-sensitive as model weights—use Bedrock Agent versioning with aliases for deployment control and IaC (CloudFormation/CDK) to make agent configurations testable and rollback-capable in CI/CD pipelines.

Question 12 - Automated FM Fine-Tuning Pipeline with Version Tracking

An enterprise is customizing Amazon Bedrock foundation models through fine-tuning to improve domain-specific performance. The fine-tuning process must be fully automated: when new training data is uploaded to S3, a fine-tuning job should start, the resulting customized model should be registered with a version identifier, and the model version should be traceable back to the exact training data and configuration used.

Which architecture should the team implement? (Select ONE)

A) Use SageMaker Pipelines with a TrainingStep that creates a fine-tuning job, followed by a RegisterModel step that stores the fine-tuned model artifact ARN and training metadata in the Model Registry.
B) Use a Lambda function triggered by S3 events to call the Bedrock CreateModelCustomizationJob API, store the job ID in DynamoDB, and manually copy the fine-tuned model ARN to a spreadsheet.
C) Use AWS Glue ETL jobs to preprocess training data, then use SageMaker Training to fine-tune, and store model artifacts in a versioned S3 bucket using S3 object versioning for traceability.
D) Use Amazon Bedrock Provisioned Throughput to automatically fine-tune the model on new data and expose it via a new endpoint, relying on CloudTrail logs for version tracking.
E) Use AWS Step Functions to orchestrate the Bedrock fine-tuning job and store the resulting model ARN in SSM Parameter Store, with manual version numbers assigned by the team.

Click to reveal Answer & Explanation

Correct Answer: A

Correct Option Analysis

A is Correct: While Bedrock fine-tuning can be triggered via API, SageMaker Pipelines provides a unified orchestration layer that can call Bedrock fine-tuning APIs (via CallbackStep or Lambda integration), then register the resulting customized model ARN in SageMaker Model Registry with full metadata (training data URI, hyperparameters, evaluation metrics). This creates an end-to-end automated pipeline with complete lineage tracking.

Distractor Analysis

B is Incorrect: Storing job IDs in DynamoDB and manually updating spreadsheets breaks automation and auditability. This approach requires constant human intervention, doesn't provide lineage to training data, and doesn't integrate with downstream deployment automation.
C is Incorrect: S3 object versioning tracks object changes but doesn't provide model-level lineage (which training data version was used, which hyperparameters were selected, what metrics resulted). Glue + SageMaker Training is appropriate for traditional ML models but doesn't leverage Bedrock fine-tuning capabilities for FMs.
D is Incorrect: Provisioned Throughput provisions capacity for an existing customized or fine-tuned model—it does not perform fine-tuning. Relying on CloudTrail logs for version tracking is not a structured approach and doesn't provide the lineage and approval workflow of a model registry.
E is Incorrect: Step Functions can orchestrate Bedrock fine-tuning, but SSM Parameter Store with manual versioning lacks the rich lineage tracking, metadata storage, and approval workflow of SageMaker Model Registry. Manual version numbers are error-prone.

Key Takeaway: For FM fine-tuning automation, orchestrate the entire workflow in SageMaker Pipelines and register results in Model Registry—this ensures automated job execution, complete lineage, and version traceability for audit purposes.

Question 13 - Pipeline Troubleshooting: Differentiating Build vs. Deployment Stage Failures

An MLOps engineer is troubleshooting a failed deployment in their AWS CodePipeline. The pipeline has three stages: Source → Build → Deploy. The Source stage successfully retrieved code from CodeCommit. The Build stage's CodeBuild action ran unit tests, built the Docker container image, pushed it to ECR, and generated a SageMaker model artifact. The Deploy stage uses SageMaker API calls to create or update a SageMaker endpoint. The pipeline failed at the Deploy stage with an ValidationException error stating that the model data URL is invalid.

What is the MOST LIKELY root cause of this failure? (Select ONE)

A) The CodeBuild IAM role lacks permissions to pull the Docker image from ECR.
B) The SageMaker endpoint's IAM role lacks permission to access the model artifact in S3, causing the deployment to fail when SageMaker tries to download the model.
C) The CodePipeline service role does not have permission to invoke the SageMaker API.
D) The build artifact contains a model data S3 URI that was not output correctly or was overwritten during the Build stage, causing SageMaker to receive an invalid path.
E) The CodeCommit repository was not properly configured as a source for the pipeline.

Click to reveal Answer & Explanation

Correct Answer: D

Correct Option Analysis

D is Correct: The ValidationException from SageMaker with "invalid model data URL" indicates the model data URI (S3 path) passed to SageMaker is malformed. This typically happens when the Build stage doesn't correctly output the model artifact path—for example, the CodeBuild output variable or artifact path contains a wrong prefix, was overwritten, or the bucket/key combination doesn't match what was uploaded. Since the Build stage succeeded and Source worked, the problem is most likely in how the Build stage passed data to the Deploy stage.

Distractor Analysis

A is Incorrect: ECR pull permission issues would cause the Build stage to fail during Docker image build/push, not the Deploy stage. The Build stage succeeded, so ECR permissions were fine.
B is Incorrect: SageMaker endpoint IAM role errors would manifest as an AccessDeniedException or similar during endpoint creation/update, not a ValidationException about an invalid model data URL. The endpoint role might be a secondary issue, but the primary error points to a malformed URI, not permissions.
C is Incorrect: If CodePipeline couldn't invoke SageMaker APIs due to missing IAM permissions, the failure would be an AccessDeniedException at the pipeline level, and the error message would reference the pipeline role's permissions, not a ValidationException about model data.
E is Incorrect: The Source stage succeeded (code was retrieved from CodeCommit), so the repository configuration is correct. A source configuration issue would fail in the Source stage, not the Deploy stage.

Key Takeaway: When troubleshooting CodePipeline failures, identify which stage failed and match the error type to the stage—ValidationException at Deploy typically indicates bad input (incorrect artifact path/URI), while AccessDeniedException indicates IAM permission issues.

Question 14 - Automated Model Quality Gate with SageMaker Clarify Integration

A bank is deploying a credit scoring model and must comply with regulatory requirements for model fairness. The CI/CD pipeline must automatically evaluate the model for bias (demographic parity, equal opportunity) before deployment, and block deployment if the model exhibits statistically significant bias against protected groups. The evaluation must use SageMaker Clarify and the results must be recorded in the SageMaker Model Registry for audit purposes.

Which pipeline configuration achieves these requirements? (Select TWO)

A) Add a SageMaker Clarify processing step in the pipeline that computes bias metrics on a validation dataset, and use a ConditionStep to compare metrics against regulatory thresholds before registering the model.
B) Skip the bias evaluation in the pipeline and rely on post-deployment monitoring via SageMaker Model Monitor to detect bias after the model is live.
C) Store bias metrics in the model registry as part of the model version metadata, and use them as evidence for regulatory compliance audits.
D) Manually run a SageMaker Clarify job after deployment and send the report to regulators via email.
E) Use only accuracy as a model quality metric in the pipeline, since bias detection is not required for credit scoring models under regulation.

Click to reveal Answer & Explanation

Correct Answers: A and C

Correct Options Analysis

A is Correct: SageMaker Clarify provides bias detection metrics (e.g., demographic parity, equal opportunity, DPL, DI). Integrating Clarify as a pipeline step with a ConditionStep creates an automated quality gate—if bias metrics exceed regulatory thresholds, the pipeline stops before registration and deployment.
C is Correct: Recording bias metrics in the Model Registry as part of model version metadata provides auditable evidence that regulatory compliance checks were performed and passed before deployment. This is critical for regulated industries where demonstrating compliance is as important as achieving it.

Distractor Analysis

B is Incorrect: Post-deployment monitoring only detects bias after the model is already serving predictions, potentially exposing the bank to regulatory penalties and customer harm. The requirement specifies bias evaluation before deployment.
D is Incorrect: Manual Clarify runs and email reports do not provide automated gating, are not scalable across frequent model updates, and don't integrate with the CI/CD pipeline. They also lack the structured metadata storage that the Model Registry provides for audit tracking.
E is Incorrect: Credit scoring models are subject to regulations like the Equal Credit Opportunity Act (ECOA) and Fair Housing Act in the US, requiring fairness in lending decisions. Using only accuracy ignores legally required fairness metrics and would put the bank at significant regulatory risk.

Key Takeaway: For regulated ML workloads, fairness testing is not optional—integrate SageMaker Clarify bias metrics as a pipeline condition step before registration, and store results in the Model Registry metadata for auditability.

Question 15 - MLflow on SageMaker AI for Experiment Tracking and Model Registration

A research team at a large enterprise frequently runs experiments with different hyperparameters, model architectures, and feature engineering approaches to improve an image classification model. Currently, they track experiments in scattered notebooks and spreadsheets, making it difficult to reproduce results or identify the best model configuration. The MLOps team wants to adopt a solution where experiment runs are automatically logged with their parameters and metrics, and the best-performing run can be registered directly to the SageMaker Model Registry without manual artifact copying.

Which service or feature should the team adopt? (Select ONE)

A) Use MLflow on Amazon SageMaker AI for experiment tracking—automatically logging parameters and metrics via MLflow's tracking API—and use its built-in model registration functionality to register selected runs into the Model Registry.
B) Store experiment results in a CSV file in S3 with columns for parameters and metrics, and have a Lambda function periodically parse the CSV and register models.
C) Use AWS Glue Data Catalog to store experiment metadata and register models through a custom Python script.
D) Manually document experiment results in a wiki page and upload the best model artifacts to the Model Registry using the SageMaker console.
E) Use Amazon QuickSight dashboards to visualize experiment results and rely on data analysts to identify the best model for manual registration.

Click to reveal Answer & Explanation

Correct Answer: A

Correct Option Analysis

A is Correct: MLflow on Amazon SageMaker AI provides a managed MLflow tracking server that automatically logs experiment parameters, metrics, artifacts, and source code. Its integration with SageMaker Model Registry allows selected runs to be registered as model versions with full lineage (training run → model version), eliminating manual artifact handling and ensuring reproducibility.

Distractor Analysis

B is Incorrect: CSV files in S3 are not a scalable experiment tracking solution—they lack automatic logging, real-time updates, querying capabilities, and native integration with SageMaker Model Registry. Custom Lambda parsing code adds operational overhead and is error-prone.
C is Incorrect: Glue Data Catalog is designed for metadata management of data assets (databases, tables, schemas), not for ML experiment tracking or model registry integration. Building custom experiment tracking on Glue is architecturally inappropriate.
D is Incorrect: Manual documentation and console-based registration are exactly the practices that lead to irreproducible results, lost lineage, and audit failures. This approach also doesn't scale as experiment frequency and team size grow.
E is Incorrect: QuickSight is a BI and visualization tool, not an experiment tracking system. While it can visualize metrics, it doesn't automatically capture parameters, artifacts, or lineage, nor does it integrate with Model Registry for seamless model registration.

Key Takeaway: Use MLflow on SageMaker AI for experiment tracking—it automates parameter/metric logging and provides direct model registration into the Model Registry, maintaining end-to-end lineage from experiment to deployment.