Skill 1.1.1: Extract data from data sources

Question 1 - Automated Agent Deployment with Version Control

A financial services company has developed a customer service generative AI agent using Amazon Bedrock that includes a system prompt, tool configurations, and an action group that queries a customer database. The team needs to implement a deployment pipeline that automatically deploys updated agent versions to production when changes are pushed to their GitHub repository, with the ability to quickly roll back to a previous agent version if issues are detected. The pipeline must minimize operational overhead.

Which TWO services or features should the solution include to meet these requirements? (Select TWO)

A) Use AWS CodePipeline with a CodeConnections source stage pointing to the GitHub repository, triggering on commits to automatically build and deploy agent changes.
B) Store agent configurations as JSON files in GitHub and use AWS CloudFormation to manually deploy them to Bedrock when approval is granted.
C) Use Amazon Bedrock Agent versioning to create immutable versions of the agent for deployment, enabling rollback to a previous version identifier.
D) Store agent prompt templates in AWS Secrets Manager and rotate them on a schedule.
E) Use AWS CodeDeploy to deploy the Bedrock agent configuration directly to the Bedrock API endpoint.

Click to reveal Answer & Explanation

Correct Answers: A and C

Correct Options Analysis

A is Correct: AWS CodePipeline with CodeConnections establishes a secure link to GitHub (OAuth-based, no stored credentials) and triggers pipeline runs on code pushes, enabling automated CI/CD for agent configuration changes.
C is Correct: Amazon Bedrock Agents natively support versioning. Each deployment creates an immutable agent version identified by a unique ARN, allowing production endpoints to reference a specific version and roll back by pointing to a previous version identifier.

Distractor Analysis

B is Incorrect: Manual deployment via CloudFormation contradicts the requirement for automated deployment when changes are pushed to GitHub, and introduces operational overhead requiring human intervention.
D is Incorrect: AWS Secrets Manager is designed for managing credentials and secrets (rotated on a schedule), not for storing or versioning agent prompt templates or deployment configurations.
E is Incorrect: AWS CodeDeploy manages application deployments to compute targets like EC2, ECS, and Lambda. Bedrock agents are managed AWS resources deployed via Bedrock APIs or CloudFormation—not CodeDeploy.

Key Takeaway: Bedrock agent deployment pipelines combine CodePipeline/CodeConnections for CI/CD automation with native Bedrock agent versioning for immutable versions and rollback capability.

Question 2 - Prompt Version Management in CI/CD

A healthcare technology team is deploying a generative AI application that uses Amazon Bedrock. The application behavior is driven significantly by prompts that include patient-specific instructions, compliance reminders, and clinical terminology context. The team needs to ensure prompt changes go through the same code review, testing, and deployment process as application code, with the ability to track which prompt version was live during an audit.

Which TWO actions should the engineering team take to meet these requirements? (Select TWO)

A) Create versioned prompts using Amazon Bedrock Prompt Management, with each update generating a new prompt version that the application references by version ID.
B) Embed prompts directly in the application code so code review processes automatically cover them.
C) Store prompts in a CodeCommit repository and deploy them using CodePipeline with a stage that updates the Bedrock prompt resource.
D) Include prompts in Amazon ECR as container images for deployment.
E) Use AWS Config rules to validate prompt syntax before deployment.

Click to reveal Answer & Explanation

Correct Answers: A and C

Correct Options Analysis

A is Correct: Amazon Bedrock Prompt Management natively supports versioned prompts. Each prompt update creates a new version with an immutable identifier, allowing audit tracking of which prompt version was live at any time.
C is Correct: CodeCommit stores prompt definitions as code/JSON artifacts, and CodePipeline automatically deploys them through defined stages (dev → test → prod), ensuring prompt changes follow the same CI/CD process as application code.

Distractor Analysis

B is Incorrect: Embedding prompts in application code makes prompt changes require application code deployments, couples prompt iteration with app release cycles, and doesn't provide prompt-native versioning for audit purposes.
D is Incorrect: Amazon ECR stores container images (Docker/OCI), not prompt text or configuration. It's the wrong service for prompt storage and versioning.
E is Incorrect: AWS Config evaluates resource compliance against rules, but does not manage prompt deployment, versioning, or CI/CD pipeline execution.

Key Takeaway: Treat prompts as versioned, deployable artifacts using Bedrock Prompt Management (for native versioning) combined with CodePipeline (for automated deployment) to maintain audit trails.

Question 3 - Rollback Configuration for Agent Deployment

An e-commerce company is using Amazon Bedrock Agents for an order-assistance chatbot. The ML engineering team has configured a CI/CD pipeline using AWS CodePipeline that deploys new agent versions to production. During the latest deployment, the new agent version began returning incorrect order status due to a tool configuration error. However, the pipeline marked the deployment as successful and all production traffic continued hitting the new version. They need to prevent this from happening in future deployments.

What should the team configure to ensure automatic rollback when the new agent version misbehaves? (Select ONE)

A) Add a manual approval stage in CodePipeline after the production deployment.
B) Configure Amazon CloudWatch alarms on agent invocation metrics and integrate them with a deployment guardrail that monitors the new version during a baking period.
C) Deploy the agent using AWS CodeDeploy with blue/green deployment settings.
D) Configure an EventBridge rule that alerts the team when the agent returns errors.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

B is Correct: Automated rollback requires a monitoring mechanism to detect degradation. CloudWatch alarms (e.g., on error rates or latency of Bedrock agent invocations) linked to a deployment guardrail with a baking period will automatically revert traffic to the previous agent version if an alarm trips during the validation window.

Distractor Analysis

A is Incorrect: Manual approval stages are pre-deployment gates (human validation before deployment proceeds), not post-deployment rollback mechanisms. They don't automatically revert traffic based on runtime behavior.
C is Incorrect: AWS CodeDeploy manages deployments for compute targets (EC2, ECS, Lambda), not Bedrock agents. Even if it were applicable, blue/green without monitoring would not prevent the issue alone.
D is Incorrect: EventBridge rules that send alerts are reactive notifications requiring manual intervention. They do not perform automated rollback or traffic redirection.

Key Takeaway: Automated rollback is only as effective as the alarm that triggers it—configuring CloudWatch alarms on agent metrics and baking periods ensures bad deployments revert automatically.

Question 4 - Agent Alias and Version Management Strategy

A logistics company is running a production Bedrock agent for shipment tracking. The company wants to deploy updates weekly with zero downtime, run A/B tests between agent versions, and maintain the ability to instantly revert to a previous version in case of errors. The deployment must minimize manual intervention.

Which strategy best achieves these requirements? (Select ONE)

A) Deploy each new agent version to a new Bedrock agent resource and update the application to point to the new agent ID each week.
B) Use Bedrock agent aliases to route production traffic to a specific agent version, updating the alias to point to new versions when promoting, and switching the alias back to a previous version for rollback.
C) Deploy agent versions to separate AWS accounts for isolation and use Route 53 weighted routing to distribute traffic.
D) Use AWS Step Functions to trigger agent updates and manage rollback manually through the AWS Console.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Options Analysis

B is Correct: Amazon Bedrock agent aliases provide a stable endpoint identifier that abstract underlying agent versions. Production applications point to the alias, and the team updates the alias to point to a new agent version for promotion or a previous version for instant rollback. Aliases support version-independent routing, enabling A/B testing by directing different traffic to different versions.

Distractor Analysis

A is Incorrect: Creating new Bedrock agent resources for each version duplicates configuration and requires application updates (pointing to new agent IDs) each week, violating the zero-downtime and minimal-manual-intervention constraints.
C is Incorrect: Multi-account isolation adds significant operational overhead and complexity. Route 53 weighted routing is for DNS traffic management, not Bedrock agent version routing, and doesn't leverage native Bedrock alias capabilities.
D is Incorrect: AWS Step Functions orchestrates workflow steps but doesn't replace Bedrock's native alias/version management. Manual Console rollback violates the automation requirement.

Key Takeaway: Bedrock agent aliases decouple applications from specific agent versions, enabling zero-downtime promotion, instant rollback, and A/B testing through alias pointer updates.

Question 5 - CI/CD Pipeline Stage Configuration for Agent Testing

A media company is building a CI/CD pipeline for their content recommendation agent built on Amazon Bedrock. The pipeline must automatically run unit tests on agent configuration code, validate the agent's tool definitions, execute a prompt evaluation against a test corpus, and compare the responses against a baseline version. If the evaluation score drops below 90% of the baseline, the pipeline must stop before deployment.

Which sequence of pipeline stages correctly implements this workflow? (Select ONE)

A) Source → Build (unit tests + tool validation) → Test (prompt evaluation against baseline) → Deploy (with approval gate)
B) Source → Build (compile application code) → Deploy → Test (evaluate in production)
C) Deploy → Source → Build (unit tests) → Test (prompt evaluation)
D) Source → Build (unit tests) → Deploy (with approval gate) → Test (prompt evaluation)

Click to reveal Answer & Explanation

Correct Answer: A

Correct Options Analysis

A is Correct: The pipeline follows best practices: (1) Source stage pulls code from the repository, (2) Build stage runs deterministic code tests (unit tests, tool definition validation), (3) Test stage evaluates the generative AI's non-deterministic responses against a baseline using statistical/semantic scoring, and (4) Deploy stage ships the validated version. The Test stage acts as a quality gate that blocks deployment if the score drops below threshold.

Distractor Analysis

B is Incorrect: Deploying before testing is not only a poor practice but also defeats the purpose of CI/CD gates—testing in production can expose users to untested agents, and configuration errors cannot be caught before release.
C is Incorrect: Stage ordering is completely backwards—starting with deployment defeats the purpose of testing before release, and the Source stage must come after the pipeline is initialized from code.
D is Incorrect: The approval gate is placed before the Test stage, which means testing happens after deployment—this is not a proper pipeline design as the gate should be after the quality gate (test results) and before production release.

Key Takeaway: In ML/AI CI/CD pipelines, separate deterministic code tests (Build stage) from non-deterministic evaluation tests (Test stage) and place the deployment gate after the test stage.

Question 6 - Automated Retraining Pipeline Integration

A manufacturing company uses Amazon SageMaker to train a defect-detection model that runs on the production line. The company needs to set up an automated retraining pipeline that triggers when new labeled image data is uploaded to an S3 bucket, trains the model, evaluates it against a held-out dataset, and registers it in the Model Registry for approval. The pipeline must NOT automatically deploy the new model without human approval.

Which combination of services and configurations achieves this? (Select TWO)

A) Configure an S3 event notification that triggers an AWS Lambda function to invoke the SageMaker Pipeline's StartPipelineExecution API when new objects are created in the training data bucket.
B) Configure the SageMaker Pipeline to use a RegisterModel step that registers the new model version to the Model Registry with ModelApprovalStatus set to PendingManualApproval.
C) Configure the SageMaker Pipeline to use a CreateModel step that automatically creates the model and a TransformJob step to deploy to production.
D) Configure an EventBridge rule that triggers an automated deployment through AWS CodeDeploy when the model is registered.
E) Configure the Model Registry to automatically approve all new model versions to ensure the latest version is always deployed.

Click to reveal Answer & Explanation

Correct Answers: A and B

Correct Options Analysis

A is Correct: S3 event notifications (specifically s3:ObjectCreated:*) trigger a Lambda function that calls StartPipelineExecution—this is the standard event-driven retraining pattern for SageMaker pipelines.
B is Correct: The RegisterModel step in SageMaker Pipeline with ModelApprovalStatus: PendingManualApproval ensures the model is registered but not automatically approved for deployment. A human must approve the model in the Model Registry or via an approval step in a CI/CD tool before deployment, satisfying the requirement.

Distractor Analysis

C is Incorrect: CreateModel and TransformJob steps directly deploy the model (for batch transform) without a human approval gate, violating the requirement that new models wait for human approval.
D is Incorrect: This configuration automatically deploys the model upon registration, bypassing human approval—a dangerous pattern for production models that could ship a defective model.
E is Incorrect: Auto-approving all model versions eliminates the human approval gate, contradicting the requirement and creating risk of deploying bad models without review.

Key Takeaway: A proper retraining pipeline ends at Model Registry with PendingManualApproval—this separates model training from deployment and ensures human oversight.

Question 7 - Model Version Lineage and Audit

An insurance company has undergone a regulatory audit. Auditors need to trace back a specific model version that was serving in production during the month of March 2024 to determine: (1) which training dataset was used, (2) which hyperparameters were configured, and (3) which experiment run produced it. The company uses Amazon SageMaker for training and deployment.

Which combination of AWS services provides this end-to-end lineage? (Select ONE)

A) Use the SageMaker Model Registry to catalog model versions, combined with Amazon SageMaker Experiments (or MLflow tracking) to record experiment metadata, hyperparameters, and training data references for each run.
B) Store the model artifacts in S3 and rely on the S3 object metadata for training details.
C) Use Amazon CloudWatch Logs to manually search for training job parameters.
D) Use AWS CloudTrail to track all training and deployment activities.

Click to reveal Answer & Explanation

Correct Answer: A

Correct Options Analysis

A is Correct: SageMaker Model Registry groups model versions together (model groups), records metadata including training job ARNs, and maintains lineage. Combined with SageMaker Experiments or MLflow tracking, which capture hyperparameters, training data references, and metrics for each experiment run, auditors can trace a model version back to its exact training run with full reproducibility.

Distractor Analysis

B is Incorrect: S3 object metadata only tracks file properties (size, checksum, timestamps), not training hyperparameters, dataset locations, or experiment lineage. However, S3 is the storage layer for the actual artifacts—but it doesn't provide audit lineage alone.
C is Incorrect: CloudWatch Logs capture raw log output during training. While they may contain some configuration details, searching through logs manually is inefficient and doesn't provide structured, queryable lineage. Additionally, logs expire based on retention policies.
D is Incorrect: AWS CloudTrail records API calls and account activity (who called what and when), but doesn't store training job hyperparameters, dataset lineage, or experiment metadata in a structured queryable format.

Key Takeaway: For audit-grade lineage, combine SageMaker Model Registry (catalog versions with metadata) with Experiments/MLflow tracking (record training run details) for end-to-end reproducibility.

Question 8 - RAG System Incremental Data Refresh

A legal research company has deployed a RAG (Retrieval-Augmented Generation) system using Amazon Bedrock Knowledge Bases and OpenSearch Serverless. The underlying document corpus in S3 contains thousands of legal documents, and new court opinions are added daily. The company wants to refresh the knowledge base to reflect new documents and modifications to existing documents without reprocessing the entire corpus, ensuring minimal latency and cost. Compliance requires that deleted documents are removed from the vector store.

Which approach should the company use? (Select ONE)

A) Use the Bedrock Knowledge Bases incremental ingestion sync (resync) operation that automatically detects new, modified, and deleted documents and processes only those changes.
B) Manually delete the existing vector collection and recreate it by re-ingesting all documents from S3.
C) Copy the existing OpenSearch Serverless collection to a new collection using snapshots.
D) Use AWS Glue ETL jobs to process all documents and write vectors to OpenSearch.

Click to reveal Answer & Explanation

Correct Answer: A

Correct Options Analysis

A is Correct: Amazon Bedrock Knowledge Bases natively supports incremental synchronization. When triggered, the resync operation compares the S3 data source metadata (timestamps, object hashes) with the vector index, processing only documents that were added, modified, or deleted since the last run. This ensures compliance with deletion requirements while minimizing cost and latency.

Distractor Analysis

B is Incorrect: Deleting and recreating the vector collection means re-parsing, re-chunking, and re-embedding every document—this is a full reindex that wastes significant compute and time, especially for a corpus with thousands of documents where only a small subset changed.
C is Incorrect: Copying an OpenSearch Serverless collection using snapshots creates a duplicate of the existing index including stale vectors. It does not process new, modified, or deleted documents from the S3 source and therefore fails the compliance requirement for removing deleted documents.
D is Incorrect: Building custom Glue ETL jobs to process documents and write vectors manually introduces unnecessary operational complexity and effort when Bedrock Knowledge Bases provides the same functionality natively with automatic chunking, embedding, and indexing.

Key Takeaway: Bedrock Knowledge Bases supports incremental sync for cost-efficient RAG refreshes—only changed or new documents are re-embedded, while deleted documents are automatically removed from the index.

Question 9 - Foundation Model Fine-Tuning Pipeline with Versioning

A retail analytics company is building a custom foundation model for product categorization using Amazon Bedrock. The company has fine-tuned a model using a custom training dataset and wants to automate the deployment pipeline so that: (1) each fine-tuned model version is uniquely identified, (2) the production application references the model by a version identifier rather than updating in place, and (3) rollback to a previous fine-tuned version is instant.

Which TWO actions are required? (Select TWO)

A) Create a custom model in Amazon Bedrock using the fine-tuning job, which generates a unique custom model ARN for each fine-tuned version.
B) Use Amazon Bedrock Provisioned Throughput to deploy the fine-tuned custom model, specifying the custom model ARN for a specific fine-tuned version.
C) Use Amazon SageMaker Model Registry to track the fine-tuned model versions.
D) Store the fine-tuned model weights in Amazon S3 and update the file in place.
E) Use AWS CodeBuild to compile the fine-tuned model into a Docker image.

Click to reveal Answer & Explanation

Correct Answers: A and B

Correct Options Analysis

A is Correct: Amazon Bedrock fine-tuning jobs create custom models with unique ARNs for each fine-tuned version. This provides immutable version identifiers that can be referenced by applications, and enables coexistence of multiple versions for rollback scenarios.
B is Correct: Amazon Bedrock Provisioned Throughput allows you to deploy fine-tuned custom models by referencing their ARN. To deploy a new version, you create a new provisioned throughput instance pointing to the new custom model ARN; to roll back, you point to the previous ARN.

Distractor Analysis

C is Incorrect: While SageMaker Model Registry is excellent for tracking SageMaker-trained models, Bedrock fine-tuned models are tracked natively through Bedrock custom model ARNs and don't require (or integrate with) SageMaker Model Registry.
D is Incorrect: Storing fine-tuned weights in S3 and updating the file in place is an anti-pattern—it doesn't provide immutable versions, breaks auditability, and prevents instant rollback (since the previous version is overwritten).
E is Incorrect: AWS CodeBuild is a build service that compiles code and creates container images. It doesn't play a role in Bedrock fine-tuning or deploying custom foundation models—Bedrock manages model storage and inference infrastructure.

Key Takeaway: Bedrock fine-tuned models are identified by unique ARNs; deploy them via Provisioned Throughput pointing to specific ARNs for versioned, rollback-capable deployments.

Question 10 - Agent Testing Framework Configuration

A technology company is building a customer support agent using Amazon Bedrock. The agent uses tool calling to query an internal CRM system and has a carefully engineered system prompt. The team wants to implement automated testing in their CI/CD pipeline to verify that: (1) tool definitions are syntactically valid and match the CRM API schema, (2) the agent's responses are helpful and non-toxic, and (3) the agent correctly invokes the right tools for different types of customer inquiries. The pipeline must handle non-deterministic responses.

Which TWO testing approaches should be included? (Select TWO)

A) Run deterministic schema validation tests on tool definitions to ensure they match the CRM API spec.
B) Run a curated set of test prompts and use an evaluation framework (e.g., LLM-as-judge or Bedrock Model Evaluation) to score response quality against criteria like helpfulness and non-toxicity.
C) Assert exact string matches for expected agent responses in test scripts.
D) Use only unit tests on the application code since agent behavior is managed by Bedrock.
E) Skip testing and rely on production monitoring to catch issues.

Click to reveal Answer & Explanation

Correct Answers: A and B

Correct Options Analysis

A is Correct: Tool definitions (action group schemas) are deterministic configurations that can and should be validated against API schemas as a standard code test. This catches configuration errors (e.g., wrong parameter names, invalid JSON) before deployment.
B is Correct: Generative AI responses are non-deterministic, so exact string assertions are impossible. A curated golden prompt dataset with evaluation metrics (helpfulness, non-toxicity, tool-selection accuracy) scored against a baseline handles the inherent variability of LLM outputs and provides meaningful quality gates.

Distractor Analysis

C is Incorrect: Exact string matching for LLM responses is ineffective because generative AI responses vary significantly between runs even for the same prompt, causing false failures (flaky tests) and preventing valid agents from shipping.
D is Incorrect: Application code unit tests alone cannot validate agent behavior, tool selection accuracy, or response quality. Agent configuration and prompt behavior need dedicated testing approaches.
E is Incorrect: Skipping testing is an unacceptable practice for production AI systems. Production monitoring detects issues after users are impacted, whereas CI/CD testing prevents bad changes from reaching users in the first place.

Key Takeaway: For agent CI/CD, combine deterministic tests (schema/config validation) with evaluation-based testing (scored quality metrics) to handle both code and non-deterministic AI behavior.