Skill 3.3.2: Configure and troubleshoot AWS CodeBuild, AWS CodeCommit, AWS CodeDeploy, AWS CodePipeline, and AWS CodeConnections

Question 1 - CodePipeline with External Git Repository

A financial services company is building an MLOps pipeline to automate model deployment for their fraud detection system. Their data science team uses GitHub Enterprise for source control, and the security team mandates that no static credentials (e.g., personal access tokens or SSH keys) may be stored in the pipeline configuration, CloudFormation templates, or application code. The pipeline must automatically trigger when a new commit is pushed to the main branch.

Which service should the ML engineer use to establish the secure connection between CodePipeline and the external Git repository?

Select ONE that applies.

A) AWS CodeCommit with repository mirroring to GitHub Enterprise.

B) AWS CodeConnections to create a managed connection using GitHub App-based OAuth handshake.

C) AWS CodeBuild with a buildspec that clones the GitHub repository using an encrypted environment variable containing a PAT.

D) AWS Secrets Manager to store GitHub credentials, with an IAM role that CodePipeline assumes to retrieve them during the source stage.

E) AWS CodeDeploy configured with a custom deployment group that pulls source code directly from GitHub Enterprise.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

B is Correct: AWS CodeConnections (formerly AWS CodeStar Connections) provides a managed, secure integration between AWS services and third-party Git repositories. It uses an OAuth or GitHub App-based handshake that establishes trust without storing credentials in pipeline definitions. CodePipeline can reference the connection ARN in the source stage, and commit events from GitHub automatically trigger the pipeline.

Distractor Analysis

A is Incorrect: CodeCommit is an AWS-native Git repository service, not a connection mechanism. Mirroring would require maintaining a synchronization process, adding unnecessary operational overhead, and does not address the secure connection requirement for triggering on external repo commits.
C is Incorrect: Storing a PAT as an encrypted CodeBuild environment variable violates the security mandate of "no static credentials" in application code or configuration. Additionally, CodeBuild clone operations with PATs require the credential to be persisted somewhere, which conflicts with the requirement.
D is Incorrect: While Secrets Manager securely stores credentials, the requirement explicitly states no static credentials may be stored anywhere. CodePipeline source stages for GitHub require either CodeConnections or a direct GitHub token, both of which rely on credential storage.
E is Incorrect: CodeDeploy is a deployment service (rollout to compute targets), not a source control integration tool. It does not provide the connection layer between external Git repositories and pipeline triggers.

Key Takeaway: AWS CodeConnections eliminates credential storage in pipeline definitions by using OAuth/GitHub App-based authentication.

Question 2 - CodeBuild Buildspec for ML Container Build

A machine learning team is automating the build of a custom training container using AWS CodeBuild. The container must be built from a Dockerfile, pushed to Amazon ECR, and include the model training dependencies specified in a requirements.txt file. The team wants the build to be secure and reproducible, avoiding the need to manage build servers.

Which of the following configurations should the ML engineer include in the CodeBuild project?

Select ONE that applies.

A) Use a buildspec file that runs docker build and docker push commands on a CodeBuild managed Ubuntu runner with Docker daemon installed.

B) Configure CodeBuild with a buildspec that uses pre-built images from ECR, specifying the AWS CLI and Docker commands to authenticate to ECR using aws ecr get-login-password before pushing.

C) Launch an EC2 instance as a dedicated build server, installing Docker and Jenkins, and connect CodeBuild as a webhook receiver.

D) Use CodeBuild with a managed Windows Server image and run PowerShell scripts to build the container.

E) Configure CodeBuild with a buildspec environment type LINUX_CONTAINER and run commands that execute ./gradlew build to create the Docker image.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

B is Correct: CodeBuild is a fully managed build service that can run Docker commands. The buildspec file specifies the commands to authenticate to Amazon ECR using the aws ecr get-login-password command (or aws ecr get-authorization-token), build the Docker image with docker build -t <image-tag> ., and push the image to ECR with docker push <image-tag>. This eliminates the need for teams to provision, manage, and scale their own build servers.

Distractor Analysis

A is Incorrect: While CodeBuild managed images include Docker, the description mentions "Docker daemon installed" which is implicit in managed CodeBuild environments but not specified explicitly. The buildspec would still need the full command sequence including ECR authentication.
C is Incorrect: Using a dedicated EC2 instance with Jenkins introduces unnecessary operational overhead (OS patching, Jenkins maintenance, scaling). CodeBuild is designed specifically to eliminate this burden.
D is Incorrect: CodeBuild also supports Windows Server builds but Linux is the standard for ML containers (TensorFlow, PyTorch), and Windows images add unnecessary licensing and runtime costs.
E is Incorrect: ./gradlew build is a Java/Gradle build command, not relevant to building Python ML containers. The LINUX_CONTAINER environment type is correct, but the toolchain is wrong.

Key Takeaway: CodeBuild eliminates build server management; the buildspec handles ECR authentication and image push as standard commands.

Question 3 - CodePipeline Stage Failures and Troubleshooting

An ML engineer is troubleshooting a failed pipeline execution. The pipeline has four stages: Source (GitHub via CodeConnections) → Build (CodeBuild running model evaluation scripts) → Deploy (CodeDeploy to SageMaker endpoint) → Post-Deploy Test (CodeBuild running integration tests). The execution failed at the Deploy stage with a CodeDeploy error stating "Deployment failed due to an invalid model artifact."

What is the most likely root cause?

Select ONE that applies.

A) The CodeBuild stage did not produce the required model.tar.gz file in the expected Amazon S3 location.

B) The CodeConnections source stage lacked permission to access the GitHub repository.

C) The Deploy stage was missing an approved action gate before deployment.

D) CodePipeline could not find the CodeBuild role due to an IAM misconfiguration.

E) The endpoint's auto-scaling policy exceeded the limit for InitialInstanceCount.

Click to reveal Answer & Explanation

Correct Answer: A

Correct Option Analysis

A is Correct: The "invalid model artifact" error from CodeDeploy indicates the deployment configurations pointed to a model artifact (e.g., S3 URI containing model.tar.gz) that does not exist at the expected location or is malformed. This typically occurs when the preceding Build stage fails to upload training/evaluation outputs to the correct S3 bucket and prefix, or the model artifact name does not match the CodeDeploy/AppSpec configuration. The failure in the Deploy stage often stems from an artifact mismatch created in the Build stage.

Distractor Analysis

B is Incorrect: A source stage failure would have prevented the execution from reaching the Deploy stage. Since the Deploy stage is invoked, the Source stage must have succeeded.
C is Incorrect: Lack of an approval gate does not cause CodeDeploy failures; approval gates simply pause the pipeline awaiting manual approval. They are not technical dependencies for successful deployment execution.
D is Incorrect: An IAM role issue would manifest as an authorization error in CodeBuild itself (e.g., AccessDenied), causing the Build stage to fail before CodeDeploy is invoked.
E is Incorrect: Scaling policy limits affect endpoint instance counts and would not produce a "invalid model artifact" error. Auto Scaling errors are different from deployment validation errors.

Key Takeaway: When troubleshooting pipeline failures, trace forward from the failed stage: Source errors stop the pipeline immediately, while deployment failures often originate from missing or mismatched artifacts created earlier in the pipeline.

Question 4 - CodeDeploy Blue/Green Deployment for SageMaker Endpoint

A retail company is deploying a new product recommendation model to a SageMaker endpoint. They require a blue/green deployment strategy with automatic rollback if the new model's error rate exceeds 2% during the first 15 minutes of serving traffic. The team has already created a CloudWatch alarm on Invocation5XXErrors.

What configuration should the ML engineer set up in CodeDeploy to meet these requirements?

Select ONE that applies.

A) Use CodeDeploy with a linear deployment strategy that shifts 10% of traffic every 10 minutes, with no rollback condition.

B) Use CodeDeploy with a blue/green deployment group that specifies the CloudWatch alarm in the AutoRollbackConfiguration and a 15-minute baking period.

C) Use CodeDeploy with an in-place deployment to the existing SageMaker endpoint and configure a manual approval gate before the bake.

D) Use CodeDeploy with a canary deployment that routes all traffic instantly to the new model, with a manual trigger for rollback.

E) Use CodeDeploy with a blue/green deployment group that only supports canary shapes and cannot integrate with CloudWatch alarms.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Option Analysis

B is Correct: CodeDeploy blue/green deployments for SageMaker endpoints support deployment guardrails with AutoRollbackConfiguration. You specify the CloudWatch alarms to monitor during the baking period, and the baking period (e.g., 15 minutes) during which the alarms are active. If the Invocation5XXErrors alarm triggers within 15 minutes, CodeDeploy automatically shifts traffic back to the original production fleet, meeting the rollback requirement.

Distractor Analysis

A is Incorrect: A linear strategy that gradually shifts traffic is valid, but the absence of a rollback condition violates the requirement for automatic rollback when error rates exceed thresholds.
C is Incorrect: In-place deployments are not supported for SageMaker endpoints in CodeDeploy; SageMaker deployments use blue/green. Additionally, manual approval gates introduce human latency and do not fulfill automatic rollback.
D is Incorrect: Canary deployment routes only a small percentage of traffic initially, not all traffic at once. A manual rollback trigger also violates the requirement for automated rollback.
E is Incorrect: CodeDeploy blue/green deployments support both canary and linear traffic shifting, and they absolutely integrate with CloudWatch alarms for automatic rollback.

Key Takeaway: Automatic rollback in CodeDeploy blue/green deployments requires linking CloudWatch alarms to the deployment group's AutoRollbackConfiguration and specifying a baking period.

Question 5 - CodeConnections vs CodeCommit for Compliance

A healthcare startup stores patient data and model training code, and must comply with HIPAA. Their compliance officer requires that all source code repositories reside in the same AWS account and region as their healthcare data stores, with no external dependencies. They currently use a GitHub repository.

Which solution meets the compliance requirements while minimizing disruption?

Select ONE that applies.

A) Migrate the repository to AWS CodeCommit and configure CodePipeline to use CodeCommit as the source stage.

B) Keep the repository in GitHub but use AWS CodeConnections to establish a secure connection.

C) Use CodeBuild to mirror the GitHub repository into S3 and use S3 as the pipeline source.

D) Maintain the GitHub repository and use a scheduled EventBridge rule to manually sync copies into AWS.

E) Use CodeDeploy to directly pull source code from GitHub and deploy to SageMaker endpoints.

Click to reveal Answer & Explanation

Correct Answer: A

Correct Option Analysis

A is Correct: AWS CodeCommit hosts Git repositories entirely within the AWS account and region, satisfying the requirement that "all source code repositories reside in the same AWS account and region" with no external dependencies. CodePipeline can natively integrate with CodeCommit as a source stage, and since CodeCommit is an AWS service, it integrates with IAM for access control and CloudTrail for auditing, supporting HIPAA compliance.

Distractor Analysis

B is Incorrect: Keeping the repository on GitHub via CodeConnections maintains an external dependency, violating the compliance mandate that no external dependencies exist.
C is Incorrect: Mirroring via CodeBuild and using S3 as a source introduces operational complexity (mirror iterations, version drift) and still involves an external GitHub dependency for the mirroring process.
D is Incorrect: A scheduled sync pattern does not meet the "minimizing disruption" or "no external dependencies" requirements, and introduces delayed source updates and synchronization errors.
E is Incorrect: CodeDeploy is a deployment service and does not provide Git repository hosting. It cannot directly pull source code from GitHub for SageMaker deployments.

Key Takeaway: CodeCommit keeps source control within the AWS account boundary, satisfying HIPAA and other compliance mandates that require no external dependencies.

Question 6 - CodeBuild Environment Variables and Secrets

An ML team is configuring a CodeBuild project to run a training script that requires access to an Amazon RDS database and an Amazon S3 bucket for input data. The team wants to follow AWS best practices for secret management, avoiding hardcoded credentials in buildspec.yml.

How should the ML engineer configure the database access for the CodeBuild environment?

Select ONE that applies.

A) Store the database password as an encrypted environment variable in the CodeBuild project configuration using AWS KMS, and reference it as $DB_PASSWORD in the buildspec.

B) Hardcode the database credentials directly in the buildspec.yml file for simplicity, since the CodeBuild logs are private.

C) Store the credentials in a plain text file in Amazon S3 and use aws s3 cp to download during the build.

D) Embed the credentials in an IAM policy attached to the CodeBuild service role for automatic access.

E) Use AWS Secrets Manager and retrieve the secret at runtime in the buildspec using the AWS CLI.

Click to reveal Answer & Explanation

Correct Answer: E

Correct Option Analysis

E is Correct: AWS Secrets Manager is the recommended service for storing, rotating, and managing database credentials. In the buildspec, the AWS CLI command (aws secretsmanager get-secret-value --secret-id <secret-name>) retrieves the credential at runtime. This avoids storing secrets in environment variables, which can appear in build logs or CloudFormation outputs.

Distractor Analysis

A is Incorrect: While CodeBuild supports encrypted environment variables, they are still stored in the CodeBuild project configuration and can be exposed if the project definition is leaked. AWS Secrets Manager provides better secret lifecycle management (rotation, access auditing) as a best practice.
B is Incorrect: Hardcoding credentials in buildspec files is a security anti-pattern and violates AWS best practices. Secrets should never appear in source-controlled files, regardless of log privacy settings.
C is Incorrect: Storing plain text credentials in S3 is insecure and violates compliance standards. S3 does not natively encrypt object content with secrets-managed keys, and downloading plain text secrets during builds is a practice to avoid.
D is Incorrect: IAM policies grant permissions to access resources but cannot store database passwords. Credential material belongs in a secrets manager, not embedded within IAM policies.

Key Takeaway: AWS Secrets Manager is the preferred service for storing database credentials retrieved at runtime during CodeBuild executions.

Question 7 - CodePipeline Cross-Account Deployment

A large enterprise with multiple AWS accounts wants to centralize the CI/CD pipeline in a tooling account but deploy machine learning models to SageMaker endpoints in multiple production accounts. The pipeline includes Source (CodeCommit), Build (CodeBuild for training and evaluation), and Deploy (CodeDeploy to SageMaker) stages.

Which solution enables secure cross-account deployment with least operational overhead?

Select ONE that applies.

A) Create a single CodePipeline in the tooling account, use cross-account IAM roles for the CodeBuild and CodeDeploy stages, and configure the target production accounts to trust the tooling account's role.

B) Duplicate the pipeline in each production account and use CodeConnections to link them to a single CodeCommit repository in the tooling account.

C) Create a CodePipeline in each production account that reads from shared S3 buckets in the tooling account, with manual synchronization scripts to copy artifacts.

D) Use a single CodePipeline in the tooling account and store all production account credentials in a shared Secrets Manager secret for cross-account deployment.

E) Deploy directly from CodePipeline to SageMaker endpoints in each production account using a cross-account IAM user access key hardcoded in the pipeline configuration.

Click to reveal Answer & Explanation

Correct Answer: A

Correct Option Analysis

A is Correct: Cross-account deployments are supported in CodePipeline by creating an IAM role in each target production account and configuring a trust policy that allows the tooling account's pipeline role to assume it. The CodePipeline in the tooling account can reference the assumed role's ARN for cross-account CodeBuild and CodeDeploy stages. This approach centralizes pipeline management while maintaining least privilege.

Distractor Analysis

B is Incorrect: Duplicating the pipeline in each production account increases operational overhead, requires maintaining consistent pipeline definitions across many accounts, and does not centralize pipeline management in the tooling account.
C is Incorrect: Using S3 buckets with manual synchronization introduces significant operational complexity, drift risks, and human error. This is not a managed cross-account deployment pattern.
D is Incorrect: Storing production account credentials in a Secrets Manager secret is a security risk (exposes multiple environments from one secret) and does not follow IAM best practices for cross-account access via roles.
E is Incorrect: Hardcoding access keys is a serious security anti-pattern and violates AWS security best practices. Cross-account access should use IAM roles and trust policies, not long-term credentials.

Key Takeaway: Cross-account CodePipeline deployments use IAM role assumption; the tooling account's pipeline role assumes a role in each target account.

Question 8 - CodePipeline with Bedrock Prompt Management

A company is automating the release of generative AI prompts stored in Amazon Bedrock Prompt Management. The prompt testing pipeline must validate prompt variants against a golden dataset, and if the aggregate quality score is below the threshold, fail the pipeline execution before deployment. The prompts are versioned and stored in Amazon S3 alongside model artifacts.

Which Pipeline and Build configuration achieves this requirement?

Select ONE that applies.

A) Create a CodePipeline with stages: Source (S3 prompt versions) → Build (CodeBuild running prompt evaluation against the golden dataset) → Deploy (CodeDeploy to update Bedrock prompt version). The Build stage fails if quality score is below threshold.

B) Create a CodePipeline with stages: Source (S3) → Manual Approval → Deploy (CodeBuild to execute evaluation and deployment in single stage).

C) Use CodeBuild to run both evaluation and deployment in a single step, without CodePipeline, to reduce complexity.

D) Create a CodePipeline with stages: Source (CodeCommit) → Build (CodeBuild to evaluate prompts) but skip the Deploy stage and manually update Bedrock prompts after review.

E) Use CodeDeploy to directly deploy prompts to Bedrock without intermediate testing, relying on Bedrock's built-in safety checks.

Click to reveal Answer & Explanation

Correct Answer: A

Correct Option Analysis

A is Correct: The pipeline structure separates source, build/test, and deploy stages. CodeBuild executes evaluation scripts that score prompt variants against a golden dataset. If the aggregate score fails the threshold, the Build stage fails, preventing progression to Deploy. This is consistent with automated testing in ML CI/CD pipelines where CodeBuild gates deployments on evaluation metrics.

Distractor Analysis

B is Incorrect: Manual approval gates require human intervention and do not automate testing. Combining evaluation and deployment in a single CodeBuild stage eliminates the separation of gating and releasing.
C is Incorrect: Using only CodeBuild eliminates the orchestration, versioning, and staging capabilities that CodePipeline provides. Deployment without a pipeline lacks reproducibility and versioned release control.
D is Incorrect: Skipping the deploy stage and manually updating prompts defeats the purpose of CI/CD automation, introduces human latency, and lacks repeatability for prompt versions.
E is Incorrect: Bedrock safety checks (Guardrails) handle content filtering and safety, but do not evaluate prompt response quality against a golden dataset. They cannot replace automated quality gating.

Key Takeaway: CodePipeline stages enforce build-test-deploy sequencing, allowing CodeBuild evaluation to gate deployments based on prompt quality scores.

Question 9 - CodeBuild Troubleshooting: IAM Permission

An ML engineer runs a CodeBuild project that executes a SageMaker training job submission using the AWS CLI. The build fails with the error: An error occurred (AccessDeniedException) when calling the CreateTrainingJob operation: User: arn:aws:sts::123456789012:assumed-role/CodeBuildRole is not authorized to perform: sagemaker:CreateTrainingJob.

What is the root cause and the correct fix?

Select ONE that applies.

A) The CodeBuild project's service role lacks the sagemaker:CreateTrainingJob permission. Attach the AmazonSageMakerFullAccess managed policy to the role.

B) The SageMaker training job requires DescribeTrainingJob permission instead of CreateTrainingJob.

C) The buildspec.yml file is missing the sagemaker client import statement.

D) The SageMaker endpoint does not exist in the same region as the CodeBuild project.

E) The IAM role used by the training job does not have S3 read permissions.

Click to reveal Answer & Explanation

Correct Answer: A

Correct Option Analysis

A is Correct: The error message explicitly indicates that the IAM role assumed by CodeBuild (the CodeBuild service role) lacks the sagemaker:CreateTrainingJob permission. The fix is to attach a policy that grants this permission, such as AmazonSageMakerFullAccess or a custom inline policy with the required action.

Distractor Analysis

B is Incorrect: The error shows the action being attempted is CreateTrainingJob. DescribeTrainingJob is a different API call and is not relevant to submitting a training job.
C is Incorrect: Missing Python SDK imports would result in a Python import error or syntax error, not an IAM AccessDenied error. The AWS CLI or boto3 would raise AccessDenied for IAM issues.
D is Incorrect: If the endpoint does not exist, the error would be ResourceNotFoundException, not AccessDeniedException. The training job does not require a pre-existing SageMaker endpoint.
E is Incorrect: The role used by the training job itself (execution role) is different from the CodeBuild service role. This error indicates CodeBuild's role lacks the permission to create the training job, not the training job's inability to read S3.

Key Takeaway: IAM AccessDenied errors identify the specific API action and the role that lacks permission; attach the accurate policy to the CodeBuild service role.

Question 10 - CI/CD Pipeline for Bedrock Fine-Tuned Model with Versioning

A media company wants to automate the deployment of a fine-tuned Amazon Bedrock foundation model. The pipeline must track model versions, associate each version with the exact training dataset and hyperparameters, and support rollback to any previous version for audit purposes. The team already uses Amazon SageMaker Pipelines for training and SageMaker Model Registry for model versioning.

Which CodePipeline and CodeBuild configuration should the ML engineer implement to meet these requirements?

Select ONE that applies.

A) Use CodePipeline stages: Source (S3 training data) → Build (CodeBuild running SageMaker Pipeline that trains and registers model version) → Deploy (CodeDeploy to deploy the Bedrock model version using the registry's approved version). Model Registry tracks lineage and metadata for audit.

B) Store model artifacts in S3 and use CodeDeploy to deploy models directly from S3 without a registry, as CodePipeline can handle versioning automatically.

C) Use CodePipeline with a single Source stage that deploys fine-tuned Bedrock models directly to production without Model Registry.

D) Rely on Bedrock's built-in versioning and bypass both SageMaker Pipelines and Model Registry for simplification.

E) Use CodeBuild to manually update model configurations in Bedrock and create a version tag in the application code.

Click to reveal Answer & Explanation

Correct Answer: A

Correct Option Analysis

A is Correct: This architecture integrates CodePipeline as the release orchestration layer, CodeBuild as the build/validation stage that triggers SageMaker Pipelines for training and registers the model version in SageMaker Model Registry, and CodeDeploy for deployment. The Model Registry stores lineage metadata (training data, hyperparameters, evaluation metrics) enabling audit and rollback across any previous version. This aligns with AWS best practices for MLOps CI/CD with foundation models.

Distractor Analysis

B is Incorrect: Deploying directly from S3 without a registry loses version lineage and audit capability. CodePipeline does not inherently track model versions; the Model Registry is essential for version management.
C is Incorrect: Skipping Model Registry eliminates version lineage and audit records, which are explicitly required. Fine-tuned Bedrock models require version tracking that only a registry provides.
D is Incorrect: Bedrock built-in versioning is limited and does not capture training lineage (dataset, hyperparameters). AWS best practice recommends using SageMaker Model Registry for comprehensive version management.
E is Incorrect: Manual configuration updates violate automation principles and lack the reproducibility and audit trail of a CI/CD pipeline with Model Registry.

Key Takeaway: A complete MLOps pipeline integrates CodePipeline for orchestration, CodeBuild for validation, and SageMaker Model Registry for version lineage and audit.