Skill 4.3.5: Troubleshoot and debug security issues in ML and AI systems
| Question | Answer & Explanation |
|---|---|
| ### Question 1 - Troubleshooting SageMaker Endpoint Access Denied A machine learning engineer is troubleshooting an AccessDeniedException when an AWS Lambda function attempts to invoke an Amazon SageMaker real-time endpoint. The Lambda function's execution role currently has the AmazonSageMakerFullAccess managed policy attached, but the security team mandates strict least privilege and requires the error to be resolved with the minimum necessary permissions.Which ONE action should the engineer take to debug and resolve this issue securely? - A) Add a resource-based policy to the SageMaker endpoint granting lambda.amazonaws.com full access.- B) Remove the managed policy and create a custom inline policy granting sagemaker:InvokeEndpoint specifically on the endpoint ARN.- C) Attach the AdministratorAccess managed policy to the Lambda execution role temporarily to clear the error.- D) Configure an interface VPC endpoint for SageMaker so the traffic bypasses IAM checks. |
Correct Answer: B Correct Answer Analysis: - B is Correct: The error is an IAM authorization failure. The best practice for least privilege is to grant only the specific action ( sagemaker:InvokeEndpoint) on the specific resource (the endpoint ARN) rather than using overly broad managed policies.Distractor Analysis: - A is Incorrect: While resource-based policies can be used, granting lambda.amazonaws.com full access without scoping it to the specific Lambda role violates the least-privilege mandate.- C is Incorrect: Attaching AdministratorAccess temporarily is an anti-pattern and violates compliance policies. It masks the root cause rather than fixing the specific permission gap.- D is Incorrect: VPC endpoints route traffic privately over the AWS network but do not bypass IAM authentication and authorization checks. > Exam Tip / AWS Best Practice: When troubleshooting AccessDenied errors, scope down permissions to the exact required action (sagemaker:InvokeEndpoint) on the specific resource ARN rather than expanding access. |
| ### Question 2 - Troubleshooting Private Subnet Connectivity An Amazon SageMaker processing job is configured to run in a private subnet with no internet route. The job attempts to read training data from an Amazon S3 bucket and pull a custom Docker image from Amazon ECR, but it fails with a network timeout error. The security team requires that the subnet remains completely isolated from the internet. Which ONE combination of actions will resolve this issue while maintaining the network isolation constraint? (Select TWO) - A) Add a NAT Gateway to the private subnet to allow outbound internet traffic. - B) Create a VPC Gateway Endpoint for Amazon S3. - C) Create a VPC Interface Endpoint for Amazon ECR. - D) Attach an Internet Gateway to the VPC and route the private subnet traffic through it. - E) Configure an S3 bucket policy to allow public read access for the training data. |
Correct Answers: B and C Correct Answer Analysis: - B is Correct: A VPC Gateway Endpoint for S3 allows resources in a private subnet to access S3 without traversing the internet. - C is Correct: ECR requires interface endpoints (for the API and Docker registries) to pull container images privately from an isolated subnet. Distractor Analysis: - A is Incorrect: A NAT Gateway routes traffic to the internet, violating the constraint that the subnet must remain isolated from the internet. - D is Incorrect: An Internet Gateway provides public internet access, violating the isolation requirement. - E is Incorrect: Making the S3 bucket public does not solve the network routing timeout from the private subnet, and it introduces a massive security risk. > Exam Tip / AWS Best Practice: Private subnets without internet access must use VPC endpoints (Gateway for S3, Interface for ECR) to communicate with AWS services. |
| ### Question 3 - Troubleshooting Missing Model Invocation Logs A security auditor is investigating an Amazon SageMaker real-time endpoint and requests the specific payload data sent to and received from the model over the last 30 days. The engineering team has an AWS CloudTrail trail active in the account, but when they search for InvokeEndpoint events, no results are returned.Which ONE action must the engineer take to ensure these specific calls are recorded for future auditing? - A) Enable data events for Amazon SageMaker on the existing CloudTrail trail. - B) Enable AWS Config rules for Amazon SageMaker. - C) Turn on Amazon CloudWatch Container Insights for the SageMaker endpoint. - D) Update the IAM execution role attached to the SageMaker endpoint to allow cloudtrail:PutEvent. |
Correct Answer: A Correct Answer Analysis: - A is Correct: By default, CloudTrail records management events (like CreateEndpoint). InvokeEndpoint is a high-volume data event. It must be explicitly enabled on the trail to capture model invocation requests.Distractor Analysis: - B is Incorrect: AWS Config evaluates resource configuration states and compliance, not individual API invocation payloads. - C is Incorrect: CloudWatch Container Insights monitor performance metrics (CPU, memory), not API request payloads for security auditing. - D is Incorrect: SageMaker does not push CloudTrail events based on its execution role; CloudTrail is an account-level service that records events automatically when configured correctly. > Exam Tip / AWS Best Practice: InvokeEndpoint and InvokeModel are data-plane operations (data events) in CloudTrail and must be explicitly enabled on a trail to be logged. |
| ### Question 4 - Troubleshooting Vulnerable Model Containers A DevOps team is building a CI/CD pipeline for a custom machine learning model. The pipeline builds a Docker container image containing the model and its dependencies. A recent security audit revealed that an outdated, vulnerable version of the numpy library was deployed to production, despite the team using Amazon CodeGuru Security.Which ONE service should the team add to the pipeline to detect vulnerabilities in the assembled container image layers? - A) Amazon Inspector - B) AWS Secrets Manager - C) Amazon Macie - D) AWS Config |
Correct Answer: A Correct Answer Analysis: - A is Correct: Amazon Inspector scans container images (e.g., in Amazon ECR) for software vulnerabilities and CVEs in the operating system and installed packages (like numpy).Distractor Analysis: - B is Incorrect: Secrets Manager is for storing and rotating credentials, not scanning code or packages. - C is Incorrect: Amazon Macie is for discovering and protecting sensitive data (PII) in Amazon S3, not for scanning container images for library vulnerabilities. - D is Incorrect: AWS Config tracks resource configuration drift, not software dependencies inside a Docker image. - Note on CodeGuru: CodeGuru Security performs static analysis on source code, but it does not scan assembled binaries or container image OS layers for known CVEs. > Exam Tip / AWS Best Practice: Use CodeGuru for static code analysis (hardcoded secrets, logic flaws) and Amazon Inspector for dynamic image/vulnerability scanning (CVEs in assembled packages). |
| ### Question 5 - Troubleshooting PII Leakage in Bedrock Responses An enterprise is using an Amazon Bedrock-powered chatbot to summarize customer support transcripts. The compliance team notices that the model occasionally outputs customer credit card numbers and Social Security Numbers (SSNs) in its summaries. The team mandates that this sensitive data must be masked before returning the response to the user. Which ONE action will resolve this issue without retraining the model? - A) Implement a Contextual Grounding check in Amazon Bedrock Guardrails. - B) Configure a Denied Topic in Amazon Bedrock Guardrails. - C) Apply a Sensitive Information filter in Amazon Bedrock Guardrails. - D) Enable AWS CloudTrail data events for the InvokeModel API. |
Correct Answer: C Correct Answer Analysis: - C is Correct: Bedrock Guardrails Sensitive Information filters can detect, mask, or block PII (like credit cards and SSNs) in both user prompts and model responses. Distractor Analysis: - A is Incorrect: Contextual Grounding checks evaluate whether the response is supported by the provided reference text (hallucination prevention), not PII masking. - B is Incorrect: Denied Topics block specific subject matters (e.g., investment advice); they do not parse text to mask specific data types like SSNs. - D is Incorrect: CloudTrail data events record the API calls for auditing, but they do not redact or block the payload sent to the user in real-time. > Exam Tip / AWS Best Practice: To prevent PII leakage in LLM outputs, use Amazon Bedrock Guardrails' Sensitive Information filters to mask or block the data in real-time. |
| ### Question 6 - Troubleshooting RAG Hallucinations in Bedrock An AI application uses Amazon Bedrock Knowledge Bases for a Retrieval-Augmented Generation (RAG) workflow. The application retrieves documents from an OpenSearch Serverless index, but users report that the model frequently returns factual claims that are not supported by the retrieved documents (hallucinations). The team needs to block these unsupported responses automatically. Which ONE Bedrock Guardrails feature should be configured to address this specific issue? - A) Sensitive Information filters - B) Content filters - C) Contextual Grounding checks - D) Word filters |
Correct Answer: C Correct Answer Analysis: - C is Correct: Contextual Grounding checks in Bedrock Guardrails evaluate if the model's response is factually grounded in the provided source references (retrieved documents). If the response is not supported, it can be blocked. Distractor Analysis: - A is Incorrect: Sensitive Information filters mask PII; they do not evaluate the factual accuracy of a response against a source. - B is Incorrect: Content filters block harmful or toxic content (hate speech, violence), not factual hallucinations. - D is Incorrect: Word filters block specific terms or profanity, not unsupported factual claims. > Exam Tip / AWS Best Practice: For RAG applications, use Contextual Grounding checks to validate that model responses are derived directly from the retrieved context, preventing hallucinations. |
| ### Question 7 - Troubleshooting Hardcoded API Keys in Code A developer committed a Python script to an AWS CodePipeline repository that triggers an Amazon SageMaker training job. The script failed a mandatory security gate because it contained a hardcoded AWS access key and secret key used to assume an IAM role. Which ONE AWS service is explicitly designed to detect this hardcoded credential vulnerability during the code build phase? - A) Amazon Inspector - B) AWS Secrets Manager - C) Amazon CodeGuru Security - D) AWS IAM Access Analyzer |
Correct Answer: C Correct Answer Analysis: - C is Correct: Amazon CodeGuru Security performs static application security testing (SAST) and can automatically detect hardcoded secrets and credentials within the source code. Distractor Analysis: - A is Incorrect: Amazon Inspector scans running EC2 instances, Lambda functions, and container images for OS/package vulnerabilities, not source code text for hardcoded strings. - B is Incorrect: Secrets Manager is where you should store the secrets securely, but it does not scan the code to find the hardcoded credentials. - D is Incorrect: IAM Access Analyzer identifies resources shared with external entities, but it does not scan application code for hardcoded strings. > Exam Tip / AWS Best Practice: Use CodeGuru Security to scan code repositories for hardcoded credentials and insecure code patterns before they are deployed. |
| ### Question 8 - Troubleshooting Bedrock Access from On-Premises A financial institution has a strict compliance policy that requires all traffic between their on-premises data center and AWS services to traverse a private network connection, avoiding the public internet. They are using an AWS Direct Connect connection and need to invoke Amazon Bedrock models from their on-premises applications. Which ONE action should the network engineer take to securely route this traffic? - A) Create a VPC Interface Endpoint for Amazon Bedrock in a private subnet. - B) Set up an Internet Gateway in the VPC and route traffic through a public subnet. - C) Use a NAT Gateway to translate the on-premises IP addresses to a public IP. - D) Configure a VPC Gateway Endpoint for Amazon Bedrock. |
Correct Answer: A Correct Answer Analysis: - A is Correct: Amazon Bedrock supports VPC Interface Endpoints (AWS PrivateLink). This allows traffic from an on-premises network (via Direct Connect) to reach Bedrock API endpoints entirely over the private AWS network without traversing the public internet. Distractor Analysis: - B is Incorrect: An Internet Gateway routes traffic over the public internet, directly violating the compliance requirement. - C is Incorrect: A NAT Gateway allows private subnets to access the internet, but it still uses the public internet pathway, violating the compliance mandate. - D is Incorrect: VPC Gateway Endpoints are only supported for Amazon S3 and Amazon DynamoDB. Bedrock uses Interface Endpoints. > Exam Tip / AWS Best Practice: To access AWS AI services like Bedrock privately from on-premises over Direct Connect, use a VPC Interface Endpoint (AWS PrivateLink). |
| ### Question 9 - Troubleshooting Credential Expiration in Production A production ML inference application running on Amazon EC2 instances is making calls to Amazon Bedrock. The application was initially configured using a long-term AWS access key stored in an environment variable. The security team has mandated that this static key be removed. After replacing it with an IAM role attached to the EC2 instance, the application is failing to authenticate with Bedrock after 1 hour of running. Which ONE issue is the most likely cause of this failure? - A) The EC2 instance profile role does not have bedrock:InvokeModel permissions.- B) The application is caching the temporary credentials locally and not refreshing them before they expire. - C) The IAM role's trust policy does not allow ec2.amazonaws.com to assume it.- D) The application is attempting to use the temporary credentials to call an AWS service in a different Region. |
Correct Answer: B Correct Answer Analysis: - B is Correct: IAM roles provide temporary credentials via the EC2 metadata service, which expire and rotate regularly (typically every 1 hour). If the application caches these credentials in memory instead of requesting them from the metadata service when needed, it will fail with an authentication error once they expire. Distractor Analysis: - A is Incorrect: If the role lacked bedrock:InvokeModel permissions, the application would fail immediately from the start, not after 1 hour.- C is Incorrect: If the trust policy was incorrect, the EC2 instance would not have been able to assume the role and retrieve the initial credentials at all. - D is Incorrect: IAM temporary credentials are valid globally across all Regions, not just the Region where the role was assumed. > Exam Tip / AWS Best Practice: When migrating from static keys to IAM roles, ensure the application code relies on the AWS SDK's default credential provider chain, which automatically handles temporary credential refresh cycles. |
| ### Question 10 - Troubleshooting Security Group Rules for SageMaker Endpoints A company has deployed an Amazon SageMaker real-time endpoint inside a private VPC subnet. A front-end application running on an EC2 instance in a different public subnet needs to send inference requests to this endpoint. However, the requests are timing out. The network team confirms that the routing tables are correctly configured between the subnets. Which ONE network configuration step must be verified to resolve this timeout issue? - A) The security group attached to the SageMaker endpoint must have an inbound rule allowing traffic on port 443 from the EC2 instance's security group. - B) The security group attached to the SageMaker endpoint must have an outbound rule allowing traffic on port 80 to the EC2 instance's security group. - C) The EC2 instance must have a public IP address to route traffic to the private subnet. - D) The SageMaker endpoint must be attached to an Internet Gateway. |
Correct Answer: A Correct Answer Analysis: - A is Correct: SageMaker real-time endpoints in a VPC operate over HTTPS (port 443). If traffic is timing out despite correct routing, it indicates a security group blocking the traffic. The endpoint's security group must allow inbound TCP 443 from the application's security group. Distractor Analysis: - B is Incorrect: The outbound rule from the endpoint would be relevant for the endpoint's response traffic, but security groups are stateful, meaning return traffic is automatically allowed. Furthermore, port 80 (HTTP) is not used for SageMaker endpoints. - C is Incorrect: Traffic within a VPC does not require a public IP address; private IP routing is sufficient and preferred for security. - D is Incorrect: A SageMaker endpoint in a private subnet does not need an Internet Gateway to receive traffic from another subnet within the same VPC. > Exam Tip / AWS Best Practice: Security groups are stateful. When troubleshooting VPC connectivity to SageMaker endpoints, ensure the endpoint's security group allows inbound TCP 443 (HTTPS) from the client's security group. |