Task 4.3 - Secure ML and AI workloads and model endpoints

Overview

Task 4.3, Secure ML and AI workloads and model endpoints, is about controlling who and what can reach a model, the artifacts around it, and the pipeline that produced it.

The controls are the ones already familiar from other workloads, applied to new objects such as training data, an inference endpoint, or a model invocation call. Each object needs its own answer.

As you review these lessons for Task 4.3, check that you understand how to do the following.

Secure continuous integration and continuous delivery (CI/CD) pipelines by checking for code and image vulnerabilities - (for example, by using Amazon CodeGuru, Amazon Inspector).
Configure least privilege access to ML and AI artifacts.
Configure IAM policies and roles for users and applications in ML and AI systems.
Configure comprehensive monitoring, auditing, compliance, and logging for ML and AI systems (for example, AWS - CloudTrail, AWS Config).
Troubleshoot and debug security issues in ML and AI systems.
Create VPCs, subnets, and security groups to securely isolate ML and AI systems.
Identify and mitigate security risks and vulnerabilities in ML and AI systems.
Select the appropriate credential type to access FMs (for example, Amazon Bedrock API keys, IAM credentials).
Implement safeguards and sensitive data protection to meet application requirements and responsible AI policies (for example, by using Amazon Bedrock Guardrails).

AWS services overview

These services group into four jobs: granting access, isolating the network path, recording what happened for later, and inspecting code, images, and model traffic for risk. A gap in this task is almost always one of those four jobs missing entirely rather than a control configured slightly wrong. Review each service to learn more.

AWS Service / Feature	Description
AWS Identity and Access Management (IAM)	Defines which identities and services may call which API operations on which resources. This is where a permission boundary for a model, a dataset, or an endpoint is actually written.
Amazon Virtual Private Cloud (Amazon VPC)	The private network where subnets and security groups decide what a training job, a notebook, or an endpoint is able to reach and what is able to reach it.
AWS PrivateLink	Connects a VPC to a service through an interface endpoint, so calls reach the service over private addresses instead of traversing the internet.
AWS Key Management Service (AWS KMS)	Manages the keys that encrypt datasets, model artifacts, and stored output, and records key use so decryption is attributable.
AWS CloudTrail	Records API activity for an account: which principal called which operation, when, and from where. Trails record management events, and data events are configured separately.
AWS Config	Records how resources are configured over time and evaluates them against rules, which turns a drifted security setting into a finding rather than a surprise.
Amazon Bedrock Guardrails	Applies configurable policies to prompts and responses, including content filters, denied topics, word filters, sensitive information filters, contextual grounding checks, and Automated Reasoning checks.
Amazon CodeGuru Security	Static analysis that finds security policy violations and vulnerabilities in application code, including credentials that were hardcoded into it.
Amazon Inspector	Continually scans workloads it discovers, including Amazon EC2 instances, container images in Amazon Elastic Container Registry, and AWS Lambda functions, for software vulnerabilities and unintended network exposure.
AWS Secrets Manager	Stores and rotates secrets such as an API key, so an application retrieves the credential at run time instead of carrying one in code or in an image.

Knowledge Check

An audit asks which principal invoked a model, at what time, and from where. Which service records that activity? (Choose 1)

A. AWS CloudTrail
B. AWS Config
C. Amazon Inspector
D. AWS Secrets Manager

Answer & Explanation

Correct Answer: A. AWS CloudTrail

Explanation:
AWS CloudTrail captures and records API activity and events across your AWS account. It logs event metadata including the user or role identity (principal), the exact timestamp of the call, the source IP address (from where), and the specific API operation invoked (such as SageMaker InvokeEndpoint or Bedrock InvokeModel).

Why the other options are incorrect: * B. AWS Config: AWS Config monitors and evaluates resource configurations over time against compliance rules; it does not log individual runtime API request events or caller details. * C. Amazon Inspector: Amazon Inspector continually scans EC2 instances, ECR container images, and Lambda functions for software vulnerabilities and unintended network exposures, rather than tracking API access logs. * D. AWS Secrets Manager: AWS Secrets Manager stores, manages, and rotates database credentials and API keys securely, having no role in logging account-wide service invocations.

AWS skills overview

In this lesson, you review the skills behind Task 4.3 as a light refresher. Each task groups a handful of related skills into a few topics. Read through them and notice which ones already feel familiar, and which ones feel shaky enough that you want to come back to them before exam day.

Granting access and choosing credentials

In practice, this group of skills lets the machine learning engineer give a person or an application exactly the access the job needs and nothing beyond it.

Be comfortable doing the following:

Scoping access to machine learning and AI artifacts to the smallest set of - operations and resources that the work requires, and revisiting that scope when - the work changes.
Writing policies and roles for the users and the applications in a machine - learning system, and keeping a human identity separate from a workload identity.
Selecting a credential type for reaching a foundation model, and knowing which - type belongs in production.

The credential question is the newest part of this group. A short-term API key for a foundation model inherits the permissions of the identity that generated it and expires with that session, which makes it convenient for exploration. A long-term API key is backed by its own identity with an expiry that is set for it, and AWS documents it for exploration and development rather than for production. Production access belongs on a role with temporary credentials, which is the same conclusion the rest of the platform reaches, and where a static secret genuinely has to exist it belongs in a secrets store that rotates it rather than in code, an image, or an environment file.

Knowledge Check

An application running in production needs to invoke a foundation model. Which credential approach fits that requirement? (Choose 1)

A. A role that the application assumes, providing temporary credentials
B. A long-term API key generated once and stored in the application image
C. A short-term API key generated by an engineer at the start of each shift
D. A shared key held in an environment variable on each host

Answer & Explanation

Correct Answer: A. A role that the application assumes, providing temporary credentials

Explanation:
AWS security best practices mandate using IAM Roles (such as EC2 Instance Profiles, ECS Task Roles, or Lambda Execution Roles) to grant production workloads temporary, automatically rotated security credentials via AWS Security Token Service (STS). This avoids embedding long-lived credentials into application code or host environments while adhering to the principle of least privilege when invoking foundation models on Amazon Bedrock or SageMaker.

Why the other options are incorrect: * B. A long-term API key generated once and stored in the application image: Baking static, long-lived credentials into container images creates a critical security risk, makes key rotation difficult, and risks exposure if the image is shared or leaked. * C. A short-term API key generated by an engineer at the start of each shift: Manual credential management introduces significant operational overhead and human error, improperly coupling automated production systems to human user identities. * D. A shared key held in an environment variable on each host: Static shared keys lack auditability across individual hosts, require manual rotation strategies, and risk exposing sensitive secrets via host process environment dumps.

Isolating ML and AI systems on the network

In practice, this group of skills lets the machine learning engineer put a model and its data somewhere the internet cannot reach.

Be comfortable doing the following:

Building the network a machine learning system runs in, with subnets that - separate what is reachable from what is not, and security groups that state - which traffic is allowed.
Attaching training jobs, notebooks, and hosted endpoints to that network, so - the containers behind them communicate inside it.
Reaching AWS service APIs from inside that network through an interface endpoint, so a private subnet does not need a route to the internet in order to work.

Network isolation is where two requirements meet and sometimes collide. Removing internet access from a subnet is straightforward, and it also removes the path the workload used to call the service APIs it depends on, so an interface endpoint, or a managed path out, has to replace it or the job simply fails. The other half is direction. A security group that permits inbound traffic broadly is the finding that gets written up, and the outbound direction is where data leaves, so both need stating. Private connectivity controls the path traffic takes and says nothing about the content that travels on it, which is why this topic and the safeguards topic later in this lesson are separate controls rather than substitutes.

Knowledge Check

A training job runs in a private subnet with no internet route and fails when it tries to read from object storage. Which change resolves it without opening the subnet to the internet? (Choose 1)

A. Attach a broader inbound rule to the job's security group
B. Add an interface endpoint for the service so the call stays on the private network
C. Encrypt the object storage bucket with a customer managed key
D. Grant the job's role permission to call the storage API

Answer & Explanation

Correct Answer: B. Add an interface endpoint for the service so the call stays on the private network

Explanation:
When resources in an isolated private subnet without internet access (no NAT Gateway or Internet Gateway) need to communicate with AWS public services like Amazon S3 (object storage), creating a VPC Endpoint (Gateway or Interface Endpoint via AWS PrivateLink) establishes a private network route. Traffic travels entirely over the secure AWS network backbone rather than the public internet.

Why the other options are incorrect: * A. Attach a broader inbound rule to the job's security group: Security group inbound rules regulate inbound connections to the training instances; they do not resolve outbound routing failures to external AWS service endpoints. * C. Encrypt the object storage bucket with a customer managed key: AWS KMS encryption protects data at rest within object storage, but it does not address underlying network connectivity or routing issues. * D. Grant the job's role permission to call the storage API: The scenario describes a network-level routing failure ("no internet route" prevents connecting to the endpoint), not an IAM authorization denial (AccessDenied).

Auditing, logging, and monitoring for compliance

In practice, this group of skills lets the machine learning engineer answer what happened, when, and whether the system was configured the way policy requires.

Be comfortable doing the following:

Recording API activity for a machine learning system, so an action can be - attributed to a principal after the fact.
Recording resource configuration over time and evaluating it against rules, so - a setting that drifts out of policy becomes a finding.
Capturing model invocation input and output where policy requires it, and - holding those records under the same protections as the data itself.
Reading the three records together, because each answers a different question and none answers all of them.

The distinction to keep is what each record is for. An activity record answers who did it. A configuration record answers what the resource looked like at that moment. An invocation record answers what went in and what came back. A compliance question usually needs two of the three, and the common gap is assuming one of them covers more than it does: a trail records management events, and the data-plane operations that read an object or invoke a model are configured separately, so an audit that expects to see model invocation calls in a trail that was never configured for data events finds nothing and concludes wrongly that nothing happened.

Knowledge Check

An investigation needs the record of every model invocation call made in an account, and the existing trail was created without additional configuration. What has to change? (Choose 1)

A. Data events have to be configured on the trail
B. A configuration rule has to be added for the endpoint
C. The trail has to be moved into the same Region as the endpoint
D. Encryption has to be enabled on the trail's storage location

Answer & Explanation

Correct Answer: A. Data events have to be configured on the trail

Explanation:
By default, a standard AWS CloudTrail trail records only management events (control-plane administrative actions, such as CreateEndpoint, CreateModel, or DeleteEndpoint). Model invocation calls (such as Amazon SageMaker InvokeEndpoint or Amazon Bedrock InvokeModel / Converse) are classified as data events (data-plane operations). Because data events represent high-volume operational activity, they must be explicitly enabled and configured on the CloudTrail trail to be captured.

Why the other options are incorrect: * B. A configuration rule has to be added for the endpoint: AWS Config rules evaluate resource compliance and configuration drift over time; they do not record individual API invocation calls. * C. The trail has to be moved into the same Region as the endpoint: A single multi-Region CloudTrail trail records events across all AWS Regions automatically without needing to be moved. * D. Encryption has to be enabled on the trail's storage location: Enabling AWS KMS encryption on the destination S3 bucket secures the log files at rest, but it has no effect on which event types (management vs. data) CloudTrail captures.

Finding and fixing risks in pipelines and models

In practice, this group of skills lets the machine learning engineer catch a weakness in the build path before it reaches a production endpoint.

Be comfortable doing the following:

Scanning the code and the container images that a delivery pipeline produces, and treating a finding as a gate rather than a report nobody reads.
Identifying the risks that are specific to a machine learning system, including an over-permissive role on a training job, an artifact store that is readable more widely than intended, an unvalidated model artifact pulled from outside the organization, and an inference path reachable from the internet.
Working a security problem back to its cause, by reading the denial or the finding rather than widening a permission until the error disappears.

Two points are worth naming. Static analysis of application code and vulnerability scanning of a built image answer different questions, so a pipeline needs both: one looks for insecure patterns and hardcoded credentials in what was written, the other looks for known vulnerable packages in what was assembled. And the fastest wrong fix in this task is widening a policy to clear an access-denied error. The error names the operation and the resource that were refused, which is the information needed to grant that operation on that resource, and a broader grant that makes the message disappear has changed the symptom rather than the cause.

Knowledge Check

A delivery pipeline for a machine learning application already runs static analysis on the application code. Which gap does adding image scanning close? (Choose 1)

A. Known vulnerabilities in the packages assembled into the built image
B. Over-permissive permissions on the training job's role
C. Prompts that attempt to make the model produce disallowed content
D. Configuration drift on the deployed endpoint

Answer & Explanation

Correct Answer: A. Known vulnerabilities in the packages assembled into the built image

Explanation:
While static code analysis (such as Amazon CodeGuru Security) inspects custom application source code for syntax flaws, logic bugs, and hardcoded secrets, it does not inspect underlying operating system layers or pre-compiled third-party binaries. Container image scanning (such as Amazon Inspector or Amazon ECR image scanning) analyzes the assembled container image layers, scanning base OS packages and Python/system dependencies against Common Vulnerabilities and Exposures (CVE) databases to detect known software vulnerabilities.

Why the other options are incorrect: * B. Over-permissive permissions on the training job's role: IAM permissions and least-privilege policies are evaluated using IAM Access Analyzer, IAM policy validators, or security audits—not container image scanning. * C. Prompts that attempt to make the model produce disallowed content: Adversarial prompts and harmful inputs/outputs are intercepted and filtered at runtime using Amazon Bedrock Guardrails or prompt safety frameworks. * D. Configuration drift on the deployed endpoint: Resource state changes and configuration drift over time are monitored and flagged by AWS Config.

Applying safeguards to model inputs and outputs

In practice, this group of skills lets the machine learning engineer put a policy between a user and a model, in both directions.

Be comfortable doing the following:

Configuring safeguards that evaluate a prompt before the model sees it and a response before a user sees it, and deciding for each policy whether a match is blocked or masked.
Matching the policy to the requirement: harmful content categories, topics the application refuses to discuss, specific terms, sensitive values such as personal information, a claim unsupported by the source passage, and a statement that has to be checked against stated rules.
Protecting sensitive data through the whole path, so a value that is masked in a response is not sitting unmasked in a log.

Safeguards are configured per policy, and the policy list is the part to recognize rather than memorize: content filters, denied topics, word filters, sensitive information filters, contextual grounding checks, and Automated Reasoning checks. Two of them are worth separating in your head, because they sound alike and answer different questions. A contextual grounding check asks whether the response is supported by the source material the application retrieved, which is the check a retrieval application needs. An Automated Reasoning check asks whether a statement is consistent with rules that were expressed formally, which is the check a policy-bound answer needs. Both apply to output, and neither replaces the access controls earlier in this lesson, because a safeguard filters content and an access control decides who could call the model at all.

Knowledge Check

A retrieval application must stop returning answers that the retrieved passages do not support. Which safeguard addresses that? (Choose 1)

A. A contextual grounding check on the response
B. A denied topic for the subject area
C. A word filter listing the terms to block
D. A sensitive information filter on the response

Answer & Explanation

Correct Answer: A. A contextual grounding check on the response

Explanation:
In Retrieval-Augmented Generation (RAG) applications, Amazon Bedrock Guardrails Contextual Grounding checks evaluate hallucination risks by validating that generated responses are factually grounded in the reference text (the retrieved passages provided in the prompt context). If the model generates claims or answers unsupported by those source passages, the contextual grounding policy flags or blocks the response.

Why the other options are incorrect: * B. A denied topic for the subject area: Denied topic filters block conversations and requests surrounding specific predefined domain categories or prohibited business subjects (e.g., medical diagnoses or investment advice), regardless of whether the answer is factually grounded. * C. A word filter listing the terms to block: Word filters block specific profanity, offensive language, or designated custom terms; they cannot evaluate semantic factual alignment between a model response and source passages. * D. A sensitive information filter on the response: Sensitive information filters detect, mask, or block Personally Identifiable Information (PII) such as Social Security numbers, phone numbers, or credit card details, rather than detecting hallucinations.

Knowledge Check

An AI assistant must be prevented from returning customer account numbers in its responses. Which configuration addresses that requirement? (Choose 1)

A. An interface endpoint, so that model invocation calls stay on the private network
B. Data events configured on the trail that records model invocation calls
C. An Amazon Inspector scan of the container image that hosts the application
D. A sensitive information filter in a guardrail, applied to model responses

Answer & Explanation

Correct Answer: D. A sensitive information filter in a guardrail, applied to model responses

Explanation:
Amazon Bedrock Guardrails provides sensitive information filters designed specifically to detect, mask, or block Personally Identifiable Information (PII)—such as customer account numbers, Social Security numbers, credit card details, or custom regex patterns—in both user inputs and generated model outputs before they reach the user.

Why the other options are incorrect: * A. An interface endpoint, so that model invocation calls stay on the private network: Interface endpoints (AWS PrivateLink) restrict API traffic to the private network backbone, preventing exposure to the public internet, but they do not inspect or redact text content within payload responses. * B. Data events configured on the trail that records model invocation calls: AWS CloudTrail data events record data-plane API calls for audit logging and security analysis; they operate passively after the API call completes and cannot redact or block text outputs in real time. * C. An Amazon Inspector scan of the container image that hosts the application: Amazon Inspector inspects container images and OS layers for software package vulnerabilities (CVEs) and network exposure risks, having no visibility into runtime conversational AI outputs.