Skip to content

Task Statement 5.1: Explain methods to secure AI systems

Objectives:

  • Identify AWS services and features to secure AI systems (for example, IAM roles, policies, and permissions; encryption; Amazon Macie; AWS PrivateLink; AWS shared responsibility model; Amazon Bedrock AgentCore Identity; Policy in AgentCore; Amazon Bedrock Guardrails).
  • Describe the concept of source citation and documenting data origins (for example, data lineage, data cataloging, Amazon SageMaker Model Cards).
  • Describe best practices for secure data engineering (for example, assessing data quality, implementing privacy-enhancing technologies, data access control, data integrity).
  • Describe security and privacy considerations for AI systems (for example, application security, threat detection, vulnerability management, infrastructure protection, prompt injection, encryption at rest and in transit, data leakage prevention, output filtering and validation, audit trail and logging requirements for AI interactions, toxicity).
  • Describe hallucination detection methods and grounding techniques to improve output accuracy (for example, Retrieval Augmented Generation [RAG] grounding, output validation, confidence scoring).

Lesson 1

et's get started with the first task statement from Domain 5, which is to explain methods to secure AI systems. This task statement is split into six lessons.

On AWS, security and compliance is a shared responsibility between AWS and the customer. To explain the division of responsibility, AWS created the Shared Responsibility Model. The Shared Responsibility Model shows the division of customer and AWS responsibilities. These divided responsibilities are commonly referred to as customer responsibilities, or security in the cloud, and AWS responsibilities, or security of the cloud. The cloud itself, the data centers, networking and physical hardware is secured by AWS. AWS makes these resources available to customers as cloud-based web services. The customers are responsible for using these services in a secure manner by limiting access, using encryption and following recommended best practices. AWS protects and secures the global infrastructure, including AWS Regions, Availability Zones, and data centers down to the physical security of the buildings.

AWS manages the hardware, software, and networking components that run AWS services, such as physical servers, host operating systems, virtualization layers and AWS networking components. When you use any AWS service, you are responsible for properly configuring the service and your applications in addition to ensuring that your data is secure. The level of responsibility that you have depends on the service. For example, you can choose to deploy your AI models on Amazon EC2, but then you would be responsible for the instance operating system, security patches, scaling, the security of the applications running on it. However, you could also host your model with SageMaker Serverless Inferencing. This fully managed service requires little management from customers. AWS manages all the underlying infrastructure, so you have no need to manage instances or scaling policies.

As we've seen, customers are responsible for using AWS in a secure manner. AWS Identity and Access Management is how customers manage permissions in their account. AWS Identity and Access Management, or IAM, is a web service that helps you manage and secure access to your AWS accounts and resources. With IAM, you can create and manage AWS users and grant them permissions to use services in your account. You control what they can do by assigning IAM permission policies that determine which actions a user can take on a resource. IAM is a global resource and not specific to any one Region. However, it is possible to use IAM to restrict a user's permissions to certain Regions. IAM is integrated with AWS services. IAM is used to manage the actions that users can take on the resources in your account. You can grant other people permissions to administer and use resources in your AWS account without having to share your password and access keys. IAM supports multifactor authentication, or MFA. You can add MFA to your account and to individual users for extra security, IAM supports identity federation. You can allow users who already have passwords elsewhere, for example, in your corporate network or within an internet identity provider, to get temporary access to your AWS account. IAM is a feature of your AWS account that is offered at no additional charge. You are only charged when you access other AWS services by using your IAM users.

When you first create an AWS account, you begin with a single identity that has complete access to all AWS services and resources in the account. This identity is called the AWS root user. The root user is accessed by signing in with the email address and password that you use to create the account. Because the root user permissions cannot be restricted, you need to keep its credentials safe. To ensure the safety of the root user, you should choose a strong password when creating your account and enable multifactor authentication for the root user. Never share your root user password and access keys with anyone. Disable or the delete the access keys associated with a root user because you won't need them after you create other users. Best practice is to not use the root user except for very few things that require it, like billing account management. Instead, you should create an IAM user and use that user for everyday tasks. You can assign an administrative access permissions policy to that user to give you all the permissions that you'll need. Using a username and password to log in into an AWS account is a single-factor authentication. Single-factor authentication is the simplest and most common form of authentication. Even if you choose a strong password for your root account, your account is still considered at risk.

For example, if a bad actor figures out your password through social engineering, bots, or scripts, then they can access your account and delete valuable data. They can also spin up an expensive cryptocurrency mining operation, leaving you with a bill. To help protect your account from malicious acts, you can use multifactor authentication, or MFA. With MFA, even if someone guesses or figures out your username and password, they cannot gain access to your account. They would need the numbers generated by the physical or virtual MFA device in your possession. AWS recommends you to enable MFA immediately after you create your AWS account. An IAM user is an identity that you create in AWS. It represents a person that interacts with AWS services and resources. It consists of a name and credentials. As a best practice, create individual IAM users for each person who must access AWS, even if multiple users require the same level of access. Provide each user their unique security credentials. Users should not share their credentials. If they do, you lose visibility into which user is taking actions in your account. By default, when you create a new IAM user in AWS, it has no permissions associated with it. To allow an IAM user to perform specific actions in AWS, such as configuring resources, you must grant the IAM user the necessary permissions. I'm going to pause this lesson here, and in the next lesson we'll continue talking about task statement 5.1.

Lesson 2

Let's continue with task statement 5.1. An IAM policy is a JSON document that allows or denies permissions to AWS services and resources. IAM policies permit you to customize users' levels of access to resources. As a best practice, follow the security principle of least privilege, which is granting only the permissions required to complete a task. Here is a JSON document example of an IAM policy that permits a user to start and stop EC2 instances inside an AWS account. Granting permission by assigning policies at the user level is a good idea, but what if you have 3,000 users in your AWS account? Administering access can become a challenge. To ease this challenge, AWS created IAM groups.

An IAM group is a collection of IAM users. When you assign an IAM policy to a group, all users in the group are granted permissions specified by the policy. To manage groups, you might want to organize IAM groups by job functions. For example, suppose that your company is building a software application, and you want to control the process. You could organize your developers into an IAM developer group, your testers into an IAM QA group and your administrators into an IAM admin group, then you could create and assign unique IAM policies to the groups instead of individual users. IAM groups can have many users, and users can belong to many groups. IAM groups cannot belong to other groups.

Best practice is to attach policies to groups and only attach to users any unique permissions that they should have. Permissions policies attached to users and groups use long-lived credentials to access AWS resources. If these get compromised, then someone else could use them. For example, a software developer could include their AWS credentials in software that they're writing and forget to remove them before sharing their code. A better way is to use IAM roles. An IAM role is an identity that a person or an AWS service can assume to gain temporary access to other AWS resources or services. When you assume an IAM role, you get temporary security credentials for your session, which will automatically expire. IAM roles can be used by an IAM user, an AWS service, or a user authenticated by an external identity provider. Each IAM role has an associated trust policy that determines which entities can assume the role.

Permissions policies that are associated with IAM users, groups, and roles are called identity-based policies. Permissions policies can also be applied at the resource level. For example, an AWS policy can be applied to an S3 bucket that controls which users or services can access the bucket and its objects. The policy is known as a resource-based policy. The resulting permissions are the total permissions of the two types. If an action is allowed by an identity-based policy, a resource-based policy or both, then AWS allows the action. An explicit deny in either of these policies overrides the allow. I'm going to pause this lesson here, and in the next lesson, we will continue talking about task statement 5.1.

Lesson 3

Let's continue with task statement 5.1. An alternative to creating and managing IAM users in your account is to use identity federation. Identity federation is when your users authenticate with an identity provider like Active Directory. After they authenticate, they are given temporary credentials for AWS. AWS IAM Identity Center lets you use an external identity provider like Active Directory to authenticate users, or you can create a directory in IAM Identity Center and use that to authenticate. IAM Identity Center refers to users as workforce users or workforce identities. Once authenticated, workforce users are directed to a portal where they can choose to access an AWS console, or they can obtain temporary access keys for any AWS account for which they have permissions. This is a big advantage for organizations that use multiple AWS accounts. Instead of managing IAM users in each account, they can add them to AWS Identity Center and manage permissions for all accounts in one central repository. You can also use AWS Identity Center to put users into groups and assign permission sets at the group level. AWS Identity Center uses roles to grant temporary permissions, which is a better practice than long-lived credentials. As a result, AWS recommends using IAM Identity Center for managing users instead of IAM. To be able to audit and validate your security configuration, you must be able to log and review the actions that users are taking.

AWS CloudTrail captures API calls and related events that are made by or on behalf of your AWS account. CloudTrail delivers the log files to an Amazon S3 bucket that you specify. Amazon SageMaker is integrated with CloudTrail. CloudTrail captures all API calls for SageMaker except for invoking endpoints. For example, when users create a training job or notebook instance, this action is logged in the trail. Using the information collected by CloudTrail, you can determine the request that was made to SageMaker. You can also find out the IP address from which the request was made, who made the request, and when it was made, and also additional details. Under the AWS shared responsibility model, customers are responsible for managing access to their data, which means keeping training data and artifacts private and secure. Amazon S3 Block Public Access is a feature that you can use to block public access to all your objects. You can block them at the bucket or account level now, and in the future. If enabled at the bucket level, some buckets in your account might be open to the public. When enabled at the account level, no buckets, existing or new, can grant public access.

When S3 Block Public Access is enabled, it will override any public permissions granted by bucket policies or access control lists. Instead of having to create IAM permissions policies yourself, Amazon SageMaker Role Manager greatly simplifies creating IAM roles that grant permissions to perform ML activities. SageMaker Role Manager provides three pre-configured role personas and predefined permissions for 12 ML activities. They include permissions to access other services like Amazon S3, AWS Glue, Amazon Athena, and Amazon CloudWatch. They also give permissions to manage and monitor SageMaker models, endpoints, training jobs, pipelines, and experiments. The data scientist persona is for someone who needs to use SageMaker to perform general machine learning development and experimentation. The MLOps persona is for someone who is managing models, pipelines, experiments, and endpoints, but doesn't need to access the data in Amazon S3. The SageMaker compute persona is used for creating a role that SageMaker compute resources can use to perform tasks such as training and inference.

When you create a role by using SageMaker Role Manager, you choose a persona that will have the appropriate activities for that persona preselected. You can customize which activities are enabled if you like. In this way, it will create the permissions policy for the role for you. You also have the option to add additional IAM policies. I'm going to pause this lesson here, and in the next lesson, we will continue talking about task statement 5.1.

Lesson 4

Let's continue with task statement 5.1. Another customer responsibility to keep your data secure is to ensure that it is encrypted. This requirement is also common in compliance standards. All AWS services offer the ability to encrypt data at rest and in transit. By encrypting your data at rest, even if someone could access your data on a storage volume, they would not be able to read it. AWS provides several features that customers can use to easily encrypt data and manage the encryption keys. Encryption keys are used together with an encryption algorithm to encrypt data before it's written to storage. To decrypt the data, you again need access to encryption keys. So getting access to the data isn't enough to be able to read it. Encryption can be accomplished on the client's side where the customer encrypts the data before sending it to an AWS service. Or encryption can be accomplished on the server side, where the AWS service encrypts the data.

Server-side encryption is the easiest way for a customer to ensure that encryption is implemented correctly and applied consistently. It is usually an option that you enable when configuring a resource. However, some AWS services, like Amazon S3, Amazon DynamoDB, and Amazon SageMaker, will encrypt your data by default without your having to enable it. For example, SageMaker will encrypt all the data on ML storage volumes, including the ones that are used by notebook instances, SageMaker jobs, and endpoints. Services with default encryption use keys that are owned by the service and not by the customer. If you need more control, you can use the AWS Key Management Service, or AWS KMS, for using encryption keys that belong to your account. You can use AWS KMS to create, manage, and use encryption keys in your AWS account. Using IAM policies, you can control the use of the keys.

By using policies to control which user or service can access a key, you're adding another layer of protection to your data. For example, suppose that someone is accidentally granted read permissions to your S3 bucket. They won't be able to read the data unless they also have permissions to use the AWS KMS key that is needed to decrypt it. Services with default encryption, like Amazon S3 and SageMaker, will use their own keys unless you specify an AWS KMS key to use. AWS KMS lets you use keys that AWS manages or ones that you control. Use customer-managed keys to control the key policies and IAM policies for the key, the key rotation, and to enable or disable the key. All AWS service endpoints support TLS to create a secure HTTPS connection to make API requests. As a result, all requests to Amazon S3 and SageMaker through the APIs and console are made over a secure encrypted connection. SageMaker distributed training, uses multiple nodes in a cluster.

By default, inter-node traffic is not encrypted, but an option exists to enable it. Although this encryption might be required for very sensitive data, enabling inter-node encryption can increase training times for some algorithms, particularly deep learning ones. Amazon Macie continually evaluates your S3 buckets and automatically generates an inventory of their size and state. It includes private or public access, shared access with other AWS accounts, and the encryption status. Macie also uses ML and pattern matching to identify and alert you to sensitive data, such as personal identifiable information, or PII. These alerts can be integrated in your machine learning workflow to take automated remediation actions. In general, PII should be removed from training data at the point of ingestion and transformation. For example, there is no reason to train a model on credit card numbers, names, addresses, or social security numbers. Macie will alert you if it finds sensitive data in your S3 bucket, and if it is in your training data, you must remove it. Customers are also responsible for managing the security configuration of their cloud infrastructure.

AWS provides virtual private clouds, or VPCs, for customers to be able to configure their own private networks on AWS. Amazon SageMaker Studio and SageMaker notebook instances are launched in the VPC that is managed by SageMaker. They are allowed direct internet access by default, which means that you can download popular packages from the internet. However you risk downloading malicious code that would send your data out over the internet. The best practice recommendation is to create a VPC in your account and specify your VPC when launching SageMaker Studio and notebooks. This will create an elastic network interface in your VPC and attach it to the notebook instance.

By using your own VPC, you can control which traffic can access the internet by configuring security groups, network access lists, and network firewalls. You can prevent SageMaker from giving your notebook instances internet access by specifying VPC only for the network access type. SageMaker Studio normally reaches required services like Amazon S3, Amazon CloudWatch, the SageMaker runtime, and the SageMaker API by using the public network. But when you use VPC only mode, the public endpoints for these services are no longer reachable. To keep all network traffic going over only a private network, you can use VPC interface endpoints. VPC interface endpoints connect your VPC directly to AWS services by using AWS PrivateLink. Because SageMaker Studio is running in your VPC, it can access external services by using the VPC interface endpoints. I'm going to pause this lesson here. And in the next lesson, we will continue talking about task statement 5.1.

Lesson 5

Let's continue with task statement 5.1. AI systems might have specific vulnerabilities that you should be aware of so that you can detect and mitigate potential security threats. An AI model learns its behavior from training data. If a malicious actor gains access to the training data, they can introduce data that will change the model's predictions.

For example, suppose that a fraud detection model is trained on financial transactions that are labeled as fraud and not fraud, then an attacker could add certain fraudulent transactions to the data that are labeled as not fraud. The model would then misclassify these transactions, which permits the attacker to commit fraud without being detected. Even without gaining access to training data, an attacker can cause a model to make mistakes. An attacker can slightly manipulate input data in a way that will cause the model to misclassify it. For example, a company uses a face recognition model to recognize employees. An attacker can make subtle but carefully designed modifications to their image to cause the model to recognize them as someone else. This kind of attack is known as adversarial inputs. A sophisticated attacker can cause a model's output to infer the training data. Known as model inversion, the attacker keeps feeding data into the model and studying the outputs. For example, the facial recognition model is trained on employee images, and its output is the employee's name and the confidence score. The attacker can repeatedly feed the model facial images, making changes until the output is an employee's name and a high-confidence score. The hacker then has a good image of an employee that they can use to pretend to be the employee.

Also, with enough input and output pairs, an attacker can create a new model that works in reverse. That is they can train a new model on the original model's outputs, and use it to infer the training input data. Similarly, the hacker can reverse engineer the model and make their own model that is very similar to original model. The more information the hacker has about the model, the more accurate their copy will be.

Large language models are vulnerable to a type of attack known as prompt injection. In this kind of attack, an attacker gives malicious instructions to the model in the prompt with the goal of influencing its output. For example, the attacker can prompt the LLM to ignore or alter its prompt template, which would permit the attacker to gain sensitive information. Customers who deploy AI systems must proactively mitigate against potential attacks by securing and limiting access to data and models. They also need to monitor models in production for drift and anomalies. Access should always follow the principle of least privilege. On AWS, this means configuring appropriate permissions policies and blocking public access. As an extra layer of protection, data and artifacts should be encrypted. You should limit and control access to the model to block reverse engineering. Input data from users must be inspected and validated. The input to your model should be looking for unusual patterns.

For example, you can teach an LLM to detect prompt injection by using key attack patterns and return the response prompt attack detected. Also, don't provide any unnecessary information in the model output. If you do, an attacker might be able to use it to infer information about the model. To help a model avoid being tricked, you should train models with adversarial input. Also, you can train your models frequently on new data so that any damage from corrupted training data will be undone. You should keep a separate set of data for validation purposes and validate your model after each re-training before deploying. You should also routinely scan and monitor your data for quality and detect anomalies before using it for training.

Finally, your model's predictions could change from their historical pattern. This occurrence should be investigated to determine whether the root cause is low data quality or an attack. Amazon SageMaker Model Monitor monitors the quality of Amazon SageMaker machine learning models in production. After deploying a model into your production environment, use Amazon SageMaker Model Monitor to continuously monitor the quality of your models in real time. You can use Amazon SageMaker Model Monitor to set up an automated alert system for deviations in model quality, such as data drift or anomalies. Amazon SageMaker Model Monitor also can be used for monitoring data quality. SageMaker Model compares the data and model with baselines. It generates statistics and metrics that are visible on SageMaker Studio and also sent to Amazon CloudWatch.

Amazon CloudWatch Logs collect files of monitoring the model status. It notifies you when the quality of your model hits preset thresholds. CloudWatch stores the log files to an Amazon S3 bucket you specify. With early and proactive detection of model deviations through AWS model monitor products, you can take prompt actions. In this way, you can maintain and improve the quality of your deployed model. To use SageMaker Model Monitor to monitor data quality, you must enable data capture. This captures the inference input and output from an inference endpoint or batch transform job and stores the data in S3. To create a baseline, you can usually use the training dataset. You then schedule data quality monitoring jobs that generate statistics for the baseline dataset and the current dataset that is being analyzed. You can view the results in SageMaker Studio. A similar process uses SageMaker Model Monitor to monitor model performance. You use labeled data to create a baseline. After the model quality job runs, it'll compare the inferences with label data from SageMaker Ground Truth for the same inputs, then it will evaluate the performance. You can view the results in SageMaker Studio.

Lesson 6

Let's wrap up task statement 5.1. Tracking all the artifacts that are used for model production is an essential requirement for reproducing the model to meet regulatory and control requirements. To be able to recreate a model, all the various artifacts that went into its development must be versioned and tracked. Code repositories, such as GitHub and AWS CodeCommit automatically retain versions of source code. This code might include training code, inference code, experiments, and notebooks. Datasets should be stored in Amazon S3 and partitioned with prefixes to uniquely identify the training dataset. Container images stored in Amazon Elastic Container Registry or Amazon ECR are uniquely identified with a unique ID and can also be tagged with additional information.

SageMaker automatically uniquely identifies each training job and stores other metadata such as hyperparameters and the unique identifiers for the container dataset and model output. Model versions can be stored in a model catalog by using SageMaker Model Registry. SageMaker endpoints all have a unique identifier and metadata. For example, the model that is used is automatically tracked as part of the endpoint configuration. Using SageMaker Model Registry, you can catalog models in model groups that contain different versions of a model. Each model package in a model group corresponds to a trained model. You can associate and view model metadata, including training metrics. Models can be deployed directly from the model registry. Model Registry also lets you maintain the status of a model such as spending, approved, or rejected.

Use Amazon SageMaker Model Cards to document, retrieve, and share essential model information from conception to deployment. Risk managers, data scientists, and ML engineers can use model cards to create an immutable record of intended model uses, risk ratings, training details, and evaluation results. Model cards can be exported to PDF and shared with relevant stakeholders. Amazon SageMaker ML Lineage Tracking automatically creates a graphical representation of all the elements of your end-to-end machine learning workflow. You can use this representation to establish model governance, reproduce your workflow, and maintain a record of your work history. Lineage tracking automatically creates tracking entities for trial components and their associated trials and experiments. You can use lineage tracking when you create SageMaker jobs such as processing jobs, training jobs, and batch transform jobs. You can run queries against the lineage data to discover relationships between the entities.

For example, query the lineage to retrieve all the models that use a particular dataset or retrieve datasets that use a container image artifact. Let's recall that a feature is a data property that is used as one of the inputs to train or make predictions with your model. For example, it can describe a column in a data table. Amazon SageMaker Feature Store is a centralized store for features and associated metadata, so features can be easily discovered and reused. Feature Store makes it less complicated to create, share, and manage features for ML development. Feature Store accelerates this process by reducing repetitive data processing and curation work required to convert raw data into features for training and ML algorithm.

You can create workflow pipelines that convert raw data into features and add them to feature groups. You can also view the lineage of a feature group. The lineage includes the information about the execution code of your feature processing workflow. It also includes which data sources were used and how they were ingested to the feature group or feature. Feature Store also supports point in time queries to retrieve the state of each feature as the historical time of interest. Amazon SageMaker Model Dashboard is a centralized portal accessible from the SageMaker console where you can view, search, and explore all models in your account. It aggregates model-related information from several features including Model Monitor and Model Cards. You can visualize workflow lineage and track your endpoint performance. You can track which models are deployed for inference and whether they're used in batch transform jobs, or hosted on endpoints.

If you set up monitors using Model Monitor, you can also track the performance of your models as they make real-time predictions on live data. You can use the dashboard to find models that violate thresholds that you set for data quality, model quality, bias, and explainability. The dashboard's comprehensive presentation of all your monitoring results helps you quickly identify models that don't have these metrics configured. In the next lesson, we'll get started with task statement 5.2.