Task Statement 5.2: Recognize governance and compliance regulations for AI systems

Objectives:

• Identify AWS services and features to assist with governance and regulation compliance (for example, AWS Config, Amazon Inspector, AWS Artifact, AWS CloudTrail, AWS Trusted Advisor).
• Describe data governance strategies (for example, data lifecycles, logging, residency, monitoring, observation, retention).
• Describe processes to follow governance protocols (for example, policies, review cadence, review strategies, governance frameworks such as the Generative AI Security Scoping Matrix, transparency standards, team training requirements).

Lesson 1

Now let's review the second task statement from Domain 5, which is recognize governance and compliance regulations for AI systems. This task statement is split into six lessons. Concerns about the risks of AI have resulted in the development of several standards of compliance for AI workloads. Following these standards will help protect your business and customers and ensure fairness in decision making. Depending on your company's industry, you might also need to uphold standards specific to your industry. Regular audits or inspections will ensure that the company has met those standards. Recall that on AWS, security and compliance are a shared responsibility between AWS and the customer.

To explain the division of responsibility, AWS created the AWS Shared Responsibility Model. The AWS Shared Responsibility Model applies to both security and compliance. AWS is responsible for meeting the compliance requirements of the cloud by securing the underlying physical data centers and infrastructure. The customer is responsible for securing their workloads in the cloud to meet the compliance requirements. AWS maintains rigorous security and compliance certifications and attestations for data center operations, technology, and security. Compliance standards typically include a list of security and operational controls that must be tested and validated by external auditors. With a shared responsibility model, AWS customers inherit some of these controls from AWS. AWS has those controls audited by a third party and makes the auditor's reports available to customers.

AWS Artifact is a service that makes compliance reports from third-party auditors available to customers. Though you can't send your own auditors to an AWS data center, you can give these reports to them. This practice reduces the scope of their audit because the compliance controls in the reports are inherited by the customer. Your auditors only need to focus on your processes and procedures as you deploy and run workloads in the cloud. Auditors have tested and verified that AWS is compliant with a variety of global, regional, and industry-specific security standards and regulations. You can find these compliance reports and regulations in AWS Artifact. Each report includes a description of its contents, and the reporting period for which the documentation is valid. As a global cloud provider, AWS must maintain compliance with standards from around the world. By doing so, it gains the trust of customers who are deploying potentially sensitive workloads on AWS.

For example, a service organization controls, SOC report, is a way to verify that a third party is following some specific best practices. This information is important to a customer before outsourcing a business function to that organization. SOC 2 reports and controls related to security, availability, processing, integrity, confidentiality, and privacy. If a company wants to achieve SOC 2 for their customers, they use the AWS SOC 2 report as a starting point. Then they have an auditor verify that they have configured the security controls they are responsible for correctly. As another example, ISO 27001 is an international security management standard that specifies security management best practices and comprehensive security controls. Because AWS has achieved a certification for compliance with the standards, it can help a customer's organization to get the certification. The customer compliance center is a set of resources to help you learn about AWS compliance. In the customer compliance center, you can read customer compliance stories about how companies in regulated industries have solved various compliance, governance, and audit challenges. You can access compliance whitepapers and documentation on various topics. These topics include AWS answers to key compliance questions, an overview of AWS risk and compliance, and an auditing security checklist.

Additionally, the customer compliance center includes an auditor learning path. Individuals in auditing compliance and legal roles might want to learn more about how their internal operations can use the AWS Cloud to demonstrate compliance. I'm going to pause this lesson here, and in the next lesson we will continue to talk about task statement 5.1 and compliance.

Lesson 2

Let's continue talking about compliance and task statement 5.2.

When it comes to AI, compliance standards are still emerging and only beginning to see adoption. ISO 402001 and ISO 23894, both published in 2023, established mechanisms for assessing and managing risk in AI systems.

They establish a framework for organizations to systematically address and control the risks related to the development and deployment of AI. They emphasize a commitment to responsible AI practices, encouraging organizations to adopt controls specific to their AI systems. They are also committed to fostering global interoperability and setting a foundation for the development and deployment of responsible AI.

Keep in mind that although compliance standards are highly recommended, they are not legal requirements. However, the EU AI Act is a European regulation on artificial intelligence, AI. It's the first comprehensive regulation on AI by a major regulator anywhere. The act assigns applications of AI to three risk categories.

First, applications and systems that create an unacceptable risk are banned outright. Some examples of banned applications include social scoring applications that classify individuals or groups based on social behavior They also include facial recognition databases that scrape facial images from the internet and applications that infer emotions in workplaces or educational institutions.

Second, high-risk applications such as CV scanning tool that ranks job applicants are subject to specific legal requirements. Finally, applications not explicitly banned or listed as high risk are largely left unregulated. Most AI systems will fall under the high-risk category. The requirements for these applications include establishing a risk management system, conducting data governance, and documenting compliance.

Even if you will not make your AI system available to EU citizens, EU regulations often become global standards, like the General Data Protection Regulation, or GDPR. Therefore, you should pay attention and attempt to meet them The AI Risk Management framework, RMF, released by the National Institute of Standards and Technology, NIST, provides guidance on promoting trustworthy, responsible development and use of AI systems.

The goal of AI RMF is to offer a resource to organizations that design, develop, deploy, or use AI systems to help manage the risks of AI. It promotes trustworthy and responsible development and use of AI systems. The framework is intended to be voluntary. It describes four specific functions, govern, map, measure, and manage to help organizations address the risk of AI systems in practice.

As we've just seen, the AI compliance standards are mostly about measuring and managing risk of AI systems. According to the NIST, RMF, a risk can be estimated by multiplying the likelihood of an event with the severity of the consequences if it occurs. Using this risk matrix, we can consider an event with low severity and rare likelihood of occurring as very low risk. To estimate the risk of an AI system, you should start by identifying the AI use case and the relevant stakeholders for the use case.

For example, consider how certain users will use the system to achieve a particular goal. Next, you identify potentially harmful events associated with the use case and use the risk matrix to determine the risk for each event. Then you address the inherent risk, which are those things that can be mitigated with proper security controls and monitoring.

For example, by increasing security controls, you reduce the likelihood of an event so it lowers its overall risk. After you mitigate the inherent risk, what remains is the residual risk, which is the risk that is left after mitigations. Finally, you summarize the risk levels for an overall risk rating for the system, which is determined by focusing on the highest residual risk ratings.

The Algorithmic Accountability Act has been introduced several times for consideration by the United States Congress. If enacted, it will require companies to assess the impacts of the AI systems that they use and sell. It will create new transparency about when and how such systems are used and empower consumers to make informed choices when interacting with AI systems.

Its goal is to try to protect Americans from AI systems that can lead to unfair and unexplained results. For example, it expects that consumers have a right to know why their loan application was rejected. They have a right to know whether it was because of some inherent and unfair bias.

Because generative AI models are complex, it is difficult to understand how a model arrives at an output at a deep level. As a minimum level of transparency, consider when to disclose to your users that they are using generative AI and not engaging with a human. However, for many applications, you might need to go deeper and be able to explain how a model arrived at an inference.

Understanding how an AI system arrived at an output is known as its explainability. One visible approach to explainability is to observe a model's behavior with regard to the output and input. With this model agnostic approach, the practitioner can determine which features are given the most consideration and treat the model as a black box. However, by using more interpretable algorithms and techniques such as decision trees or rule-based systems, you can observe the model weights and understand the inner workings.

In the initial stages of AI and ML development, you should consider tradeoffs between interpretability and performance with regulatory requirements in mind. The second major reason for algorithmic accountability is to remove bias from decision making. The goal is to be certain that the results are not influenced by the improper use of a person's various personal attributes. These attributes might include race, sex, gender identity, religious beliefs, political affiliations, and whether they live in the city or a rural community.

This goal of bias removal requires you to test and monitor for bias in the model's results and in its training data. Amazon SageMaker Clarify can help you understand how different variables influence a model's behavior and monitor for bias and feature attribution drift.

I'm going to pause this lesson here, and in the next lesson we will continue talking about task statement 5.2 and AWS services for AI compliance.

Lesson 3

Let's continue talking about compliance and task statement 5.2. To help customers achieve compliance, AWS provides services and features to help audit, monitor, control and report on relevant security controls that customers must configure correctly.

AWS Audit Manager maps your compliance requirements to your AWS usage data. It collects evidence of compliance or noncompliance and produces assessment reports that can be given to auditors. Audit Manager will create an assessment based on a chosen framework, which is a grouping of controls that are related to your audit. It has built-in frameworks including ones for generative AI best practices and SOC 2.

You also can define your own framework tailored to assess your specific controls of concern. When you create an assessment, Audit Manager automatically assesses resources in your AWS accounts and services based on controls that are defined in the framework. Then it collects the relevant evidence and converts it into an auditor-friendly format. Next, it attaches the evidence to the controls in your assessment. When it's time for an audit, you can review the collected evidence and then add it to an assessment report. This report helps you demonstrate that your controls are working as intended.

Guardrails is a feature for Amazon Bedrock that you use to implement application-specific safeguards based on your use cases and responsible AI policies. Guardrails for Amazon Bedrock provides content filters with configurable thresholds to filter harmful content across hate, insults, sexual, and violence categories. Using short natural language description, Guardrails for Amazon Bedrock permits you to define a set of topics to avoid within the context of your application.

You can optionally provide some example phrases to help guardrails recognize the topics. Guardrails detects and blocks user inputs and FM responses that fall into the restricted topics. Guardrails for Amazon Bedrock also allows you to detect personally identifiable information, PII, in user inputs and FM responses. Based on the use case, you can selectively reject inputs that contain PII or redact PII in FM responses.

You can use guardrails to configure thresholds across the different categories to filter out harmful interactions. Guardrails will automatically evaluate both user queries and FM responses to detect and help prevent content that falls into restricted categories.

For example, an ecommerce site can design its online assistant to avoid using inappropriate language such as hate speech or insults. Also, you can easily block specific topics. For example, if you want to block your model from giving investment advice, you can add a denied topic and describe it in simple terms.

You can also specify two messages to give as responses. One, when prompt is blocked and one when a response by the model is blocked. One example response could be, "We are sorry, but that topic is outside our area of expertise." While achieving compliance is the goal, be aware if something changes in a resource configuration such as misconfiguration that makes it noncompliant.

AWS Config provides a detailed inventory of the current configuration of AWS resources. Whenever a resource configuration change occurs, AWS Config captures and records the change in a configuration history snapshot. The change is evaluated by configuration rules. If a change is noncompliant with the rules, it can be automatically remediated with an AWS Systems Manager automation document.

You can choose to use prebuilt rules or create a custom rule by using a Lambda function. Conformance packs package together AWS Config rules and remediation actions to help deploy the rules and remediations to meet your compliance needs. You can create a conformance pack yourself or select from a library of conformance pack templates. Two useful conformance packs are the operational best practices for AI and ML and the security best practices for Amazon SageMaker.

Whereas AWS Config monitors configurations at the resource level, Amazon Inspector works at the application level. It checks applications and containers for security vulnerabilities and deviations from security best practices, such as open access to EC2 instances and installations of vulnerable software versions.

Amazon Inspector helps to improve the security and compliance of applications by running automated security assessments. After Amazon Inspector has performed an assessment, it provides you with a list of security findings. The list prioritizes by severity level, including a detailed description of each security issue and the recommendation of how to fix it.

AWS Trusted Advisor helps you optimize costs, increase performance, improve security and resilience, and operate at scale in the cloud. Trusted Advisor continuously evaluates your AWS environment by using best practice checks across the categories of cost optimization, performance, resilience, security, operational excellence, and service limits. It recommends actions to remediate any deviations from best practices.

I'm going to stop this lesson here, and in the next lesson, we will continue talking about task statement 5.2.

Lesson 4

Let's continue talking about compliance and task statement 5.2. Data governance is the combination of people, process, and technology that is used to manage the availability, usability, integrity, and security of enterprise system data. Effective data governance ensures the data is consistent and trustworthy without being misused. Data governance can be broken down into three major parts.

Curation, discovery and understanding, and protection. Curating your data at scale means identifying and managing your most valuable data sources, including databases, data lakes, and data warehouses. By curating your data, you can limit the proliferation and transformation of critical data assets. Curating your data at scale means identifying and managing your most valuable data sources, including databases, data lakes, and data warehouses. By curating your data, you can limit the proliferation and transformation of critical data assets. Curating data also means ensuring that the data is accurate, fresh, and free of sensitive information so that users can have confidence in data-driven decisions.

Understanding your data in context means that users can discover and comprehend the meaning of their data to use it confidently to drive business value. With a centralized data catalog, data can be found easily. Access can be requested and data can be used to make business decisions.

Protecting your data means being able to strike the right balance between data privacy, security, and access. It's essential to be able to govern data across organizational boundaries with tools that are intuitive for both business and engineering users. Data governance focuses on ensuring the data is treated as a strategic asset and on developing competencies to put that strategic asset to effective use, exercise authority and control over your data to meet stakeholder expectations. Start with the data domains that are necessary to succeed with the targeted business initiatives. Define key data governance roles and responsibilities such as data owners, stewards, and IT.

While taking segregation of duties into account, assign key roles to appropriate individuals in your organization. Data owners should be recognized at an organizational level that includes both technology and business representatives. Data stewardship should be a responsibility of all data-facing business personnel. A data steward is a person from the business who has detailed knowledge of the data needed to support targeted business initiatives. Data stewards are involved in the details of projects day to day. They help understand the data issues that are likely to cause challenges with business initiatives.

A data owner is an executive-level person who makes data policy decisions, including regulatory and compliance policies. They decide, for example, who has access to claims data and who has access to customer data. A direct relationship exists between the data owner and the data steward. The data steward completes daily tasks for projects and the data owner owns policies about the data that guides the work of the data steward. Roles in IT help navigate systems that produce and consume data and provide data stewards with the right tools and capabilities. For example, IT might be responsible for managing and deploying data governance tools in AWS.

With data profiling, data is systematically examined to determine whether anything is wrong with the data and to understand data characteristics for various purposes. A data catalog makes sure that the data is available for people who need access to it. Data lineage identifies where specific data elements originated and how the data was moved, transformed, and stored. When data consumers see data in a report, they often question where the data came from and any calculations made along the way. AWS Glue DataBrew is a visual data preparation tool that users can use to clean and normalize data without writing any code. It has two features that are helpful for data governance.

The first is data profiling. You can use AWS Glue DataBrew to run profiling jobs against your dataset to create a data profile. This profile tells you about the existing shape of your data, including the context of the content, the structure of the data, and its relationship. You can define data quality rules and these rules are validated during the profiling. to detect problems with the data.

AWS Glue DataBrew can analyze many types of datasets, including data that's stored in Amazon S3 and in relational data bases and data warehouses. In addition to creating a data profile, AWS Glue DataBrew tracks your data in a visual interface to determine its origin called the data lineage. This view shows you how the data flows through different entities from where it originally came. You can see its origin, other entities that influenced it, what happened to it over time, and where it was stored. I'm going to stop this lesson here. And in the next lesson we will continue talking about task statement 5.2.

Lesson 5

Let's continue talking about compliance and task statement 5.2.

Data quality management addresses data quality issues that are found in data profiling or through other means. Wherever possible, we need to get to the root of what's causing the data quality problems. To this end, we need business and technical knowledge of the data and its role in the targeted initiative.

If the problem involves the source, we must alert and report the issues to the data steward and business users who can correct the issues. Data integration is about collecting and merging data from a variety of sources. We need to ensure that data from different sources links together coherently so that they can be combined to gain a more complete picture.

An AWS Glue Data Catalog stores metadata about your data sources. This metadata includes information about the locations and schemas, including data types and table definitions. You can populate the data catalog manually, or you can define an AWS Glue crawler job that scans data sources and populates the catalog automatically.

AWS Glue Data Quality evaluates objects that are stored in the AWS Glue Data Catalog. It offers non-coders a straightforward way to set up data quality rules, including recommending quality rules and automatically using machine learning to detect data anomalies.

After you define the rule set, you run an AWS Glue Data Quality job. You then review the results in the console that show which rules did or did not pass. Data security means defining who can have access to data and when they should have access to data.

For example, certain customer data needs to be automatically accessible by certain roles like customer service or sales. Typically, the data steward helps to enable a role-based and temporary access guided by policy decisions established by the data owner.

Compliance means understanding government regulations and making sure that we comply with those regulations. To ensure compliance, data owners must work with the security and legal teams to make policy decisions for sensitive data domains. The rules often require interpretation, judgment, and knowledge of how the data is used in the business.

Data lifecycle management means being intentional about storing data for straightforward access and optimized cost.

Data governance is about making data available to the right people and applications when they need it, while keeping the data safe and secure with appropriate controls in place. So it's a balance between control and access. Too much control and your data becomes locked in silos and difficult to use for insights and innovation that can help the business. Not enough control and your data and business are at risk.

With AWS Lake Formation, you can manage fine-grained access control for a data lake built in Amazon S3 and cataloged using AWS Glue Data Catalog. Lake Formation permissions are enforced using granular controls at the column, row, and cell levels across AWS analytics and machine learning services. Lake Formation helps you break down data in silos and combine different types of structured and unstructured data into a centralized repository.

You can identify existing data stores in Amazon S3 or relational and NoSQL databases, and move the data into your data lake. Then you catalog the data and set up the permissions. A user might try to access the data by using an integrated analytical engine like Athena, AWS Glue, Amazon EMR, or Amazon Redshift.

In that case, the AWS Glue Data Catalog checks the permissions with Lake Formation before granting access. Training data for your models is typically stored in S3 buckets. After using it to train your model, you might not need to access it again, but still need to retain it for compliance reasons. Amazon S3 offers different storage classes, which are optimized for cost based on the frequency of access.

Amazon S3 Standard is the storage class that you use for frequently accessed data. S3 Standard-IA and S3 One Zone-IA are for storing data that is less frequently accessed, but must be quickly retrievable when needed. These classes are less costly for storage, but more costly for retrieval than S3 Standard.

The S3 Intelligent-Tiering class is for data with unknown or changing access patterns. It will automatically move less frequently accessed objects into lower-cost infrequent access tiers. S3 Glacier storage classes are for rarely accessed data and used primarily to retain data in a long-term archive for data retention or compliance reasons.

The different classes of S3 Glacier provide options for how quickly you can retrieve your data from an archive when needed. When you know your data's access patterns, you should create lifecycle rules on your buckets. These rules will move your data into the infrequent access and archive tiers as the rule conditions are met.

For each lifecycle rule, you specify the target storage class and the number of days after creation that the data should be transitioned. You can also choose to create a rule that deletes the data when it is no longer needed to be retained.

For example, you could create a lifecycle rule that transitions your data from S3 Standard to S3 Standard-IA 5 days after creation. Then it would move the data into S3 Glacier Deep Archive, 120 days after creation and delete the data after 5 years.

I'm going to pause this lesson here, and in the next lesson we will continue talking about task statement 5.2.

Lesson 6

Let's continue talking about compliance and task statement 5.2. Let's now discuss the steps for implementing an AI governance strategy.

An AI governance strategy begins with identifying the scope of your responsibility. This responsibility can include governance and compliance, legal and privacy, risk management, implementing security controls, and the model resilience.

The Generative AI Security Scoping Matrix shows increased levels of scope, depending on the way AI is consumed or implemented. Scopes 1 and 2 carry the least responsibility because you are consuming a third-party consumer or enterprise application.

With scopes 3, 4, and 5, you are building your own AI solution. Your data can be used in the training, fine-tuning, or output of the model. You are responsible for classifying the data and model for risk, implementing threat modeling, limiting access, implementing security controls, and assuring the model endpoint's resilience.

Here is an example of how the responsibilities for governance and compliance change as the scope increases. If a third-party model has all the data and functionality that you need, scope 1 and scope 2 applications might fit your requirements.

We've seen throughout this course that AWS offers services in each of these scopes. The main takeaway is to look for a solution from left to right, beginning with fully trained AI services, for example, Amazon Comprehend or Amazon Translate. If those services don't meet your needs, then look for pre-trained models, for example, Amazon Bedrock, which can be enhanced with retrieval augmented generation.

Or consider models that are pre-trained, but can be fine-tuned with your own data, such as those that are available in SageMaker JumpStart. Minimizing your scope will minimize your responsibilities for governance and compliance, along with legal and privacy, risk management, implementing security controls, and model resilience.

As soon as your scope has been determined, the next step is to document your AI governance policies and train your employees on their responsibilities. Their responsibilities would be according to their job roles and level of access. This policy includes establishing standards for data governance, access requests, and model transparency.

Use the compliances and certifications that the business requires to guide the policies and best practices. Define mechanisms to monitor AI systems' performance, compliance, and bias, and determine actions based on pre-defined thresholds. Finally, frequently review results and revise existing policies as necessary to ensure alignment with business goals and AI safety.

Let's get started with our ninth walkthrough question.