Task Statement 4.1: Explain the development of AI systems that are responsible

Objectives:

• Identify features of responsible AI (for example, bias, fairness, inclusivity, robustness, safety, veracity).
• Explain how to use tools to identify features of responsible AI (for example, Amazon Bedrock Guardrails).
• Define responsible practices to select a model (for example, environmental considerations, sustainability).
• Identify legal risks of working with generative AI (GenAI) (for example, intellectual property infringement claims, biased model outputs, loss of customer trust, end user risk, hallucinations).
• Identify characteristics of datasets (for example, inclusivity, diversity, curated data sources, balanced datasets).
• Describe effects of bias and variance (for example, effects on demographic groups, inaccuracy, overfitting, underfitting).
• Describe tools to detect and monitor bias, trustworthiness, and truthfulness (for example, analyzing label quality, human audits, subgroup analysis, Amazon Augmented AI [Amazon A2I]).

Lesson 1

Let's get started with the first task statement from domain 4, which is to explain the development of AI systems that are responsible. This task statement is split into three lessons.

Let's get started by describing what responsible AI is and the core dimensions of a responsible AI model. Responsible AI is a set of guidelines and principles to ensure that AI systems operate in a safe, trustworthy, and responsible manner.

One of the core dimensions of responsible AI is fairness. Fairness aims to ensure that models treat everyone equitably and impartially regardless of their age, where they live, their gender, or their ethnicity. It's important to be able to explain in human terms why a model made a particular decision, which is known as explainability.

For example, what was the main reason or reasons that a loan application was rejected? As users put their trust in and rely on AI, robustness makes sure that AI systems are tolerant of failures and minimize errors. Privacy and security are primarily about protecting user privacy and not exposing personal identifiable information, or PII.

Governance is about meeting and auditing compliance with industry standards and best practices, including estimating properly and mitigating risk. Transparency is about providing clear information about model capabilities, limitations, and potential risks to stakeholders.

Transparency includes making sure that users know when they are interacting with AI. Fairness avoids perpetuating or amplifying societal biases and discrimination through AI systems. Fairness of a model is measured by the bias and variance of outcomes across different groups.

Demographic disparities can lead to unequal outcomes or treatment of different groups based on age, race, sex, and more. And model's accuracy could also show bias, but by being more accurate for some groups and less accurate for others.

Overfitting becomes a problem when the training dataset is not representative of the real world. As a result, the model only performs well on inputs that resemble the training data. Groups not properly represented in the training data will potentially experience more negative outcomes.

Underfitting can occur for some groups when there wasn't enough training data that matched their characteristics so the model doesn't perform well for them. User trust can erode because of biased or inconsistent output, and ethical concerns arise when AI systems violate principles of fairness, non-discrimination, and equal treatment.

It is crucial that we understand and address the effects of bias and variance in our AI models. By doing so, we can ensure that our systems are accurate, fair, and trustworthy. One of the main reasons for model bias is class imbalance.

Class imbalance occurs when a feature value has fewer training samples when compared with another value in the dataset. In this example, the feature for sex shows that women constitute 32.4% of the training data, whereas men constitute 67.6%. As a result.

The model saw more data records for men than for women during training, so it performs better with inferences associated with men, and the model could overfit the women records because it might have undersampled them.

As a result, the model could have a higher error rate for women. If this was for predicting whether someone has a disease, it could cause a higher rate of inaccurately diagnosing women.

Responsible datasets are the foundation of responsible AI because biases in the training dataset will become biases in the output. Responsible datasets need to avoid class imbalances and demonstrate other important characteristics.

Inclusivity is about representing diverse populations, perspectives, and experiences in our training data. Diversity means incorporating a wide range of attributes, features, and variables to avoid bias. Curated data sources are carefully selected and varied to ensure quality and integrity.

Balanced datasets ensure equal representation of different groups and avoid skewed distributions. Privacy protection is a must, safeguarding sensitive information and adhering to data protection regulations.

Consent and transparency involve obtaining informed consent from data subjects and providing clear information about data usage. And regular audits are crucial for conducting periodic reviews of datasets to identify and address potential issues or biases.

As AI experts, it's our responsibility to ensure that the datasets we use for training our models possess these characteristics. By doing so, we can build AI systems that are fair, unbiased, and respectful of individual privacy and consent.

When it comes to selecting AI models, several practices and other factors must be considered. Environmental impact is a big one because the compute resources for training a large and complex model are substantial. Therefore, we need to assess the carbon footprint and energy consumption of our models.

Consider using an already trained model as a starting point to reduce the amount of training that your model needs. Sustainability is about prioritizing models with minimal environmental impact and long-term viability. Reuse of existing work is the key principle of sustainability.

Transparency is about providing clear information about model capabilities, limitations, and potential risks. It also means making sure that users know when they are using AI. Accountability means establishing clear lines of responsibility for AI model outcomes and decision-making.

Stakeholder engagement is all about involving diverse perspectives in the model selection and deployment process. It's your responsibility to consider these practices when choosing AI models to ensure that systems are not only technically sound but also socially responsible.

I'm going to pause this lesson here, and in the next lesson, we'll continue talking about task statement 4.1.

Lesson 2

Let's continue with task statement 4.1, which is to explain the development of AI systems that are responsible.

Let's talk about services and features from AWS that you can use to measure and monitor a model's bias, trustworthiness, and truthfulness.

Biases are imbalances in data or disparities in the performance of a model across different groups. SageMaker Clarify helps you mitigate bias by detecting potential bias during the data preparation, after model training, and in your deployed model, by examining specific attributes. A model's explainability is important to be certain that the decisions it makes are unbiased.

SageMaker Clarify can improve explainability by looking at the inputs and outputs for your model, treating the model itself as a black box. By making these observations, it determines the relative importance of each feature.

For example, it can say a loan application was rejected because two of the most important features, income and outstanding debt, did not meet the thresholds. Because SageMaker Clarify treats the model as a black box, it can understand the basis for how deep learning models are making predictions without understanding the inner workings. It can even explain computer vision and natural language processing models, which are using unstructured data. SageMaker Clarify examines your dataset and model by using processing jobs.

A SageMaker Clarify processing job uses this SageMaker Clarify processing container to interact with an Amazon S3 bucket, the S3 bucket would contain your input datasets and a model that is deployed to a SageMaker inference endpoint. The SageMaker Clarify processing container obtains the input dataset and configuration for analysis from an S3 bucket.

For feature analysis, the SageMaker Clarify processing container sends requests to the model container and retrieves model predictions from the response from the model container. After that step, the processing container computes and saves analysis results to the S3 bucket. These results include a JSON file with bias metrics and global feature attributions, a visual report, and additional files for local feature attributions.

You can download the results from the output location and view them. Here are a few examples of metrics that are measured by SageMaker Clarify when analyzing the training dataset. As we've seen, a balanced dataset is important to avoid errors that occur for a particular class.

For example, an ML model trained primarily on data from middle-aged individuals might be less accurate when making predictions that involve younger and older people. Another type of imbalance occurs when the labels favor positive outcomes for one class over another.

For our example, training data might exhibit an unwanted pattern in showing that loans are approved at a higher rate for middle-aged individuals. The demographic disparity metric indicates whether a particular class has a larger proportion of the rejected outcomes in the dataset than the accepted outcomes.

For example, consider the case of college admissions. Women applicants comprised 46% of the rejected applicants, but made up only 32% of the accepted applicants. We say that this result shows demographic disparity, because the rate at which women were rejected exceeds the rate at which women were accepted. Here are some of the metrics that SageMaker Clarify looks at with a trained model.

The difference in positive proportions in predictions metric indicates whether the model predicts positive outcomes differently for each class. This metric can be compared with the label imbalance in the training data. The goal is to see whether the bias in positive proportions changes after the training or whether the bias is also present in the data.

Remember that specificity measures how often the model correctly predicts a negative outcome. If the specificity is lower for middle-aged men than other age groups, the model is showing bias against the other age groups.

The recall difference metric is the difference in recall of the model between two classes. Any difference in these recalls is a potential form of bias. Recall is the true positive rate, TPR, which measures how often the model correctly predicts the cases that should receive a positive outcome.

If the recall rate is high for one class, but low for another, the difference provides a measure for this bias. Model accuracy can also be different between classes, and this difference is also a form of bias.

The accuracy difference metric is the difference between the prediction accuracies for different classes. This result can occur when the data contains class imbalance. The treatment equality is the difference in the ratio of false negatives to false positives. Even if the accuracy of the model is the same for two classes, this ratio could have differences.

A difference in the type of errors that occur for different classes can constitute bias. Consider the loan approval example. More incorrect loan denials for one class and more incorrect loan approvals for another are two very different outcomes that show bias between the classes.

I'm going to pause this lesson here, and in the next lesson, we will wrap up task statement 4.1.

Lesson 3

Let's continue with task statement 4.1, which is to explain the development of AI systems that are responsible.

Now, let's talk about the challenges and risks of working with generative AI models. Though all AI models are only as reliable as the data they're trained on, generative AI involves a risk of hallucination. This outcome occurs when the model makes up something that might sound factual, but is actually fiction.

Hallucination is the result of the AI model attempting to fill in the gaps when something is missing in its training data. Depending on how the generated content is used, hallucinations can be disastrous.

In 2023, lawyers submitted a brief that contained extract and case citations, which they got from generative AI to a New York court. However, the case and citations were fake. The court dismissed the client's case, sanctioned the lawyers, and fined them and their firm. Interestingly, AI-generated works cannot be copyrighted because they are not the work of a human.

However, a generative AI model might have been trained with data that's protected by copyrights, patents, or trademarks, which can then be included in outputs. Also, a user can submit a copyrighted work as an input, and the generative AI can create an unlicensed derivative.

In 2023, Getty Images filed a lawsuit against the creators of Stable Diffusion, a text-to-image generative AI model. The lawsuit was for allegedly infringing on the use of more than 12 million photographs, their associated captions, and metadata in building the model. Biased model outputs can lead to discriminatory or unfair treatment of individuals or groups, which can become a major legal risk.

The Equal Employment Opportunity Commission sued three companies for using an AI hiring program that discriminated against older applicants. The program automatically rejected applicants if the person was a female over age 55 or a male over age 60.

Generative AI models are capable of generating content that is offensive, disturbing, or even obscene if that of content was in their training data. Toxic content can cause real-world harm to users, such as mental and emotional health problems. It can even cause increased propensity toward violence against specific individuals or marginalized groups.

Data privacy is a risk because sensitive data that makes its way into a large language model can leak and be incorporated into its output. This output could include PII, intellectual property, trade secrets, and healthcare records, to name a few. This information could end up in the model by being present in training data or input as a prompt by a user.

One problem is that as soon as an FM is trained or sees some data, you can't make it forget by deleting the data. Any of these risks can result in the loss of customer trust and with reputational damage because of irresponsible AI practices or unintended consequences.

Fortunately, for foundation models in Amazon Bedrock, you can configure guardrails to filter and block inappropriate content. You can use guardrails to define threshold for content filters for hate, insults, sexual content, or violence. You can also block topics altogether.

For these topics, you can use plaintext to describe the topics that should be denied. When a user submits a prompt, it must pass through guardrails first. If the prompt is not allowed by guardrails, the user receives a violation response that you configure. The prompt never makes it to the model. Guardrails can be set on both the prompt and the model response.

So, if a prompt passes the guardrail, the response can still be blocked. Otherwise, it passes through as is. Another feature in SageMaker Clarify is the ability to run evaluation jobs of large language models so that you can compare models.

It can run four different types of tasks, including text generation, text classification, question and answering, and text summarization. It can then evaluate the performance for five different dimensions. Prompt stereotyping measures the probability of your model, including biases in its response. It includes biases for race, gender, sexual orientation, religion, age, nationality, disability, physical appearance, and socioeconomic status.

Toxicity checks your model for sexual references, rude, unreasonable, hateful, or aggressive comments, profanity, insults, flirtations, attacks on identities, and threats. Factual knowledge checks the veracity of the model responses.

Semantic robustness checks whether your model output changes because of keyword typos, random changes to uppercase, and random additions or deletions of white spaces. Accuracy compares the model output to the expected responses, such as classifying and summarizing the data correctly.

You can use a built-in prompt dataset or supply your own. You can also get a human workforce, such as employees or subject matter experts, to provide their feedback. If you are using Amazon Bedrock, the same capabilities are available in the Bedrock console to help evaluate the pre-trained LLMs in Amazon Bedrock.

I'm going to pause this lesson here, and in the next lesson, we will get started with task statement 4.2.