Skip to content

Task Statement 2.3: Describe AWS infrastructure & technologies for building GenAI applications

Objectives:

  • Identify AWS services and features to develop GenAI applications (for example, Amazon Bedrock, Amazon SageMaker AI, SageMaker JumpStart, Amazon Quick, Kiro, Strands Agents, Amazon Bedrock AgentCore).
  • Describe the advantages of using AWS GenAI services to build applications (for example, accessibility, lower barrier to entry, efficiency, cost-effectiveness, speed to market, ability to meet business objectives).
  • Describe the benefits of AWS infrastructure for GenAI applications (for example, security, compliance, responsibility, safety).
  • Describe cost tradeoffs of AWS GenAI services (for example, responsiveness, availability, redundancy, performance, regional coverage, token-based pricing, provision throughput, custom models).

Lesson 1

Let's get started with the third task statement from Domain 2, which is to: Describe AWS infrastructure and technologies for building generative AI applications. This task statement is split into two lessons. For this task statement, let's continue to talk about advantages of using AWS generative AI services to build applications. These advantages include accessibility, lower barrier to entry, efficiency, cost-effectiveness, speed to market, and ability to meet business objectives.

It's important to understand that it takes a long time to train an LLM and then to process all the data. This data might contain millions of calculations. The process of fine-tuning and training a pre-trained model on a new dataset without training from scratch is also known as transfer learning. It can produce accurate models with smaller datasets and less training time. You can accelerate the learning for some types of models by using transfer learning, then use a pre-trained model as your starting point for the training of a new dataset. You can create your own pre-trained set or get them from the internet. The generalized model from the generalized dataset can be transferred into your training process with a new dataset.

Then, when you train your model, the algorithm already knows some information and understands what it needs to do. Then it only needs to figure out how those elements map to your new dataset. SageMaker JumpStart helps to find projects that are already built and are quick builds with datasets, models, algorithm types, and solutions based on industry best practices. This task statement also includes understanding the benefits of AWS infrastructure for generative AI applications, especially security, compliance, responsibility, and safety. In task statement 2.2, we talked about customer experiences using your generative AI applications. But customers are also building generative AI applications by using large language models, LLMs, and other foundation models, FMs, to enhance experiences. These models can also transform operations, improve employee productivity, and create new revenue.

I added a link for the AWS Cloud Adoption Framework for Artificial Intelligence, Machine Learning, and Generative AI, CAF-AI. It serves us both a starting point and a guide for your journey in AI, ML, and generative AI. The framework can be used as a resource for discussions about your AI strategy with your team, and also when collaborating with coworkers and AWS Partners. Foundation models and the applications built on them are valuable investments for organizations.

One of the biggest concerns that we hear about with generative AI applications is securing sensitive business data. For example, this data includes personal data, compliance data, operational data, and financial information. At AWS, our top priority is security and ensuring confidentiality of our customers' workloads. We think about security across the three layers of generative AI stack. The bottom layer provides the tools for building and training LLMs and other FMs. AI can have large hardware demands for training, large amounts of compute resources for inference, and guardrails for consumption. Price compared to performance is one of the key factors in setting standards for your organization. With AWS, you can use specialized hardware that can help reduce costs, with better price performance than traditional CPUs or GPUs.

The AWS Nitro System has specialized hardware and associated firmware that are designed to enforce security restrictions. These restrictions are to ensure that no one can access your workloads or data running on Amazon Elastic Compute Cloud, Amazon EC2, instances. This protection applies to all Nitro-based applications and instances. They include instances with ML accelerator, such as AWS Inferentia, AWS Trainium, and instances with GPUs, like P4, P5, G5, and G6.

At AWS, securing AI infrastructure means zero access to sensitive AI data, such as AI model weights and data processed with those models by any unauthorized person. The middle layer provides access to all the models, along with tools that you need to build and scale generative AI applications. You can design how your platform supports the development, deployment, and iteration of AWS ML and AI services. For example, ML services train or tune foundation models. And in this layer, all services consume models and capabilities, such as the large and costly to train foundation models in the generative AI domain. The top layer includes applications that use LLMs and other FMs to write and debug code, generate content, derive insights, and take actions. It might be a dashboard or a foundation model that you can use for prompt engineering, or it might be a specific generative AI architecture, such as Retrieval-Augmented Generation, RAG, applications.

What are the three critical components of any AI system? It's input, model, and output. These components can be protected by establishing security policies, standards, and guidelines, along with the roles and responsibilities related to AI workloads. AI systems can have specific vulnerabilities, such as prompt injection, data poisoning, and model inversion vulnerabilities. Remember to validate policies, because risks associated with an AI can have consequences, including privacy breaches, data manipulation, abuse, and compromised decision-making. Ensure that you understand why it's important to implement encryption, multi-factor authentication, continuous monitoring, and alignment to your tolerance and frameworks. I'm going to pause this lesson here, and in the next lesson, we will continue with task statement 2.3.

Lesson 2

Let's continue with task statement 2.3. And let's talk more about cost tradeoffs for AWS generative AI services. Here is a question, what are the pricing models for LLMs? One pricing model is to host LLMs on your own infrastructure. You are responsible for paying for the computing resources that are required to run these models. And you might have to pay a license fee for the LLM itself. You can also choose to pay by token pricing model.

What is token-based pricing? It is the number of tokens processed, and each token represents a discrete unit of information. For example, units of information might include a character or word in text, or a pixel in an image, or both for input and output. Tokens are the units that vendors use to price calls in their APIs. If you use the AWS and the pay-by-token model, it adds scalability. By hosting your own model will require infrastructure investment and maintenance. We have talked about the AWS ML stack. The AWS Global Infrastructure has architectural components, such as Regions, edge locations, and Availability Zones, and services that are globally resilient, regionally resilient and availability zone resilient.

Many AWS managed services, AMS, are built for specific purpose but have high availability built in. Dive deeper and ensure you know how and why the AWS Global Infrastructure adds high availability and fault tolerance. On top of the infrastructure layer are the AWS machine learning services, such as Amazon SageMaker. And on top of the machine learning layer are the AWS AI services, which are prebuilt services, prebuilt algorithms, models and services that you can use. You can use these services and integrate them into your application. If you know how to integrate with an API, write code and work with the AWS SDK, you can use these services without ML and AI.

Let's talk about AWS services that can help you build applications by using LLMs. SageMaker JumpStart is a model hub and it helps quickly deploy foundation models that are available within the service and integrate them into your applications. It provides fine-tuning and deploying models, and also helps you get quickly into production to operate at scale. JumpStart also provides a lot of resources in terms of blogs, videos, and example notebooks. JumpStart models require GPUs to fine-tune and deploy. You should refer to the SageMaker pricing page before selecting the compute to use. Also, remember to delete the SageMaker model endpoints when not in use and follow cost-monitoring best practices to optimize cost.

Amazon Bedrock is a managed AWS service that lets you use and access a number of different foundation models, FMs, using APIs. These foundation models include models that are curated by AWS in addition to foundation models provided by third parties, such as Cohere and Stability AI, which gives you the ability to interact with different models for different use cases. And you can develop generative AI applications at scale compared to having to build your own FMs from scratch. Amazon Bedrock adds the capability to import custom weights for supportive model architectures, and serve the custom model by using on-demand mode. You only pay for what you use with no time-based term commitment.

Amazon provides its own foundation models too. Currently, the Amazon Titan foundation model provided by Amazon acts as a general-purpose foundation model and is a great option if you require tax generation capabilities. Many of the foundation models offer similar use cases, especially when it comes to tax generation. Therefore, it's important to identify which foundation model would be most suitable for your use case. Amazon Bedrock offers features that can help you do just that through the use of playgrounds and model evaluations. Playgrounds in Amazon Bedrock let you experiment by running model inference against different base foundation models that are supported within the service to help you align your use cases with the highest accuracy. Depending on the model selected for your playground, it will determine the right types of inference parameters that you can adjust. And remember, you can vary the number of inference parameters to determine different completion results. PartyRock is a playground built on Amazon Bedrock. You can use it to build generative AI applications to learn fundamental techniques, such as understanding how a foundation model responds to different prompts. Some examples of applications that you can build in PartyRock are creating playlists, trivia games, recipes, and more. Let's wrap up this lesson and continue talking about the advantages of using AWS generative AI services to build applications. Most of us will not train our own LLMs because training requires investing, research, collecting and cleaning quality data, time, and more. Also, most of us will not host our own model because of the hardware expenses and the expense to store the data.

Remember that with generative AI, you can use vector databases, and that data is stored as embeddings. These embeddings are vectors that can be compressed, stored and indexed for advanced searches. AWS can help you build and scale generative AI applications that deliver new customer and employee experiences. You can customize your data and your use cases. You can use industry-leading FMs and LLMs of all sizes and types and generative AI-powered duplications with enterprise-grade security and privacy. I'm going to wrap up this lesson. Now, let's get started with our walkthrough questions.