AWS AI Practitioner Exam Notes
Task Statement 5.1: Explain Methods to Secure AI Systems
This task statement focuses on protecting AI data, models, applications, infrastructure, identities, and outputs. It also covers governance, traceability, privacy, hallucination mitigation, and monitoring.
1. AWS Shared Responsibility Model
AWS security and compliance follow the Shared Responsibility Model.
AWS responsibility: Security of the cloud
AWS is responsible for protecting the underlying infrastructure that runs AWS services, including:
- AWS Regions and Availability Zones
- Data centers and physical facilities
- Physical servers and storage
- Networking infrastructure
- Hardware
- Virtualization layer
- Managed service infrastructure
- Host operating systems for many managed services
AWS provides secure cloud infrastructure, but customers must configure and use AWS services securely.
Customer responsibility: Security in the cloud
Customers are generally responsible for:
- IAM users, roles, policies, and permissions
- Protecting credentials
- Configuring security groups and network controls
- Encrypting data
- Protecting training data, prompts, and model artifacts
- Configuring applications and APIs securely
- Managing operating systems and patches when using services such as EC2
- Monitoring activity and responding to threats
- Configuring logging and audit trails
- Preventing public access to data
- Validating model inputs and outputs
- Managing privacy and regulatory requirements
The exact customer responsibility depends on the service.
Service-specific responsibility
Amazon EC2
If a model is hosted on EC2, the customer is responsible for more components, including:
- Guest operating system
- Security patches
- Installed software
- Application code
- Model-serving framework
- Instance configuration
- Scaling configuration
- Network configuration
Managed services
With managed services such as Amazon SageMaker managed endpoints or serverless inference, AWS manages more of the underlying infrastructure. However, the customer is still responsible for:
- IAM configuration
- Data access
- Model and application configuration
- Network settings
- Encryption choices
- Data privacy
- Monitoring and governance
Exam tip
The Shared Responsibility Model does not mean AWS is responsible for all security. AWS secures the infrastructure, while the customer secures data, identities, applications, configurations, and workloads.
2. IAM for Securing AI Systems
AWS Identity and Access Management, or IAM, controls access to AWS resources and APIs.
IAM can be used to control access to:
- Amazon S3 training data
- SageMaker notebooks and jobs
- SageMaker endpoints
- Amazon Bedrock models
- KMS encryption keys
- CloudWatch logs
- ECR container images
- Glue catalogs
- Feature Store resources
- Agent tools and external services
Principle of least privilege
Grant only the permissions required to perform a task.
For example:
- A data scientist may need read access to a specific S3 prefix.
- A training job may need read access to training data and write access to a model-output location.
- An inference endpoint may need access only to a feature store or model artifact.
- An MLOps user may manage endpoints but not read sensitive data.
- A logging role may write logs but not delete them.
Avoid broad permissions such as:
Action: \"*\"
Resource: \"*\"
unless they are genuinely required.
IAM users
An IAM user represents a person or application identity created in an AWS account.
Best practices:
- Create separate users for separate people.
- Do not share credentials.
- Require MFA.
- Avoid long-lived access keys.
- Do not embed credentials in source code.
- Rotate or remove unnecessary access keys.
- Prefer federation and IAM roles for workforce access.
New IAM users have no permissions by default unless permissions are granted.
IAM groups
An IAM group is a collection of IAM users.
Groups are useful for assigning permissions according to job function, such as:
- Developers
- Data scientists
- Security administrators
- MLOps engineers
- Auditors
Policies should generally be attached to groups rather than individually to users.
Important facts:
- A user can belong to multiple groups.
- A group can contain many users.
- Groups cannot contain other groups.
IAM roles
An IAM role provides temporary credentials when assumed by an authorized identity.
A role can be assumed by:
- An IAM user
- An AWS service
- An application
- A SageMaker job
- An EC2 instance
- A Lambda function
- A federated workforce user
- An external identity provider
Roles are preferred over long-lived access keys because temporary credentials expire automatically.
Every role has a trust policy, which specifies who or what can assume the role.
The role also has one or more permissions policies, which specify what the role can do after it is assumed.
Example
A SageMaker training job can assume a role that allows it to:
- Read input data from a specific S3 bucket
- Write model artifacts to a specific S3 location
- Publish logs to CloudWatch
- Use a specific KMS key
It should not automatically have administrator access to the account.
Identity-based and resource-based policies
Identity-based policy
Attached to an identity, such as:
- User
- Group
- Role
It specifies which actions the identity can perform.
Resource-based policy
Attached to a resource, such as:
- S3 bucket policy
- KMS key policy
- SNS topic policy
- SQS queue policy
It specifies which principals can access that resource.
Important policy evaluation rule
An action generally requires an applicable allow and must not be blocked by an explicit deny.
An explicit deny overrides an allow.
Other policy mechanisms can further restrict access, including:
- Service Control Policies, or SCPs
- Permissions boundaries
- Session policies
- VPC endpoint policies
- Resource policies
- Identity policies
Exam trap
Do not assume that an allow in one policy always grants access. An explicit deny, permissions boundary, SCP, or other restriction can prevent the action.
Root user
The AWS root user has complete account access and its permissions cannot be restricted by IAM policies.
Best practices:
- Enable MFA on the root user.
- Use a strong, unique password.
- Never share root credentials.
- Do not use the root user for everyday work.
- Delete or disable root access keys.
- Use IAM roles or IAM Identity Center instead.
The root user is reserved for limited account-level tasks that require root access.
IAM Identity Center and federation
AWS IAM Identity Center allows users to authenticate through:
- Microsoft Active Directory
- An external identity provider
- A directory managed by IAM Identity Center
Users can receive temporary access to multiple AWS accounts through a central portal.
Advantages:
- Centralized workforce identity management
- Group-based access
- Permission sets
- Temporary credentials
- Reduced need for IAM users in each account
- Better integration with enterprise identity systems
Exam tip
For human workforce access across multiple AWS accounts, IAM Identity Center and federation are generally preferred over creating individual IAM users in every account.
3. SageMaker Role Manager
Amazon SageMaker Role Manager helps create IAM roles for common machine learning activities.
It provides predefined personas, such as:
- Data scientist
- MLOps
- SageMaker compute
It can generate permissions for activities involving:
- S3
- AWS Glue
- Amazon Athena
- CloudWatch
- SageMaker training jobs
- SageMaker endpoints
- Pipelines
- Experiments
- Model management
Role Manager simplifies role creation, but the resulting role should still be reviewed and reduced to the minimum necessary permissions.
Exam trap
A predefined role or managed policy is convenient, but convenience does not automatically equal least privilege.
4. Encryption for AI Systems
Encryption protects data so that possession of the storage or network path is not sufficient to read the data.
Encryption at rest
Protects stored data, including:
- Training datasets
- Validation datasets
- Model artifacts
- Feature data
- Notebook storage
- Endpoint storage
- Logs
- Backups
- Prompt and response records
Examples include encryption in:
- Amazon S3
- Amazon EBS
- SageMaker storage
- Amazon DynamoDB
- Amazon RDS
- CloudWatch Logs
Encryption in transit
Protects data while moving between systems.
AWS service APIs and endpoints generally support TLS/HTTPS.
Use encrypted connections for:
- Client-to-API communication
- Application-to-model communication
- Data transfers
- Service-to-service communication
- Administrative access
Client-side and server-side encryption
Client-side encryption
The customer encrypts data before sending it to AWS.
Advantages:
- Customer controls encryption before transmission.
- AWS services never receive the unencrypted plaintext.
The application is responsible for encryption and decryption.
Server-side encryption
The AWS service encrypts data after receiving it and before storing it.
Advantages:
- Easier to implement
- Usually enabled through a service configuration
- More consistent across applications
Amazon S3, SageMaker, and other services may provide default encryption.
AWS Key Management Service
AWS KMS allows customers to:
- Create and manage encryption keys
- Control key usage
- Define key policies
- Control which roles and services can use a key
- Enable or disable keys
- Configure key rotation
- Audit key usage
A customer-managed KMS key provides more control than a service-owned or AWS-managed key.
Important security concept
Access to encrypted data usually requires both:
- Permission to access the data resource, such as an S3 object
- Permission to use the KMS key for decryption
Therefore, accidentally granting S3 read access may not be sufficient to read data encrypted with a customer-managed KMS key.
SageMaker distributed training
Distributed training uses multiple compute nodes.
Inter-node traffic may not be encrypted by default in some configurations. If required, enable inter-node encryption.
Trade-off:
- Improves confidentiality.
- May increase training time, especially for deep learning workloads.
Exam trap
Encryption at rest does not protect data while it is moving. Encryption in transit does not protect stored data. For strong protection, use both.
5. Amazon Macie
Amazon Macie is a data security and privacy service that helps discover and protect sensitive data in Amazon S3.
Macie can:
- Inventory S3 buckets
- Identify whether buckets are public or private
- Identify cross-account sharing
- Report encryption status
- Detect sensitive data
- Use machine learning and pattern matching
- Identify data such as personally identifiable information, or PII
- Generate findings and alerts
Potential sensitive data includes:
- Names
- Addresses
- Social Security numbers
- Credit card numbers
- Account numbers
- Other regulated or confidential information
AI security use case
Use Macie to scan S3 data before it is used for:
- Model training
- Fine-tuning
- RAG knowledge bases
- Feature engineering
- Evaluation datasets
Sensitive information should generally be:
- Removed
- Masked
- Tokenized
- Anonymized
- Replaced with synthetic values
- Restricted through access controls
Exam tip
Macie is primarily associated with discovering sensitive data in Amazon S3. It is not a general-purpose IAM replacement, model-monitoring service, or network firewall.
6. Network Security and AWS PrivateLink
AI workloads should avoid unnecessary public network exposure.
VPC controls
A VPC can be used to control network connectivity for SageMaker and other workloads.
Security controls include:
- Security groups
- Network ACLs
- Route tables
- Network firewalls
- NAT gateways
- VPC endpoints
- Private subnets
- Egress restrictions
SageMaker Studio and notebook instances can be configured with VPC networking.
Internet access risk
Direct internet access may allow a notebook or model-development environment to:
- Download packages
- Access external websites
- Send data outside the organization
- Download malicious or compromised code
- Exfiltrate training data or credentials
A more secure approach is to use a VPC with controlled egress.
VPC-only mode
VPC-only mode helps prevent direct public internet access from a SageMaker environment.
However, if the environment cannot access public service endpoints, it still needs a private method to reach required AWS services.
AWS PrivateLink and VPC interface endpoints
AWS PrivateLink allows private connectivity from a VPC to supported AWS services without requiring traffic to traverse the public internet.
VPC interface endpoints can provide private access to services such as:
- Amazon S3, where supported through the appropriate endpoint type
- SageMaker APIs
- SageMaker Runtime
- CloudWatch
- Other AWS services
Benefits:
- Keeps traffic within private AWS networking
- Reduces public exposure
- Supports private subnet architectures
- Helps satisfy network isolation requirements
Exam trap
A VPC alone does not automatically make every service interaction private. You may need to configure the appropriate VPC endpoints and routing.
7. Amazon Bedrock Guardrails
Amazon Bedrock Guardrails help control and filter generative AI inputs and outputs.
They can help with:
- Denying unwanted topics
- Blocking harmful or unsafe content
- Detecting and filtering sensitive information
- Applying content filters
- Redacting or blocking certain information
- Reducing toxicity
- Controlling prompt and response behavior
Guardrails can be used with supported foundation model interactions and generative AI applications.
Guardrail objectives
Guardrails help reduce:
- Toxic or abusive output
- Hate or violence-related content
- Sexual content
- Harassment
- Sensitive data disclosure
- Unsafe responses
- Responses on prohibited topics
Guardrails are a defense layer, not a replacement for:
- IAM
- Encryption
- Input validation
- Output validation
- Application authorization
- Monitoring
- Human review
Exam tip
Guardrails govern model input and output behavior. IAM governs access to AWS resources. They solve different security problems.
8. Amazon Bedrock AgentCore Identity and Policy
Agent-based AI systems can call tools, APIs, databases, and external services. These systems require identity and authorization controls.
AgentCore Identity
Amazon Bedrock AgentCore Identity is used to help agents securely authenticate to tools and resources.
It can help manage:
- Agent workload identities
- User identities
- Authentication to external services
- OAuth-based access
- Credential and token handling
- Secure access to tools and APIs
- Temporary or delegated access
The goal is to avoid placing static credentials directly in prompts, code, or agent configuration.
Policy in AgentCore
AgentCore policy controls what an agent is authorized to do.
Policies should define:
- Which agent can use which tool
- Which user can invoke which agent
- Which resources the agent may access
- Which actions are allowed
- Which actions require approval
- Which parameters or destinations are permitted
- Whether sensitive operations are blocked
Use least privilege for agent tools. For example, an agent that reads customer data should not automatically be allowed to delete records or send external email.
Agent security risks
Agents introduce additional risks because they can:
- Invoke tools
- Access private data
- Make changes in business systems
- Follow instructions from untrusted content
- Chain multiple actions
- Take actions without direct human involvement
Controls should include:
- Identity-based authorization
- Tool-level permissions
- Input validation
- Output validation
- Human approval for high-risk actions
- Logging of tool calls
- Rate limiting
- Explicit allowlists
- Isolation of sensitive tools
Exam trap
An agent’s ability to call a tool is not the same as the tool’s authorization to perform every possible operation. Permissions must be limited at both the agent and resource levels.
9. AI-Specific Threats and Vulnerabilities
Data poisoning
An attacker modifies training, validation, or fine-tuning data to influence model behavior.
Example:
- Fraudulent transactions are labeled as legitimate.
- Malicious examples are inserted into a fine-tuning dataset.
- A backdoor pattern is added so the model behaves incorrectly for specific inputs.
Controls:
- Restrict write access to training data.
- Version and approve datasets.
- Validate data quality and labels.
- Compare new data to trusted baselines.
- Scan for anomalies.
- Maintain immutable or protected copies.
- Review changes before training.
- Use separate validation data.
- Retrain and validate after suspected corruption.
Adversarial inputs
An attacker makes small, carefully designed changes to an input to cause an incorrect prediction.
Examples:
- Slight image modifications cause incorrect facial recognition.
- Small changes cause a classifier to misclassify a transaction.
- Crafted text causes an LLM to produce an unsafe result.
Controls:
- Input validation
- Adversarial testing
- Robust model training
- Rate limiting
- Anomaly detection
- Restricting confidence-sensitive actions
- Human review for high-impact predictions
Model inversion
An attacker studies model outputs to infer information about the training data.
Examples:
- Repeated queries reveal characteristics of employee facial images.
- Model responses expose whether a person’s record was included in training.
- Confidence scores help reconstruct sensitive information.
Controls:
- Minimize output detail.
- Restrict model access.
- Avoid unnecessarily exposing confidence scores.
- Apply query throttling.
- Monitor unusual query patterns.
- Use privacy-enhancing techniques.
- Avoid training on unnecessary PII.
Model extraction
An attacker uses many input-output pairs to create a copy or approximation of the original model.
Controls:
- Restrict endpoint access.
- Authenticate and authorize users.
- Rate-limit requests.
- Monitor query volume and patterns.
- Avoid exposing unnecessary model details.
- Protect model artifacts.
- Detect suspicious repeated probing.
Prompt injection
Prompt injection occurs when malicious instructions influence a generative AI model to ignore intended instructions or reveal information.
Examples:
- “Ignore previous instructions.”
- Instructions hidden in a retrieved document.
- A webpage telling an agent to disclose secrets.
- A user attempting to override system instructions.
- Malicious content causing an agent to invoke an unauthorized tool.
Prompt injection can be:
- Direct: The user includes the malicious instruction.
- Indirect: Instructions are embedded in retrieved documents, web pages, emails, or files.
Controls:
- Treat user and retrieved content as untrusted.
- Separate system instructions from user content.
- Validate and classify prompts.
- Detect known attack patterns.
- Restrict tools and permissions.
- Do not place secrets in prompts.
- Use guardrails.
- Require approval for sensitive actions.
- Filter model output.
- Log prompts, responses, and tool calls where appropriate.
- Use allowlists for tools and destinations.
Exam trap
Prompt injection is not solved solely by encrypting data. Encryption protects confidentiality while stored or transmitted; prompt injection manipulates model behavior.
Data leakage
Sensitive information may leak through:
- Prompts
- Retrieved documents
- Model responses
- Logs
- Traces
- Error messages
- Training data
- Public S3 buckets
- Model artifacts
- Tool outputs
Controls:
- Data classification
- PII detection and redaction
- Macie
- IAM least privilege
- Guardrails
- Output filtering
- Encryption
- Private networking
- Log redaction
- DLP policies
- Avoiding unnecessary data collection
Toxicity and harmful content
AI systems may generate:
- Abusive language
- Hate speech
- Harassment
- Violent content
- Sexual content
- Discriminatory responses
Controls:
- Content filters
- Amazon Bedrock Guardrails
- Prompt restrictions
- Output moderation
- Human review
- Red-team testing
- Monitoring user feedback
- Clear acceptable-use policies
10. Secure Data Engineering Best Practices
Secure AI begins with secure data.
Assess data quality
Before training or retrieval, assess:
- Accuracy
- Completeness
- Consistency
- Timeliness
- Duplicate records
- Missing values
- Outliers
- Label correctness
- Schema consistency
- Class imbalance
- Data drift
- Unexpected distributions
Poor-quality data can cause both security and model-performance problems.
Protect privacy
Use privacy-enhancing techniques where appropriate:
- Data minimization
- PII removal
- Masking
- Tokenization
- Anonymization
- Pseudonymization
- Aggregation
- Differential privacy
- Synthetic data
- Access-controlled views
Only collect and use data necessary for the AI use case.
Control data access
Use:
- IAM roles
- S3 bucket policies
- S3 Block Public Access
- KMS policies
- VPC endpoints
- Lake Formation permissions where applicable
- Separate development, test, and production accounts
- Data classification
- Dataset versioning
- Private repositories
- Approval workflows
Protect data integrity
Controls include:
- Checksums and hashes
- Digital signatures
- Versioning
- Immutable storage
- Object Lock where appropriate
- Audit logs
- Data validation pipelines
- Source approval
- Schema validation
- Controlled write access
S3 Block Public Access
S3 Block Public Access can block public access at:
- Bucket level
- Account level
It overrides public access granted through bucket policies or access control lists.
Exam tip
When training data is stored in S3, check:
- Is the bucket private?
- Is Block Public Access enabled?
- Is encryption enabled?
- Are access policies least privilege?
- Is cross-account sharing intentional?
- Are sensitive data findings reviewed?
11. Source Citation, Data Lineage, and Data Origins
AI systems should document where data, models, prompts, and outputs came from.
This supports:
- Trust
- Reproducibility
- Compliance
- Debugging
- Bias investigation
- Copyright and licensing review
- Incident response
- Regulatory reporting
- Hallucination investigation
Source citation
Source citation identifies the documents or sources used to generate an answer.
For a RAG application, a response may cite:
- Document name
- Document version
- Page number
- Section
- URL
- Database record
- Timestamp
- Retrieved passage
Citations allow users to verify the answer against the source.
Important distinction
A citation is evidence of the source used. It does not guarantee that the generated answer is correct. The application should still validate whether the answer is supported by the cited content.
Data lineage
Data lineage records how data moves and changes through a system.
It can identify:
- Original data source
- Data ingestion process
- Transformations
- Feature engineering steps
- Training dataset version
- Code version
- Container image
- Hyperparameters
- Training job
- Model artifact
- Deployment endpoint
- Inference workflow
Amazon SageMaker ML Lineage Tracking can automatically track relationships between ML artifacts and workflow components.
Example lineage query:
- Which models used a particular dataset?
- Which datasets were processed by a particular container?
- Which endpoint is using a specific model version?
Data cataloging
A data catalog helps organizations discover and understand datasets.
Metadata may include:
- Data owner
- Business purpose
- Schema
- Data classification
- Quality metrics
- Retention period
- Lineage
- Sensitivity
- Access rules
- Source and update frequency
Cataloging makes it easier to identify approved and trusted datasets.
SageMaker Model Cards
Amazon SageMaker Model Cards document important model information, including:
- Intended use
- Out-of-scope use
- Model overview
- Training data details
- Evaluation data
- Evaluation metrics
- Known limitations
- Bias or fairness considerations
- Risk rating
- Ethical considerations
- Deployment information
- Responsible-use guidance
Model Cards help risk managers, data scientists, ML engineers, and auditors understand how a model should and should not be used.
They can be exported and shared with stakeholders.
SageMaker Model Registry
Model Registry helps catalog and version models.
It can track:
- Model versions
- Model packages
- Training metrics
- Approval status
- Deployment history
- Model groups
Typical approval states may include:
- Pending
- Approved
- Rejected
Exam distinction
- Model Registry: manages model versions and approval lifecycle.
- Model Cards: documents model purpose, risks, limitations, and evaluation.
- Lineage Tracking: records relationships between data, code, jobs, models, and deployments.
- Feature Store: manages reusable features and feature metadata.
12. Audit Trails and Logging
AI systems should record enough information to investigate security incidents, quality problems, and inappropriate usage.
Useful records include:
- User or workload identity
- Timestamp
- Source IP or network context
- API call
- Model or endpoint used
- Model version
- Prompt or input, subject to privacy requirements
- Retrieved sources
- Generated output, subject to privacy requirements
- Tool calls
- Parameters
- Approval decisions
- Errors
- Guardrail actions
- Latency and token usage
- Evaluation results
AWS CloudTrail
AWS CloudTrail records AWS API activity, including:
- Who made a request
- What action was performed
- Which resource was accessed
- When it occurred
- Source IP address
- Request details
CloudTrail logs can be delivered to Amazon S3 and integrated with monitoring and security workflows.
SageMaker API activity is integrated with CloudTrail. However, not every type of model invocation is necessarily captured as a standard CloudTrail management event.
Exam trap
CloudTrail is primarily for AWS API activity and account governance. It is not automatically a complete record of every prompt and model response. Application-level AI interaction logging may be required.
CloudWatch
CloudWatch can collect:
- Logs
- Metrics
- Alarms
- Model-monitoring results
- Application events
- Endpoint metrics
Use alarms to notify teams when thresholds are exceeded.
Logging must be designed carefully because prompts, retrieved documents, and outputs may contain sensitive data. Apply:
- Redaction
- Access control
- Encryption
- Retention policies
- Data minimization
13. Monitoring AI Systems
Production AI systems should be monitored for both security and quality.
SageMaker Model Monitor
SageMaker Model Monitor can monitor:
- Data quality
- Model quality
- Data drift
- Model performance
- Anomalies
- Baseline deviations
Results can be viewed in SageMaker Studio and sent to CloudWatch.
Data quality monitoring
Data capture records inference inputs and outputs and stores them in Amazon S3.
A baseline can be created from:
- Training data
- A trusted reference dataset
The monitoring job compares current inference data with the baseline.
Possible findings include:
- Missing features
- Unexpected distributions
- Invalid values
- Schema changes
- Statistical drift
Model quality monitoring
Model quality monitoring compares predictions against labeled ground-truth data.
It can measure metrics such as:
- Accuracy
- Precision
- Recall
- F1 score
- AUC
- Regression error
Important distinction
- Data quality monitoring checks whether incoming data has changed or become invalid.
- Model quality monitoring checks whether predictions remain accurate against labeled outcomes.
14. Hallucinations and Grounding
A hallucination is an AI-generated response that is incorrect, unsupported, or fabricated.
Examples:
- Invented facts
- Fake citations
- Incorrect calculations
- Claims unsupported by source documents
- Confident answers when the model lacks sufficient information
Retrieval-Augmented Generation grounding
RAG grounds generation in retrieved enterprise or trusted documents.
Typical flow:
- User submits a question.
- The system searches a knowledge base.
- Relevant passages are retrieved.
- The passages are included in the model context.
- The model generates an answer based on that context.
- The application may return citations.
Benefits:
- Uses current or organization-specific information
- Reduces reliance on model memory
- Improves traceability
- Enables source citation
- Can reduce hallucinations
RAG limitations
RAG does not guarantee correctness. Hallucinations can still occur when:
- Retrieval returns irrelevant documents
- Important documents are missing
- Documents are outdated
- The context is too large
- The model misinterprets the source
- Retrieved content contains prompt injection
- The system does not require evidence-based answers
Grounding techniques
Useful techniques include:
- Use approved and authoritative sources.
- Improve document chunking.
- Use metadata filters.
- Use hybrid or semantic search.
- Rerank retrieved passages.
- Include source references.
- Require the model to answer only from retrieved context.
- Instruct the model to say “I do not know” when evidence is insufficient.
- Use structured output formats.
- Validate claims against source passages.
- Use human review for high-impact decisions.
Output validation
Validate generated output before showing it to users or taking action.
Validation can include:
- Schema validation
- Type checking
- Required-field checking
- Business-rule validation
- Citation verification
- Source entailment checks
- PII detection
- Toxicity detection
- Safety filtering
- Prohibited-content detection
- Fact verification
- SQL or code safety checks
For an agent, validate tool arguments before execution.
Example:
- Verify that a payment amount is within limits.
- Verify that an email recipient is approved.
- Verify that a database query is read-only.
- Verify that a generated URL belongs to an approved domain.
Confidence scoring
Confidence scores estimate how certain a system is about an answer or prediction.
They can be used to:
- Route low-confidence results to human review
- Abstain from making a prediction
- Request additional information
- Restrict automated actions
- Trigger a fallback process
Important caution
A model’s confidence is not necessarily the same as correctness. Generative AI models can be highly confident while hallucinating.
Use confidence scoring together with:
- Retrieval quality
- Citation support
- Output validation
- Ground-truth evaluation
- Human review
15. End-to-End Secure AI Architecture
A secure AI solution commonly applies multiple layers:
Data layer
- Classify data.
- Remove unnecessary PII.
- Use Macie to discover sensitive S3 data.
- Apply S3 Block Public Access.
- Encrypt data with S3 encryption and KMS.
- Version and validate datasets.
- Restrict access using IAM roles.
Network layer
- Use private subnets where appropriate.
- Restrict internet egress.
- Configure security groups and network ACLs.
- Use VPC endpoints and PrivateLink.
- Avoid exposing model endpoints publicly unless required.
Identity layer
- Use IAM roles instead of long-lived credentials.
- Use IAM Identity Center for workforce access.
- Require MFA.
- Apply least privilege.
- Use AgentCore Identity for agent authentication and delegated access.
- Apply AgentCore policies to control tool and resource access.
Model layer
- Protect model artifacts.
- Control endpoint access.
- Test for adversarial inputs.
- Monitor for drift and anomalies.
- Maintain model versions.
- Document model risks with Model Cards.
- Track lineage.
Application layer
- Detect prompt injection.
- Validate user input.
- Treat retrieved content as untrusted.
- Filter toxic and sensitive output.
- Validate structured responses.
- Verify citations.
- Apply human approval to high-risk actions.
Governance layer
- Use CloudTrail for AWS API auditing.
- Log AI interactions where appropriate.
- Protect logs from unauthorized access.
- Maintain data, model, and code lineage.
- Monitor model and data quality.
- Define retention and incident-response procedures.
16. High-Value Exam Tips and Traps
Tip 1: Match the service to the problem
| Requirement | Relevant AWS service or feature |
|---|---|
| Control AWS permissions | IAM |
| Temporary credentials | IAM roles |
| Workforce federation | IAM Identity Center |
| MFA | IAM |
| Encrypt and manage keys | AWS KMS |
| Discover PII in S3 | Amazon Macie |
| Block public S3 access | S3 Block Public Access |
| Private access to AWS services | VPC endpoints / AWS PrivateLink |
| Log AWS API calls | AWS CloudTrail |
| Monitor model/data drift | SageMaker Model Monitor |
| Document model risks | SageMaker Model Cards |
| Track ML artifact relationships | SageMaker Lineage Tracking |
| Version and approve models | SageMaker Model Registry |
| Manage reusable ML features | SageMaker Feature Store |
| Filter generative AI content | Amazon Bedrock Guardrails |
| Authenticate agent workloads and tools | Amazon Bedrock AgentCore Identity |
| Restrict agent actions and tools | Policy in AgentCore |
| Reduce hallucinations | RAG grounding, citations, validation |
Tip 2: Encryption does not equal authorization
Encryption protects confidentiality. IAM determines who can access resources. Both are required.
Tip 3: Private networking does not automatically mean secure access
A private network path does not replace:
- IAM
- Authorization
- Encryption
- Input validation
- Monitoring
Tip 4: Guardrails are not IAM
Guardrails filter content and help control model behavior. IAM controls access to AWS resources and APIs.
Tip 5: CloudTrail is not the same as application logging
CloudTrail records AWS API activity. It does not necessarily record every prompt, response, retrieved document, or tool invocation. Application-level audit logging may be needed.
Tip 6: RAG reduces hallucinations but does not eliminate them
Retrieval quality, source quality, prompt design, output validation, and citation verification still matter.
Tip 7: Root user permissions cannot be restricted
Protect the root user with MFA and avoid using it for normal operations.
Tip 8: Explicit deny wins
An explicit deny overrides an allow in IAM or related policy controls.
Tip 9: Model monitoring has different purposes
- Data quality: Has the input distribution or schema changed?
- Model quality: Are predictions still accurate?
- Drift: Has the data or relationship between data and predictions changed?
- Security monitoring: Is the system being probed, abused, or attacked?
Tip 10: Protect logs too
AI logs may contain prompts, PII, confidential documents, secrets, and generated responses. Logs must be access-controlled, encrypted, redacted, and retained appropriately.