Skip to content

Skill 3.1.5: Deploy models that were built outside of AWS into AWS environments (for example, Amazon SageMaker AI, Amazon Bedrock Custom Model Import)

Question 1 - Selecting Deployment Path for External Customized Foundation Model

A financial services company has fine-tuned a Llama 3 8B foundation model in their on-premises GPU cluster for sentiment analysis. The data science team wants to serve the customized model through a fully managed API, with no responsibility for provisioning or operating inference endpoints. The solution must integrate with existing AWS Identity and Access Management (IAM) policies for model invocation, and the team wants to pay for inference on-demand rather than committing to reserved capacity.

Which deployment strategy should the ML engineer use?

A) Export the fine-tuned model weights to Amazon S3 and use Amazon Bedrock Custom Model Import to serve it via the Bedrock InvokeModel API. B) Register the model in Amazon SageMaker Model Registry and deploy to a real-time SageMaker endpoint with auto scaling enabled. C) Deploy the model as a containerized service on Amazon Elastic Kubernetes Service (EKS) with a Kubernetes Ingress controller. D) Upload the model artifacts to Amazon SageMaker JumpStart and deploy through the JumpStart managed inference pipeline.

Click to reveal Answer & Explanation

Correct Answer: A

Correct Answer Analysis

Option A is correct. Amazon Bedrock Custom Model Import is specifically designed for this scenario. It allows organizations to import supported open-weight model architectures (including Llama 3) that were fine-tuned outside of AWS. The model is served through the fully managed Bedrock InvokeModel API with no endpoint management, integrates natively with IAM policies for access control, and supports on-demand invocation without requiring Provisioned Throughput reservations. The team also benefits from a serverless pricing model where they only pay for tokens processed.

Distractor Analysis

  • Option B is incorrect: SageMaker real-time endpoints require the team to provision, manage, and scale instances (even with auto scaling), which violates the constraint of "no responsibility for provisioning or operating inference endpoints" and "fully managed API."
  • Option C is incorrect: Deploying on EKS introduces significant operational overhead—the team would manage pods, nodes, networking, and cluster upgrades. This contradicts the requirement for a fully managed solution with zero endpoint operations.
  • Option D is incorrect: SageMaker JumpStart is a catalog for deploying pre-trained foundation models, not a mechanism for importing custom fine-tuned weights from external training environments. There is no JumpStart "managed inference pipeline" for externally customized models.

Key Takeaway: Amazon Bedrock Custom Model Import is the AWS-native path for serving externally fine-tuned open-weight models via a serverless, on-demand managed API.

Question 2 - Validating Custom Model Import Compatibility

A machine learning engineering team at a healthcare analytics company has trained a custom object detection model using PyTorch and wants to explore serving it through Amazon Bedrock Custom Model Import to reduce operational overhead. Before committing to this migration path, the team must verify their model CAN be imported into Amazon Bedrock.

Which TWO compatibility requirements must the team confirm? (Select TWO)

A) The model architecture belongs to one of the supported open-weight foundation model families (e.g., Llama, Mistral, Flan-T5). B) The model artifacts are packaged in the expected file layout (e.g., Safetensors weights and tokenizer files) and stored in Amazon S3. C) The model is exported in ONNX format for cross-platform compatibility. D) The model is smaller than 100 MB when compressed. E) The model was originally trained using Amazon SageMaker distributed training.

Click to reveal Answer & Explanation

Correct Answers: A and B

Correct Options Analysis

  • A is Correct: Amazon Bedrock Custom Model Import accepts only specific open-weight foundation model architectures. Before planning a migration, teams must verify their model architecture is on the supported list—otherwise the import will fail.
  • B is Correct: The model artifacts must follow specific file format conventions (e.g., Safetensors for weights and standard tokenizer configuration files) and be uploaded to Amazon S3 before import. Bedrock validates the format during the import process.

Distractor Analysis

  • C is Incorrect: Custom Model Import does not require ONNX conversion. It expects weights in the native format of supported architectures (e.g., Safetensors), not a hardware-agnostic intermediate representation.
  • D is Incorrect: There is no 100 MB size limit for imported models. While S3 objects have limits individually, multipart upload allows large model files (many GBs) to be stored and imported.
  • E is Incorrect: Imported models can be trained anywhere (on-premises, EC2, other cloud providers). SageMaker training is not a prerequisite for Custom Model Import.

Key Takeaway: Always verify architecture support and artifact format/layout before planning a Bedrock Custom Model Import migration.

Question 3 - Selecting Deployment Strategy for Low-Latency Custom CV Model

An autonomous robotics company has developed a custom, lightweight computer vision model using a modified ResNet architecture for real-time obstacle detection. The model must be served with single-digit millisecond latency to robots over a WebSocket connection. The company wants minimal operational overhead but cannot tolerate cold-start delays, and the model requires GPU acceleration for inference.

Which hosting option should the ML engineer select?

A) Amazon Bedrock Custom Model Import with on-demand invocation. B) Amazon SageMaker Serverless Inference with auto scaling. C) Amazon SageMaker Real-Time Endpoint on a GPU instance with auto scaling. D) A batch transform job that processes sensor data periodically. E) AWS Lambda with a container image for inference.

Click to reveal Answer & Explanation

Correct Answer: C

Correct Answer Analysis

Option C is correct. The scenario demands single-digit millisecond latency over persistent WebSocket connections, GPU acceleration, and no tolerance for cold starts. A SageMaker Real-Time Endpoint on GPU-backed instances satisfies all constraints: - GPU acceleration is available (unlike serverless inference). - Real-time endpoints maintain warm instances with sub-10ms p99 latency for lightweight models. - Auto scaling handles robot fleet traffic fluctuations while preserving availability. - WebSocket connections can be handled through an API Gateway WebSocket API that invokes the endpoint.

Distractor Analysis

  • Option A is incorrect: Amazon Bedrock Custom Model Import is designed for foundation models (e.g., Llama, Mistral), not custom vision models like a modified ResNet. Even if it supported such architectures, Bedrock's invocation model is HTTP request/response oriented and is not ideal for persistent WebSocket connections with strict millisecond budgets.
  • Option B is incorrect: Serverless Inference supports scaling to zero but introduces cold starts (seconds) when traffic resumes after idle periods. It also does NOT support GPU acceleration—a direct violation of the GPU requirement.
  • Option D is incorrect: Batch transform is offline bulk processing, not real-time streaming inference over WebSocket connections.
  • Option E is incorrect: While Lambda supports container images, it does not provide GPU instances. Lambda's 15-minute maximum runtime is fine, but the lack of GPU acceleration makes it unsuitable for vision model inference at this latency. Lambda's cold starts also introduce unpredictable latency.

Key Takeaway: GPU acceleration is a hard requirement that rules out serverless inference options (SageMaker Serverless or Lambda); choose real-time endpoints when cold starts are unacceptable.

Question 4 - Configuring Multi-Model Endpoint for External Models

A retail analytics startup has built 300 lightweight fraud detection models (XGBoost) in a local development environment. These models share the same framework and serving stack. Traffic patterns are highly uneven—20 models are called constantly while the remaining 280 are invoked infrequently. The startup wants to minimize total infrastructure cost while serving all models.

Which hosting architecture BEST satisfies this requirement?

A) Deploy all 300 models to individual SageMaker real-time endpoints, each with its own compute instance. B) Deploy all 300 models to a single SageMaker Multi-Model Endpoint (MME), accepting load penalties for infrequently used models. C) Deploy the 20 frequently used models to individual endpoints and the remaining 280 to an MME. D) Deploy all 300 models to a single multi-container endpoint with one container per model. E) Deploy the models to Amazon Bedrock Custom Model Import for serverless, on-demand inference.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

Option B is correct. A SageMaker Multi-Model Endpoint (MME) is the ideal fit for: - Many models sharing the same framework/serving stack (XGBoost), which MME explicitly supports. - Highly uneven traffic patterns with a few hot models and many cold models. - Cost minimization: One shared compute fleet serves all 300 models rather than 300 separate instances. SageMaker dynamically loads/unloads models from memory, so frequently used models stay warm while rarely used models incur a one-time load penalty on first invocation.

Distractor Analysis

  • Option A is incorrect: 300 individual endpoints means 300 instances running continuously. This is extremely costly, especially when 280 of those models are rarely invoked and would run idle most of the time.
  • Option C is incorrect: While hybrid architectures can work, this answer violates the "minimize total infrastructure cost" constraint. Running 20 always-on endpoints plus an MME adds complexity and cost compared to a single MME serving everything. The MME handles hot and cold models together efficiently.
  • Option D is incorrect: Multi-container endpoints (MCEs) are designed for up to 15 different containers with potentially different frameworks or inference pipelines—not 300 models sharing the same framework. Deploying 300 containers to one endpoint exceeds MCE limits and introduces unnecessary complexity.
  • Option E is incorrect: Bedrock Custom Model Import only supports open-weight foundation model architectures (e.g., Llama, Mistral). XGBoost models are traditional ML models and cannot be imported into Bedrock.

Key Takeaway: Multi-Model Endpoints are the cost-optimal solution for many small models that share a framework, accepting load penalties for rarely used models.

Question 5 - Deploying External Model to Managed Foundation Model API

A legal technology company has fine-tuned a Mistral 7B model on local GPU hardware for contract summarization. The legal team wants the model served through its managed API so it can be invoked from internal applications. However, the company also has strict latency SLAs that require consistent throughput and immediate availability regardless of traffic.

Which deployment approach should the ML engineer select?

A) Import to Amazon Bedrock Custom Model Import and invoke on-demand. B) Import to Amazon Bedrock Custom Model Import and configure Provisioned Throughput for the imported model. C) Deploy to a SageMaker real-time endpoint with auto scaling and GPU instances. D) Deploy to a SageMaker serverless endpoint with provisioned concurrency.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

Option B is correct. Amazon Bedrock Custom Model Import supports Provisioned Throughput for imported models. This configuration: - Uses the fully managed Bedrock API (no endpoint operations). - Reserves dedicated model processing capacity to ensure consistent throughput and latency. - Eliminates cold starts that might occur with on-demand invocation. - Meets the "strict latency SLAs" and "immediate availability" requirements.

Distractor Analysis

  • Option A is incorrect: On-demand invocation for imported models in Bedrock may introduce unpredictable latency as capacity fluctuates. It does not guarantee "consistent throughput and immediate availability regardless of traffic" as required.
  • Option C is incorrect: While a SageMaker real-time endpoint with GPUs could meet the latency SLA, it violates the requirement for a "fully managed API"—the company would need to manage instances, scaling, health checks, and updates.
  • Option D is incorrect: SageMaker Serverless Inference does not support GPU acceleration (important for 7B model inference), and provisioned concurrency for serverless endpoints is not available in SageMaker (this is a Lambda feature). Even with warm capacity, the lack of GPU support makes this option unsuitable for foundation model inference.

Key Takeaway: For imported custom models in Amazon Bedrock, choose Provisioned Throughput when consistent performance is required; choose on-demand when cost optimization matters more than latency guarantees.

Question 6 - Migrating External Model with Custom Container Dependencies

A manufacturing company has built a custom defect detection model using a proprietary C++ serving stack and custom preprocessing logic that depends on specific system libraries. The team wants to run this model in a managed AWS service but cannot use Amazon SageMaker's built-in containers or serverless inference because of the specialized dependencies.

Which TWO options would enable the team to deploy this model to AWS with minimal operational overhead? (Select TWO)

A) Package the model and its dependencies into a Docker container and deploy to Amazon SageMaker Real-Time Endpoint using the bring-your-own-container (BYOC) approach. B) Package the model into a Docker container and deploy to Amazon Elastic Container Service (ECS) with Auto Scaling. C) Import the model into Amazon Bedrock Custom Model Import and serve through the Bedrock API. D) Deploy the model to Amazon SageMaker Multi-Model Endpoint (MME) using built-in XGBoost container. E) Convert the C++ serving code to run on AWS Lambda with a custom runtime.

Click to reveal Answer & Explanation

Correct Answers: A and B

Correct Options Analysis

  • A is Correct: SageMaker Real-Time Endpoint supports BYOC (Bring Your Own Container), allowing the team to package their proprietary C++ serving stack, custom preprocessing logic, and any system libraries into a Docker image. SageMaker manages the underlying infrastructure (instances, health checks, rolling updates) while the customer supplies the container.
  • B is Correct: Amazon ECS is a container orchestration service that can run any Docker image. With Auto Scaling enabled, ECS provides managed container orchestration while still allowing full control over the runtime environment and dependencies inherent to the Docker container itself.

Distractor Analysis

  • Option C is Incorrect: Amazon Bedrock Custom Model Import only accepts specific open-weight foundation model architectures (e.g., Llama, Mistral). A custom defect detection model using C++ serving logic cannot be imported into Bedrock.
  • Option D is Incorrect: Multi-Model Endpoints with built-in XGBoost containers assume all models use XGBoost and its standard serving container. The custom C++ stack and specialized libraries cannot run inside the built-in XGBoost container.
  • Option E is Incorrect: While Lambda supports custom runtimes, it imposes constraints: no GPU access, 10 GB max container size (uncompressed), limited disk, and potential cold starts. A proprietary C++ defect detection model with specialized libraries may not fit Lambda's constraints.

Key Takeaway: When a model requires specialized serving stacks or libraries, use SageMaker BYOC or container services (ECS/EKS) rather than Bedrock Custom Model Import, which is limited to supported foundation model architectures.

Question 7 - Cost-Optimal Deployment for External NLP Model with Intermittent Traffic

A startup has built a lightweight NLP classification model (DistilBERT) on a local workstation. The model is used by internal tools that make predictions only during business hours, with long idle periods overnight and on weekends. The team wants the lowest possible operational overhead and is willing to accept occasional cold-start latency on the first request after idle periods. GPU acceleration is NOT required.

Which hosting option should the ML engineer select?

A) Amazon SageMaker Real-Time Endpoint on CPU instances with auto scaling. B) Amazon SageMaker Serverless Inference. C) Amazon Bedrock Custom Model Import with on-demand invocation. D) A scheduled batch transform job that runs every 15 minutes during business hours.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

Option B is correct. Amazon SageMaker Serverless Inference is the optimal choice for this workload: - Low operational overhead: No instances to manage—serverless handles all infrastructure automatically. - Intermittent traffic: Scales to zero during idle periods (nights, weekends), minimizing cost. - Cold-start tolerance: The team explicitly accepts cold-start latency on first invocations after idle. - CPU-only: The model does not need GPU acceleration, so serverless constraints are acceptable. - Pay-per-use: The team pays only for actual inference processing time during business hours.

Distractor Analysis

  • Option A is incorrect: Real-time endpoints keep instances running continuously (or require scaling policies), which means paying for idle capacity during nights and weekends. Auto scaling does not scale to zero by default and introduces operational overhead for policy management.
  • Option C is incorrect: Amazon Bedrock Custom Model Import only accepts specific open-weight foundation model architectures. DistilBERT is a traditional NLP model and is not one of the supported foundation model families for Custom Model Import. Even if supported, it would require model weights in specific formats.
  • Option D is incorrect: Batch transform processes pre-assembled bulk datasets offline. It is not designed to accept real-time requests from internal tools during business hours—it processes static data from S3 on a schedule, not dynamic incoming prediction requests.

Key Takeaway: Serverless inference is ideal for intermittent, CPU-only workloads where cold starts are acceptable and the team wants zero instance management.

Question 8 - Selecting Multi-Model vs. Multi-Container Endpoint for Heterogeneous Models

A media streaming company has just acquired two startups. Each startup brings ML models that need to be served:

  • One model (a PyTorch recommendation embedder) that is called on every user request at high volume.
  • Two models (a TensorFlow content classifier and a PyTorch content similarity model) that are called occasionally during content ingestion.

All models were built outside AWS. The company wants to serve all three models from a compute-efficient architecture while maintaining strict latency requirements for the recommendation embedder.

Which hosting architecture should the ML engineer choose?

A) Deploy all three models to a single SageMaker Multi-Model Endpoint using the PyTorch serving container. B) Deploy the recommendation embedder to a dedicated real-time endpoint and deploy the two content models to a multi-container endpoint. C) Deploy all three models to a single multi-container endpoint with inference pipeline mode. D) Import all three models into Amazon Bedrock Custom Model Import for serverless inference.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

Option B is correct. The recommendation embedder handles every user request at high volume, so it requires dedicated compute to guarantee strict latency. Hosting it on its own real-time endpoint ensures it does not compete for resources with other models. The two content models are invoked occasionally and have less strict latency requirements, so they can share infrastructure. A multi-container endpoint (MCE) can host multiple heterogeneous containers (TensorFlow + PyTorch), and each container can be invoked directly by name. This approach balances performance (dedicated resources for hot model) with cost efficiency (shared resources for cold models).

Distractor Analysis

  • Option A is incorrect: A Multi-Model Endpoint (MME) assumes all models use the same framework and serving stack—but here we have PyTorch and TensorFlow, which use different containers. Additionally, MME shares a single compute fleet across all models, meaning the high-volume recommendation embedder would compete with the other models for compute, potentially causing latency spikes.
  • Option C is incorrect: Inference pipeline mode in a multi-container endpoint chains containers in sequence for a single prediction (e.g., preprocessing → inference → postprocessing). These three models serve different purposes and are not part of a single prediction pipeline.
  • Option D is incorrect: Amazon Bedrock Custom Model Import only accepts specific open-weight foundation model architectures (e.g., Llama, Mistral). PyTorch or TensorFlow custom recommendation/classification models are not supported for Custom Model Import.

Key Takeaway: For heterogeneous models with different frameworks and traffic patterns, separate dedicated endpoints for hot models and multi-container or MMEs for cold models is the best practice.

Question 9 - Deploying External Model from S3 Using SageMaker

A startup has trained a custom XGBoost model on a local machine and saved the model artifacts as model.tar.gz to Amazon S3. The model needs to be deployed to a SageMaker Endpoint using the built-in XGBoost serving container. The team writes a deployment script but encounters an error when calling create_model.

Which TWO steps are necessary before calling create_model to ensure successful deployment? (Select TWO)

A) Create a SageMaker Model Registry entry and approve the model version. B) Specify the Amazon S3 path to the model artifact and the Amazon Elastic Container Registry (ECR) path to the correct inference container. C) Create a model package group and register the model artifact. D) Create an execution role that grants SageMaker permission to access the model artifact in Amazon S3. E) Initialize a SageMaker endpoint configuration with auto scaling parameters for XGBoost instances.

Click to reveal Answer & Explanation

Correct Answers: B and D

Correct Options Analysis

  • B is Correct: The create_model API call requires the PrimaryContainer parameter, which specifies both the model artifact URL in S3 and the inference container image URI in Amazon ECR. The built-in XGBoost container must be specified so SageMaker knows what serving stack to use.
  • D is Correct: SageMaker needs an execution role (IAM role) with permissions to read the model artifact from S3 and potentially pull container images from ECR. Without a properly configured execution role, model creation fails.

Distractor Analysis

  • Option A is incorrect: Registering in SageMaker Model Registry is optional for model governance and versioning—it is not a prerequisite for calling create_model. Deployment can happen directly from S3 artifacts without registry involvement.
  • Option C is incorrect: Model package groups and model registration are features of SageMaker Model Registry, not prerequisites for create_model. These steps are used for model governance workflows, not direct endpoint deployment.
  • Option E is incorrect: Endpoint configuration and auto scaling are configured after create_model—they are not prerequisites for creating the model object itself. The correct order is: create_model → create_endpoint_config → create_endpoint → optional auto scaling.

Key Takeaway: The essentials for SageMaker model deployment are the model artifact S3 path, the inference container URI, and an execution role with appropriate permissions.

Question 10 - Migrating On-Premises Model with Real-Time vs. Batch Requirements

A telecommunications company has built a churn prediction model on-premises using scikit-learn. The model serves two purposes:

  1. Daily batch processing of 10 million customer records stored in Amazon S3.
  2. Real-time scoring for customer support agents when viewing customer profiles.

The company wants to migrate to AWS and is evaluating SageMaker hosting options. The model requires low latency for the real-time use case but batch processing is cost-sensitive. The team also wants to use the same model artifact in both places.

Which hosting strategy should the ML engineer recommend?

A) Deploy one SageMaker real-time endpoint for both batch and real-time use cases. B) Use two separate deployment paths: a SageMaker batch transform job for daily processing and a SageMaker real-time endpoint for agent queries. C) Use SageMaker serverless inference for both use cases. D) Use an asynchronous inference endpoint for batch processing and a real-time endpoint for agent queries. E) Use the same model on a SageMaker Multi-Model Endpoint (MME) for both use cases.

Click to reveal Answer & Explanation

Correct Answer: B

Correct Answer Analysis

Option B is correct. The two workloads have fundamentally different requirements: - Batch processing (10 million records daily): No live caller is waiting; the dataset is pre-assembled in S3. SageMaker Batch Transform is ideal—it provisions compute, processes the entire dataset, writes results to S3, and releases resources, making it cost-effective. - Real-time scoring (support agents): Agents are actively waiting for a churn score while viewing customer profiles. A real-time endpoint with low latency is required. - Same model artifact: The same S3 model artifact can be referenced by both the batch transform job and the real-time endpoint, ensuring consistency.

Distractor Analysis

  • Option A is incorrect: Forcing real-time endpoint to process 10 million records has two problems: it lacks the throughput for efficient bulk processing without massive scaling, and it keeps expensive instances running for batch workloads when they could be released after the job completes. Batch transform is cheaper for this use case.
  • Option C is incorrect: Serverless inference has cold starts and does not provide the high-throughput batch processing capability needed for 10M records. Running such volume through serverless would be inefficient.
  • Option D is incorrect: Asynchronous inference is designed for individual requests that take longer than real-time limits, not bulk dagta processing of 10M records. It queues individual requests rather than efficiently processing pre-assembled bulk datasets like Batch Transform.
  • Option E is incorrect: A Multi-Model Endpoint is for hosting many models efficiently, not for dual-host use cases. Using an MME for both batch and real-time processing still requires a real-time endpoint instance that would not scale for efficient bulk processing and would remain running for real-time use even when batch jobs complete.

Key Takeaway: The same model artifact can serve multiple inference modes—use batch transform for offline bulk data and real-time endpoints for synchronous, low-latency use cases.