Skip to content

Skill 1.2.2: Create and manage features by using AWS tools (for example, SageMaker Feature Store)

Task 1.2: Data Preparation for ML and AI - Perform data transformation, feature engineering, and pre-processing.

Core objective: Centralize engineered features so training and inference use the same definitions, reducing training-serving skew and enabling feature reuse across models.

SageMaker Feature Store — Core Architecture

Component Role Key detail
Feature group Logical container Requires record identifier + event time; feature definitions have types (String, Integral, Fractional)
Online store Low-latency serving Latest value per record ID; TTL; KMS; for real-time inference
Offline store Historical repository S3 + Glue Data Catalog; time-travel; point-in-time joins; for training
Ingestion Write features Streaming (Lambda/Kinesis/Flink) or batch (Glue/Data Wrangler/Processing)
Retrieval Read features Online for real-time; offline via Athena/Spark for training

Online vs Offline Store — Decision Matrix

Requirement Online Offline Both
Real-time inference ✅ ❌ ✅
Training with history ❌ ✅ ✅
Point-in-time correctness ❌ ✅ ✅
Time-travel queries ❌ ✅ ✅
Latest feature value only ✅ ❌ ✅
Cost-sensitive bulk storage ❌ ✅ ✅

Feature Engineering Transforms — Purpose & Gotchas

Transform Purpose Exam gotcha
Scaling (MinMax) Bounded range Fit on train; outliers distort
Standardization (Z-score) Center + scale by spread Fit on train; assumes distribution
Normalization (L2) Unit norm Common for embeddings/vectors
Log transform Compress skew Cannot handle ≤0 without offset
Binning Continuous → ranges Bin edges must be fixed from train
Feature splitting Compound field → parts e.g., address → city/state
One-hot encoding Categorical → binary High cardinality → sparse; consider hashing
Imputation Fill missing Use train statistics; avoid leakage
Outlier handling Cap/clip/winsorize Decide thresholds from train only

AWS Tool Selection for Feature Creation & Management

Tool Primary role Use when Not for
SageMaker Feature Store Store/serve features Need online/offline consistency Raw data lake; vector search
SageMaker Data Wrangler Interactive feature engineering Low-code exploration, transforms, export to Feature Store Cluster-scale ETL
AWS Glue DataBrew Visual data prep & profiling Profile, clean, recipes without code Real-time streaming
AWS Glue Serverless ETL Scheduled/event batch Spark jobs Low-latency online serving
Amazon EMR Managed Spark cluster Large-scale/custom framework transforms Simple per-record transforms
AWS Lambda Event-driven compute Per-record or fixed-window streaming transforms Sliding-window stateful processing
Managed Service for Apache Flink Stateful stream processing Sliding windows, ordering, state across intervals Batch-only pipelines

Consistency & Anti-Patterns

  • Fit on training, apply on inference: Persist scaler/encoder parameters; never refit on inference data.
  • Feature definitions live in Feature Store: Training and inference read the same definitions; avoid reimplementing transforms in two places.
  • Use event time correctly: Offline store uses event time for point-in-time joins; out-of-order records may not update online store if event time is older.
  • Do not use online store for training: It lacks history and causes leakage.
  • Do not store raw data in Feature Store: Store engineered features; raw data belongs in S3.
  • Do not use Feature Store for embeddings: It supports String/Integral/Fractional, not vector similarity; use OpenSearch, Aurora pgvector, etc.
  • Plan schema upfront: You can add features later, but cannot remove or modify existing feature definitions; major changes require a new feature group.
  • Set TTL on online store: Prevents unbounded growth and cost.

Exam Gotcha: Feature Store does not compute transformations. Transformations must run before ingestion (Data Wrangler, Glue, EMR, Lambda).

Exam Gotcha: Offline store requires event time; online store returns only the latest value per record ID.

Exam Gotcha: Refitting a scaler at inference causes training-serving skew — the same raw value maps to a different feature value than the model was trained on.

Exam Gotcha: For RAG, chunk and embed documents into a vector store; Feature Store is for tabular features, not semantic retrieval.

Security & Compliance

  • IAM: Separate roles for ingestion and retrieval; least privilege.
  • KMS: Encrypt online store and offline S3 at rest.
  • TLS: Encrypt in transit.
  • VPC endpoints: Use PrivateLink for SageMaker Feature Store to avoid public internet.
  • Discovery before storage: Use Macie for S3 sensitive data, Comprehend for PII in text; mask/redact/anonymize before ingestion.
  • Audit: CloudTrail for API activity; Glue Data Catalog for offline metadata.

Key Trade-offs — When to Use X vs Y

Decision Choose Because
Feature Store vs S3 Feature Store Online serving + point-in-time joins + metadata
Feature Store vs DynamoDB Feature Store Integrated offline store and time-travel
Data Wrangler vs Glue Data Wrangler Interactive, low-code, visual
Data Wrangler vs Glue DataBrew Data Wrangler Feature engineering for ML; DataBrew for profiling/cleaning
Lambda vs Managed Flink Lambda Per-record or fixed non-overlapping windows
Lambda vs Managed Flink Managed Flink Sliding windows, ordering, state across intervals
Offline vs Online Offline Training, batch, history, point-in-time
Online vs Offline Online Real-time inference, low latency