Skill 1.2.4: Perform Feature Engineering (Scaling, Standardization, Feature Splitting, Binning, Log Transformation, Normalization)
Task 1.2: Data Preparation for ML and AI – Perform data transformation, feature engineering, and pre-processing
Core objective: Convert raw variables into model-ready numeric features that preserve signal, make scales comparable, reduce skew, and are applied identically at training and inference.
Transform Decision Matrix
| Transform | What it does | Use when | Exam gotchas |
|---|---|---|---|
| Scaling / Min-Max Normalization | Rescales to fixed range (e.g., 0–1) using min/max. | Distance/gradient algorithms: k-NN, SVM, k-means, PCA, neural nets. Bounded inputs needed. | Sensitive to outliers; fit min/max on train only. |
| Standardization (Z-score) | Centers to mean 0, std 1. | Algorithms assuming Gaussian or distance-based; linear models with regularization. | Still influenced by outliers; persist mean/std. |
| Vector Normalization (L1/L2) | Scales each sample vector to unit norm. | Text embeddings, cosine similarity, k-NN on high-dimensional vectors. | Different from min-max; read context. |
| Log Transformation | Compresses right-skewed positive distributions. | Income, counts, prices, multiplicative relationships. | Requires positive values; handle zeros/negatives; changes interpretability. |
| Binning / Discretization | Continuous → categorical bins (equal width/frequency/quantile/custom). | Non-linear relationships, noisy data, scorecards, some tree/linear hybrids. | Loses information; bin edges must be persisted; target-based bins risk leakage. |
| Feature Splitting | Splits compound field into parts (date → year/month/day; address → city/state; email → domain). | Subcomponents carry signal; parsing is reliable. | High cardinality; PII risk; inconsistent parsing. |
| Interaction / Polynomial Features | Combines features (products, ratios). | Linear models need non-linearity; domain-driven interactions. | Explodes dimensionality; overfitting. |
| Categorical Encoding | One-hot, ordinal, target/mean encoding. | Categorical inputs for non-tree models; target encoding for high cardinality. | Target encoding must be computed within CV folds; one-hot high cardinality explodes. |
Algorithm-Driven Choices
| Algorithm family | Scaling needed? | Preferred transforms | Avoid |
|---|---|---|---|
| Tree-based (XGBoost, RF) | No | Binning, target encoding, log for skewed targets | Wasting time on scaling; target leakage |
| Linear/Logistic | Yes | Standardization, log, interactions, one-hot | Unscaled features with L1/L2 |
| Distance-based (k-NN, k-means) | Yes | Standardization or min-max; L2 for vectors | Unscaled features dominate distance |
| Neural nets | Yes | Standardization, min-max, embeddings | Raw high-cardinality categoricals |
| PCA / SVM | Yes | Standardization | Unscaled variance |
AWS Service Mapping
| Need | Service | Notes |
|---|---|---|
| Interactive feature engineering | SageMaker Data Wrangler | Visual transforms: scaler, encoder, binning, log, split, join, group by. Export to pipeline/Feature Store. |
| Feature consistency | SageMaker Feature Store | Offline store for training, online store for inference. Prevents training-serving skew. |
| Batch ETL at scale | AWS Glue / EMR Spark | Scheduled/on-demand jobs; Spark ML transforms. |
| Visual low-code prep | AWS Glue DataBrew | Recipes, profiling, cleaning, scaling/binning without code. |
| Streaming transforms | Lambda / Managed Service for Apache Flink | Per-record transforms in Lambda; sliding windows/stateful aggregates need Flink. |
| Custom processing jobs | SageMaker Processing | Run preprocessing scripts at scale. |
| Text/image features | Amazon Bedrock embeddings | Convert unstructured data to vectors; not tabular feature engineering. |
Critical Exam Gotchas
Fit on training only. Apply the same fitted parameters to validation, test, and inference. Refitting a scaler on inference data causes training-serving skew. Scaling is algorithm-dependent. Tree models do not require scaling. Distance/gradient-based models do. Binning edges are part of the model contract. Persist and reuse them; changing bins changes feature meaning. Log transformation requires positive values. Handle zeros/negatives explicitly; otherwise use signed log or shift. Normalization is overloaded. It can mean min-max scaling or L1/L2 vector normalization. Read the context. Target encoding must be fold-safe. Compute within CV folds to avoid target leakage. Streaming global statistics need state. Per-record transforms are safe in Lambda; rolling means/std need stateful stream processing.
Feature Store & Training-Serving Consistency
- Purpose: Write feature definitions once; reuse in training and inference.
- Offline store: Historical features for batch training.
- Online store: Low-latency features for real-time inference.
- Event time: Enables point-in-time correct joins; avoids future leakage.
- Ingestion: Data Wrangler, Glue, Spark, SageMaker Processing.
- Anti-pattern: Reimplementing transforms in training and inference separately.
Anti-Patterns
- Scaling before train/test split → data leakage.
- Refitting transforms on test/inference → training-serving skew.
- Log on negative/zero values → invalid or misleading features.
- Target encoding before split → leakage.
- One-hot on high-cardinality → dimensionality explosion.
- Binning with target-derived edges → leakage.
- Scaling for tree models → unnecessary.
- Ignoring skew for linear models → poor fit.
Quick Decision Flow
- Distance/gradient algorithm? → Standardize or min-max scale.
- Right-skewed positive distribution? → Log transform.
- Non-linear relationship? → Binning, interactions, polynomial.
- Compound field? → Feature splitting.
- High-cardinality categorical? → Target encoding or embeddings.
- Need identical training/inference? → SageMaker Feature Store.
- Streaming? → Lambda for per-record; Flink for windows/state.