Domain 1: Data Preparation for ML and AI
Task 1.1: Collect and Store Data
| Trigger Keywords & Scenario Constraints | Lock-In Architecture | Immediate Rejection / Trap to Avoid | Explanation |
|---|---|---|---|
| "Distributed GPU / HPC training" + "POSIX / high-throughput parallel I/O" + "hydrate from S3" | Amazon FSx for Lustre (S3 repository / DRA) | S3 direct (API latency/throttling) or EFS (lower parallel I/O). | Lustre is the HPC POSIX scratch FS; import/export with S3 for training corpora. |
| "Shared POSIX file system" + "multi-AZ / SageMaker notebook shared home" + "moderate throughput" | Amazon EFS | FSx for Lustre (overkill/cost) or EBS (single-AZ, single instance). | EFS is regional NFS; many notebooks/jobs mount the same dataset. |
| "Single training instance" + "lowest-latency block disk" + "gp3 / io2 IOPS" | Amazon EBS | EFS (network FS latency) or S3 (object, not block). | Attach gp3/io2 to the training EC2/SageMaker instance only. |
| "Source of truth for training / lake" + "any size, 11 9s, cheapest object store" | Amazon S3 Standard (or Intelligent-Tiering if access unknown) | EFS/FSx as lake; Glacier for active training reads. | Default ML lake. Use prefixes + partitioning; never train from Glacier. |
| "Unknown / changing access patterns" + "auto cost optimize storage" | S3 Intelligent-Tiering | Manual S3-IA/Glacier lifecycle that breaks training GETs. | Auto-moves objects; no retrieval penalty vs mis-tiered Glacier. |
| "Compliance archive / WORM / rarely read" + "hours OK" | S3 Glacier Flexible Retrieval or Deep Archive (+ Object Lock if WORM) | S3 Standard (cost) or Glacier for online Feature Store/training. | Deep Archive = cheapest long-term; minutes–hours restore. |
| "On-prem NFS/SMB → S3 lake" + "incremental / scheduled" | AWS DataSync | Storage Gateway (hybrid cache, not bulk migrate) or manual aws s3 sync. | Purpose-built transfer; preserves metadata, throttles, schedules. |
| "Hybrid on-prem apps keep POSIX/SMB" + "backend is S3" | AWS Storage Gateway (File Gateway) | DataSync (one-way migration, not a mounted gateway). | Gateway presents NFS/SMB; data lands in S3. |
| "Ad-hoc SQL on S3 Parquet/CSV" + "serverless / no cluster" | Amazon Athena (+ Glue Data Catalog) | Redshift (provisioned warehouse) or EMR for simple SELECT. | Pay-per-TB scanned; partition + columnar to cut cost. |
| "Petabyte warehouse / complex SQL / BI + ML features" | Amazon Redshift (Spectrum for S3) | Athena for heavy concurrent warehouse SLAs. | Spectrum/lake queries; unload to S3 for SageMaker. |
| "Governed data lake" + "row/column LF-TBAC / fine-grained access" | AWS Lake Formation + Glue Catalog + S3 | Bucket-policy-only S3 (no cell-level) or Lake Formation as an ETL engine. | Catalog + grants; Glue/Athena/Redshift respect LF permissions. |
| "Relational OLTP source" + "SQL extract / JDBC" | Amazon RDS → Glue/Athena Federated / Spark JDBC | DynamoDB (NoSQL) or scraping RDS snapshots as the primary ETL path. | Classic structured source; don’t train in-place on OLTP. |
| "Key-value / single-digit ms" + "high-scale lookups / session features" | Amazon DynamoDB (+ Streams for CDC) | RDS (not the ms/key-value pattern) or S3 GetObject for online features. | Online feature-like lookups; Streams → Lambda/Kinesis for ingest. |
| "Document / MongoDB API" | Amazon DocumentDB | DynamoDB (different API/model) or RDS PostgreSQL JSONB unless SQL is required. | Mongo-compatible operational store as an ML source. |
| "Full-text search + logs + k-NN vectors" | Amazon OpenSearch Service | Athena (analytics SQL) or pgvector when hybrid BM25+kNN at high QPS is required. | Ingest, search, and vector recall in one engine. |
| "Healthcare FHIR / longitudinal records" | AWS HealthLake | Generic S3 + Glue without FHIR semantics. | FHIR-native ingest/store/query for clinical ML. |
| "Real-time stream ingest" + "multiple consumers / replay / <1s" | Amazon Kinesis Data Streams | Data Firehose (≥60s buffer, no fan-out replay) or SQS (queue, not stream). | Shards; 24h–365d retention; Lambda/Flink/EMR consumers. |
| "Near-real-time dump to S3/OpenSearch/Redshift" + "no servers / Parquet conversion" | Amazon Data Firehose | Kinesis Data Streams (you manage consumers) when the need is managed delivery. | Buffer 60–900s or 1–128 MB; optional Lambda transform + Glue schema → Parquet/ORC. |
| "Sub-second / event-time windows / CEP / stateful stream features" | Amazon Managed Service for Apache Flink on KDS/MSK | Firehose (no windows) or Glue Streaming (micro-batch). | Exactly-once stateful Flink; sliding windows, joins, sessionization. |
| "Existing Kafka / Kafka API / long retention / exactly-once log" | Amazon MSK (or MSK Serverless) | Kinesis Data Streams (different API/retention model). | Drop-in Kafka; MSK Connect for sinks; Flink/Spark consumers. |
| "Spark micro-batch streaming ETL" + "Glue Catalog / serverless Spark" | AWS Glue Streaming ETL | Flink when sub-second event-time state is required. | Continuous Glue Spark jobs; seconds-level micro-batches. |
| "Per-record transform" + "<15 min" + "simple filter/enrich" | AWS Lambda (KDS/MSK/DDB Streams event source) | Glue/EMR for heavy Spark/stateful windows; Lambda 6 MB sync / 15 min limits. | Fan-out simple transforms; not a replacement for Flink. |
| "Video / audio live streams" + "Rekognition / SageMaker frames" | Amazon Kinesis Video Streams | Kinesis Data Streams (not a media container) or S3-only batch video. | Time-indexed media ingest for CV/speech pipelines. |
| "Columnar analytics / SageMaker + Athena" + "compression / predicate pushdown" | Apache Parquet (or ORC) on S3 | CSV/JSON as the training-at-scale format (row scan tax). | Column projection + Snappy/ZSTD; default Feature Store offline format. |
| "Row / nested / semi-structured events" + "schema evolution in stream" | JSON / JSONL or Avro | Parquet for low-latency per-record streaming APIs. | JSONL = FM fine-tune pairs; Avro for Kafka schema registry. |
| "Human-readable export / small prototype" | CSV | CSV for TB-scale Athena/Spark training (tiny files + full scans). | Convert via Glue/Firehose/Athena CTAS → Parquet before scale. |
| "SageMaker built-in algorithms" | RecordIO-protobuf (or algorithm-native) | Raw CSV without respecting the algorithm’s expected channel format. | Built-ins expect RecordIO/protobuf on the train channel. |
| "TensorFlow native training input" | TFRecord | Parquet-only if the TF pipeline isn’t using TFX/IO adapters. | TF-standard; pair with SageMaker TF containers. |
| "Merge heterogeneous sources / catalog / Spark ETL serverless" | AWS Glue (crawlers + Spark jobs + Data Catalog) | EMR when you don’t need cluster control; Lambda for TB joins. | Serverless Spark ETL; crawl RDS/S3/DDB; write Parquet partitions. |
| "Massive Spark/Hadoop / custom AMIs / long-lived cluster / Hive/Presto" | Amazon EMR (Spark) | Glue (DPU/serverless limits / less cluster knobs) for multi-hour petabyte jobs. | Max control + frameworks; cost via Spot/instance fleets. |
| "CDC from OLTP into the lake" | RDS/DDB Streams → Kinesis/Lambda/Glue or Spark incremental | Nightly full dumps when “incremental/CDC” is stated. | Stream changes; merge into Iceberg/Hudi/Parquet partitions. |
| "Vector DB + hybrid keyword+semantic" + "sub-10 ms / high QPS RAG" | Amazon OpenSearch Service (k-NN + BM25 hybrid) | S3 Vectors (higher latency) or pgvector when hybrid search at scale is the requirement. | Faiss/Lucene k-NN; filters inside k-NN; OpenSearch Serverless option. |
| "Vector search + SQL joins / ACID / existing Aurora/RDS Postgres" | Amazon RDS/Aurora PostgreSQL + pgvector (HNSW or IVFFlat) | OpenSearch when the data already lives in Postgres and SQL+ANN in one query is enough. | HNSW = faster queries/more RAM; IVFFlat = less memory, rebuild. |
| "Billions of vectors / 90% cheaper / infrequent RAG / ~100 ms OK" | Amazon S3 Vectors | OpenSearch (cost) for cold/archival embeddings; S3 Vectors for chatbot p99 <20 ms. | Native vector object store; Bedrock KB backend; tier with OpenSearch for hot. |
| "GraphRAG / multi-hop entity relations" | Amazon Neptune | OpenSearch k-NN-only (no graph traversal). | Knowledge graph + embeddings for multi-hop retrieval. |
| "Ultra-low-latency feature cache / Redis collections" | Feature Store Online InMemory (ElastiCache Redis OSS) or ElastiCache | Standard Feature Store Online when sub-ms Redis-class + list/set/vector types are required. | InMemory: online-only, ~50 GiB default cap, no CMK, no online+offline replica. |
| "Images / PDFs / audio / text mixed modalities into the lake" | S3 prefixes by modality + Textract / Transcribe / Rekognition extractors | Forcing everything into RDS/CSV. | Object lake holds blobs; AI services emit text/labels/JSON sidecars. |
| "Online real-time features ms GetRecord" + "latest value only" | SageMaker Feature Store Online | Offline store (S3, seconds–minutes) or raw DynamoDB without FS metadata/lineage. | GetRecord / PutRecord; millisecond reads for inference enrichment. |
| "Historical features / training / batch inference / point-in-time join" | SageMaker Feature Store Offline (S3 Parquet, Athena) | Online store (no history) or ad-hoc S3 without event_time → feature leakage. | Append-only; Glue (hourly partitions) or Iceberg (time travel/schema). |
| "Avoid train/serve skew" + "same features online + offline" | Feature Store Online + Offline (dual-store group) | Separate DynamoDB + S3 pipelines that drift. | Single PutRecord fans out; point-in-time queries for labels. |
| "Streaming features into Feature Store" | KDS/MSK → Flink/Lambda → Feature Store PutRecord |
Firehose → S3 → delayed batch (stale online features). | Sync PutRecord keeps online features fresh. |
| "Offline store query / Iceberg ACID / schema evolution" | Feature Store Offline + Apache Iceberg table format | Default Glue format when you need mutable partitions/time-travel. | Glue format = immutable hourly partitions; Iceberg = lakehouse semantics. |
| "Tiny files / S3 503 Slow Down / Spark task explosion" | Firehose 64–128 MB buffers or Glue compaction | Writing 10 KB objects per event to S3. | Target 128–512 MB Parquet files for training/Athena. |
| "JSON stream → columnar lake automatically" | Firehose record format conversion (Glue Catalog schema → Parquet/ORC) | Firehose conversion to OpenSearch/Redshift (S3-only) or GZIP+conversion together. | Conversion requires S3 dest, uncompressed input, Snappy by Firehose, ≥64 MB buffer. |
| "Ingest + catalog schema automatically" | Glue Crawler → Data Catalog | Hand-written Hive DDL as the first choice when crawlers fit. | Crawlers classify CSV/JSON/Parquet and update partitions. |
| "Troubleshoot ingestion throttling / shard iterator age" | KDS scale shards / on-demand mode; check WriteProvisionedThroughputExceeded |
“Just use Firehose” when multiple replayable consumers are required. | 1 MB/s in & 1000 PUTs per shard; split shards or on-demand. |
| "Feature Store offline lag" | Expect ~15 min buffer/batch to S3 after PutRecord |
Assuming online and offline are synchronously identical at t=0. | Offline is batched; don’t use it for sub-minute labels. |
| "Cost vs performance for training data GET" | Keep hot corpus on S3 Standard / Instant Retrieval / FSx; lifecycle old versions only | Lifecycle whole prefix to Glacier then launch a training job. | Restore time + retrieval fees fail jobs; Intelligent-Tiering is the safe auto-tier. |
Task 1.2: Transformation, Feature Engineering & Pre-processing
| Trigger Keywords & Scenario Constraints | Lock-In Architecture | Immediate Rejection / Trap to Avoid | Explanation |
|---|---|---|---|
| "Visual ML data prep" + "300+ transforms" + "export to Feature Store / Pipelines / Clarify" | SageMaker Data Wrangler (Canvas/Studio) | Glue DataBrew (analytics, not ML-native exports) or raw Spark for no-code data scientists. | Built-in analysis, bias report, SMOTE; export job → Processing / Pipelines / FS. |
| "Visual prep for analysts" + "250 recipes" + "profile quality, no SageMaker" | AWS Glue DataBrew | Data Wrangler when the user is an analyst outside Studio and the dest is the lake/warehouse. | No-code recipes; profile jobs; S3/Redshift/RDS; not Feature Store-first. |
| "Production serverless Spark ETL" + "catalog / bookmarks / job bookmarks" | AWS Glue ETL (Spark) | DataBrew for complex joins/custom Spark; EMR if you need cluster-level control. | PySpark/Scala jobs, crawlers, bookmarks, Lake Formation. |
| "Interactive Spark in Studio / no cluster mgmt" | Glue Interactive Sessions or EMR Serverless from Studio | Standing up EMR clusters for exploratory notebooks. | Notebook-native Spark for prep; promote to jobs later. |
| "Scale-out sklearn / custom Python / Spark processing as a SageMaker step" | SageMaker Processing jobs | Lambda (15 min / memory) or notebook “Run All” as production. | Containerized prep; wires into Pipelines; sklearn/Spark/BYOC. |
| "Petabyte / Hadoop ecosystem / custom Spark tuning" | Amazon EMR Spark | Glue when the scenario stresses YARN/Hive/long-lived multi-framework clusters. | Maximum Spark surface; mix with SageMaker via S3. |
| "Streaming transform simple / enrich / filter" | Lambda on KDS/MSK | Glue Streaming/Flink for stateful windows. | Stateless map/filter; watch 15 min and payload caps. |
| "Streaming transform windowed aggregates / joins" | Managed Flink or Spark Structured Streaming on EMR/Glue | Lambda (no event-time state) or Firehose Lambda (per-record only). | Flink if sub-second CEP; Spark if existing Spark SQL. |
| "Standardization (z-score) vs min-max vs robust" | StandardScaler (mean/std) / MinMaxScaler / RobustScaler (IQR) | Normalizing tree models as if required; scaling after split leakage. | Fit scaler on train only; trees don’t need scaling; RobustScaler for outliers. |
| "Skewed numeric target/feature (prices, counts)" | Log / Box-Cox / Yeo-Johnson transform | StandardScaler alone on heavy-tailed revenue. | Compress skew before linear/NN models. |
| "Continuous → categorical buckets" | Binning / quantile discretization | One-hot of raw high-cardinality IDs. | Improves linear models; watch bin leakage (fit on train). |
| "High-cardinality categoricals for deep models" | Entity embeddings | Massive one-hot (curse of dimensionality). | Embedding tables; Feature Store can serve them. |
| "Low-cardinality categoricals for linear/trees" | One-hot / ordinal encoding | Embeddings for 3-level sex/country on XGBoost (unnecessary). |
One-hot for linear; ordinal/label OK for trees. |
| "Split compound strings (date, name, path)" | Feature splitting (Data Wrangler / Spark) | Leaving ISO timestamps as raw strings for tree/linear models. | Extract hour/dow/month; regex split. |
| "Reuse features across teams / lineage / discovery" | SageMaker Feature Store (catalog + groups) | Each project recomputes features into private S3. | Glue catalog search; groups; online/offline. |
| "Point-in-time correct labels" + "no future leakage" | Feature Store Offline time-travel / PIT join | Random SQL join on “latest” features as of today. | As-of event_time; critical for fraud/medical/churn. |
| "Text/image → dense vectors for k-NN/RAG" | Amazon Bedrock embeddings (Titan Text v2, Titan Multimodal, Cohere Embed) or SM JumpStart | Training a word2vec model from scratch when a managed embedder fits. | Same model for index and query; Titan v2: 8,192 tokens, dims 1024/512/256. |
| "Batch embed large corpus cheaply" | Bedrock embeddings batch / throughput-optimized | On-demand chat endpoints to embed millions of chunks. | Batch/index path vs latency-optimized retrieval path. |
| "Image + text joint embedding" | Titan Multimodal Embeddings | Text-only Titan Embed for CLIP-style image search. | One vector space for image and query text. |
| "Match tokenizer to the FM being fine-tuned" | Model-native tokenizer (BPE / SentencePiece / WordPiece) | Generic NLTK word_tokenize for LLM fine-tune packing. |
Vocab mismatch wastes tokens and hurts loss. |
| "Classical NLP bag-of-words pipeline" | Stopword / stem / lemma + TF-IDF (sklearn/Spark NLP) | Applying stemming to LLM tokenization pipelines. | Useful for linear/classical; skip for modern FMs. |
| "Domain-specific text augmentation" | Synonym / back-translation / nlpaug; domain dictionaries | Generic SMOTE on raw token IDs. | SMOTE is tabular; text needs NLP augmenters. |
| "PDFs / scans / forms → text for RAG" | Amazon Textract (+ Layout/Queries/Tables) | pdfminer as the AWS-exam answer; Rekognition (CV labels, not OCR forms). |
OCR + layout; then chunk. |
| "Audio → text corpus" | Amazon Transcribe (Medical if PHI) | Rekognition (video/images) or Comprehend (text in, not audio). | Transcripts land in S3/JSON; then NLP/RAG. |
| "Entities / PII / sentiment / topics as features" | Amazon Comprehend (Medical for PHI) | Macie (S3 discovery, not per-document NLP features). | DetectEntities/PII/Sentiment/Topic; write columns to Parquet/FS. |
| "RAG chunking: preserve meaning, cite pages" | Semantic / recursive / heading-aware chunking + overlap + metadata | Naive 2k-char slices with zero overlap and no source metadata. | Overlap ~10–20%; store {source, page, title, created_at} for filters/citations. |
| "Long docs vs embedding context window" | Chunk ≤ embedder max tokens (Titan v2 8,192; still prefer paragraphs) | Embedding entire 100-page PDFs as one vector. | Retrieval likes paragraph/section chunks; parent-doc expansion optional. |
| "Extract metadata (title, date, author, section)" | Textract + Comprehend + custom Spark/Wrangler | Embedding-only RAG with no filterable metadata. | Metadata filters in OpenSearch/pgvector/S3 Vectors cut noise. |
| "Mask/redact PII in unstructured text" | Amazon Comprehend PII (offsets + confidence) via Wrangler/Glue/Lambda | Amazon Macie as an inline redactor; regex-only for free text. | Comprehend returns start/end; replace with [NAME] etc.; auditable. |
| "PHI in clinical notes" | Amazon Comprehend Medical | Standard Comprehend PII (not medical entity types). | Rx, conditions, PII/PHI categories. |
| "Discover which S3 buckets hold PII at rest" | Amazon Macie | Comprehend (not a bucket crawler) or Macie to redact inside documents. | Classification/audit of S3; then remediate with Comprehend/Glue. |
| "Tabular sensitive-field detection in ETL" | Glue Sensitive Data Detection / DataBrew PII transforms | Macie on non-S3 JDBC sources. | Column-level detect + mask in Glue jobs. |
| "Irreversible anonymization" | Hash/HMAC (KMS-backed salt) or drop columns | Encryption (reversible) as “anonymization”. | Encryption ≠ anonymization; tokenization may be reversible. |
| "FM supervised fine-tune data format" | JSONL prompt/completion or messages/chat pairs, high-quality, de-duplicated | Dumping raw PDFs/CSV into the fine-tune job. | One example per line; validate schema; hold out eval; no test contamination. |
| "Domain continued pre-training" | Large unlabeled domain corpus (cleaned, de-duplicated text/tokens) | Tiny labeled Q-A JSONL (that’s SFT, not CPT). | CPT needs volume; SFT needs instruction pairs. |
| "Knowledge distillation data" | Teacher FM generations as labels for student | Distillation = quantization/pruning (those are compression, not distillation). | Student trains on teacher outputs; smaller/cheaper model. |
| "Prepare images for CV" | Resize/normalize; TFRecord/RecordIO; augment (flip/crop) on S3 | Storing millions of tiny files without packing/manifest. | Manifest + packed records avoid S3 small-object tax. |
| "No-code end-to-end prep + FM in one UI" | SageMaker Canvas + Data Wrangler | Glue Studio for business-analyst no-code modeling. | Canvas visual prep → train/FM; still export for production MLOps. |
| "Convert CSV lake → Parquet in SQL" | Athena CTAS or Glue ETL | Python pandas on a notebook for TB conversion. | CTAS writes compressed partitioned Parquet. |
| "Dedup / entity resolution across dirty records" | Glue FindMatches (ML transform) | Exact DISTINCT when the scenario says fuzzy names/addresses. |
Supervised matching; labeling set required. |
| "Orchestrate prep DAG" (only if asked inside data prep) | SageMaker Pipelines or Step Functions / MWAA | Cron on a notebook instance. | Processing → FS ingest → train; MWAA if Airflow-standard. |
Task 1.3: Validate Data Quality and Manage Bias
| Trigger Keywords & Scenario Constraints | Lock-In Architecture | Immediate Rejection / Trap to Avoid | Explanation |
|---|---|---|---|
| "Declarative DQ rules / DQDL / recommend + evaluate" + "Glue jobs" | AWS Glue Data Quality | DataBrew Profile when you need CI-gated rulesets in Glue ETL. | Recommend rules, DQDL, anomaly detection, fail the job on threshold. |
| "Visual profile / correlations / missingness / outliers for analysts" | Glue DataBrew profile jobs | Glue Data Quality DQDL when the user is click-ops profiling only. | Stats, value distribution, duplicate detection UI. |
| "ML-centric bias + data quality visuals inside Studio" | Data Wrangler analysis + Clarify report | DataBrew as the bias tool (it is not Clarify). | Data Wrangler: histograms, target leakage, bias report. |
| "Human labeling / annotation workflows" + "your workforce / MTurk / vendor" | SageMaker Ground Truth | Ground Truth Plus when you manage the workforce; Rekognition (auto labels, not HITL UI). | Bounding box, NER, classification, verification; custom UI. |
| "AWS-managed labeling workforce / you don’t want to run ops" | SageMaker Ground Truth Plus | Ground Truth self-service when the scenario outsources workforce mgmt. | AWS runs hiring, QA, workflow. |
| "Cut labeling cost / auto-label easy examples" | Ground Truth active learning | Label 100% with humans when “active learning / auto-annotation” is stated. | Model pre-labels high-confidence; humans get the uncertain tail. |
| "Pre-training data bias (imbalance, DPL, etc.)" | SageMaker Clarify pre-training metrics | Post-training bias metrics (need model predictions) or CloudWatch. | CI, DPL, KL/JS/LP/TVD/KS, CDD — model-agnostic on the dataset. |
| "Class imbalance (CI) on tabular" | Data Wrangler: SMOTE / random oversample / undersample + stratified split | SMOTE on text/images; shuffling alone. | SMOTE = numeric/tabular synthetic minority; fit only on train. |
| "Imbalanced images" | Augmentation (flip, crop, color jitter, MixUp) or class-aware sampling | SMOTE in pixel space. | Geometric/color augments expand the minority class. |
| "Imbalanced text" | Back-translation / synonym / random insert-delete; class weights later | Numeric SMOTE on embeddings without care, or dropping the minority. | NLP augmenters; keep a real holdout. |
| "Prevent order / batch bias" | Shuffle (except time-series) | Shuffling time-series (leaks future). | Randomize i.i.d. rows; time-based split for temporal data. |
| "Preserve class ratios in train/val/test" | Stratified split | Naive random split on a 1% fraud label. |
Stratify on the label (and key facets if needed). |
| "Time-series / leakage-safe split" | Temporal / walk-forward split | Random or stratified-by-label shuffle that mixes future into train. | Train past → validate future; PIT features. |
| "Multimodal bias across numeric + text + image" | Clarify/custom facet metrics per modality + rebalance each | Checking only the tabular facet and ignoring caption/image representation. | Measure representation per group across all asset types; augment the sparse modality. |
| "Target leakage in features" | Drop post-label columns; PIT Feature Store; Wrangler leakage analysis | Using “days_to_churn” as a feature. | Anything known only after the label timestamp is leakage. |
| "Prompt–response pair validation for SFT" | Schema checks + length/toxicity/refusal filters + human spot-check | Training on raw unvalidated chat logs. | JSON schema, empty answers, mismatched language, jailbreak content. |
| "Content safety screening of training/RAG corpus" | Comprehend toxicity / Bedrock Guardrails eval / custom classifiers | Macie (PII discovery, not toxicity) as the safety filter. | Strip hate/sexual/PII before CPT/SFT/RAG index. |
| "Outlier detection" | IQR / z-score / isolation forest; DataBrew/Wrangler | Deleting outliers after looking at the test set. | Decide rules on train; winsorize vs drop vs separate model. |
| "Missing values" | Impute median/mode/KNN (Wrangler/DataBrew/sklearn); missingness indicator | Dropping all rows when MAR/MNAR and sample size matters; mean-impute categoricals. | Fit imputer on train; trees can use native NaN; add is_missing flag. |
| "Exact duplicate rows" | Spark/Glue dropDuplicates / DataBrew |
FindMatches (overkill) for byte-identical rows. | Exact first; fuzzy next. |
| "Near-duplicate documents in FM corpus" | MinHash/LSH / embedding similarity dedup | Exact hash only (misses paraphrases). | Dedup before CPT to avoid memorization and wasted tokens. |
| "Eval contamination" | Remove train n-grams/near-dups that overlap the benchmark | Fine-tune on data that includes the test set. | Decontaminate SFT/CPT against eval. |
| "Label quality / inter-annotator agreement" | Ground Truth annotation consolidation (majority / confidence) | Single unreviewed annotator as “gold”. | Multiple workers + consolidation; audit queue. |
| "Fair dataset split across sensitive facet" | Stratify on facet + label; report Clarify CI/DPL per split | Global shuffle that leaves all minority rows in test. | Each split should reflect facet prevalence (or be explicitly rebalanced). |
| "Rebalance without fabricating minorities (strict compliance)" | Random undersample majority or class-weighted training (later) | SMOTE when synthetic persons are forbidden. | Undersample/weights vs synthetic rows. |
| "Data quality in pipelines as a gate" | Glue Data Quality Evaluate + CloudWatch / fail job | “We’ll profile in a notebook before prod.” | Treat DQ like unit tests; block Feature Store ingest. |
| "Track dataset versions for audits" | S3 versioning + Glue table versions / Feature Store + lineage | Overwriting s3://bucket/train.csv in place. |
Immutable prefixes (/date=…/v=…); Feature Store event_time. |
| "Small-sample overfit risk at prep time" | Don’t target-encode with full-data statistics; use CV encodings | Fitting TargetEncoder on train+test. | Any learned transform = fit on train folds only. |
| "Image labeling at scale with auto helpers" | Ground Truth + automatic 3D/2D tools / active learning | Pure Rekognition if human gold labels are required. | Rekognition can pre-label; humans still own gold sets. |
| "NLP label: NER / sentiment / classification" | Ground Truth text tasks or Comprehend custom after gold set | Using foundation-model labels as unvalidated gold. | LLM-as-labeler needs human audit (HITL). |
| "Validate RAG chunk quality" | Spot-check retrieval: chunk completeness, metadata, PII, overlap | Indexing raw HTML/nav boilerplate. | Strip boilerplate; enforce min/max chunk tokens. |
| "Multimodal class imbalance (e.g., rare defect photos)" | Targeted image augmentation + oversample minority hashes | Copying the same 10 JPEGs 1,000× without augmentation (overfit). | Augment then sample; keep originals for eval. |
| "Skewed numeric labels in regression" | Log-target + inverse transform; report metrics on original scale | MAE on raw dollars with extreme outliers unaddressed. | Transform for training stability; evaluate in business units. |
| "I.I.D. assumption broken (grouped customers)" | Group / patient / device split | Random row split that leaks the same customer into train and test. | Split on entity ID, not row. |
| "Automated outlier + missing + dup + schema in one lake job" | Glue ETL + Glue Data Quality + FindMatches | Three disconnected notebooks. | Production pattern: crawl → DQ → clean → Parquet → catalog. |
| "Bias report before any training job" | Clarify pre-training processing job (Studio / Pipelines) | Waiting for Model Monitor (that’s post-deploy drift). | Gate Pipelines on CI/DPL thresholds. |
| "Synthetic tabular minority, mixed types" | SMOTE-NC (or Wrangler SMOTE where numeric) | Plain SMOTE on one-hot categoricals blindly. | SMOTE-NC handles categorical+continuous. |
| "Human review of GenAI training pairs" | Ground Truth custom workflow / HITL UI | Fully automated acceptance of teacher-model dumps. | Rate helpfulness/harmlessness; drop junk before SFT. |
Cross-cutting Domain 1 Traps (always-wrong pairs)
| Trigger Keywords & Scenario Constraints | Lock-In Architecture | Immediate Rejection / Trap to Avoid | Explanation |
|---|---|---|---|
| "Real-time features <1s" | KDS + Flink/Lambda + Feature Store Online | Firehose (mandatory ≥60s) or Feature Store Offline. | Offline/Firehose cannot meet sub-second serving. |
| "Lowest $ for billions of embeddings, QPS low" | S3 Vectors | OpenSearch provisioned as cold archive. | Up to ~90% cheaper; ~100 ms; not chatbot hot-path. |
| "Hybrid BM25 + vector" | OpenSearch | pgvector or S3 Vectors as the hybrid search engine. | Keyword+k-NN fusion is OpenSearch’s exam tell. |
| "SQL + ANN in one DB you already run" | RDS/Aurora pgvector | Standing up OpenSearch when Postgres + HNSW meets latency. | Don’t add a second system without a hybrid/QPS reason. |
| "No-code ML prep in SageMaker" | Data Wrangler | DataBrew (Glue family, not Feature Store/Clarify-native). | Brew = analysts/lake; Wrangler = ML in Studio/Canvas. |
| "No-code lake/warehouse prep, no Studio" | DataBrew | Data Wrangler (needs SageMaker). | Match the persona and destination. |
| "Serverless Spark ETL" | Glue | EMR (clusters) unless custom Hadoop/YARN is required. | Glue = serverless; EMR = control. |
| "POSIX HPC training I/O" | FSx for Lustre | EFS or raw S3. | Exam triplet: GPU + POSIX + throughput → Lustre. |
| "Shared notebooks, not GPU saturation" | EFS | FSx for Lustre (cost/complexity). | Don’t over-provision Lustre for home dirs. |
| "PII redact text" | Comprehend PII | Macie | Macie finds S3 sensitive data; Comprehend redacts text. |
| "PII discover buckets" | Macie | Comprehend | Inverse of the row above. |
| "PHI clinical" | Comprehend Medical / HealthLake | Standard Comprehend / generic S3. | Medical entity types + FHIR. |
| "OCR documents" | Textract | Rekognition or Comprehend. | Rekognition = image/video labels; Comprehend needs text. |
| "Speech to text" | Transcribe | Polly (TTS opposite) or Kinesis Data Streams. | Polly is speech synthesis. |
| "Built-in SageMaker algo input" | RecordIO-protobuf | “Any CSV is fine”. | Channel format is algorithm-specific. |
| "Lake query cost" | Parquet + partition + Athena | CSV full-scan or SELECT *. | Columnar + partition pruning. |
| "Train/serve skew" | Feature Store dual online/offline | Two ad-hoc pipelines. | One write path, PIT reads. |
| "Leakage" | PIT join + temporal split + train-only fit | Shuffle + fit-on-all + latest-feature join. | Highest-yield Domain 1 gotcha. |
| "Active learning labels" | Ground Truth active learning | Ground Truth Plus (workforce outsourcing, not the algorithm). | Plus ≠ active learning. |
| "Pre-training bias" | Clarify | Model Monitor (production drift) or DataBrew. | Clarify = bias metrics; Monitor = deployed drift. |
| "SMOTE" | Tabular numeric (Wrangler) | Text/image tensors. | Wrong modality is a common distractor. |
| "Fine-tune FM" | JSONL instruction pairs | Unchunked PDFs / raw data lake. | Format + quality + decontamination. |
| "CPT / continued pre-train" | Large unlabeled domain text | Small labeled Q-A file. | Volume vs instruction signal. |
| "Same embedding model both sides" | One Bedrock/SM embedder for index and query | Titan for index, Cohere for query. | Cross-model vectors are not comparable. |
| "Firehose ≠ real-time" | Use KDS/Flink | Firehose for “sub-second dashboard features”. | Buffer floor is 60 seconds. |