Skip to content

1.2

Task 1.2: Transform data and perform feature engineering

Knowledge of:

  • Data cleaning and transformation techniques (for example, detecting and treating outliers, imputing missing data, combining, deduplication)
  • Feature engineering techniques (for example, data scaling and standardization, feature splitting, binning, log transformation, normalization)
  • Encoding techniques (for example, one-hot encoding, binary encoding, label encoding, tokenization)
  • Tools to explore, visualize, or transform data and features (for example, SageMaker Data Wrangler, AWS Glue, AWS Glue DataBrew)
  • Services that transform streaming data (for example, AWS Lambda, Spark)
  • Data annotation and labeling services that create high-quality labeled datasets

Skills in:

  • Transforming data by using AWS tools (for example, AWS Glue, DataBrew, Spark running on Amazon EMR, SageMaker Data Wrangler)
  • Creating and managing features by using AWS tools (for example, SageMaker Feature Store)
  • Validating and labeling data by using AWS services (for example, SageMaker Ground Truth, Amazon Mechanical Turk)

Let's get started with the second task statement from domain one, which is to transform data and perform feature engineering. For this task statement, let's talk about preparing your data for training and steps to take before training. For the exam, ensure you understand the different types of data, such as useless, binary, continuous, categorical, text, temporal and more, and also ensure you understand feature encoding. Here's a question. You are preparing a dataset to train a predictive model. The dataset contains numerical, categorical, and ordinal features. To increase prediction accuracy, you need to transform categorical features into numeric values. What is your solution? One solution is to use AWS Glue DataBrew and perform one-hot encoding. In AWS Glue DataBrew, you can use recipe actions to tabulate and summarize data from different perspectives or to perform advanced transformations. Another solution might be the Data Wrangler experience in Amazon SageMaker Canvas, I added links under the additional resources for more explanations of data types and encoding. For the exam, ensure you understand feature engineering and pre-processing algorithms, such as feature reduction algorithm and principle component analysis, and understand why you transform data and perform feature engineering to your data. You can use SageMaker Canvas to import, prepare, transform, visualize, and analyze data in your machine learning workflows to pre-process and feature engineer your data using little to no code. You can also add your own Python scripts and transformations to customize your data prep workflow. You can also import data from Amazon S3, Amazon Redshift, Athena, and use SageMaker Canvas to create machine learning data prep workflows with built-in and custom data transformations and analysis. After you've defined a data workflow, you can integrate it with SageMaker Processing, SageMaker Pipelines and SageMaker Feature Store to perform the task of processing, sharing, and storing the machine learning training data. You can also export your data flow to a Python script and create a custom machine learning data prep pipeline. Let's ask a few questions. You are preparing a dataset that contains numerical, categorical, and ordinal features. To train a predictive model, to increase the prediction accuracy of the model, you need to transform the categorical features into numerical values. Which feature engineering solution do you choose? One-hot encoding, feature scaling, dimensionality reduction, or date formatting. My choice would be one-hot encoding and with AWS Glue DataBrew, you can use recipe actions to tabulate and summarize data from different perspectives or to perform advanced transformations. Here's another question. You want to convert a column in your training data into binary values first, before training the model, which method do you choose, one-hot encoding, tokenization, or target encoding? Categorical data that is converted into numerical forms to be used in machine learning algorithms is encoding, and one-hot encoding is a process by which categorical variables are converted into numerical values that could be provided to the machine learning algorithms for better predictions. Target encoding is incorrect, because this type of encoding is achieved by replacing categorical variables with just one new numerical variable and replacing each category of the categorical variable with its corresponding probability of the target. This won't convert categorical variables into binary values, and tokenization is also incorrect because this method is commonly used in natural language processing where you split a string into a list of words that have a semantic meaning. Here's another question. Let's say that during data preparation, you discovered missing values on some columns of a dataset containing categorical features. You need to ensure that this will not misrepresent the data and reduce the model reliability. What is your solution? One solution would be to use Amazon Forecast because Forecast provides several filling methods such as middle filling, backfilling, and future filling to help handle missing values in your dataset. You can also use the multiple imputations by chain equations, or MICE, algorithm, which can help to deal with missing data in your dataset. MICE is usually a better imputation method than filling missing values with zero or dropping columns. Let's also ask a question around feature processing. You are preparing a dataset to be used for training a linear learner model in SageMaker. During your exploratory data analysis, you detected multiple feature columns that have missing values. The percentage of the missing data across the whole training dataset is about 15%. You want to reduce the bias to your model that might lead to inaccurate results. What is your solution to reduce the bias caused by missing values? One solution is to use supervised learning methods to estimate the missing values for each feature. If you drop the columns that include missing values, because they only account for 10% of the training data, you might remove features that might be valuable to your machine learning task. There are valid imputation techniques such as. compute the mean of non-missing values in the same row and use the result to replace missing values, and also compute the mean of non-missing values in the same column and use the result to replace missing values. But I have not seen these give better estimates than a supervised learning model, but if you have, please let me know. Let's ask one more question about SageMaker and normalization. You are training a k-nearest neighbor model and during data analysis you observe different input variables that vary significantly. You want to ensure that your dataset does not have features with a larger value that greatly influence the model's predictive capability. Which transformation do you choose? I sort of gave it away, but I would choose a normalization transformation. Let's get started with the third task statement from Domain 1, which is to ensure data integrity and prepare data for modeling.