Skip to content

Bonus Assignment: Prepare a support ticket dataset

This optional, ungraded assignment reviews Domain 1 by walking one small dataset through all three task statements.

It is hands-on: the steps below are meant to be run in your own AWS account or a sandbox account.

Disclaimer

This project is for educational purposes only and might lead to AWS service charges. You assume any risk associated with this project. You are responsible for any costs associated with your AWS account.

The scenario

AnyCompany Support keeps customer tickets in a relational database, the attached screenshots in object storage, and a separate set of resolution notes maintained by hand. The team wants two things from this data: a model that predicts whether a ticket will escalate, and an assistant that answers agent questions from the resolution notes. You are preparing the data for both.

Steps

Task 1.1: Collect and store data

  • Create an Amazon S3 bucket and upload a handful of sample documents to stand in for the resolution notes.
  • Configure a vector store and enable vector search on it, then note what the specification actually forced you to decide: dimensions, and how the store will be queried.
  • Write one sentence per source, for the tickets, the screenshots, and the notes, justifying where each belongs against cost, performance, data structure, and compliance.
  • Say which of the three sources needs a vector store and which needs a feature store, and name the question each one answers.

Task 1.2: Perform data transformation, feature engineering, and pre-processing

  • Generate embeddings for your sample documents with an embedding model available to your account. Note the number of dimensions the output carries, and what a larger or smaller dimension count would trade away.
  • Split one resolution note two ways, once by a fixed length and once on meaning. Compare which chunk you would rather have retrieved for one specific agent question.
  • Pick two ticket fields and name the feature engineering transform each needs, saying what the transform is for rather than only naming it.
  • Find any value in the sample data that would need masking or redaction before training, decide which of the two applies, and say why.
  • Decide which customization path the assistant needs, if any, and name the artifact that path would require you to prepare.

Task 1.3: Validate data quality and manage bias

  • Declare three data quality rules for the ticket dataset, each specific enough to pass or fail on a given run, and run them if your account has the tooling for it.
  • Profile the dataset and read the result. Note which defect you would not have predicted before profiling.
  • Check the proportion of escalated to non-escalated tickets. Name it as class imbalance or representation bias, and justify which.
  • Identify which bias metrics apply to this dataset across its numeric fields and its free text, then name the mitigation technique that fits the defect you found and one that would not help, and say why.

Check yourself

  • Storage reasoning Could you defend your storage choice for each source to someone who proposed putting everything in one place, using all four selection criteria?

  • Retrieval versus customization Could you say in one sentence whether the assistant needs a chunking decision, a customization decision, or both, and why?

  • Bias versus imbalance Could you state which problem the escalation indicator represents, and why the remedy for the other one would not fix it?

  • Metrics that fit Could you name which bias metrics apply to a dataset that mixes numeric fields with free text, and why the numeric ones alone are not enough?

Find your gap

If any step felt unclear, return to the skills in that task's section before moving on. Note the gap so you know what to review.

If you have no AWS account

Every step has a paper equivalent. Describe the bucket layout and the vector store specification instead of creating them, reason about embedding dimensions instead of generating embeddings, and write the three quality rules out instead of running them. The decisions are the review; the console is only where they become concrete.

Clean up

Delete anything you created, so nothing is left running or stored: remove the uploaded objects and the bucket, delete the vector store and any index on it, and remove any job or endpoint. If you used a generative AI tool to produce sample data, clear its conversation history so the sample data is not retained.