Some decisions about data are a matter of weighing cost versus benefit,
like short-term performance goals against long-term maintainability.
Others though are about right and wrong.
For example, let's say that you've trained
a model to predict the probability that a patient
has cancer from medical records and that you've selected patient age,
gender, prior medical conditions,
hospital name, vital signs,
and test results as features.
Your model had excellent performance on
held-out test data but performed terribly on new patients.
Any guesses as to why?
It turns out that the model was trained using a feature that
wasn't legitimately available at decision time,
and so when the model was deployed into production,
the distribution of this feature changed and it was no longer a reliable predictor.
In this case, that feature was hospital name.
You might think hospital name,
how could that be predictive?
Well, remember that there are some hospitals that focus on diseases like cancer.
So, the model learned that hospital name is very important.
However, at decision time this feature wasn't available to
the model because patients hadn't yet been assigned to a hospital.
But rather than throwing an error,
the model simply interpreted the hospital name as an empty string,
which it was capable of doing thanks to
out-of-vocabulary buckets and its representation of words.
We refer to this idea where the label is somehow
leaking into the training data as data leakage.
Data leakage is related to a broader class of problems that we've seen before in
the last specialization where we talked about models learning unacceptable strategies.
Previously, we learned that when there's a class imbalance,
a model might learn to predict the majority class.
In this case, the model has learned to use a feature that wouldn't actually be
known and which cannot be plausibly related to the label.
Here's a similar case,
and let's see if you can recognize the similarity.
A professor of 18th century literature believed that there was a relationship between
how an author thought about the mind and their political orientation.
So for example, perhaps authors who use language like the mind is a garden,
had one political orientation and authors who use
language like the mind is a steel trap had another.
What if we were to naively test this hypothesis with machine learning?
Some people tried that and they got some unexpected results. Here's what they did.
They took all the sentences in all of the works by number of 18th century authors,
extracted the mind metaphors and set those as their features
and then use the political orientations of the authors who wrote them as labels.
Then they randomly assign sentences to each of the training,
validation, and test sets and,
because they divided the data in this way,
some sentences from each author were distributed to each of those three sets.
The resulting model was amazing, but suspiciously amazing.
What might have gone wrong?
One way to think about it is that political orientation is linked to that person,
and if we wouldn't include person name in the feature set,
we should not include it implicitly either.
When the researchers changed the way they partition the data and
instead partitioned it by author instead of by sentence,
the model's accuracy dropped to something more reasonable.