Bias in Training Data

The idea

A cafeteria worker who always hands out pizza because it's the most popular choice overall - never once asking YOU what you actually like. A fair cafeteria worker asks your favorite first, and only defaults to 'the popular choice' if you don't have one of your own.

What it actually is

Training data can be lopsided WITHOUT anyone meaning it to be - if way more kids happened to try pizza first, pizza's count balloons for reasons that have nothing to do with whether any ONE particular kid actually likes it best. A system that blindly recommends 'whatever has the highest count' inherits that bias and treats every kid the same, ignoring what they personally said they like. The fix: check the person's own stated preferences FIRST, and only fall back to the (biased) popularity data if there's no personal match at all.

How you write it

for personal_choice in favorites:
    if personal_choice in overall_data:
        recommendation = personal_choice
        matched = True
# only fall back to the most popular overall if nothing personal matched

A worked example

snack_popularity = {"pizza": 50, "carrots": 3, "grapes": 2, "pretzels": 4}
kid_favorites = ["carrots", "grapes"]

# BIASED naive system: always recommends the most-counted snack overall
recommendation = "unknown"
best_count = -1
for snack in snack_popularity:
    if snack_popularity[snack] > best_count:
        best_count = snack_popularity[snack]
        recommendation = snack

print(recommendation)

pizza has by far the highest count (50) - not because THIS kid likes it best, but because way more kids happened to try pizza first, long before this recommendation ever ran. The naive system only ever looks at the popularity counts, so it recommends "pizza" - even though this specific kid's own stated favorites are carrots and grapes, which never even get checked. Biased training data produced biased (unfair) behavior.

snack_popularity = {"pizza": 50, "carrots": 3, "grapes": 2, "pretzels": 4}
kid_favorites = ["carrots", "grapes"]

recommendation = "unknown"
best_count = -1
for snack in snack_popularity:
    if snack_popularity[snack] > best_count:
        best_count = snack_popularity[snack]
        recommendation = snack

found_personal_match = False
for snack in kid_favorites:
    if not found_personal_match:
        if snack in snack_popularity:
            recommendation = snack
            found_personal_match = True

print(recommendation)

Same biased popularity data, same kid - but now, AFTER the naive popularity pick, the code checks kid_favorites first and overwrites recommendation with "carrots" (the kid's own stated favorite) the moment it finds a personal match. The biased popularity data still exists and still gets used as a starting point, but it no longer gets the final say - one extra check restores fairness to this specific kid without throwing away the popularity data entirely.

Mistakes children actually make

Mistake 1: assuming 'most popular' means 'best for everyone' - a high count in training data can just mean WHO happened to try something first, not universal quality or any one person's actual taste. Mistake 2: fixing the bias by deleting the popularity data entirely instead of just re-ordering the priority (check personal preferences first, THEN fall back) - throwing the aggregate data away completely loses a useful fallback for a kid who never stated any preference at all. Mistake 3: only checking the FIRST favorite in kid_favorites and giving up immediately if it isn't in the data, instead of checking every item in the list in order - a kid whose first favorite happens to be missing from the data (but whose SECOND favorite is present) would incorrectly get the biased fallback instead of their real, matching second choice.

Then they try it

In the app this lesson continues with the animated explanation, spoken aloud, and then the practice: Fix the bias: check kid_favorites first (in order) and recommend the first one that's a key in snack_popularity. Only fall back to the snack with the highest count in snack_popularity if none of kid_favorites match. Print the final recommendation. The editor runs your child’s real code and checks the result, with their coding buddy reacting to what they wrote.