Pebellslearn by building
0%
RoadmapsMatrixCoursesSandboxSign in
AI Safety and Ethics  ›  Lesson

Bias and Fairness

Bias and Fairness 12 minModels learn patterns, including unfair ones, from their training data, so an AI feature must be tested for unequal treatment and kept away from decisions it should not make alone.

Bias and Fairness

Pedal Point lets the assistant suggest whether to allow a return a few days past the deadline. A staff member notices something odd: requests written in formal English seem to get "yes" more often than requests with spelling mistakes, though the facts are the same. Nobody programmed that. The model learned it.

The rule

Models inherit patterns from their data; test for unequal treatment.

A language model learns from enormous amounts of human writing, and human writing contains stereotypes and unequal patterns. The model can reproduce them in ways that are hard to spot in any single answer:

  • treating people differently based on names, dialect or spelling
  • assuming a person's gender, age or background from small clues
  • giving more thorough help to some kinds of questions than others

Bias matters most when the model's output affects people: approving exceptions, ranking applicants, flagging accounts. For a bike-size recommendation it is a quality issue; for decisions about people it is a fairness issue, and it may also be a legal one depending on where you operate.

Practical steps:

  • Leave out what the decision does not need. If names and addresses are not in the input, they cannot sway the output.
  • Write the rule down. "Allow exceptions up to 7 days late for unused items" gives the model something better to follow than its instincts.
  • Measure, don't guess. Test many cases and compare outcome rates between groups.
  • Keep people in charge of decisions with real consequences. The model can suggest; a person or a fixed rule decides.

The shortcut

Swap one detail, keep the facts, compare outcomes. If changing only a name or a writing style changes the result, you have found bias.

Testing the returns-exception featureStep 1 of 4

  1. Build a pair

    Same item, same 3 days late, same order history. One request signed "Emma", one signed "Oluwaseun".

The trap is judging fairness from a handful of chats. Bias shows up in rates across many cases, not in one answer.

Drills

Answer fast — trust the rule.

quick drill

Where does an LLM's bias usually come from?

quick drill

What makes a good fairness test pair?

quick drill

Which input change most directly stops names from swaying the decision?

quick drill

A remaining gap is found in a decision about customers. What is the responsible design?

Remember

  • Models inherit patterns from their data; test for unequal treatment.
  • Swap one detail, keep the facts, compare outcomes.
  • Remove what the decision does not need; keep people in charge.

Practice — level up

0/1 solved

Write your answer, then compare it with a rubric and a model answer.

ExplainDesign a swap test

Pedal Point lets the assistant suggest whether to grant a returns exception for items slightly past the 30-day window. Design a test to check whether it treats customers unfairly, and explain what you would change if the test finds a problem.