Repetition Penalties
The marketing team asks the Pedal Point assistant for ten taglines for a winter sale. It returns "Ride into winter", "Ride the winter", "Winter riding made easy", "Ride winter in style"... Every line leans on the same two words. Temperature alone did not fix it. A repetition penalty will.
The rule
Penalties make tokens already used less likely to be used again.
As the model writes, the app keeps track of which tokens have appeared. A penalty lowers the score of those tokens before the next pick. There are two common flavours:
- A presence penalty applies a flat reduction to any token that has appeared at least once. It asks: "Have we used this before?" It nudges the model toward new words and topics.
- A frequency penalty grows with how many times a token has appeared. It asks: "How often have we used this?" It mostly stops one word from being hammered again and again.
Some tools, especially for running open models locally, offer a single "repetition penalty" that scales down the scores of recent tokens. Names, ranges and exact formulas differ between providers, so read the documentation for the one you use and start with small values.
Penalties are also a fix for loops, where a model gets stuck repeating a phrase or list item. They are not free. A penalty does not know which repeats are good. In a support answer, the order number, the product name and words like "the" must repeat. Push too hard and the model starts avoiding them, choosing odd synonyms or even changing a number.
The shortcut
Presence asks "ever?"; frequency asks "how often?" Use penalties for creative variety; keep them at zero for factual answers.
Diagnose
"Ride" and "winter" dominate. The problem is reuse of the same words across lines.
The trap is setting a penalty globally for every feature. Each feature needs its own settings. A tagline writer and an order-status helper can share one model and still use completely different dials.
Drills
Answer fast — trust the rule.
A token has appeared five times. Which penalty grows larger because of the count?
What does a presence penalty mainly encourage?
A strong penalty is switched on for order-status answers. What is the most likely harm?
The model gets stuck writing the same list item over and over. Which dial is designed to help?
Remember
- Penalties make tokens already used less likely to be used again.
- Presence asks "ever?"; frequency asks "how often?"
- Creative lists welcome penalties; factual answers want none.