Regular Expressions
In short: a regular expression (regex) describes the shape of text, not
its meaning. re.search finds the pattern anywhere, re.match only tries at
the start, re.fullmatch demands the pattern cover the entire string, and
re.findall returns every match as a list. Character classes (\d, \w,
[a-z]) describe one character at a time, and a quantifier straight after one
(+, *, ?, {m,n}) says how many in a row — greedy by default, taking as
much as it can. Always write a pattern as a raw string (r"..."): Python's own
backslash escapes run first, and they will quietly rewrite a pattern you never
meant to change.
Quarry's yard can now open any file and decode it correctly. What comes out is
text — a lot of it server logs, which look structured but have nothing
enforcing that structure. Somewhere in 40,000 lines there's a stack trace, a
line with two stray leading spaces, or a service that logs NOTICE because
someone thought it sounded friendlier than INFO. This section builds the
extractor, the part of Quarry that turns a line of text into typed fields.
Its first job is the humblest one: deciding whether a line is a log line.
Spot a PAN number on a KYC form
An Indian PAN looks like ABCDE1234F: five uppercase letters, four digits, one
uppercase letter. A character class names a kind of character, and a
quantifier right after it says how many:
[A-Z]{5} is "a letter A–Z, five times in a row". \d{4} is "a digit, four
times". Lowercase fails because the class only allows capitals; the third
value fails because it only has three digits where four are required. So far
so good — but re.search only needs the pattern to fit somewhere:
A PAN with junk stuck on either end is wrongly accepted. Run it, see both results print True, then fix the check so only an exact PAN passes.
A form field should hold nothing but a PAN. re.fullmatch is the honest
tool for that: it says exactly what you mean, and nobody reading the code has
to wonder whether a stray character could sneak past.
Pull every rupee amount out of a UPI SMS
+ means "one or more", so \d+ grabs a whole run of digits, not just one.
That's greedy: it takes as many digits as it can before giving up.
import re
sms = "Paid Rs.250.00 to Raju. Avl bal Rs.4200.00"
print(re.findall(r"\d+", sms))To keep an amount whole, the pattern has to say "digits, then a dot, then
digits" as one shape — \d+\.\d+:
A pattern matches characters, not the meaning you have in your head. If you want "an amount", you have to spell out its exact shape — Python doesn't know what a rupee amount looks like until you tell it.
Greedy and lazy, in one line
+, * and friends take as much as possible by default. Put a ? right
after one and it takes as little as possible instead:
Greedy .* races to the end of the line and backs off until a closing "
fits — landing on the last one. Lazy .*? stops at the first " it can.
Neither is "more correct"; they answer different questions. The next lesson
goes much deeper into what that choice costs.
Make sure a server line really is one of Quarry's logs
Quarry's format, element by element:
2026-09-14 08:15:02 WARN [auth] login failed user=maya took=312ms
└─ date ─┘ └ time ┘ └lvl┘ └comp┘ └──────── message ────────┘- date —
\d{4}-\d{2}-\d{2} - time —
\d{2}:\d{2}:\d{2} - level — one of four words.
|means "or":(DEBUG|INFO|WARN|ERROR). Parentheses keep the|from splitting the whole pattern — groups get a lesson of their own next. - component — a literal
[, a lowercase letter, then letters/digits/ hyphens, then a literal]. Brackets are special, so escape them:\[[a-z][a-z0-9-]*\]. - message — at least one more character:
.+
Each rejection fails for a different reason: leading spaces, an unknown level,
a traceback with no timestamp at all. fullmatch is what makes validation
honest — search would happily approve a line with junk around a valid core,
exactly like the PAN above.
Backslash is special to both Python strings and regex, which is where the
next bug hides. Without the r prefix, Python processes escapes first and
hands re whatever is left:
ERROR is plainly in the line, but this prints "missing". Run it, work out what Python did to the pattern, then fix it with one character.
\b is a word boundary: a zero-width position between a word character
and a non-word one. r"\bERROR\b" finds ERROR but not the ERROR hiding
inside TERRORS. Make r"..." a reflex for every pattern and this whole
class of bug disappears.
Watching the engine work
Knowing what re.search actually does explains every surprise later in this
section. For each start position it walks the pattern, consuming characters
while they fit, and gives up on that position only once nothing else can be
tried — then moves one character to the right. Here's that loop written by
hand for \d+ms against "took=312ms":
The engine is left-to-right and first-wins — it reports the leftmost
match, not the "best" one. A pattern 19 characters wide (the timestamp) has
exactly one place to sit on a 19-character line, which is all fullmatch ever
tries. A short pattern on a long line can start almost anywhere, so an
unanchored search that fails pays for every one of those attempts.
Quick practice
import re
print(re.findall(r"ab+", "abbb ab abab a"))import re
p = r"\[[a-z][a-z0-9-]*\]"
print([bool(re.fullmatch(p, s)) for s in ["[db]", "[Auth]", "[2fa]", "[disk-io]"]])import re
print(bool(re.match(r"\d+", "312ms")), bool(re.fullmatch(r"\d+", "312ms")))🎯 Your turn
Write is_log_line(line) — Quarry's first gate. It returns True only when
the whole line is in Quarry's format (date, time, one of
DEBUG/INFO/WARN/ERROR, a bracketed lowercase component, and a
non-empty message), and False otherwise:
is_log_line("2026-09-14 08:15:02 WARN [auth] login failed user=maya")→Trueis_log_line(" 2026-09-14 08:15:02 INFO [db] pool ready")→Falseis_log_line("2026-09-14 08:15:02 NOTICE [auth] hi")→False
Hint — build the pattern as a raw string, one element at a time, and use
re.fullmatch so nothing is allowed before or after it. Wrap the result in
bool(...) — the grader wants True/False, not a match object.
Then press ▶ Run and hit ✓ Check. ✅