Latency vs Throughput
"How fast is Hop?" has two honest answers. One is "a redirect takes 40 milliseconds". The other is "it handles 5,000 redirects a second". They measure different things, and improving one can hurt the other.
The rule
Latency is how long one takes; throughput is how many finish per second.
- Latency is time for a single request, measured in milliseconds. It is what one user feels.
- Throughput is completed work per unit of time, like requests per second. It is what the business feels at peak.
Think of a motorway. Latency is how long your drive takes; throughput is how many cars pass a point each minute. Adding lanes raises throughput without making your car faster.
The two are tied together by a simple rule (known as Little's law):
in flight = throughput × latency
If Hop completes 4,000 requests per second and each takes 0.05 s, then 4,000 × 0.05 = 200 requests are in progress at any instant. Those 200 need connections, memory and threads.
The design goal: the highest throughput that keeps latency acceptable. Past a point, pushing more work in only builds queues, and latency shoots up.
The shortcut
Before comparing two options, name which one you are measuring, then use in-flight = throughput × latency to see the cost.
Today
Hop writes every click to storage one at a time: 2 ms each, so one worker logs about 500 clicks per second.
Batching is the classic trade: more throughput, worse latency. The trap is reporting only one number. "Handles 10,000 per second" means little if each request takes five seconds.
Also watch averages. If 99 requests take 10 ms and 1 takes 2 s, the average (about 30 ms) hides the unlucky user. Designers quote percentiles, such as "99% of redirects finish under 50 ms".
Drills
Answer fast — trust the rule.
A user says "the link took ages to open". Which measure are they describing?
Hop completes 2,000 redirects per second, each taking 0.1 s. How many are in flight at once?
Grouping many small writes into one big write usually does what?
Which target describes latency most usefully?
Latency stays at 50 ms but Hop must now handle twice the throughput. What happens to requests in flight?
Remember
- Latency is how long one takes; throughput is how many finish per second.
- Shortcut: in flight = throughput × latency; name which one you are measuring.
- Trap: quoting only one number, or an average that hides the slow tail.