Aggregation & grouping
Aggregation collapses many rows into one. The mechanics are simple; the subtleties are all about what happens to NULLs, what the engine will let you select alongside a group, and the difference between filtering rows and filtering groups.
Aggregates and NULL
One rule covers almost everything: aggregate functions ignore NULLs. They are removed before the function runs. The consequences are larger than that sentence suggests.
| Expression | With NULLs present | Note |
|---|---|---|
COUNT(*) | Counts rows | The only one that does not skip NULLs — it counts rows, not values. |
COUNT(col) | Counts non-NULL values | Often silently different from COUNT(*). |
SUM(col) | Sums non-NULLs | NULL if every value is NULL — not 0. |
AVG(col) | SUM/COUNT(col) | Divides by non-NULL count. Not the row count. |
MIN/MAX | Ignores NULLs | NULL only if all values are NULL. |
| No rows at all | COUNT→0, everything else →NULL | The empty-set asymmetry. |
SUM of an empty set is NULL, not zero. So
total + SUM(x) becomes NULL when there is nothing to sum, and the
whole expression vanishes. Wrap it: COALESCE(SUM(x), 0).
AVG divides by the non-NULL count. If half your rows have
NULL, the average is over the other half — which may be what you want, or may
be badly wrong. If NULL means zero in your domain, write
AVG(COALESCE(x, 0)) and make the intent explicit.CREATE TABLE r (score INT);
INSERT INTO r VALUES (10),(20),(NULL),(NULL);
SELECT COUNT(*) AS rows_, -- 4
COUNT(score) AS scored, -- 2
SUM(score) AS total, -- 30
AVG(score) AS avg_, -- 15.0
AVG(COALESCE(score, 0)) AS avg_zero -- 7.5
FROM r;
-- the empty-set asymmetry
SELECT COUNT(*), SUM(score), AVG(score)
FROM r WHERE score > 1000;
-- 0, NULL, NULL ← COUNT gives 0, the others give NULL
-- so this silently produces NULL, not 100:
SELECT 100 + SUM(score) FROM r WHERE score > 1000; -- NULL
SELECT 100 + COALESCE(SUM(score), 0) FROM r WHERE score > 1000; -- 100GROUP BY and functional dependency
After GROUP BY, each group is one output row. So every expression
in SELECT must produce exactly one value per group. It can do that
two ways: by being a grouping column, or by being an aggregate.
Anything else is ambiguous — which of the group's several values should it return? — and the standard rejects it.
-- ambiguous: which name? the group may contain many. SELECT dept, name, COUNT(*) FROM emp GROUP BY dept; -- ERROR -- unambiguous: pick one explicitly SELECT dept, MAX(name), COUNT(*) FROM emp GROUP BY dept; SELECT dept, ANY_VALUE(name), COUNT(*) FROM emp GROUP BY dept;
Functional dependency, and why some "invalid" queries are fine
If you group by a primary key, every other column of that table is functionally determined — there is only one possible value per group. Modern engines recognize this:
-- legal: grouping by the PK determines name and email SELECT u.id, u.name, u.email, COUNT(o.id) FROM users u LEFT JOIN orders o ON o.user_id = u.id GROUP BY u.id; -- Postgres and MySQL 8 both accept this. -- Grouping by a non-key column would not be accepted.
ONLY_FULL_GROUP_BY is in the default sql_mode since
5.7. If you inherit a codebase that breaks when you enable it, the queries were
always ambiguous — the mode is reporting a pre-existing bug, not creating one.
Fix them with ANY_VALUE() where the ambiguity is genuinely
harmless, and with a proper aggregate where it is not.HAVING versus WHERE
Chapter 6 already answered this: WHERE is step 2 and filters
rows; HAVING is step 4 and filters groups. Two
practical corollaries follow.
WHEREcannot use an aggregate, because groups do not exist yet.HAVINGcan use non-aggregates, but should not: a non-aggregate condition inHAVINGis evaluated after grouping, so it filters far more rows than necessary.
-- WRONG PLACE: groups every department, then throws most away SELECT dept, COUNT(*) FROM emp GROUP BY dept HAVING dept <> 'temp'; -- filters AFTER aggregating -- RIGHT: excludes the rows before any grouping work happens SELECT dept, COUNT(*) FROM emp WHERE dept <> 'temp' GROUP BY dept; -- both together: the normal shape SELECT dept, COUNT(*) FROM emp WHERE hired_at >= '2026-01-01' -- which rows to consider GROUP BY dept HAVING COUNT(*) > 5; -- which groups to keep
HAVING condition down into
WHERE automatically. Writing it correctly still matters: it states
intent, it is guaranteed rather than dependent on an optimization, and it makes
the query's shape obvious to the next reader.GROUPING SETS, ROLLUP, CUBE
One pass, several levels of aggregation. The alternative is several queries
combined with UNION ALL, each re-scanning the table.
-- GROUPING SETS: name exactly the groupings you want SELECT region, product, SUM(amount) FROM sales GROUP BY GROUPING SETS ( (region, product), -- detail (region), -- subtotal per region () -- grand total ); -- ROLLUP: the hierarchical shortcut for exactly that pattern SELECT region, product, SUM(amount) FROM sales GROUP BY ROLLUP (region, product); -- (region, product), (region), () -- MySQL's older spelling of ROLLUP: -- GROUP BY region, product WITH ROLLUP -- CUBE: every combination (Postgres; MySQL has no CUBE) SELECT region, product, SUM(amount) FROM sales GROUP BY CUBE (region, product); -- (region,product), (region), (product), ()
Telling a subtotal from a real NULL
Subtotal rows carry NULL in the columns being rolled up — which is
indistinguishable from a genuine NULL in the data.
GROUPING() resolves it, returning 1 when the NULL is a subtotal
marker:
SELECT CASE WHEN GROUPING(region) = 1 THEN 'ALL REGIONS' ELSE COALESCE(region, '(unknown)') END AS region, SUM(amount) FROM sales GROUP BY ROLLUP (region);
FILTER, and its workaround
SQL:2003's FILTER clause restricts one aggregate to a subset of
rows, so several differently-filtered aggregates can share a single pass.
-- Postgres, SQLite, standard SELECT COUNT(*) AS total, COUNT(*) FILTER (WHERE status = 'paid') AS paid, COUNT(*) FILTER (WHERE status = 'refunded') AS refunded, SUM(amount) FILTER (WHERE status = 'paid') AS revenue FROM orders;
MySQL has no FILTER. The equivalent exploits the fact that
aggregates skip NULLs (chapter 22):
-- MySQL: CASE returning NULL for non-matching rows SELECT COUNT(*) AS total, COUNT(CASE WHEN status = 'paid' THEN 1 END) AS paid, SUM(CASE WHEN status = 'paid' THEN amount END) AS revenue, SUM(status = 'paid') AS paid_short FROM orders; -- note: no ELSE, so non-matching rows yield NULL and are skipped. -- SUM(boolean) works in MySQL because TRUE is 1.
Conditional aggregation and pivoting
The same technique turns rows into columns. Standard SQL has no
PIVOT; this is how everyone does it.
-- from: one row per (month, region) -- to: one row per month, one COLUMN per region SELECT month, SUM(CASE WHEN region = 'EU' THEN amount ELSE 0 END) AS eu, SUM(CASE WHEN region = 'US' THEN amount ELSE 0 END) AS us, SUM(CASE WHEN region = 'AF' THEN amount ELSE 0 END) AS af, SUM(amount) AS total FROM sales GROUP BY month ORDER BY month;
COUNT(*) vs COUNT(col) vs COUNT(DISTINCT col)
| Form | Counts | Cost |
|---|---|---|
COUNT(*) | Rows | Cheapest. Can use any index, or the smallest one. |
COUNT(1) | Rows | Identical to COUNT(*). The folklore that one is faster is false in every modern engine. |
COUNT(col) | Non-NULL values | Must examine the column. |
COUNT(DISTINCT col) | Distinct non-NULL values | Expensive — needs a sort or hash of all values. |
COUNT(*) was
instant. InnoDB cannot: with MVCC, the count depends on which transaction is
asking (MySQL module, ch 42), so it must actually scan an index.
On a hundred-million-row table that is seconds. If you need an approximate count for a UI, use
information_schema.tables.table_rows — it is
an estimate from the same sampled statistics the optimizer uses, and it is
instant. If you need an exact count frequently, maintain a counter table.-- exact, and it scans SELECT COUNT(*) FROM big_table; -- approximate, and instant SELECT table_rows FROM information_schema.tables WHERE table_schema = DATABASE() AND table_name = 'big_table'; -- Postgres equivalent -- SELECT reltuples::bigint FROM pg_class WHERE relname = 'big_table'; -- the three counts differing on the same column SELECT COUNT(*), COUNT(region), COUNT(DISTINCT region) FROM sales;
Ordered-set aggregates and percentiles
Some aggregates need their input in order — medians and percentiles. SQL:2003
gives them a distinct syntax, WITHIN GROUP.
-- Postgres / Oracle / standard SELECT PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY ms) AS p50, PERCENTILE_CONT(0.95) WITHIN GROUP (ORDER BY ms) AS p95, PERCENTILE_DISC(0.95) WITHIN GROUP (ORDER BY ms) AS p95_actual FROM requests; -- CONT interpolates between values; DISC returns an actual -- observed value. For latency SLOs, DISC is usually what you want.
MySQL has neither. The portable approach uses a window function (Part 4) to rank rows and pick the one at the right position:
-- MySQL: p95 latency
SELECT ms AS p95 FROM (
SELECT ms,
PERCENT_RANK() OVER (ORDER BY ms) AS pr
FROM requests
) t
WHERE pr >= 0.95
ORDER BY pr
LIMIT 1;AVG(response_ms) is dominated by the bulk of fast requests and
hides the tail entirely. A service with a 40ms mean can have a 4-second p99,
and the p99 is what users experience as "the site is broken". Report
percentiles; use the mean only alongside them.Part 4 is window functions — the same aggregate machinery, but computed per-row instead of collapsing the rows. It is the single largest capability gap between engineers who learned SQL before 2010 and after.