Writing SQL the engine likes
The semantics are settled; this is about execution. A small number of habits
account for most query performance problems, and nearly all of them come down to
one question: can the engine use the index, or has your WHERE
clause quietly prevented it?
Sargability
From "Search ARGument able". A predicate is sargable if the engine can use it to seek into an index rather than examining every row.
The rule is simple: the indexed column must appear alone on one side of
the comparison. An index stores created_at, not
DATE(created_at) — so wrapping the column destroys the ordering
the B+tree provides.
| Non-sargable | Sargable rewrite |
|---|---|
WHERE DATE(created_at) = '2026-09-21' | WHERE created_at >= '2026-09-21' AND created_at < '2026-09-22' |
WHERE YEAR(d) = 2026 | WHERE d >= '2026-01-01' AND d < '2027-01-01' |
WHERE UPPER(email) = '[email protected]' | A case-insensitive collation, or a generated column (Part 6, ch 54) |
WHERE price * 1.2 > 100 | WHERE price > 100 / 1.2 |
WHERE name LIKE '%son' | No rewrite — a leading wildcard cannot seek. Reverse the column, or use full-text. |
WHERE id + 0 = 42 | WHERE id = 42 |
Implicit conversions
The invisible version of the same problem. When the two sides of a comparison have different types, the engine converts one — and if it converts the column, that is a function on the column, so the index is gone.
CREATE TABLE t ( phone VARCHAR(20) PRIMARY KEY, name VARCHAR(50) ); -- string column, NUMBER literal → MySQL converts the COLUMN to a -- number for every row. full scan. EXPLAIN SELECT * FROM t WHERE phone = 8012345678; -- type: ALL ← scanning -- quote it and the index works EXPLAIN SELECT * FROM t WHERE phone = '8012345678'; -- type: const ← seeking -- worse: the comparison is also WRONG. -- '08012345678' and '8012345678' both become the number -- 8012345678, so the leading zero silently disappears and -- you match rows you did not mean to. -- Postgres does not do this — it raises a type error instead, -- which is the better behaviour.
② Joining columns of different types —
INT to
BIGINT is fine, INT to VARCHAR is not.
③ Joining columns with different collations, which is the same problem wearing a disguise (MySQL module, ch 106).
All three produce a correct-looking query, no warning, and a full scan. The only way to find them is
EXPLAIN.Keyset pagination
Chapter 12 introduced the mechanism; this is the operational case for it.
OFFSET n requires the engine to produce and discard n
rows. Page 1 is instant, page 5000 reads a hundred thousand rows to return
twenty. The cost grows linearly with page number, so the slowest pages are the
ones crawlers and power users hit.
-- O(offset) — degrades as users go deeper SELECT * FROM posts ORDER BY created_at DESC, id DESC LIMIT 20 OFFSET 100000; -- O(1) per page — seek straight to the position SELECT * FROM posts WHERE (created_at, id) < (?, ?) -- last row of the previous page ORDER BY created_at DESC, id DESC LIMIT 20; CREATE INDEX idx_page ON posts (created_at DESC, id DESC);
| OFFSET | Keyset | |
|---|---|---|
| Cost of page N | O(N × size) | O(1) |
| Under concurrent inserts | Rows shift — duplicates and skips | Stable |
| Jump to page 500 | Yes | No |
| Total page count | Needs a COUNT(*) anyway | Same |
Keep
OFFSET only for an explicit numbered pager, and cap how deep
it can go ("showing the first 1,000 results"). Search engines do exactly this,
and nobody complains.
Note the tiebreaker in both queries: sorting by
created_at alone
is not a total order, so rows with equal timestamps can appear twice or never.
The id makes it deterministic.The anti-pattern catalogue
SELECT *
Fetches columns you do not need, which prevents covering indexes
(MySQL module, ch 35), forces extra page reads for off-page
TEXT/BLOB columns (ch 21), and breaks silently when
a column is added or reordered. Name your columns.
A function on the indexed column
Chapter 73. The commonest cause of an unused index.
Leading wildcard LIKE
LIKE '%son' cannot seek — the B+tree is ordered by prefix.
Options: a reversed generated column indexed and queried with
LIKE 'nos%', a trigram index (Postgres
pg_trgm), or full-text search.
OR across different columns
WHERE a = 1 OR b = 2 usually cannot use one index well. It
becomes an index merge (MySQL module, ch 83) or a scan. A
UNION ALL of two indexed queries is frequently faster, and it
makes the intent explicit.
Implicit type mismatch
Chapter 74. Invisible in the query text; visible only in EXPLAIN.
SELECT DISTINCT to hide fan-out
Part 2, chapter 15. The duplicates come from the join; deduplicating afterwards pays for a full sort and can still give wrong aggregates.
N+1 from the application
One query for a list, then one per row for its details. A thousand round
trips where one join or one IN would do. This is the single
largest source of slow pages in ORM-based applications, and the ORM module
covers why it happens by default.
COUNT(*) for pagination on a large table
Exact counts scan (Part 3, ch 28). Use an estimate for the UI, or omit the total and show "more" instead.
ORDER BY RAND()
Part 8, chapter 69. Full scan plus full sort to return one row.
Unbounded queries
No LIMIT on a query that could match everything. It works until
the table grows, then it returns a million rows into application memory. Bound
every query that is not aggregating.
Reading your own plan
The MySQL module's chapter 85 covers EXPLAIN ANALYZE in depth.
Here is the short version — the four things to check on any query, in order.
- Access type.
constandeq_refare ideal;refandrangeare fine;indexis a full index scan;ALLis a full table scan. AnALLon a large table is where to start. - Estimated versus actual rows. An order-of-magnitude gap means a statistics problem, not an optimizer problem. This is the single most valuable comparison.
- Extra flags.
Using filesortandUsing temporaryare not always bad but are worth explaining.Using indexis good — the query was covered. - Rows examined versus rows returned. Available from
sys.statement_analysis. A ratio near 1 is perfect; 10,000 means scanning to find one row.
EXPLAIN FORMAT=TREE SELECT ...\G -- the plan EXPLAIN ANALYZE SELECT ...\G -- plan + real timings (RUNS it) EXPLAIN FORMAT=JSON SELECT ...\G -- costs and filtered % EXPLAIN FOR CONNECTION 1234\G -- a query running RIGHT NOW -- Postgres -- EXPLAIN (ANALYZE, BUFFERS, FORMAT TEXT) SELECT ...; -- BUFFERS shows shared hits vs reads — cache behaviour.
Rewriting one query five ways
The exercise that consolidates this part. One question — the most recent
order for each customer — written five ways. Run all five with
EXPLAIN ANALYZE on real data and compare.
-- ① correlated subquery
SELECT c.id, c.name,
(SELECT MAX(o.created_at) FROM orders o WHERE o.customer_id = c.id)
FROM customers c;
-- good with few customers + an index on (customer_id, created_at)
-- ② join to a grouped subquery
SELECT c.id, c.name, o.last_order
FROM customers c
LEFT JOIN (SELECT customer_id, MAX(created_at) AS last_order
FROM orders GROUP BY customer_id) o
ON o.customer_id = c.id;
-- one pass over orders. good when most customers have orders.
-- ③ window function
WITH r AS (
SELECT o.*, ROW_NUMBER() OVER (PARTITION BY customer_id
ORDER BY created_at DESC) rn
FROM orders o
)
SELECT c.id, c.name, r.id AS order_id, r.created_at
FROM customers c LEFT JOIN r ON r.customer_id = c.id AND r.rn = 1;
-- gives the whole order row, not just the date.
-- but ranks EVERY order to keep one per customer.
-- ④ LATERAL
SELECT c.id, c.name, o.id AS order_id, o.created_at
FROM customers c
LEFT JOIN LATERAL (
SELECT id, created_at FROM orders
WHERE customer_id = c.id ORDER BY created_at DESC LIMIT 1
) o ON TRUE;
-- stops after ONE row per customer. usually the winner
-- with an index on (customer_id, created_at DESC).
-- ⑤ anti-join: "no later order exists"
SELECT c.id, c.name, o.id, o.created_at
FROM customers c
JOIN orders o ON o.customer_id = c.id
WHERE NOT EXISTS (
SELECT 1 FROM orders o2
WHERE o2.customer_id = o.customer_id
AND o2.created_at > o.created_at
);
-- elegant, and often the slowest. measure it.What you should take away is the habit: when a query matters, write it two or three ways and measure, rather than reasoning about which ought to be faster. Chapter 77's four checks tell you why the winner won, and that explanation is what generalizes to the next query.
Part 10 is the capstone: a SQL engine you write yourself, which connects directly to the storage engine from the MySQL module's Part 11.