Part 8 · 8 chapters · ~20 min

Advanced & vendor frontiers

Where SQL stops being uniform. Each of these is genuinely useful, supported differently by each engine, and comes with a judgement call about whether the database is the right place for it at all.

65

JSON in SQL

MySQL JSONPostgres jsonbPostgres json
StorageBinary, parsedBinary, parsedRaw text
Key orderNot preservedNot preservedPreserved
Duplicate keysLast winsLast winsKept
IndexableVia generated column or functional indexDirectly, with GINNo
code
-- MySQL path extraction
SELECT payload->'$.user.name' AS quoted,      -- "ada"
       payload->>'$.user.name' AS unquoted     -- ada
FROM events;
--   comparing a -> result to a plain string silently fails to match.
-- Postgres
SELECT payload -> 'user' ->> 'name' FROM events;
SELECT payload #>> '{user,name}' FROM events;

-- containment — the Postgres query that GIN accelerates
SELECT * FROM events WHERE payload @> '{"type":"signup"}';
CREATE INDEX idx_payload ON events USING GIN (payload);

-- JSON_TABLE: expand an array into rows (SQL:2016, both engines)
SELECT e.id, t.sku, t.qty
  FROM events e,
  JSON_TABLE(e.payload, '$.items[*]' COLUMNS (
    sku VARCHAR(32) PATH '$.sku',
    qty INT PATH '$.qty'
  )) t;
the design rule
JSON is right for genuinely variable shapes: third-party payloads kept verbatim, sparse user-defined attributes, event bodies that differ per type. It is wrong for fields you filter, join or sort on regularly — those are columns, and using JSON for them trades one schema migration today for a permanent query-performance tax and no type checking.

A good hybrid: store the payload as JSON and promote the two or three fields you actually query into generated columns with indexes (Part 6, ch 54).
66

Arrays and composite types

A genuine Postgres capability with no MySQL equivalent.

code
-- Postgres
CREATE TABLE posts (
  id   SERIAL PRIMARY KEY,
  tags TEXT[] NOT NULL DEFAULT '{}'
);

SELECT * FROM posts WHERE 'sql' = ANY(tags);
SELECT * FROM posts WHERE tags @> ARRAY['sql','db'];   -- contains both
CREATE INDEX ON posts USING GIN (tags);

-- unnest to rows, and aggregate back
SELECT id, tag FROM posts, unnest(tags) AS tag;
SELECT array_agg(tag ORDER BY tag) FROM tags;
an array is a denormalized join table
Arrays are excellent for small, read-mostly, order-significant lists you always fetch whole. They are poor when you need to query from the element side ("which posts share a tag with this one"), enforce referential integrity on elements, or store attributes about the relationship.

The test: if the element would ever need a column of its own — a weight, a created_at, a source — it is a join table, not an array.
67

Full-text search

code
-- MySQL
CREATE TABLE docs (id INT PRIMARY KEY, body TEXT,
  FULLTEXT KEY ft_body (body));

SELECT id, MATCH(body) AGAINST ('innodb btree' IN NATURAL LANGUAGE MODE) AS score
  FROM docs
 WHERE MATCH(body) AGAINST ('innodb btree' IN NATURAL LANGUAGE MODE)
 ORDER BY score DESC;

-- Postgres: tsvector + GIN, with stemming and weighting
ALTER TABLE docs ADD COLUMN tsv tsvector
  GENERATED ALWAYS AS (to_tsvector('english', body)) STORED;
CREATE INDEX ON docs USING GIN (tsv);

SELECT id, ts_rank(tsv, q) AS rank
  FROM docs, to_tsquery('english', 'innodb & btree') q
 WHERE tsv @@ q ORDER BY rank DESC;
where the line is
Database full-text is right for a search box over a modest corpus where results must be transactionally consistent with the data — an admin console, an internal document store. It gives you no typo tolerance, weak ranking, limited language support, and surprising defaults (MySQL's minimum token length is 3, so two-letter terms silently match nothing).

When search is a product feature — relevance tuning, facets, synonyms, fuzzy matching, autocomplete — use a search engine. Knowing which side of that line you are on matters more than knowing the syntax.
68

Temporal tables

SQL:2011 defines two independent time dimensions, and conflating them is the usual mistake.

  • System time — when the database knew it. Maintained automatically; an audit trail.
  • Valid time — when the fact was true in the world. Application-maintained; a price effective from a date, a contract term.

Neither MySQL nor Postgres implements system versioning natively (MariaDB does). In practice you build it:

code
-- the common pattern: a history table plus a trigger
CREATE TABLE prices_history (
  LIKE prices,
  valid_from DATETIME(3) NOT NULL,
  valid_to   DATETIME(3) NOT NULL,
  PRIMARY KEY (id, valid_from)
);

-- "what was the price when this order was placed?"
-- a non-equi join, exactly as in Part 2 ch 21
SELECT o.id, p.amount
  FROM orders o
  JOIN prices_history p
    ON p.sku = o.sku
   AND o.created_at >= p.valid_from
   AND o.created_at <  p.valid_to;
why this matters in finance
A ledger must be able to answer "what did we believe on the 3rd?" as well as "what was true on the 3rd?" — and those differ whenever a correction is backdated. Bitemporal modelling is the formal answer, and the practical minimum is: never update a financial fact in place. Insert a new version and keep the old one.
69

TABLESAMPLE

code
-- Postgres: sample ~1% of BLOCKS, not rows. very fast.
SELECT * FROM events TABLESAMPLE SYSTEM (1);

-- BERNOULLI samples rows independently: more uniform, slower
SELECT * FROM events TABLESAMPLE BERNOULLI (1);

-- MySQL has neither. the usual approximation:
SELECT * FROM events WHERE RAND() < 0.01;   -- full scan!
ORDER BY RAND() LIMIT 1 is a trap
It assigns a random value to every row, sorts the entire table, and takes one. On a large table it is a full scan plus a full sort for a single row.

Better: pick a random id in the known range and take the next row at or above it, or maintain a random column with an index. Both are O(log n) instead of O(n log n).
70

Set operations

OperationReturnsDuplicatesMySQL
UNION ALLEverything from bothKeptYes
UNIONEverything, deduplicatedRemovedYes
INTERSECTRows in bothRemoved8.0.31+
EXCEPTIn the first, not the secondRemoved8.0.31+
code
-- the rules that trip people up
--   1. both sides need the same column COUNT and compatible types
--   2. column NAMES come from the FIRST query
--   3. ORDER BY applies to the WHOLE result and goes at the end

(SELECT id, name FROM a ORDER BY name LIMIT 5)    -- per-branch: parenthesise
UNION ALL
(SELECT id, name FROM b ORDER BY name LIMIT 5)
ORDER BY name;                                    -- overall

Before INTERSECT and EXCEPT existed in MySQL, the equivalents were an inner join and a NOT EXISTS anti-join respectively — which are still often faster, because they can use indexes rather than materializing and deduplicating both sides.

71

Views and materialized views

ViewMaterialized view
Stores dataNo — a stored queryYes
FreshnessAlways currentAs of the last refresh
Query costThe underlying query, every timeA table read
IndexableNoYes
MySQLYesNo — use a table plus a refresh job
code
-- Postgres
CREATE MATERIALIZED VIEW daily_revenue AS
SELECT DATE(created_at) AS d, SUM(total) AS revenue
    FROM orders GROUP BY 1;

CREATE UNIQUE INDEX ON daily_revenue (d);      -- required for:
REFRESH MATERIALIZED VIEW CONCURRENTLY daily_revenue;
--   without CONCURRENTLY, the refresh takes an exclusive lock
--   and the view is unreadable while it runs.
views do not make slow queries fast
A view is a macro. Selecting from a view runs its query, and selecting from a view built on views runs all of them. A three-level view stack can generate a query the optimizer handles badly, and the person debugging it sees only SELECT * FROM v_summary.

Views are excellent for encapsulating a join or enforcing a row filter for security. They are not a performance tool — that is a materialized view, or a summary table maintained on write.
72

Procedures, functions, triggers

InvokedReturnsMain risk
ProcedureCALLOut params, result setsLogic outside version control and deploy
FunctionIn an expressionOne valueCalled per row — can be catastrophic in a WHERE
TriggerAutomatically on DML—Invisible. The statement you ran is not all that ran.
the Staff position, argued both ways
For: logic next to the data runs without a network round trip per row, it cannot be bypassed by a second client, and for set-based bulk work it is dramatically faster than pulling rows into an application.

Against: it is usually outside your repository, has no tests, no code review, no staged rollout, and no observability. Debugging means reading SHOW CREATE PROCEDURE in production. Triggers are the worst of this — a cascade of them turns one INSERT into work nobody can see from the query log.

The workable line: use them for things that genuinely must be atomic with the write and enforced regardless of client — an audit row, a denormalized counter. Keep business logic in the application. And whatever you use, keep the definitions in migration files so they are versioned like everything else.
the function-in-WHERE trap
A user-defined function in a WHERE clause is called once per examined row, and it makes the predicate non-sargable so the index is not used (Part 9). A function that itself runs a query turns one statement into an N+1 inside the database. This is a common cause of a query that is fast on 1,000 rows and unusable on 1,000,000.

Part 9 turns to performance: writing SQL the engine can execute well, and the handful of habits that account for most query problems.