← All digests

DBMS Weekly — 2026-11-02 (week of Nov 2–8)

Days before 19 RC2, Andrew Dunstan reverted the to_json/to_jsonb immutability change after it let JSON expression indexes go stale; anything built on 19rc1 that depends on it must be redefined before dump/restore or pg_upgrade. The 18.7/17.12/16.16/15.20/14.25 notes collect two checkpoint-race fixes, Memoize and Sort wrong results, a pg_trgm GiST fix that needs a REINDEX, and a walsender lost wakeup. AI-found bugs became the list's meta-story, from a tester holding over 1,000 to Peter Geoghegan's LLM-written rival to the nbtree prefix-compression patch. On the blogs failover dominated, alongside UUIDv4's real insert cost, a TOAST page-size trap and PostHog leaving DuckDB.

PostgreSQL

  • UUIDv4 vs UUIDv7: what a random primary key costs in Postgres — on 18.6, once the primary key outgrew shared_buffers (~26 M rows), UUIDv4 inserts ran at 0.62–0.69× an ordered key and wrote 1.5–1.9× the WAL per row; generate v7 server-side. (Nikolay Samokhvalov · postgres.ai)
  • Postgres logical replication beyond one writer — four regions of pg_auto_failover groups linked only by core logical replication: a planned failover lost nothing, kill -9 gave 2–4 duplicates and no holes, emptied synchronized_standby_slots lost 3–13 rows; DDL remains the gap. (Dimitri Fontaine · tapoueh.org) [by pg_auto_failover / pgcopydb author]
  • Fencing a Postgres primary that might still be alive — promotion alone doesn't stop an old primary that a session can switch to synchronous_commit = local, so Ubicloud cuts its network (host veth, force-detached AWS NIC, deleted GCP NIC) and promotes anyway if every fence fails. (Furkan Sahin · clickhouse.com / Ubicloud) [vendor blog — substantive]
  • Designing a reliable unplanned failover — the opposite answer: no lease; Neki fails over only when a reachable replica reports broken replication and none contradicts it, with keepalives and wal_receiver_timeout tuned so crashes, drops and freezes look alike. (Shlomi Noach · planetscale.com) [vendor blog — substantive]
  • PostgreSQL document compression: TOAST thresholds, storage policies, and page size — TOAST tries compression only past a hardcoded quarter-page gate that toast_tuple_target can't lower, so on 16/32 KB builds of 19rc1 3.9 KB and 7.7 KB JSONB documents stop compressing. (Franck Pachot · dev.to)
  • Patroni, Ubuntu 24.04, and two unplanned PostgreSQL failovers — unattended-upgrades updated zlib, needrestart restarted the custom Patroni unit, and with KillMode=control-group the stop signal reached Postgres too: two failovers in a day, traced through journal and dpkg logs. (Franck Pachot · dev.to)
  • systemd-sysusers and directory-managed Postgres accounts — PGDG RPMs now create postgres via sysusers.d at early boot, before AD/LDAP answers, so a directory-managed account gets a local twin with another UID and the cluster fails to start after reboot. (Nick Ivanov · enterprisedb.com)
  • Establishing the PostgreSQL standard: PGConf.dev 2026 part two — postgres-compat, a reference test for "Postgres-compatible": Postgres's own 200 regression tests and 117 isolation specs run unchanged against the target, plus 74 probes for silent failures and contrib behaviour. (Jimmy Angelakos · vyruss.org)
  • Building a table access method, part 7: storing and reading rows — pg_memtable gains INSERT and a full scan over its DSM arena: how slots carry rows into the AM, what a TID must encode, arena allocation, and visibility with two concurrent writers. (Mats Kindahl · dbmsdrops.kindahl.net)
  • "server closed the connection unexpectedly" in PostgreSQL — five causes behind libpq's message, each reproduced on 13–18 across six drivers and PgBouncer; no library surfaced the 57P02 crash notice, and an OOM kill logs exactly like kill -9. (Chris van Eijk · now-next.nl)
  • A month with WalShadow on ClickHouse Managed Postgres — decoding physical WAL outside Postgres, with no slot, held a 0.37–0.41 s median commit-to-visible latency up to 200,000 rows/s, while logical-replication ClickPipes tracked its 60 s sync interval. (Hidehiko Asahi · clickhouse.com) [vendor blog] [unverified: private preview]
  • Why I'm maintaining a VectorChord fork and pg_mooncake — VectorChord, silent since April, lives on as pgsty/* forks for PG 14–18; a pg_mooncake port to pg_duckdb 1.2 went upstream once Mooncake Labs said it would keep maintaining it. (Ruohang Feng · vonng.com)
  • OrioleDB v1.10.1 and v1.10.2 — releases drop betaN numbering a week after the public beta, with no storage or WAL format change. Fixes cover lost updates under REPEATABLE READ, wrong rows through bridged (GIN etc.) indexes, checkpointer and undo-chain PANICs, and decoding crashes on TOASTed tables. Transaction streaming is now disabled at decoding startup cluster-wide. (orioledb · GitHub) [release]
  • pg_duckdb 1.2.0 — moves to DuckDB 2.0.0 and is explicitly marked experimental, "not ready for production use", over unresolved user reports. Adds initial PostgreSQL 19 support, INSERT INTO <pg table> SELECT … through DuckDB, and installation into a non-public schema. (duckdb · GitHub) [release]
  • psqlODBC 18.00.0006 — a security release that "fixes multiple security issues"; the announcement points only to the changelog. (pgsql-announce) [release] [unverified: individual issues not read]
  • AgensGraph 2.18 — rebased on PostgreSQL 18.6, it adds the first ISO GQL clauses (LET, FOR, FILTER, NEXT, FINISH) and graph subqueries. Variable-length traversals now use the edge label's index at each hop; a two-hop query that timed out at 20 s takes milliseconds. (pgsql-announce) [release] [unverified: self-reported]
  • pgBadger 13.3 — no longer counts a query twice when both log_min_duration_statement and auto_explain log it. Also parses auto_explain's query parameters, fixes multiline statements and --dbuser filtering, and uses per-run temp files so concurrent runs don't collide. (pgsql-announce) [release]
  • HypoPG 1.4.5 — fixes relid handling in hypothetical indexes that could crash the backend, and a long-standing thinko in hypo_estimate_index. (pgsql-announce) [release]
  • Brume 2.1 — an Apache-2.0 tool that copies a subset of a database for testing with personal data pseudonymised. Format-preserving encryption rewrites primary keys and the foreign keys pointing at them, so referential integrity holds without disabling triggers. (pgsql-announce) [release]

PostgreSQL mailing lists

CommitFest (open: PG20-4, #63 · in progress: PG20-3, #62)

  • Balance (Nov 2–8): 37 new · 17 closed (13 committed / 4 withdrawn) → net +20 (vs +44 last week; complete counts). Every new entry went to PG20-4; 19 promotions to Ready for Committer. Active at scan: PG20-3 565 (395 needs review / 50 waiting on author / 120 ready for committer), PG20-4 37.
  • New this week: Periodically refresh database frozen IDs during autovacuum (Jingtang Zhang) — datfrozenxid advances only after a worker finishes its whole table list, so with 600,000+ tables it stays stale; v2, after Andres Freund's same-day review, refreshes it between tables and in manual VACUUM. Also Fix ReadStream handling when heap rescan strategy changes (#7579), Improve row mark handling in self-join elimination (#7598), MERGE loses UPDATE transition rows when moving rows between partitions (#7597), Add pg_stat_progress overview view (#7608).
  • Closed: committed Fix missing tts_tid in ExecForceStoreHeapTuple (#7371), Teach pg_upgrade to deal with invalid databases (#7262), Fix pg_trgm GiST union (#7385), Fix reuse of stale hashed SubPlan tables (#7384), Avoid resetting subscription two_phase state (#7576, filed and committed the same week) and 8 more; withdrawn #7528, #7584, #7585 and a duplicate.

Community pulse

Wider DBMS & distributed data

  • Next-level benchmarking overfitting — GenDB-V's 22–61× over Umbra comes from per-query structures precomputed at load time and dbgen's key layout hardcoded, while Umbra runs all 22 SF10 queries in ~540 ms. (Thomas Neumann · databasearchitects.blogspot.com)
  • How Valkey 9.0 reduces the large-object noisy-neighbor problem — 10 MB values are no longer copied into reply buffers; a 16-byte reference plus writev() on I/O threads cut neighbouring 1 KB GETs' p99.9 from 26.2 ms to 3.10 ms. (Khawaja Shams · valkey.io)
  • Can you even fsync correctly — the write path from fprintf down to NVMe FLUSH/FUA, ending in rules: fsync the parent directory after create or rename, and never retry a failed fsync, because the pages are already clean. (deferworks.org)
  • Five months treating bugs like patients and coding agents like a medical team — agents added IBM Db2 as a MOLT migration source in under two days for $4,172 in tokens; Oracle support took 9 months and ~$160k in 2024. (cockroachlabs.com) [vendor blog] [unverified: self-reported costs]
  • Apache DataFusion Comet 1.2.0 — five weeks and 399 commits spent mostly on matching Spark semantics: timestamp timezone labels, −0.0/NaN rules in hashing, min/max and sort keys, collation fallbacks. Adds opt-in native Iceberg v3 writes and an experimental native Delta scan. (Apache DataFusion PMC · datafusion.apache.org) [release]
  • CockroachDB 26.4 — skip-level upgrades between yearly .4 releases are GA. Transactional-mode logical data replication applies multi-statement transactions atomically across two data centers and sends failures to a dead-letter queue; online restore serves reads while restoring (preview). (cockroachlabs.com) [release] [vendor blog]

Commercial engines (SQL Server, Oracle, MySQL, …)

Migration experience

  • Why we're moving our warehouse product off DuckDB — five months after leaving ClickHouse for DuckDB, PostHog leaves again: at ~2 PB, DuckLake's catalog made single-table writes take over 3 minutes, so it moves to a Trino fork and its own table format. (Benjamin Knofe-Vider · posthog.com; HN 187 pts, 119 comments)

Research & cutting edge

International (non-English sources)

  • A 1C month-close went from 12 minutes to 8 hours after leaving SQL Server — the culprit was hash-join bucket estimation: COALESCE join keys have no statistics, and the planner takes the smallest per-clause bucket fraction; stock Postgres needs an index covering every join column. (alex7six · Tantor on Habr) [ru] [vendor blog — substantive] (orig: Регистр бухгалтерии, итоги и боль закрытия месяца: разбор кейса оптимизации)
  • Three storage-integrity incidents, each repaired by what the object can be rebuilt from — a broken pg_statistic TOAST value, an all-zero B-tree page, and a lost TOAST chunk that REPLICA IDENTITY FULL kept from being fixed by setting NULL. (瓶盖吃芥末 · modb.pro) [zh] (orig: PostgreSQL 三类存储完整性故障:TOAST 缺块、B-tree 零页与业务字段恢复)
  • Why one reconciliation query returns a different count every run — HAVING sum(amount) <> 0 over double precision found 19,596 accounts serially and a different ~17,800 each parallel run, because partial aggregation reorders the additions. (badcasedaily1 · OTUS on Habr) [ru] (orig: Три места, где Postgres отдаёт разные цифры на один и тот же отчёт)
  • pg_walex, an offline WAL→SQL / undo-SQL extractor, tested on PolarDB 15 — rebuilds the catalog from WAL without connecting; it auto-detected 1 GB segments and produced correct undo even at wal_level=replica, thanks to full-page images. (岳麓丹枫 · modb.pro) [zh] (orig: pg_walex —— WAL 解析为 SQL / undo SQL 工具使用经验)
  • postgres_fdw from 27 minutes to 1 second — ANALYZE on foreign tables took a join from 1,622 s to 2.2 s and fetch_size 10,000 to 1.0 s; enum filters aren't shipped, so use = ANY(array(subquery)). (伊藤 寿浩 · FORCIA on Zenn) [ja] (orig: データ連携をボトルネックにさせないためのpostgres_fdwチューニング)
  • Supabase's transaction-mode pooler carries SET into the next client — session-level set_config leaked to the next client 8 of 8 times, exposing rows an RLS policy meant to hide; set_config(…, true) never leaked. (tomozilla · Qiita) [ja] (orig: Supabaseの接続数は「掛け算」:満杯になると何から止まるか、上限の先まで試した)
  • MySQL drops loose index scan as soon as IN has two or more placeholders — 2 ms with literals, 1.6 s with bound parameters from Laravel, on both 8.0.46 and 9.2; emulated prepares restore the fast plan. (DanielSpravtsev · Habr) [ru] (orig: Один запрос: 4 мс с литералами и 3 секунды из Laravel. Как плейсхолдеры в IN выключают loose index scan в MySQL)
  • Why fallback replication from GreatSQL 8.0 to MySQL 5.7 stopped on collation #255 — a routine's binlog events carry its definition-time character_set_client, so recreate routines under a collation 5.7 knows. (GreatSQL community · modb.pro) [zh] (orig: GreatSQL 8.0回流MySQL 5.7主从复制中断分析排查过程)
  • DuckDB's parallel executor read against the morsel-driven paper and its pipeline splitting — plans carry no degree of parallelism, only thread pinning survives of the paper's NUMA design, and pipelines split at sink operators, acyclic by construction. (杨泽 / 勉仁 · Alibaba 数据库内核月报) [zh] (orig: 从 Volcano 到 Morsel:DuckDB 并行执行器的设计原理 / 树怎么变成一组流水线:DuckDB 的 Pipeline 切分)
  • Inside Oracle 7.2–9.2 WRAP — pre-10g wrapping stores the PL/SQL front end's DIANA tree with names in a plain lexicon, so V1 wrapped units can be reversed; the format was mapped from 332 reference units. (softwind · Habr) [ru] (orig: Wrap PL/SQL в Oracle 7.2–9.2: DIANA, слои преобразования и формат V1)
  • Snowflake's classic clustering looked at only the first 5 bytes of a key — that is 1.3 Japanese characters, so 東京都… prefix filters pruned to every Tokyo partition; Optima Clustering's 1 KB keys prune to one. (indigo13love · Snowflake Japan on Zenn) [ja] (orig: Snowflake: Optima Clustering による 1KB クラスタリングキーでの日本語フィルタの改善)
  • A MySQL collation let séc_allow slip past a sec_ deny-list — the accented key matched under utf8mb4_0900_ai_ci, even utf8mb4_bin ignores trailing spaces, and only utf8mb4_0900_bin kept all six variants distinct. (Container CMS team · Zenn) [ja] (orig: MySQL の照合順序で、設定キーの拒否を「séc」の1文字ですり抜けられた)_
  • Masking unique columns without duplicate checks — random replacement collides like a birthday problem (~500 pairs per 1 M rows); an 8-round AES Feistel permutation with cycle-walking keeps uniqueness and format without lookups. (shroud_tdm · Habr) [ru] (orig: Маскирование без дублей: перестановка вместо проверок на уникальность)

New sources added this week

  • Jean-François Gagné's MySQL blog — long-running MySQL replication/InnoDB blog that checks vendor benchmark framing against the numbers. (Jean-François Gagné)
  • PostHog engineering blog — first-hand warehouse-architecture reversals with production numbers. (PostHog)
  • Azure SQL dev blog — its "Under the Hood" series recaps the papers behind SQL Server features (recaps, so judged per post). (Microsoft)
  • FORCIA Tech Blog [ja] — reproducible database tuning write-ups from production. (FORCIA)
  • GreatSQL community [zh] — MySQL-fork replication and binlog case studies from production tickets, syndicated to modb.pro. (GreatSQL)

86 items · yield — mailing lists: 1,204 messages in window (703 hackers / 157 bugs / 0 performance / 23 general, plus 311 pgsql-committers and 10 pgsql-announce) → 79 shortlisted → 38 published (plus 5 releases from pgsql-announce) · blogs: ~115 posts in window → 59 shortlisted → 24 published · community: ~260 threads viewed → 32 shortlisted → 3 published (one folded into the Migration item) · research: 60 cs.DB announcements in window (none on the weekend) → 18 shortlisted → 10 published · international: ru 28→18→5, zh ~305→34→4, ja ~110→17→4, fr 1→1→0, de 0→0→0 (polyglot search unavailable this run; native feeds, site APIs and the browser used instead).