Skip to main content

Browse the archive

https://github.com/postgres/postgres
07 April 2024, 13:56:23 UTC
sort by:
Revision Author Date Message Commit Date
5aa3189 Tom Lane 08 February 2021, 21:59:29 UTC Stamp 10.16. 08 February 2021, 21:59:29 UTC
019a3a9 Peter Eisentraut 08 February 2021, 17:07:32 UTC Translation updates Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: dde3cdf8d0aa350a829bda50a0fb498b25ea19d1 08 February 2021, 17:07:32 UTC
fde8f1d Tom Lane 07 February 2021, 20:46:38 UTC Release notes for 13.2, 12.6, 11.11, 10.16, 9.6.21, 9.5.25. 07 February 2021, 20:46:38 UTC
d6759ac Tom Lane 07 February 2021, 17:54:08 UTC Revert "Propagate CTE property flags when copying a CTE list into a rule." This reverts commit ed290896335414c6c069b9ccae1f3dcdd2fac6ba and equivalent back-branch commits. The issue is subtler than I thought, and it's far from new, so just before a release deadline is no time to be fooling with it. We'll consider what to do at a bit more leisure. Discussion: https://postgr.es/m/CAJcOf-fAdj=nDKMsRhQzndm-O13NY4dL6xGcEvdX5Xvbbi0V7g@mail.gmail.com 07 February 2021, 17:54:08 UTC
56ff63c Tom Lane 07 February 2021, 00:28:39 UTC Propagate CTE property flags when copying a CTE list into a rule. rewriteRuleAction() neglected this step, although it was careful to propagate other similar flags such as hasSubLinks or hasRowSecurity. Omitting to transfer hasRecursive is just cosmetic at the moment, but omitting hasModifyingCTE is a live bug, since the executor certainly looks at that. The proposed test case only fails back to v10, but since the executor examines hasModifyingCTE in 9.x as well, I suspect that a test case could be devised that fails in older branches. Given the nearness of the release deadline, though, I'm not going to spend time looking for a better test. Report and patch by Greg Nancarrow, cosmetic changes by me Discussion: https://postgr.es/m/CAJcOf-fAdj=nDKMsRhQzndm-O13NY4dL6xGcEvdX5Xvbbi0V7g@mail.gmail.com 07 February 2021, 00:28:39 UTC
ad85e5e Tom Lane 06 February 2021, 20:17:02 UTC Disallow converting an inheritance child table to a view. Generally, members of inheritance trees must be plain tables (or, in more recent versions, foreign tables). ALTER TABLE INHERIT rejects creating an inheritance relationship that has a view at either end. When DefineQueryRewrite attempts to convert a relation to a view, it already had checks prohibiting doing so for partitioning parents or children as well as traditional-inheritance parents ... but it neglected to check that a traditional-inheritance child wasn't being converted. Since the planner assumes that any inheritance child is a table, this led to making plans that tried to do a physical scan on a view, causing failures (or even crashes, in recent versions). One could imagine trying to support such a case by expanding the view normally, but since the rewriter runs before the planner does inheritance expansion, it would take some very fundamental refactoring to make that possible. There are probably a lot of other parts of the system that don't cope well with such a situation, too. For now, just forbid it. Per bug #16856 from Yang Lin. Back-patch to all supported branches. (In versions before v10, this includes back-patching the portion of commit 501ed02cf that added has_superclass(). Perhaps the lack of that infrastructure partially explains the missing check.) Discussion: https://postgr.es/m/16856-0363e05c6e1612fd@postgresql.org 06 February 2021, 20:17:02 UTC
c06632e Heikki Linnakangas 05 February 2021, 09:14:56 UTC Fix backslash-escaping multibyte chars in COPY FROM. If a multi-byte character is escaped with a backslash in TEXT mode input, and the encoding is one of the client-only encodings where the bytes after the first one can have an ASCII byte "embedded" in the char, we didn't skip the character correctly. After a backslash, we only skipped the first byte of the next character, so if it was a multi-byte character, we would try to process its second byte as if it was a separate character. If it was one of the characters with special meaning, like '\n', '\r', or another '\\', that would cause trouble. One such exmple is the byte sequence '\x5ca45c2e666f6f' in Big5 encoding. That's supposed to be [backslash][two-byte character][.][f][o][o], but because the second byte of the two-byte character is 0x5c, we incorrectly treat it as another backslash. And because the next character is a dot, we parse it as end-of-copy marker, and throw an "end-of-copy marker corrupt" error. Backpatch to all supported versions. Reviewed-by: John Naylor, Kyotaro Horiguchi Discussion: https://www.postgresql.org/message-id/a897f84f-8dca-8798-3139-07da5bb38728%40iki.fi 05 February 2021, 09:17:07 UTC
2671125 Tom Lane 02 February 2021, 18:49:08 UTC Fix ancient memory leak in contrib/auto_explain. The ExecutorEnd hook is invoked in a context that could be quite long-lived, not the executor's own per-query context as I think we were sort of assuming. Thus, any cruft generated while producing the EXPLAIN output could accumulate over multiple queries. This can result in spectacular leakage if log_nested_statements is on, and even without that I'm surprised nobody complained before. To fix, just switch into the executor's context so that anything we allocate will be released when standard_ExecutorEnd frees the executor state. We might as well nuke the code's retail pfree of the explain output string, too; that's laughably inadequate to the need. Japin Li, per report from Jeff Janes. This bug is old, so back-patch to all supported branches. Discussion: https://postgr.es/m/CAMkU=1wCVtbeRn0s9gt12KwQ7PLXovbpM8eg25SYocKW3BT4hg@mail.gmail.com 02 February 2021, 18:49:08 UTC
1797751 Noah Misch 30 January 2021, 08:00:27 UTC Fix CREATE INDEX CONCURRENTLY for simultaneous prepared transactions. In a cluster having used CREATE INDEX CONCURRENTLY while having enabled prepared transactions, queries that use the resulting index can silently fail to find rows. Fix this for future CREATE INDEX CONCURRENTLY by making it wait for prepared transactions like it waits for ordinary transactions. This expands the VirtualTransactionId structure domain to admit prepared transactions. It may be necessary to reindex to recover from past occurrences. Back-patch to 9.5 (all supported versions). Andrey Borodin, reviewed (in earlier versions) by Tom Lane and Michael Paquier. Discussion: https://postgr.es/m/2E712143-97F7-4890-B470-4A35142ABC82@yandex-team.ru 30 January 2021, 08:02:11 UTC
c448423 Tom Lane 28 January 2021, 22:18:23 UTC Silence another gcc 11 warning. Per buildfarm and local experimentation, bleeding-edge gcc isn't convinced that the MemSet in reorder_function_arguments() is safe. Shut it up by adding an explicit check that pronargs isn't negative, and by changing MemSet to memset. (It appears that either change is enough to quiet the warning at -O2, but let's do both to be sure.) 28 January 2021, 22:18:42 UTC
4637160 Tom Lane 28 January 2021, 16:17:13 UTC Make ecpg's rjulmdy() and rmdyjul() agree with their declarations. We had "short *mdy" in the extern declarations, but "short mdy[3]" in the actual function definitions. Per C99 these are equivalent, but recent versions of gcc have started to issue warnings about the inconsistency. Clean it up before the warnings get any more widespread. Back-patch, in case anyone wants to build older PG versions with bleeding-edge compilers. Discussion: https://postgr.es/m/2401575.1611764534@sss.pgh.pa.us 28 January 2021, 16:17:41 UTC
be2729d Alvaro Herrera 28 January 2021, 15:50:40 UTC pgbench: Remove dead code doConnect() never returns connections in state CONNECTION_BAD, so checking for that is pointless. Remove the code that does. This code has been dead since ba708ea3dc84, 20 years ago. Discussion: https://postgr.es/m/20210126195224.GA20361@alvherre.pgsql Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> 28 January 2021, 15:50:40 UTC
11a708f Alvaro Herrera 26 January 2021, 19:42:13 UTC Report the true database name on connection errors When reporting connection errors, we might show a database name in the message that's not the one we actually tried to connect to, if the database was taken from libpq defaults instead of from user parameters. Fix such error messages to use PQdb(), which reports the correct name. (But, per commit 2930c05634bc, make sure not to try to print NULL.) Apply to branches 9.5 through 13. Branch master has already been changed differently by commit 58cd8dca3de0. Reported-by: Robert Haas <robertmhaas@gmail.com> Discussion: https://postgr.es/m/CA+TgmobssJ6rS22dspWnu-oDxXevGmhMD8VcRBjmj-b9UDqRjw@mail.gmail.com 26 January 2021, 19:42:13 UTC
564cb25 Tom Lane 26 January 2021, 18:04:52 UTC Code review for psql's helpSQL() function. The loops to identify word boundaries could access past the end of the input string. Likely that would never result in an actual crash, but it makes valgrind unhappy. The logic to try different numbers of words didn't work when the input has two words but we only have a match to the first, eg "\h with select". (We must "continue" the pass loop, not "break".) The logic to compute nl_count was bizarrely managed, and in at least two code paths could end up calling PageOutput with nl_count = 0, resulting in failing to paginate output that should have been fed to the pager. Also, in v12 and up, the nl_count calculation hadn't been updated to account for the addition of a URL. The PQExpBuffer holding the command syntax details wasn't freed, resulting in a session-lifespan memory leak. While here, improve some comments, choose a more descriptive name for a variable, fix inconsistent datatype choice for another variable. Per bug #16837 from Alexander Lakhin. This code is very old, so back-patch to all supported branches. Kyotaro Horiguchi and Tom Lane Discussion: https://postgr.es/m/16837-479bcd56040c71b3@postgresql.org 26 January 2021, 18:04:52 UTC
d09e878 Tom Lane 25 January 2021, 18:03:11 UTC Fix broken ruleutils support for function TRANSFORM clauses. I chanced to notice that this dumped core due to a faulty Assert. To add insult to injury, the output has been misformatted since v11. Obviously we need some regression testing here. Discussion: https://postgr.es/m/d1cc628c-3953-4209-957b-29427acc38c8@www.fastmail.com 25 January 2021, 18:03:11 UTC
712ce07 David Rowley 25 January 2021, 07:00:16 UTC Fix hypothetical bug in heap backward scans Both heapgettup() and heapgettup_pagemode() incorrectly set the first page to scan in a backward scan in which the number of pages to scan was specified by heap_setscanlimits(). The code incorrectly started the scan at the end of the relation when startBlk was 0, or otherwise at startBlk - 1, neither of which is correct when only scanning a subset of pages. The fix here checks if heap_setscanlimits() has changed the number of pages to scan and if so we set the first page to scan as the final page in the specified range during backward scans. Proper adjustment of this code was forgotten when heap_setscanlimits() was added in 7516f5259 back in 9.5. However, practice, nowhere in core code performs backward scans after having used heap_setscanlimits(), yet, it is possible an extension uses the heap functions in this way, hence backpatch. An upcoming patch does use heap_setscanlimits() with backward scans, so this must be fixed before that can go in. Author: David Rowley Discussion: https://postgr.es/m/CAApHDvpGc9h0_oVD2CtgBcxCS1N-qDYZSeBRnUh+0CWJA9cMaA@mail.gmail.com Backpatch-through: 9.5, all supported versions 25 January 2021, 07:00:16 UTC
728971d Tom Lane 24 January 2021, 21:29:48 UTC Update time zone data files to tzdata release 2021a. DST law changes in Russia (Volgograd zone) and South Sudan. Historical corrections for Australia, Bahamas, Belize, Bermuda, Ghana, Israel, Kenya, Nigeria, Palestine, Seychelles, and Vanuatu. Notably, the Australia/Currie zone has been corrected to the point where it is identical to Australia/Hobart. 24 January 2021, 21:29:48 UTC
0fceac7 Tom Lane 22 January 2021, 23:58:25 UTC Doc: improve directions for building on macOS. In light of recent discussions, we should instruct people to install Apple's command line tools; installing Xcode is secondary. Also, fix sample command for finding out the default sysroot, as we now know that the command originally recommended can give a result that doesn't match your OS version. Also document the workaround to use if you really don't want configure to select a sysroot at all. Discussion: https://postgr.es/m/20210119111625.20435-1-james.hilliard1@gmail.com 22 January 2021, 23:58:54 UTC
0d6903e Tom Lane 22 January 2021, 16:29:43 UTC Doc: remove misleading claim in documentation of PQreset(). This text claimed that the reconnection would occur "to the same server", but there is no such guarantee in the code, nor would insisting on that be an improvement. Back-patch to v10 where multi-host connection strings were added. Discussion: https://postgr.es/m/1095901.1611268376@sss.pgh.pa.us 22 January 2021, 16:30:06 UTC
a334538 Tom Lane 20 January 2021, 17:07:23 UTC Further tweaking of PG_SYSROOT heuristics for macOS. It emerges that in some phases of the moon (perhaps to do with directory entry order?), xcrun will report that the SDK path is /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk which is normally a symlink to a version-numbered sibling directory. Our heuristic to skip non-version-numbered pathnames was rejecting that, which is the wrong thing to do. We'd still like to end up with a version-numbered PG_SYSROOT value, but we can have that by dereferencing the symlink. Like the previous fix, back-patch to all supported versions. Discussion: https://postgr.es/m/522433.1611089678@sss.pgh.pa.us 20 January 2021, 17:07:46 UTC
3ae247d Tom Lane 20 January 2021, 16:49:29 UTC Disable vacuum page skipping in selected test cases. By default VACUUM will skip pages that it can't immediately get exclusive access to, which means that even activities as harmless and unpredictable as checkpoint buffer writes might prevent a page from being processed. Ordinarily this is no big deal, but we have a small number of test cases that examine the results of VACUUM's processing and therefore will fail if the page of interest is skipped. This seems to be the explanation for some rare buildfarm failures. To fix, add the DISABLE_PAGE_SKIPPING option to the VACUUM commands in tests where this could be an issue. In passing, remove a duplicated query in pageinspect/sql/page.sql. Back-patch as necessary (some of these cases are as old as v10). Discussion: https://postgr.es/m/413923.1611006484@sss.pgh.pa.us 20 January 2021, 16:49:29 UTC
74ee3ab Michael Paquier 20 January 2021, 02:39:24 UTC Fix ALTER DEFAULT PRIVILEGES with duplicated objects Specifying duplicated objects in this command would lead to unique constraint violations in pg_default_acl or "tuple already updated by self" errors. Similarly to GRANT/REVOKE, increment the command ID after each subcommand processing to allow this case to work transparently. A regression test is added by tweaking one of the existing queries of privileges.sql to stress this case. Reported-by: Andrus Author: Michael Paquier Reviewed-by: Álvaro Herrera Discussion: https://postgr.es/m/ae2a7dc1-9d71-8cba-3bb9-e4cb7eb1f44e@hot.ee Backpatch-through: 9.5 20 January 2021, 02:39:24 UTC
fac54bd Tom Lane 19 January 2021, 18:25:33 UTC Remove faulty support for MergeAppend plan with WHERE CURRENT OF. Somebody extended search_plan_tree() to treat MergeAppend exactly like Append, which is 100% wrong, because unlike Append we can't assume that only one input node is actively returning tuples. Hence a cursor using a MergeAppend across a UNION ALL or inheritance tree could falsely match a WHERE CURRENT OF query at a row that isn't actually the cursor's current output row, but coincidentally has the same TID (in a different table) as the current output row. Delete the faulty code; this means that such a case will now return an error like 'cursor "foo" is not a simply updatable scan of table "bar"', instead of silently misbehaving. Users should not find that surprising though, as the same cursor query could have failed that way already depending on the chosen plan. (It would fail like that if the sort were done with an explicit Sort node instead of MergeAppend.) Expand the clearly-inadequate commentary to be more explicit about what this code is doing, in hopes of forestalling future mistakes. It's been like this for awhile, so back-patch to all supported branches. Discussion: https://postgr.es/m/482865.1611075182@sss.pgh.pa.us 19 January 2021, 18:25:54 UTC
0efe2a0 Bruce Momjian 18 January 2021, 23:48:25 UTC doc: adjust alignment of doc file list for "pg_waldump.sgml" Backpatch-through: 10 18 January 2021, 23:48:25 UTC
5cd94de Tom Lane 18 January 2021, 23:32:30 UTC Avoid crash with WHERE CURRENT OF and a custom scan plan. execCurrent.c's search_plan_tree() assumed that ForeignScanStates and CustomScanStates necessarily have a valid ss_currentRelation. This is demonstrably untrue for postgres_fdw's remote join and remote aggregation plans, and non-leaf custom scans might not have an identifiable scan relation either. Avoid crashing by ignoring such nodes when the field is null. This solution will lead to errors like 'cursor "foo" is not a simply updatable scan of table "bar"' in cases where maybe we could have allowed WHERE CURRENT OF to work. That's not an issue for postgres_fdw's usages, since joins or aggregations would render WHERE CURRENT OF invalid anyway. But an otherwise-transparent upper level custom scan node might find this annoying. When and if someone cares to expend work on such a scenario, we could invent a custom-scan-provider callback to determine what's safe. Report and patch by David Geier, commentary by me. It's been like this for awhile, so back-patch to all supported branches. Discussion: https://postgr.es/m/0253344d-9bdd-11c4-7f0d-d88c02cd7991@swarm64.com 18 January 2021, 23:32:55 UTC
b933c9c Noah Misch 16 January 2021, 20:21:35 UTC Fix pg_dump for GRANT OPTION among initial privileges. The context is an object that no longer bears some aclitem that it bore initially. (A user issued REVOKE or GRANT statements upon the object.) pg_dump is forming SQL to reproduce the object ACL. Since initdb creates no ACL bearing GRANT OPTION, reaching this bug requires an extension where the creation script establishes such an ACL. No PGXN extension does that. If an installation did reach the bug, pg_dump would have omitted a semicolon, causing a REVOKE and the next SQL statement to fail. Separately, since the affected code exists to eliminate an entire aclitem, it wants plain REVOKE, not REVOKE GRANT OPTION FOR. Back-patch to 9.6, where commit 23f34fa4ba358671adab16773e79c17c92cbc870 first appeared. Discussion: https://postgr.es/m/20210109102423.GA160022@rfd.leadboat.com 16 January 2021, 20:21:39 UTC
9afba9b Noah Misch 16 January 2021, 20:21:35 UTC Prevent excess SimpleLruTruncate() deletion. Every core SLRU wraps around. With the exception of pg_notify, the wrap point can fall in the middle of a page. Account for this in the PagePrecedes callback specification and in SimpleLruTruncate()'s use of said callback. Update each callback implementation to fit the new specification. This changes SerialPagePrecedesLogically() from the style of asyncQueuePagePrecedes() to the style of CLOGPagePrecedes(). (Whereas pg_clog and pg_serial share a key space, pg_serial is nothing like pg_notify.) The bug fixed here has the same symptoms and user followup steps as 592a589a04bd456410b853d86bd05faa9432cbbb. Back-patch to 9.5 (all supported versions). Reviewed by Andrey Borodin and (in earlier versions) by Tom Lane. Discussion: https://postgr.es/m/20190202083822.GC32531@gust.leadboat.com 16 January 2021, 20:21:39 UTC
c225397 Tomas Vondra 15 January 2021, 22:24:19 UTC Disallow CREATE STATISTICS on system catalogs Add a check that CREATE STATISTICS does not add extended statistics on system catalogs, similarly to indexes etc. It can be overriden using the allow_system_table_mods GUC. This bug exists since 7b504eb282c, adding the extended statistics, so backpatch all the way back to PostgreSQL 10. Author: Tomas Vondra Reported-by: Dean Rasheed Backpatch-through: 10 Discussion: https://postgr.es/m/CAEZATCXAPrrOKwEsyZKQ4uzzJQWBCt6QAvOcgqRGdWwT1zb%2BrQ%40mail.gmail.com 15 January 2021, 22:34:02 UTC
5fa060c Tom Lane 15 January 2021, 16:28:51 UTC Improve our heuristic for selecting PG_SYSROOT on macOS. In cases where Xcode is newer than the underlying macOS version, asking xcodebuild for the SDK path will produce a pointer to the SDK shipped with Xcode, which may end up building code that does not work on the underlying macOS version. It appears that in such cases, xcodebuild's answer also fails to match the default behavior of Apple's compiler: assuming one has installed Xcode's "command line tools", there will be an SDK for the OS's own version in /Library/Developer/CommandLineTools, and the compiler will default to using that. This is all pretty poorly documented, but experimentation suggests that "xcrun --show-sdk-path" gives the sysroot path that the compiler is actually using, at least in some cases. Hence, try that first, but revert to xcodebuild if xcrun fails (in very old Xcode, it is missing or lacks the --show-sdk-path switch). Also, "xcrun --show-sdk-path" may give a path that is valid but lacks any OS version identifier. We don't really want that, since most of the motivation for wiring -isysroot into the build flags at all is to ensure that all parts of a PG installation are built against the same SDK, even when considering extensions built later and/or on a different machine. Insist on finding "N.N" in the directory name before accepting the result. (Adding "--sdk macosx" to the xcrun call seems to produce the same answer as xcodebuild, but usually more quickly because it's cached, so we also try that as a fallback.) The core reason why we don't want to use Xcode's default SDK in cases like this is that Apple's technology for introducing new syscalls does not play nice with Autoconf: for example, configure will think that preadv/pwritev exist when using a Big Sur SDK, even when building on an older macOS version where they don't exist. It'd be nice to have a better solution to that problem, but this patch doesn't attempt to fix that. Per report from Sergey Shinderuk. Back-patch to all supported versions. Discussion: https://postgr.es/m/ed3b8e5d-0da8-6ebd-fd1c-e0ac80a4b204@postgrespro.ru 15 January 2021, 16:29:26 UTC
1c9791f Tom Lane 14 January 2021, 21:19:38 UTC pg_dump: label PUBLICATION TABLE ArchiveEntries with an owner. This is the same fix as commit 9eabfe300 applied to INDEX ATTACH entries, but for table-to-publication attachments. As in that case, even though the backend doesn't record "ownership" of the attachment, we still ought to label it in the dump archive with the role name that should run the ALTER PUBLICATION command. The existing behavior causes the ALTER to be done by the original role that started the restore; that will usually work fine, but there may be corner cases where it fails. The bulk of the patch is concerned with changing struct PublicationRelInfo to include a pointer to the associated PublicationInfo object, so that we can get the owner's name out of that when the time comes. While at it, I rewrote getPublicationTables() to do just one query of pg_publication_rel, not one per table. Back-patch to v10 where this code was introduced. Discussion: https://postgr.es/m/1165710.1610473242@sss.pgh.pa.us 14 January 2021, 21:19:38 UTC
8916036 Tom Lane 13 January 2021, 18:30:04 UTC Doc: clarify behavior of back-half options in pg_dump. Options that change how the archive data is converted to SQL text are ignored when dumping to archive formats. The documentation previously said "not meaningful", which is not helpful. Discussion: https://postgr.es/m/161052021249.12228.9598689907884726185@wrigleys.postgresql.org 13 January 2021, 18:30:04 UTC
636b6f9 Amit Kapila 13 January 2021, 03:01:45 UTC Fix memory leak in SnapBuildSerialize. The memory for the snapshot was leaked while serializing it to disk during logical decoding. This memory will be freed only once walsender stops streaming the changes. This can lead to a huge memory increase when master logs Standby Snapshot too frequently say when the user is trying to create many replication slots. Reported-by: funnyxj.fxj@alibaba-inc.com Diagnosed-by: funnyxj.fxj@alibaba-inc.com Author: Amit Kapila Backpatch-through: 9.5 Discussion: https://postgr.es/m/033ab54c-6393-42ee-8ec9-2b399b5d8cde.funnyxj.fxj@alibaba-inc.com 13 January 2021, 03:31:57 UTC
a2c5a43 Tom Lane 12 January 2021, 17:52:14 UTC Doc: fix description of privileges needed for ALTER PUBLICATION. Adding a table to a publication requires ownership of the table (in addition to ownership of the publication). This was mentioned nowhere. 12 January 2021, 17:52:37 UTC
6adec1e Alvaro Herrera 12 January 2021, 14:48:45 UTC Fix thinko in comment This comment has been wrong since its introduction in commit 2c03216d8311. Author: Masahiko Sawada <sawada.mshk@gmail.com> Discussion: https://postgr.es/m/CAD21AoAzz6qipFJBbGEaHmyWxvvNDp8httbwLR9tUQWaTjUs2Q@mail.gmail.com 12 January 2021, 14:48:45 UTC
09777bd Bruce Momjian 09 January 2021, 17:11:15 UTC doc: expand description of how non-SELECT queries are processed The previous description of how the executor processes non-SELECT queries was very dense, causing lack of clarity. This expanded text spells it out more simply. Reported-by: fotis.koutoupas@gmail.com Discussion: https://postgr.es/m/160912275508.676.17469511338925622905@wrigleys.postgresql.org Backpatch-through: 9.5 09 January 2021, 17:11:15 UTC
2d540ac Tom Lane 08 January 2021, 17:16:00 UTC Fix ancient bug in parsing of BRE-mode regular expressions. brenext(), when parsing a '*' quantifier, forgot to return any "value" for the token; per the equivalent case in next(), it should return value 1 to indicate that greedy rather than non-greedy behavior is wanted. The result is that the compiled regexp could behave like 'x*?' rather than the intended 'x*', if we were unlucky enough to have a zero in v->nextvalue at this point. That seems to happen with some reliability if we have '.*' at the beginning of a BRE-mode regexp, although that depends on the initial contents of a stack-allocated struct, so it's not guaranteed to fail. Found by Alexander Lakhin using valgrind testing. This bug seems to be aboriginal in Spencer's code, so back-patch all the way. Discussion: https://postgr.es/m/16814-6c5e3edd2bdf0d50@postgresql.org 08 January 2021, 17:16:00 UTC
28489f1 Tom Lane 07 January 2021, 16:45:09 UTC Further second thoughts about idle_session_timeout patch. On reflection, the order of operations in PostgresMain() is wrong. These timeouts ought to be shut down before, not after, we do the post-command-read CHECK_FOR_INTERRUPTS, to guarantee that any timeout error will be detected there rather than at some ill-defined later point (possibly after having wasted a lot of work). This is really an error in the original idle_in_transaction_timeout patch, so back-patch to 9.6 where that was introduced. 07 January 2021, 16:45:09 UTC
8358910 Michael Paquier 07 January 2021, 11:50:49 UTC doc: Remove reference to recovery params for divergence lookup in pg_rewind The documentation of pg_rewind mentioned the use of restore_command and primary_conninfo as options available at startup to fetch missing WAL segments that could be used to find the point of divergence for the rewind. This is confusing because when finding the point of divergence the target cluster is offline, so this option is not available. Issue introduced by 878bd9a, so backpatch down to 9.6. The documentation of 13 and HEAD was already right as this sentence has been changed by a7e8ece when introducing -c/--restore-target-wal. Reported-by: Amine Tengilimoglu Discussion: https://postgr.es/m/CADTdw-w_0MP=iQrfizeU4QU5fcZb+w8P_oPeLL+WznWf0kbn3w@mail.gmail.com Backpatch-through: 9.6 07 January 2021, 11:50:49 UTC
4c20e78 Fujii Masao 06 January 2021, 03:31:55 UTC Detect the deadlocks between backends and the startup process. The deadlocks that the recovery conflict on lock is involved in can happen between hot-standby backends and the startup process. If a backend takes an access exclusive lock on the table and which finally triggers the deadlock, that deadlock can be detected as expected. On the other hand, previously, if the startup process took an access exclusive lock and which finally triggered the deadlock, that deadlock could not be detected and could remain even after deadlock_timeout passed. This is a bug. The cause of this bug was that the code for handling the recovery conflict on lock didn't take care of deadlock case at all. It assumed that deadlocks involving the startup process and backends were able to be detected by the deadlock detector invoked within backends. But this assumption was incorrect. The startup process also should have invoked the deadlock detector if necessary. To fix this bug, this commit makes the startup process invoke the deadlock detector if deadlock_timeout is reached while handling the recovery conflict on lock. Specifically, in that case, the startup process requests all the backends holding the conflicting locks to check themselves for deadlocks. Back-patch to v9.6. v9.5 has also this bug, but per discussion we decided not to back-patch the fix to v9.5. Because v9.5 doesn't have some infrastructure codes (e.g., 37c54863cf) that this bug fix patch depends on. We can apply those codes for the back-patch, but since the next minor version release is the final one for v9.5, it's risky to do that. If we unexpectedly introduce new bug to v9.5 by the back-patch, there is no chance to fix that. We determined that the back-patch to v9.5 would give more risk than gain. Author: Fujii Masao Reviewed-by: Bertrand Drouvot, Masahiko Sawada, Kyotaro Horiguchi Discussion: https://postgr.es/m/4041d6b6-cf24-a120-36fa-1294220f8243@oss.nttdata.com 06 January 2021, 03:32:32 UTC
22aee14 Bruce Momjian 05 January 2021, 19:26:37 UTC doc: improve NLS instruction wording Reported-by: "Tang, Haiying" Discussion: https://postgr.es/m/bbbccf7a3c2d436e85d45869d612fd6b@G08CNEXMBPEKD05.g08.fujitsu.local Author: "Tang, Haiying" Backpatch-through: 9.5 05 January 2021, 19:26:37 UTC
0097193 Dean Rasheed 05 January 2021, 11:46:44 UTC Add an explicit cast to double when using fabs(). Commit bc43b7c2c0 used fabs() directly on an int variable, which apparently requires an explicit cast on some platforms. Per buildfarm. 05 January 2021, 11:46:44 UTC
275b190 Dean Rasheed 05 January 2021, 11:04:41 UTC Fix numeric_power() when the exponent is INT_MIN. In power_var_int(), the computation of the number of significant digits to use in the computation used log(Abs(exp)), which isn't safe because Abs(exp) returns INT_MIN when exp is INT_MIN. Use fabs() instead of Abs(), so that the exponent is cast to a double before the absolute value is taken. Back-patch to 9.6, where this was introduced (by 7d9a4737c2). Discussion: https://postgr.es/m/CAEZATCVd6pMkz=BrZEgBKyqqJrt2xghr=fNc8+Z=5xC6cgWrWA@mail.gmail.com 05 January 2021, 11:04:41 UTC
b847ec7 Bruce Momjian 02 January 2021, 18:06:24 UTC Update copyright for 2021 Backpatch-through: 9.5 02 January 2021, 18:06:24 UTC
f70e899 Tom Lane 01 January 2021, 20:51:09 UTC Doc: improve explanation of EXTRACT(EPOCH) for timestamp without tz. Try to be clearer about what computation is actually happening here. Per bug #16797 from Dana Burd. Discussion: https://postgr.es/m/16797-f264b0b980b53b8b@postgresql.org 01 January 2021, 20:51:09 UTC
7ca285c Tom Lane 30 December 2020, 22:48:43 UTC Doc: spell out comparison behaviors for the date/time types. The behavior of cross-type comparisons among date/time data types was not really explained anywhere. You could probably infer it if you recognized the applicability of comments elsewhere about datatype conversions, but it seems worthy of explicit documentation. Per bug #16797 from Dana Burd. Discussion: https://postgr.es/m/16797-f264b0b980b53b8b@postgresql.org 30 December 2020, 22:48:43 UTC
6054930 Noah Misch 30 December 2020, 09:43:43 UTC In pg_upgrade cross-version test, handle lack of oldstyle_length(). This suffices for testing v12 -> v13; some other version pairs need more changes. Back-patch to v10, which removed the function. 30 December 2020, 09:43:47 UTC
a8c68e7 Michael Paquier 29 December 2020, 09:19:20 UTC doc: Improve some grammar and sentences 90fbf7c has taken care of that for HEAD. This includes the portion of the fixes that applies to the documentation, where needed depending on the branch. Author: Justin Pryzby Discussion: https://postgr.es/m/20201227202604.GC26311@telsasoft.com Backpatch-through: 9.5 29 December 2020, 09:19:20 UTC
1c7ca61 Michael Paquier 28 December 2020, 13:17:11 UTC Fix inconsistent code with shared invalidations of snapshots The code in charge of processing a single invalidation message has been using since 568d413 the structure for relation mapping messages. This had fortunately no consequence as both locate the database ID at the same location, but it could become a problem in the future if this area of the code changes. Author: Konstantin Knizhnik Discussion: https://postgr.es/m/8044c223-4d3a-2cdb-42bf-29940840ce94@postgrespro.ru Backpatch-through: 9.5 28 December 2020, 13:17:11 UTC
744357d Fujii Masao 28 December 2020, 11:00:15 UTC postgres_fdw: Fix connection leak. In postgres_fdw, the cached connections to foreign servers will not be closed until the local session exits if the user mappings or foreign servers that those connections depend on are dropped. Those connections can be leaked. To fix that connection leak issue, after a change to a pg_foreign_server or pg_user_mapping catalog entry, this commit makes postgres_fdw close the connections depending on that entry immediately if current transaction has not used those connections yet. Otherwise, mark those connections as invalid and then close them at the end of current transaction, since they cannot be closed in the midst of the transaction using them. Closed connections will be remade at the next opportunity if necessary. Back-patch to all supported branches. Author: Bharath Rupireddy Reviewed-by: Zhihong Yu, Zhijie Hou, Fujii Masao Discussion: https://postgr.es/m/CALj2ACVNcGH_6qLY-4_tXz8JLvA+4yeBThRfxMz7Oxbk1aHcpQ@mail.gmail.com 28 December 2020, 11:00:15 UTC
d176b9d Noah Misch 25 December 2020, 19:02:56 UTC Fix back-patch of "Invalidate acl.c caches when pg_authid changes." Test script role names and error messages differed in v10, 9.6 and 9.5. Discussion: https://postgr.es/m/20201221095028.GB3777719@rfd.leadboat.com 25 December 2020, 19:02:56 UTC
d7c197f Noah Misch 25 December 2020, 18:41:59 UTC Invalidate acl.c caches when pg_authid changes. This makes existing sessions reflect "ALTER ROLE ... [NO]INHERIT" as quickly as they have been reflecting "GRANT role_name". Back-patch to 9.5 (all supported versions). Reviewed by Nathan Bossart. Discussion: https://postgr.es/m/20201221095028.GB3777719@rfd.leadboat.com 25 December 2020, 18:42:03 UTC
1a6b1c4 Tom Lane 24 December 2020, 22:00:43 UTC Fix race condition between shutdown and unstarted background workers. If a database shutdown (smart or fast) is commanded between the time some process decides to request a new background worker and the time that the postmaster can launch that worker, then nothing happens because the postmaster won't launch any bgworkers once it's exited PM_RUN state. This is fine ... unless the requesting process is waiting for that worker to finish (or even for it to start); in that case the requestor is stuck, and only manual intervention will get us to the point of being able to shut down. To fix, cancel pending requests for workers when the postmaster sends shutdown (SIGTERM) signals, and similarly cancel any new requests that arrive after that point. (We can optimize things slightly by only doing the cancellation for workers that have waiters.) To fit within the existing bgworker APIs, the "cancel" is made to look like the worker was started and immediately stopped, causing deregistration of the bgworker entry. Waiting processes would have to deal with premature worker exit anyway, so this should introduce no bugs that weren't there before. We do have a side effect that registration records for restartable bgworkers might disappear when theoretically they should have remained in place; but since we're shutting down, that shouldn't matter. Back-patch to v10. There might be value in putting this into 9.6 as well, but the management of bgworkers is a bit different there (notably see 8ff518699) and I'm not convinced it's worth the effort to validate the patch for that branch. Discussion: https://postgr.es/m/661570.1608673226@sss.pgh.pa.us 24 December 2020, 22:00:43 UTC
f3ee99d Michael Paquier 23 December 2020, 03:51:46 UTC Fix portability issues with parsing of recovery_target_xid The parsing of this parameter has been using strtoul(), which is not portable across platforms. On most Unix platforms, unsigned long has a size of 64 bits, while on Windows it is 32 bits. It is common in recovery scenarios to rely on the output of txid_current() or even the newer pg_current_xact_id() to get a transaction ID for setting up recovery_target_xid. The value returned by those functions includes the epoch in the computed result, which would cause strtoul() to fail where unsigned long has a size of 32 bits once the epoch is incremented. WAL records and 2PC data include only information about 32-bit XIDs and it is not possible to have XIDs across more than one epoch, so discarding the high bits from the transaction ID set has no impact on recovery. On the contrary, the use of strtoul() prevents a consistent behavior across platforms depending on the size of unsigned long. This commit changes the parsing of recovery_target_xid to use pg_strtouint64() instead, available down to 9.6. There is one TAP test stressing recovery with recovery_target_xid, where a tweak based on pg_reset{xlog,wal} is added to bump the XID epoch so as this change gets tested, as per an idea from Alexander Lakhin. Reported-by: Alexander Lakhin Discussion: https://postgr.es/m/16780-107fd0c0385b1035@postgresql.org Backpatch-through: 9.6 23 December 2020, 03:51:46 UTC
542248f Tom Lane 21 December 2020, 18:11:30 UTC Remove "invalid concatenation of jsonb objects" error case. The jsonb || jsonb operator arbitrarily rejected certain combinations of scalar and non-scalar inputs, while being willing to concatenate other combinations. This was of course quite undocumented. Rather than trying to document it, let's just remove the restriction, creating a uniform rule that unless we are handling an object-to-object concatenation, non-array inputs are converted to one-element arrays, resulting in an array-to-array concatenation. (This does not change the behavior for any case that didn't throw an error before.) Per complaint from Joel Jacobson. Back-patch to all supported branches. Discussion: https://postgr.es/m/163099.1608312033@sss.pgh.pa.us 21 December 2020, 18:11:30 UTC
efaeabc Tom Lane 20 December 2020, 20:28:22 UTC Doc: fix description of how to use src/tutorial files. The separate "cd" command before invoking psql made sense (or at least I thought so) when it was added in commit ed1939332. But 4e3a61635 removed the supporting text that explained when to use it, making it just confusing. So drop it. Also switch from four-dot to three-dot filler for the unsupplied part of the path, since at least one person has read the four-dot filler as a typo for "../..". And fix these/those inconsistency. Discussion: https://postgr.es/m/160837647714.673.5195186835607800484@wrigleys.postgresql.org 20 December 2020, 20:28:22 UTC
fafc6f7 Tom Lane 20 December 2020, 18:37:25 UTC Doc: improve description of pgbench script weights. Point out the workaround to be used if you want to write a script file name that includes "@". Clean up the text a little. Fabien Coelho, additional wordsmithing by me Discussion: https://postgr.es/m/1c4e81550d214741827a03292222db8d@G08CNEXMBPEKD06.g08.fujitsu.local 20 December 2020, 18:37:25 UTC
b70cab8 Tom Lane 18 December 2020, 20:46:44 UTC Avoid memcpy() with same source and destination during relmapper init. A narrow reading of the C standard says that memcpy(x,x,n) is undefined, although it's hard to envision an implementation that would really misbehave. However, analysis tools such as valgrind might whine about this; accordingly, let's band-aid relmapper.c to not do it. See also 5b630501e, d3f4e8a8a, ad7b48ea0, and other similar fixes. Apparently, none of those folk tried valgrinding initdb? This has been like this for long enough that I'm surprised it hasn't been reported before. Back-patch, just in case anybody wants to use a back branch on a platform that complains about this; we back-patched those earlier fixes too. Discussion: https://postgr.es/m/161790.1608310142@sss.pgh.pa.us 18 December 2020, 20:46:44 UTC
9595b06 Bruce Momjian 16 December 2020, 00:20:14 UTC doc: clarify COPY TO for partitioning/inheritance It was not clear how COPY TO behaved with partitioning/inheritance because the paragraphs were so far apart. Also reword to simplify. Discussion: https://postgr.es/m/20201203211723.GR24052@telsasoft.com Author: Justin Pryzby Backpatch-through: 10 16 December 2020, 00:20:14 UTC
f57c6cd Andrew Dunstan 15 December 2020, 15:00:18 UTC Use native methods to open input in TestLib::slurp_file on Windows. This is a backport of commits 114541d58e and 6f59826f0 to the remaining live branches. 15 December 2020, 15:26:10 UTC
ac7df6e Jeff Davis 15 December 2020, 07:49:29 UTC Revert "Cannot use WL_SOCKET_WRITEABLE without WL_SOCKET_READABLE." This reverts commit 3a9e64aa0d96c8ffb6c682b082d0f72b1d373327. Commit 4bad60e3 fixed the root of the problem that 3a9e64aa worked around. This enables proper pipelining of commands after terminating replication, eliminating an undocumented limitation. Discussion: https://postgr.es/m/3d57bc29-4459-578b-79cb-7641baf53c57%40iki.fi Backpatch-through: 9.5 15 December 2020, 07:49:29 UTC
ab96c4c Bruce Momjian 12 December 2020, 17:51:16 UTC initdb: properly alphabetize getopt_long options in C string Backpatch-through: 9.5 12 December 2020, 17:51:16 UTC
3470caa Tom Lane 08 December 2020, 22:50:54 UTC Teach contain_leaked_vars that assignment SubscriptingRefs are leaky. array_get_element and array_get_slice qualify as leakproof, since they will silently return NULL for bogus subscripts. But array_set_element and array_set_slice throw errors for such cases, making them clearly not leakproof. contain_leaked_vars was evidently written with only the former case in mind, as it gave the wrong answer for assignment SubscriptingRefs (nee ArrayRefs). This would be a live security bug, were it not that assignment SubscriptingRefs can only occur in INSERT and UPDATE target lists, while we only care about leakproofness for qual expressions; so the wrong answer can't occur in practice. Still, that's a rather shaky answer for a security-related question; and maybe in future somebody will want to ask about leakproofness of a tlist. So it seems wise to fix and even back-patch this correction. (We would need some change here anyway for the upcoming generic-subscripting patch, since extensions might make different tradeoffs about whether to throw errors. Commit 558d77f20 attempted to lay groundwork for that by asking check_functions_in_node whether a SubscriptingRef contains leaky functions; but that idea fails now that the implementation methods of a SubscriptingRef are not SQL-visible functions that could be marked leakproof or not.) Back-patch to 9.6. While 9.5 has the same issue, the code's a bit different. It seems quite unlikely that we'd introduce any actual bug in the short time 9.5 has left to live, so the work/risk/reward balance isn't attractive for changing 9.5. Discussion: https://postgr.es/m/3143742.1607368115@sss.pgh.pa.us 08 December 2020, 22:50:54 UTC
567d6d1 Tom Lane 08 December 2020, 18:09:48 UTC Doc: clarify that CREATE TABLE discards redundant unique constraints. The SQL standard says that redundant unique constraints are disallowed, but we long ago decided that throwing an error would be too user-unfriendly, so we just drop redundant ones. The docs weren't very clear about that though, as this behavior was only explained for PRIMARY KEY vs UNIQUE, not UNIQUE vs UNIQUE. While here, I couldn't resist doing some copy-editing and markup-fixing on the adjacent text about INCLUDE options. Per bug #16767 from Matthias vd Meent. Discussion: https://postgr.es/m/16767-1714a2056ca516d0@postgresql.org 08 December 2020, 18:09:48 UTC
b4a5d54 Tom Lane 08 December 2020, 17:06:19 UTC Doc: explain that the string types can't store \0 (ASCII NUL). This restriction was mentioned in connection with string literals, but it wasn't made clear that it's a general restriction not just a syntactic limitation in query strings. Per unsigned documentation comment. Discussion: https://postgr.es/m/160720552914.710.16625261471128631268@wrigleys.postgresql.org 08 December 2020, 17:06:19 UTC
5ba1df0 Michael Paquier 08 December 2020, 06:22:53 UTC pgcrypto: Detect errors with EVP calls from OpenSSL The following routines are called within pgcrypto when handling digests but there were no checks for failures: - EVP_MD_CTX_size (can fail with -1 as of 3.0.0) - EVP_MD_CTX_block_size (can fail with -1 as of 3.0.0) - EVP_DigestInit_ex - EVP_DigestUpdate - EVP_DigestFinal_ex A set of elog(ERROR) is added by this commit to detect such failures, that should never happen except in the event of a processing failure internal to OpenSSL. Note that it would be possible to use ERR_reason_error_string() to get more context about such errors, but these refer mainly to the internals of OpenSSL, so it is not really obvious how useful that would be. This is left out for simplicity. Per report from Coverity. Thanks to Tom Lane for the discussion. Backpatch-through: 9.5 08 December 2020, 06:22:53 UTC
d137b14 Heikki Linnakangas 07 December 2020, 12:44:34 UTC Fix more race conditions in the newly-added pg_rewind test. pg_rewind looks at the control file to check what timeline a server is on. But promotion doesn't immediately write a checkpoint, it merely writes an end-of-recovery WAL record. If pg_rewind runs immediately after promotion, before the checkpoint has completed, it will think think that the server is still on the earlier timeline. We ran into this issue a long time ago already, see commit 484a848a73f. It's a bit bogus that pg_rewind doesn't determine the timeline correctly until the end-of-recovery checkpoint has completed. We probably should fix that. But for now work around it by waiting for the checkpoint to complete before running pg_rewind, like we did in commit 484a848a73f. In the passing, tidy up the new test a little bit. Rerder the INSERTs so that the comments make more sense, remove a spurious CHECKPOINT call after pg_rewind has already run, and add --debug option, so that if this fails again, we'll have more data. Per buildfarm failure at https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=rorqual&dt=2020-12-06%2018%3A32%3A19&stg=pg_rewind-check. Backpatch to all supported versions. Discussion: https://www.postgresql.org/message-id/1713707e-e318-761c-d287-5b6a4aa807e8@iki.fi 07 December 2020, 12:55:25 UTC
45d3631 Heikki Linnakangas 04 December 2020, 16:20:18 UTC Fix race conditions in newly-added test. Buildfarm has been failing sporadically on the new test. I was able to reproduce this by adding a random 0-10 s delay in the walreceiver, just before it connects to the primary. There's a race condition where node_3 is promoted before it has fully caught up with node_1, leading to diverged timelines. When node_1 is later reconfigured as standby following node_3, it fails to catch up: LOG: primary server contains no more WAL on requested timeline 1 LOG: new timeline 2 forked off current database system timeline 1 before current recovery point 0/30000A0 That's the situation where you'd need to use pg_rewind, but in this case it happens already when we are just setting up the actual pg_rewind scenario we want to test, so change the test so that it waits until node_3 is connected and fully caught up before promoting it, so that you get a clean, controlled failover. Also rewrite some of the comments, for clarity. The existing comments detailed what each step in the test did, but didn't give a good overview of the situation the steps were trying to create. For reasons I don't understand, the test setup had to be written slightly differently in 9.6 and 9.5 than in later versions. The 9.5/9.6 version needed node 1 to be reinitialized from backup, whereas in later versions it could be shut down and reconfigured to be a standby. But even 9.5 should support "clean switchover", where primary makes sure that pending WAL is replicated to standby on shutdown. It would be nice to figure out what's going on there, but that's independent of pg_rewind and the scenario that this test tests. Discussion: https://www.postgresql.org/message-id/b0a3b95b-82d2-6089-6892-40570f8c5e60%40iki.fi 04 December 2020, 16:25:23 UTC
673b3c7 Bruce Momjian 03 December 2020, 15:28:58 UTC docs: list single-letter options first in command-line summary In a few places, the long-version options were listed before the single-letter ones in the command summary of a few commands. This didn't match other commands, and didn't match the option ordering later in the same reference page. Backpatch-through: 9.5 03 December 2020, 15:28:58 UTC
8b5c353 Heikki Linnakangas 03 December 2020, 13:57:48 UTC Fix pg_rewind bugs when rewinding a standby server. If the target is a standby server, its WAL doesn't end at the last checkpoint record, but at minRecoveryPoint. We must scan all the WAL from the last common checkpoint all the way up to minRecoveryPoint for modified pages, and also consider that portion when determining whether the server needs rewinding. Backpatch to all supported versions. Author: Ian Barwick and me Discussion: https://www.postgresql.org/message-id/CABvVfJU-LDWvoz4-Yow3Ay5LZYTuPD7eSjjE4kGyNZpXC6FrVQ%40mail.gmail.com 03 December 2020, 13:58:00 UTC
d98e27d Tom Lane 01 December 2020, 19:02:28 UTC Ensure that expandTableLikeClause() re-examines the same table. As it stood, expandTableLikeClause() re-did the same relation_openrv call that transformTableLikeClause() had done. However there are scenarios where this would not find the same table as expected. We hold lock on the LIKE source table, so it can't be renamed or dropped, but another table could appear before it in the search path. This explains the odd behavior reported in bug #16758 when cloning a table as a temp table of the same name. This case worked as expected before commit 502898192 introduced the need to open the source table twice, so we should fix it. To make really sure we get the same table, let's re-open it by OID not name. That requires adding an OID field to struct TableLikeClause, which is a little nervous-making from an ABI standpoint, but as long as it's at the end I don't think there's any serious risk. Per bug #16758 from Marc Boeren. Like the previous patch, back-patch to all supported branches. Discussion: https://postgr.es/m/16758-840e84a6cfab276d@postgresql.org 01 December 2020, 19:02:28 UTC
2270175 Thomas Munro 01 December 2020, 00:46:27 UTC Free disk space for dropped relations on commit. When committing a transaction that dropped a relation, we previously truncated only the first segment file to free up disk space (the one that won't be unlinked until the next checkpoint). Truncate higher numbered segments too, even though we unlink them on commit. This frees the disk space immediately, even if other backends have open file descriptors and might take a long time to get around to handling shared invalidation events and closing them. Also extend the same behavior to the first segment, in recovery. Back-patch to all supported releases. Bug: #16663 Reported-by: Denis Patron <denis.patron@previnet.it> Reviewed-by: Pavel Borisov <pashkin.elfe@gmail.com> Reviewed-by: Neil Chen <carpenter.nail.cz@gmail.com> Reviewed-by: David Zhang <david.zhang@highgo.ca> Discussion: https://postgr.es/m/16663-fe97ccf9932fc800%40postgresql.org 01 December 2020, 01:43:04 UTC
d3bd36a Alvaro Herrera 30 November 2020, 21:24:55 UTC Document concurrent indexes waiting on each other Because regular CREATE INDEX commands are independent, and there's no logical data dependency, it's not immediately obvious that transactions held by concurrent index builds on one table will block the second phase of concurrent index creation on an unrelated table, so document this caveat. Backpatch this all the way back. In branch master, mention that only some indexes are involved. Author: James Coleman <jtc331@gmail.com> Reviewed-by: David Johnston <david.g.johnston@gmail.com> Discussion: https://postgr.es/m/CAAaqYe994=PUrn8CJZ4UEo_S-FfRr_3ogERyhtdgHAb2WG_Ufg@mail.gmail.com 30 November 2020, 21:24:55 UTC
785f446 Tom Lane 30 November 2020, 17:22:43 UTC Fix miscomputation of direct_lateral_relids for join relations. If a PlaceHolderVar is to be evaluated at a join relation, but its value is only needed there and not at higher levels, we neglected to update the joinrel's direct_lateral_relids to include the PHV's source rel. This causes problems because join_is_legal() then won't allow joining the joinrel to the PHV's source rel at all, leading to "failed to build any N-way joins" planner failures. Per report from Andreas Seltenreich. Back-patch to 9.5 where the problem originated. Discussion: https://postgr.es/m/87blfgqa4t.fsf@aurora.ydns.eu 30 November 2020, 17:22:43 UTC
946277f Tom Lane 29 November 2020, 20:22:04 UTC Fix recently-introduced breakage in psql's \connect command. Through my misreading of what the existing code actually did, commits 85c54287a et al. broke psql's behavior for the case where "\c connstring" provides a password in the connstring. We should use that password in such a case, but as of 85c54287a we ignored it (and instead, prompted for a password). Commit 94929f1cf fixed that in HEAD, but since I thought it was cleaning up a longstanding misbehavior and not one I'd just created, I didn't back-patch it. Hence, back-patch the portions of 94929f1cf having to do with password management. In addition to fixing the introduced bug, this means that "\c -reuse-previous=on connstring" will allow re-use of an existing connection's password if the connstring doesn't change user/host/port. That didn't happen before, but it seems like a bug fix, and anyway I'm loath to have significant differences in this code across versions. Also fix an error with the same root cause about whether or not to override a connstring's setting of client_encoding. As of 85c54287a we always did so; restore the previous behavior of overriding only when stdin/stdout are a terminal and there's no environment setting of PGCLIENTENCODING. (I find that definition a bit surprising, but right now doesn't seem like the time to revisit it.) Per bug #16746 from Krzysztof Gradek. As with the previous patch, back-patch to all supported branches. Discussion: https://postgr.es/m/16746-44b30e2edf4335d4@postgresql.org 29 November 2020, 20:22:04 UTC
bc3b4c8 Tom Lane 29 November 2020, 18:58:30 UTC Doc: clarify behavior of PQconnectdbParams(). The documentation omitted the critical tidbit that a keyword-array entry is simply ignored if its corresponding value-array entry is NULL or an empty string; it will *not* override any previously-obtained value for the parameter. (See conninfo_array_parse().) I'd supposed that would force the setting back to default, which is what led me into bug #16746; but it doesn't. While here, I couldn't resist the temptation to do some copy-editing, both in the description of PQconnectdbParams() and in the section about connection URI syntax. Discussion: https://postgr.es/m/931505.1606618746@sss.pgh.pa.us 29 November 2020, 18:58:30 UTC
f6324bb Tom Lane 28 November 2020, 19:03:40 UTC Fix a recently-introduced race condition in LISTEN/NOTIFY handling. Commit 566372b3d fixed some race conditions involving concurrent SimpleLruTruncate calls, but it introduced new ones in async.c. A newly-listening backend could attempt to read Notify SLRU pages that were in process of being truncated, possibly causing an error. Also, the QUEUE_TAIL pointer could become set to a value that's not equal to the queue position of any backend. While that's fairly harmless in v13 and up (thanks to commit 51004c717), in older branches it resulted in near-permanent disabling of the queue truncation logic, so that continued use of NOTIFY led to queue-fill warnings and eventual inability to send any more notifies. (A server restart is enough to make that go away, but it's still pretty unpleasant.) The core of the problem is confusion about whether QUEUE_TAIL represents the "logical" tail of the queue (i.e., the oldest still-interesting data) or the "physical" tail (the oldest data we've not yet truncated away). To fix, split that into two variables. QUEUE_TAIL regains its definition as the logical tail, and we introduce a new variable to track the oldest un-truncated page. Per report from Mikael Gustavsson. Like the previous patch, back-patch to all supported branches. Discussion: https://postgr.es/m/1b8561412e8a4f038d7a491c8b922788@smhi.se 28 November 2020, 19:03:40 UTC
d965f69 Peter Eisentraut 25 November 2020, 08:49:00 UTC doc: Fix typos Author: Justin Pryzby <pryzby@telsasoft.com> Discussion: https://www.postgresql.org/message-id/20201121194105.GO24784@telsasoft.com 25 November 2020, 08:57:43 UTC
321c87e Andrew Gierth 24 November 2020, 20:58:32 UTC Properly check index mark/restore in ExecSupportsMarkRestore. Previously this code assumed that all IndexScan nodes supported mark/restore, which is not true since it depends on optional index AM support functions. This could lead to errors about missing support functions in rare edge cases of mergejoins with no sort keys, where an unordered non-btree index scan was placed on the inner path without a protecting Materialize node. (Normally, the fact that merge join requires ordered input would avoid this error.) Backpatch all the way since this bug is ancient. Per report from Eugen Konkov on irc. Discussion: https://postgr.es/m/87o8jn50be.fsf@news-spur.riddles.org.uk 24 November 2020, 21:18:59 UTC
1dbe627 Heikki Linnakangas 20 November 2020, 12:41:14 UTC Skip allocating hash table in EXPLAIN-only mode. This is a backpatch of commit 2cccb627f1, backpatched due to popular demand. Backpatch to all supported versions. Author: Alexey Bashtanov Discussion: https://www.postgresql.org/message-id/36823f65-050d-ae24-aa4d-a37726998240%40imap.cc 20 November 2020, 12:49:42 UTC
153bf8d Tom Lane 20 November 2020, 05:58:26 UTC On macOS, use -isysroot in link steps as well as compile steps. We previously put the -isysroot switch only into CPPFLAGS, theorizing that it was only needed to find the right copies of include files. However, it seems that we also need to use it while linking programs, to find the right stub ".tbd" files for libraries. We got away without that up to now, but apparently that was mostly luck. It may also be that failures are only observed when the Xcode version is noticeably out of sync with the host macOS version; the case that's prompting action right now is that builds fail when using latest Xcode (12.2) on macOS Catalina, even though it's fine on Big Sur. Hence, add -isysroot to LDFLAGS as well. (It seems that the more common practice is to put it in CFLAGS, whence it'd be included at both compile and link steps. However, we can't mess with CFLAGS in the platform template file without confusing configure's logic for choosing default CFLAGS.) Back-patch of 49407dc32 into all supported branches. Report and patch by James Hilliard (some cosmetic mods by me) Discussion: https://postgr.es/m/20201120003314.20560-1-james.hilliard1@gmail.com 20 November 2020, 05:58:26 UTC
f533250 Thomas Munro 19 November 2020, 21:44:09 UTC Adjust DSM and DSA slot usage constants (back-patch). 1. Previously, a DSA area would create up to four segments at each size before doubling the size. After this commit, it will create only two at each size, so it ramps up faster and therefore needs fewer slots. 2. Previously, the total limit on DSM slots allowed for 2 per connection. Switch to 5 per connection. This back-patches commit d061ea21 from release 13 into 10-12 based on a field complaint. Discussion: https://postgr.es/m/CAO03teA%2BjE1qt5iWDWzHqaufqBsF6EoOgZphnazps_tr_jDPZA%40mail.gmail.com Discussion: https://postgr.es/m/CA%2BhUKGL6H2BpGbiF7Lj6QiTjTGyTLW_vLR%3DSn2tEBeTcYXiMKw%40mail.gmail.com 19 November 2020, 21:53:08 UTC
513db7b Tom Lane 19 November 2020, 20:03:17 UTC Further fixes for CREATE TABLE LIKE: cope with self-referential FKs. Commit 502898192 was too careless about the order of execution of the additional ALTER TABLE operations generated by expandTableLikeClause. It just stuck them all at the end, which seems okay for most purposes. But it falls down in the case where LIKE is importing a primary key or unique index and the outer CREATE TABLE includes a FOREIGN KEY constraint that needs to depend on that index. Weird as that is, it used to work, so we ought to keep it working. To fix, make parse_utilcmd.c insert LIKE clauses between index-creation and FK-creation commands in the transformed list of commands, and change utility.c so that the commands generated by expandTableLikeClause are executed immediately not at the end. One could imagine scenarios where this wouldn't work either; but currently expandTableLikeClause only makes column default expressions, CHECK constraints, and indexes, and this ordering seems fine for those. Per bug #16730 from Sofoklis Papasofokli. Like the previous patch, back-patch to all supported branches. Discussion: https://postgr.es/m/16730-b902f7e6e0276b30@postgresql.org 19 November 2020, 20:03:17 UTC
214f228 Tom Lane 17 November 2020, 01:32:35 UTC Don't Insert() a VFD entry until it's fully built. Otherwise, if FDDEBUG is enabled, the debugging output fails because it tries to read the fileName, which isn't set up yet (and should in fact always be NULL). AFAICT, this has been wrong since Berkeley. Before 96bf88d52, it would accidentally fail to crash on platforms where snprintf() is forgiving about being passed a NULL pointer for %s; but the file name intended to be included in the debug output wouldn't ever have shown up. Report and fix by Greg Nancarrow. Although this is only visibly broken in custom-made builds, it still seems worth back-patching to all supported branches, as the FDDEBUG code is pretty useless as it stands. Discussion: https://postgr.es/m/CAJcOf-cUDgm9qYtC_B6XrC6MktMPNRby2p61EtSGZKnfotMArw@mail.gmail.com 17 November 2020, 01:32:35 UTC
c8961de Bruce Momjian 16 November 2020, 18:13:43 UTC doc: update bgwriter description This clarifies exactly what the bgwriter does, which should help with tuning. Reported-by: Chris Wilson Discussion: https://postgr.es/m/160399562040.7809.7335281028960123489@wrigleys.postgresql.org Backpatch-through: 9.5 16 November 2020, 18:13:43 UTC
0adbd3e Bruce Momjian 16 November 2020, 17:36:16 UTC doc: clarify how to find pg_type_d.h in the install tree Followup to patch 152ed04799. Reported-by: Alvaro Herrera Discussion: https://postgr.es/m/20201112202900.GA28098@alvherre.pgsql Backpatch-through: 9.5 16 November 2020, 17:36:16 UTC
2ed099e Bruce Momjian 16 November 2020, 16:14:54 UTC doc: adjust expression index analyze working some more This applies the fix bcbd771332 to skipped branches. Reported-by: Erik Rijkers Discussion: https://postgr.es/m/e92b3fba98a0c0f7afc0a2a37e765954@xs4all.nl Backpatch-through: 9.5-11 16 November 2020, 16:14:54 UTC
97b4f85 Bruce Momjian 16 November 2020, 15:26:16 UTC doc: improve wording of the need for analyze of exp. indexes This is a followup commit on 3370207986. Reported-by: Justin Pryzby Discussion: https://postgr.es/m/20201112211143.GL30691@telsasoft.com Backpatch-through: 9.5 16 November 2020, 15:26:16 UTC
6585770 Bruce Momjian 12 November 2020, 20:13:01 UTC doc: clarify where to find pg_type_d.h (PG 11+) and pg_type.h These files are in compiled directories and install directories. Reported-by: e.indrupskaya@postgrespro.ru Discussion: https://postgr.es/m/160379609706.24746.7506163279454026608@wrigleys.postgresql.org Backpatch-through: 9.5 12 November 2020, 20:13:01 UTC
64ac51a Bruce Momjian 12 November 2020, 20:00:44 UTC docs: mention that expression indexes need analyze Expression indexes can't benefit from pre-computed statistics on columns. Reported-by: Nikolay Samokhvalov Discussion: https://postgr.es/m/CANNMO++5rw9RDA=p40iMVbMNPaW6O=S0AFzTU=KpYHRpCd1voA@mail.gmail.com Author: Nikolay Samokhvalov, modified Backpatch-through: 9.5 12 November 2020, 20:00:44 UTC
fe904ac Bruce Momjian 12 November 2020, 19:33:28 UTC doc: wire protocol data type for history file content is bytea Document that though the history file content is marked as bytea, it is the same a text, and neither is btyea-escaped or encoding converted. Reported-by: Brar Piening Discussion: https://postgr.es/m/6a1b9cd9-17e3-df67-be55-86102af6bdf5@gmx.de Backpatch-through: 13 - 9.5 (not master) 12 November 2020, 19:33:28 UTC
068de9d Andrew Gierth 12 November 2020, 14:34:37 UTC pg_trgm: fix crash in 2-item picksplit Whether from size overflow in gistSplit or from secondary splits, picksplit is (rarely) called with exactly two items to split. Formerly, due to special-case handling of the last item, this would lead to access to an uninitialized cache entry; prior to PG 13 this might have been harmless or at worst led to an incorrect union datum, but in 13 onwards it can cause a backend crash from using an uninitialized pointer. Repair by removing the special case, which was deemed not to have been appropriate anyway. Backpatch all the way, because this bug has existed since pg_trgm was added. Per report on IRC from user "ftzdomino". Analysis and testing by me, patch from Alexander Korotkov. Discussion: https://postgr.es/m/87k0usfdxg.fsf@news-spur.riddles.org.uk 12 November 2020, 14:57:08 UTC
1186a9d Tomas Vondra 11 November 2020, 17:37:36 UTC Remove duplicate code in brin_memtuple_initialize Commit 8bf74967dab moved some of the code from brin_new_memtuple to brin_memtuple_initialize, but this resulted in some of the code being duplicate. Fix by removing the duplicate lines and backpatch to 10. Author: Tomas Vondra Backpatch-through: 10 Discussion: https://postgr.es/m/5eb50c97-9a8e-b691-8c40-1b2a55611c4c%40enterprisedb.com 11 November 2020, 17:59:41 UTC
e87139b Tom Lane 11 November 2020, 03:51:19 UTC Fix and simplify some usages of TimestampDifference(). Introduce TimestampDifferenceMilliseconds() to simplify callers that would rather have the difference in milliseconds, instead of the select()-oriented seconds-and-microseconds format. This gets rid of at least one integer division per call, and it eliminates some apparently-easy-to-mess-up arithmetic. Two of these call sites were in fact wrong: * pg_prewarm's autoprewarm_main() forgot to multiply the seconds by 1000, thus ending up with a delay 1000X shorter than intended. That doesn't quite make it a busy-wait, but close. * postgres_fdw's pgfdw_get_cleanup_result() thought it needed to compute microseconds not milliseconds, thus ending up with a delay 1000X longer than intended. Somebody along the way had noticed this problem but misdiagnosed the cause, and imposed an ad-hoc 60-second limit rather than fixing the units. This was relatively harmless in context, because we don't care that much about exactly how long this delay is; still, it's wrong. There are a few more callers of TimestampDifference() that don't have a direct need for seconds-and-microseconds, but can't use TimestampDifferenceMilliseconds() either because they do need microsecond precision or because they might possibly deal with intervals long enough to overflow 32-bit milliseconds. It might be worth inventing another API to improve that, but that seems outside the scope of this patch; so those callers are untouched here. Given the fact that we are fixing some bugs, and the likelihood that future patches might want to back-patch code that uses this new API, back-patch to all supported branches. Alexey Kondratov and Tom Lane Discussion: https://postgr.es/m/3b1c053a21c07c1ed5e00be3b2b855ef@postgrespro.ru 11 November 2020, 03:51:57 UTC
ceb883d Bruce Momjian 11 November 2020, 00:18:35 UTC doc: fix spelling "connction" to "connection" Was wrong in commit 1a9388bd0f. Reported-by: Tom Lane, Justin Pryzby Discussion: https://postgr.es/m/20201102063333.GE22691@telsasoft.com Backpatch-through: 9.5 11 November 2020, 00:18:35 UTC
253f02c Tom Lane 10 November 2020, 23:32:36 UTC Work around cross-version-upgrade issues created by commit 9e38c2bb5. Summarily changing the STYPE of regression-test aggregates that depend on array_append or array_cat is an issue for the buildfarm's cross-version-upgrade tests, because those aggregates (as defined in the back branches) now won't load into HEAD. Although this seems like only a minimal risk for genuine user-defined aggregates, we need to do something for the buildfarm. Hence, adjust the aggregate definitions, in both HEAD and the back branches. Discussion: https://postgr.es/m/1401824.1604537031@sss.pgh.pa.us Discussion: https://postgr.es/m/E1kaQ2c-0005lx-Eg@gemulon.postgresql.org 10 November 2020, 23:32:36 UTC
e6c792a Tom Lane 09 November 2020, 22:29:52 UTC Stamp 10.15. 09 November 2020, 22:29:52 UTC
b052979 Tom Lane 09 November 2020, 18:02:13 UTC Last-minute updates for release notes. Security: CVE-2020-25694, CVE-2020-25695, CVE-2020-25696 09 November 2020, 18:02:13 UTC
ca0c8ea Tom Lane 09 November 2020, 17:02:24 UTC Doc: clarify data type behavior of COALESCE and NULLIF. After studying the code, NULLIF is a lot more subtle than you might have guessed. Discussion: https://postgr.es/m/160486028730.25500.15740897403028593550@wrigleys.postgresql.org 09 November 2020, 17:02:24 UTC
a498db8 Noah Misch 09 November 2020, 15:32:09 UTC Ignore attempts to \gset into specially treated variables. If an interactive psql session used \gset when querying a compromised server, the attacker could execute arbitrary code as the operating system account running psql. Using a prefix not found among specially treated variables, e.g. every lowercase string, precluded the attack. Fix by issuing a warning and setting no variable for the column in question. Users wanting the old behavior can use a prefix and then a meta-command like "\set HISTSIZE :prefix_HISTSIZE". Back-patch to 9.5 (all supported versions). Reviewed by Robert Haas. Reported by Nick Cleaton. Security: CVE-2020-25696 09 November 2020, 15:32:13 UTC
f97ecea Noah Misch 09 November 2020, 15:32:09 UTC In security-restricted operations, block enqueue of at-commit user code. Specifically, this blocks DECLARE ... WITH HOLD and firing of deferred triggers within index expressions and materialized view queries. An attacker having permission to create non-temp objects in at least one schema could execute arbitrary SQL functions under the identity of the bootstrap superuser. One can work around the vulnerability by disabling autovacuum and not manually running ANALYZE, CLUSTER, REINDEX, CREATE INDEX, VACUUM FULL, or REFRESH MATERIALIZED VIEW. (Don't restore from pg_dump, since it runs some of those commands.) Plain VACUUM (without FULL) is safe, and all commands are fine when a trusted user owns the target object. Performance may degrade quickly under this workaround, however. Back-patch to 9.5 (all supported versions). Reviewed by Robert Haas. Reported by Etienne Stalmans. Security: CVE-2020-25695 09 November 2020, 15:32:13 UTC
back to top