Skip to main content

Browse the archive

https://forge.softwareheritage.org/source/swh-scheduler.git
13 March 2024, 03:48:39 UTC
Revision e051b320e4050bdc75502bf23de8b5d53d368809 authored by Nicolas Dandrimont on 09 December 2021, 13:54:09 UTC, committed by Nicolas Dandrimont on 09 December 2021, 14:16:06 UTC 
Use a temporary table to update scheduler metrics
 
When using ``insert into <...> select <...>``, PostgreSQL disables
parallel querying. Under some circumstances (in our large production
database), this makes updating the scheduler metrics take a (very) long
time.

Parallel querying is allowed for ``create table <...> as select <...>``,
and doing so restores the small(er) runtimes for this query (15 minutes
instead of multiple hours). To use that, we have to turn the function
into plpgsql instead of plain sql.
1 parent a8edbdb
Raw File
Tip revision: e051b320e4050bdc75502bf23de8b5d53d368809 authored by Nicolas Dandrimont on 09 December 2021, 13:54:09 UTC
Use a temporary table to update scheduler metrics
Tip revision: e051b32
requirements-test.txt 
pytest
pytest-mock
celery >= 4.3
hypothesis >= 3.11.0
swh.lister
swh.storage[testing]
types-click
types-flask
types-pyyaml
types-requests
types-Deprecated
back to top