Skip to main content

Browse the archive

https://forge.softwareheritage.org/source/swh-scheduler.git
13 March 2024, 03:48:39 UTC
Revision e051b320e4050bdc75502bf23de8b5d53d368809 authored by Nicolas Dandrimont on 09 December 2021, 13:54:09 UTC, committed by Nicolas Dandrimont on 09 December 2021, 14:16:06 UTC 
Use a temporary table to update scheduler metrics
 
When using ``insert into <...> select <...>``, PostgreSQL disables
parallel querying. Under some circumstances (in our large production
database), this makes updating the scheduler metrics take a (very) long
time.

Parallel querying is allowed for ``create table <...> as select <...>``,
and doing so restores the small(er) runtimes for this query (15 minutes
instead of multiple hours). To use that, we have to turn the function
into plpgsql instead of plain sql.
1 parent a8edbdb
Raw File
Tip revision: e051b320e4050bdc75502bf23de8b5d53d368809 authored by Nicolas Dandrimont on 09 December 2021, 13:54:09 UTC
Use a temporary table to update scheduler metrics
Tip revision: e051b32
requirements.txt 
# Add here external Python modules dependencies, one per line. Module names
# should match https://pypi.python.org/pypi names. For the full spec or
# dependency lines, see https://pip.readthedocs.org/en/1.1/requirements.html

attrs
attrs-strict
celery >= 4.3, != 5.0.3
click < 8.0
flask
humanize
pika >= 1.1.0
psycopg2
pyyaml
requests
setuptools
typing-extensions
back to top