Skip to main content

Browse the archive

https://forge.softwareheritage.org/source/swh-scheduler.git
13 March 2024, 03:48:39 UTC
Revision ee162fe9659b79694d4ce130bb4d2f4cb3b5b8d1 authored by Nicolas Dandrimont on 13 December 2019, 10:29:22 UTC, committed by Nicolas Dandrimont on 13 December 2019, 10:32:33 UTC 
Use a btree of (task_type, md5(arguments)) to match task arguments
 
The former index on hash(arguments->'args') has lost relevance as about half the
tasks (the ones for the loader) have the same value (an empty list) for this
field.

This index is more universal, faster, and also easier to convince the planner of
using.

If we want more specific indexes (e.g. on specific keyword arguments) we'll be
able to add that separately.
1 parent 0b04220
History
Tip revision: ee162fe9659b79694d4ce130bb4d2f4cb3b5b8d1 authored by Nicolas Dandrimont on 13 December 2019, 10:29:22 UTC
Use a btree of (task_type, md5(arguments)) to match task arguments
Tip revision: ee162fe
File Mode Size
bin
data
docs
sql
swh
.gitignore -rw-r--r-- 129 bytes
.pre-commit-config.yaml -rw-r--r-- 1.1 KB
AUTHORS -rw-r--r-- 112 bytes
CODE_OF_CONDUCT.md -rw-r--r-- 3.3 KB
CONTRIBUTORS -rw-r--r-- 14 bytes
LICENSE -rw-r--r-- 34.3 KB
LICENSE.Celery -rw-r--r-- 2.6 KB
MANIFEST.in -rw-r--r-- 154 bytes
Makefile -rw-r--r-- 163 bytes
README.md -rw-r--r-- 292 bytes
conftest.py -rw-r--r-- 292 bytes
mypy.ini -rw-r--r-- 505 bytes
pytest.ini -rw-r--r-- 54 bytes
requirements-swh.txt -rw-r--r-- 51 bytes
requirements-test.txt -rw-r--r-- 92 bytes
requirements.txt -rw-r--r-- 349 bytes
setup.py -rwxr-xr-x 2.2 KB
tox.ini -rw-r--r-- 588 bytes

README.md

back to top