Skip to main content

Browse the archive

https://forge.softwareheritage.org/source/swh-scheduler.git
13 March 2024, 03:48:39 UTC
Revision 0dba862bef1ff5b1d275dbb7363068af10207687 authored by Antoine R. Dumont (@ardumont) on 02 October 2023, 15:33:13 UTC, committed by Antoine R. Dumont (@ardumont) on 02 October 2023, 15:34:12 UTC 
Provide a default max_queue_length value to task_type
 
Because the register task type routine does not provide the value, it's left unchecked.
Once a new lister starts listing origins, the scheduler keeps on scheduling new origins
in the queue without limits. As the consumption may be slower than the production, that
tends towards too much resources usage in rabbitmq.

This should limit the issue for new deployments.
1 parent 9a91b8d
History
Tip revision: 0dba862bef1ff5b1d275dbb7363068af10207687 authored by Antoine R. Dumont (@ardumont) on 02 October 2023, 15:33:13 UTC
Provide a default max_queue_length value to task_type
Tip revision: 0dba862
File Mode Size
data
docs
sql
swh
.git-blame-ignore-revs -rw-r--r-- 140 bytes
.gitignore -rw-r--r-- 129 bytes
.pre-commit-config.yaml -rw-r--r-- 882 bytes
AUTHORS -rw-r--r-- 112 bytes
CODE_OF_CONDUCT.md -rw-r--r-- 3.3 KB
CONTRIBUTORS -rw-r--r-- 14 bytes
LICENSE -rw-r--r-- 34.3 KB
LICENSE.Celery -rw-r--r-- 2.6 KB
MANIFEST.in -rw-r--r-- 199 bytes
Makefile -rw-r--r-- 163 bytes
README.md -rw-r--r-- 292 bytes
conftest.py -rw-r--r-- 664 bytes
mypy.ini -rw-r--r-- 750 bytes
pyproject.toml -rw-r--r-- 237 bytes
pytest.ini -rw-r--r-- 62 bytes
requirements-journal.txt -rw-r--r-- 12 bytes
requirements-simulator.txt -rw-r--r-- 21 bytes
requirements-swh.txt -rw-r--r-- 48 bytes
requirements-test.txt -rw-r--r-- 301 bytes
requirements.txt -rw-r--r-- 518 bytes
setup.cfg -rw-r--r-- 342 bytes
setup.py -rwxr-xr-x 2.5 KB
tox.ini -rw-r--r-- 1.6 KB

README.md

back to top