https://forge.softwareheritage.org/source/swh-scheduler.git

sort by:
Revision Author Date Message Commit Date
11851ef Updated backport on buster-swh from debian/1.7.0-1_swh1 (unstable-swh) 21 March 2023, 13:47:34 UTC
bb6cc8f Merge tag 'debian/1.7.0-1_swh1' into debian/buster-swh 21 March 2023, 13:47:33 UTC
9e790d4 Updated debian changelog for version 1.7.0 21 March 2023, 13:44:56 UTC
e6629da Update upstream source from tag 'debian/upstream/1.7.0' Update to upstream version '1.7.0' with Debian dir 5ffe6676732b6589e3350b3f65ec83e5b32720ea 21 March 2023, 13:44:55 UTC
707698a New upstream version 1.7.0 21 March 2023, 13:44:54 UTC
5936ae1 add-forge-now: Allow scheduling of cgit task type Refs. swh/infra/sysadm-environment#4813 21 March 2023, 12:00:04 UTC
c24b0c8 mypy: Bump to 1.0.1 and fix new typing errors Related to swh/meta#4960 17 February 2023, 16:59:03 UTC
4cb605e Update and clean tox configuration for version 4 Related to swh/meta#4959 16 February 2023, 16:10:00 UTC
e33d0ad pre-commit: Bump isort from 5.10.1 to 5.11.5 This fixes python 3.7 support due to poetry, a dependency of isort, that removed support for that Python version in a recent release. 02 February 2023, 10:07:36 UTC
72ab97a Updated backport on buster-swh from debian/1.6.0-1_swh1 (unstable-swh) 31 January 2023, 18:26:33 UTC
8cb8bab Merge tag 'debian/1.6.0-1_swh1' into debian/buster-swh 31 January 2023, 18:26:32 UTC
195b832 Updated debian changelog for version 1.6.0 31 January 2023, 17:19:09 UTC
c57e758 Update upstream source from tag 'debian/upstream/1.6.0' Update to upstream version '1.6.0' with Debian dir 0eaa4da2fc97d458abc4a235f88c53ad26c969b8 31 January 2023, 17:19:08 UTC
cb428ae New upstream version 1.6.0 31 January 2023, 17:19:07 UTC
9beef90 Configure logging from environment variable SWH_LOG_CONFIG When not provided, this uses the logging configuration coded in the scheduler (as before). Refs. swh/infra/sysadm-environment#4524 31 January 2023, 16:59:14 UTC
bebf298 swh.scheduler.cli: Pass initialization exceptions to subcommands 30 January 2023, 15:27:11 UTC
7d7eeb2 Updated backport on buster-swh from debian/1.5.1-1_swh1 (unstable-swh) 27 January 2023, 11:34:07 UTC
7d91381 Merge tag 'debian/1.5.1-1_swh1' into debian/buster-swh 27 January 2023, 11:34:06 UTC
3546c1c Updated debian changelog for version 1.5.1 27 January 2023, 11:26:14 UTC
9cf76b6 Update upstream source from tag 'debian/upstream/1.5.1' Update to upstream version '1.5.1' with Debian dir c4cdd8eb7810cfde921d809d5bf37bddfc587c53 27 January 2023, 11:26:13 UTC
f7947dd New upstream version 1.5.1 27 January 2023, 11:26:12 UTC
a65c4ed celery_backend/config: Fix missing comma in setup_log_handler Because of that missing comma, an exception was raised (tuple object is not callable) but it was caught and displayed by the _print_errors decorator so tests could not detect it. As a consequence, the logging configuration of celery workers was broken. Add a test to check if an exception was raised by the setup_log_handler function to avoid bad surprises when deploying to production or in docker. 26 January 2023, 15:11:11 UTC
7d3e9ae require pytest-postgresql < 4.0.0 25 January 2023, 13:48:56 UTC
037946a Add missing dependency on pytest-postgresql It is used by the pytest plugin 25 January 2023, 13:37:42 UTC
849bc08 Updated backport on buster-swh from debian/1.5.0-1_swh1 (unstable-swh) 24 January 2023, 13:30:48 UTC
a3cbfb6 Merge tag 'debian/1.5.0-1_swh1' into debian/buster-swh 24 January 2023, 13:30:47 UTC
d68d03c Updated debian changelog for version 1.5.0 24 January 2023, 13:26:22 UTC
00fd130 Update upstream source from tag 'debian/upstream/1.5.0' Update to upstream version '1.5.0' with Debian dir 5c5abea93e5496c1e4ce76325e777e811f41bb4a 24 January 2023, 13:26:21 UTC
5a97137 New upstream version 1.5.0 24 January 2023, 13:26:20 UTC
8f0849a Allow logging configuration from configuration yaml file This will allow proper logging configuration for the services which are currently running in the dynamic infrastructure. Their logs are current written in the wrong elasticsearch indices. Ref. swh/infra/sysadm-environment#4524 23 January 2023, 17:03:12 UTC
fccf944 Add missing __init__.py so find_packages keep finding sql modules Otherwise, at some point, this will get discarded as per the debian build warning [1] [1] https://jenkins.softwareheritage.org/view/swh-debian%20(draft)/job/debian/job/packages/job/DSCH/job/gbp-buildpackage/182/console 02 January 2023, 09:21:57 UTC
d521ab7 docs: Include module indices only when building standalone package doc In order to remove warnings about /apidoc/*.rst files being included multiple times in toc when building full swh documentation, prefer to include module indices only when building standalone package documentation. Also include them the proper sphinx way. Related to T4496 19 December 2022, 14:10:54 UTC
761bd63 Updated backport on buster-swh from debian/1.4.0-1_swh1 (unstable-swh) 14 December 2022, 13:00:44 UTC
c8e09df Merge tag 'debian/1.4.0-1_swh1' into debian/buster-swh 14 December 2022, 13:00:44 UTC
3ca9293 Updated debian changelog for version 1.4.0 12 December 2022, 10:51:31 UTC
0f46f3a Update upstream source from tag 'debian/upstream/1.4.0' Update to upstream version '1.4.0' with Debian dir 0fc297ff329f9f004f363b713e958febf0acc324 12 December 2022, 10:51:30 UTC
76030a1 New upstream version 1.4.0 12 December 2022, 10:51:30 UTC
8e125f1 cli.add_forge_now: Open `register-lister` with sensible defaults This will ease scheduling of new add-forge-now requests, on: - staging: this will list a subset of disabled origins once - production: this will register recurring tasks (full, incremental if any) to list that new forge This also unifies the previous subcommand schedule-first-visits with the --preset flag. So, the following would be enough to list appropriately in staging/production: ``` swh scheduler add-forge-now \ ( --preset [production|staging] \ # to enable a pre-defined set of rules ) register-lister \ gitea \ url=https://git.afpy.org/api/v1/ ``` Related to https://gitlab.softwareheritage.org/infra/sysadm-environment/-/issues/4674 08 December 2022, 17:51:45 UTC
1c34e98 cli.add_forge_now: Open `schedule-first-visits` with sensible defaults This should ease scheduling the first visits for add-forge-now request. The following would be enough to fetch and schedule the forge just listed (be it in production or staging): ``` swh scheduler add-forge-now \ schedule-first-visits \ --visit-type git \ (--visit-type svn \ # if a lister lists multiple kinds of visit, we can mention it ) --lister-name gitea \ --lister-instance-name git.afpy.org \ ( --production | --staging ) # to list only enabled | disabled origins ``` Related to https://gitlab.softwareheritage.org/infra/sysadm-environment/-/issues/4674 07 December 2022, 15:44:28 UTC
03c0d1b Updated debian changelog for version 1.3.0 07 December 2022, 12:50:33 UTC
80c1df4 Update upstream source from tag 'debian/upstream/1.3.0' Update to upstream version '1.3.0' with Debian dir 4b8c2ba3ff41fa515c60acf0dd8a33c9e97e7600 07 December 2022, 12:50:32 UTC
354f2d4 New upstream version 1.3.0 07 December 2022, 12:50:31 UTC
e2878b5 task add: Ensure task type provided exist and raise otherwise Related to https://gitlab.softwareheritage.org/infra/sysadm-environment/-/issues/4674 07 December 2022, 11:57:04 UTC
cd16fce grab_next_visits: Open lister name and instance name filtering Related to https://gitlab.softwareheritage.org/infra/sysadm-environment/-/issues/4674 06 December 2022, 16:03:32 UTC
a776963 send-to-celery: Adapt to schedule from lister name & instance_name This allows to bypass the lister id retrieval step using directly the name and instance name of the lister to discover the uuid. This also drops the --lister-uuid flag which is somewhat difficult to use. Related to https://gitlab.softwareheritage.org/infra/sysadm-environment/-/issues/4674 06 December 2022, 15:54:02 UTC
ff75e74 Ensure origins are not visited faster than twice a day The scheduled_cooldown only applies to tasks that have not been executed yet. absolute_cooldown avoids archiving objects faster than that. 25 October 2022, 14:48:51 UTC
1f9109f Refresh task type data from the database every time recurrent tasks are run Avoids inconsistencies between the database state and an ongoing recurrent task scheduler. 25 October 2022, 14:48:51 UTC
bde27a9 Use json instead of msgpack for serializers Recent celery versions generate serialized messages with mime types incompatible with older versions when using msgpack 25 October 2022, 13:51:01 UTC
aeb870a pre-commit, tox: Bump pre-commit, codespell, black and flake8 - pre-commit from 4.1.0 to 4.3.0, - codespell from 2.2.1 to 2.2.2, - black from 22.3.0 to 22.10.0 and - flake8 from 4.0.1 to 5.0.4. Also freeze flake8 dependencies. Also change flake8's repo config to github (the gitlab mirror being outdated). 18 October 2022, 16:53:38 UTC
d589194 Updated backport on buster-swh from debian/1.2.3-1_swh1 (unstable-swh) 03 October 2022, 12:10:15 UTC
91b2f57 Merge tag 'debian/1.2.3-1_swh1' into debian/buster-swh 03 October 2022, 12:10:15 UTC
87ff3db Updated debian changelog for version 1.2.3 03 October 2022, 12:07:44 UTC
edc1a2e Update upstream source from tag 'debian/upstream/1.2.3' Update to upstream version '1.2.3' with Debian dir 692868b76356cffe70463fb90e278de55a0e035c 03 October 2022, 12:07:43 UTC
0929c07 New upstream version 1.2.3 03 October 2022, 12:07:42 UTC
17c6d48 Fix compatibility issue with latest dependency version This currently fails all swh related builds which depend on the celery/kombu stack due to that dependency's latest version release. 03 October 2022, 11:58:46 UTC
6d0b1d1 backend: Prevent query exception when lister ids is empty Related to T4545 23 September 2022, 07:49:04 UTC
9e68b73 Updated backport on buster-swh from debian/1.2.2-1_swh1 (unstable-swh) 15 September 2022, 12:02:44 UTC
d621c90 Merge tag 'debian/1.2.2-1_swh1' into debian/buster-swh 15 September 2022, 12:02:44 UTC
4604ff8 Updated debian changelog for version 1.2.2 15 September 2022, 12:00:17 UTC
889cf7b Update upstream source from tag 'debian/upstream/1.2.2' Update to upstream version '1.2.2' with Debian dir fb5e868766d116ab788888d47d0f6258440a68b2 15 September 2022, 12:00:16 UTC
cfa5a6f New upstream version 1.2.2 15 September 2022, 12:00:15 UTC
b1afdab recurrent_visits: Allow to set no origins scheduled backoff in config The send_visits_for_visit_type function uses a default schedule backoff of 20 minutes where there is no origins to schedule for a given visit type. It exists use cases when we would like that schedule backoff to be shorter in order to schedule listed origins for loading into the archive more rapidly, typically in the docker environment. So allow to set that backoff value through configuration. 15 September 2022, 08:41:20 UTC
7cfaa98 sql/Makefile: Fix swh-scheduler SQL file paths Those files have been renamed so the database could not be filled. 22 August 2022, 13:19:50 UTC
fd6df6a api/server: Clarify load and check configuration backend This adds type to the function, update its docstring and clarify its associated tests as well. 29 July 2022, 08:12:23 UTC
07bd0b2 Updated backport on buster-swh from debian/1.2.1-1_swh1 (unstable-swh) 08 July 2022, 14:57:04 UTC
f5a9137 Merge tag 'debian/1.2.1-1_swh1' into debian/buster-swh 08 July 2022, 14:57:03 UTC
4b6972c Updated debian changelog for version 1.2.1 08 July 2022, 14:53:11 UTC
b462eab Update upstream source from tag 'debian/upstream/1.2.1' Update to upstream version '1.2.1' with Debian dir 8876e6480e8b880ad9f16a682166f4ad7951a25b 08 July 2022, 14:53:11 UTC
58b365e New upstream version 1.2.1 08 July 2022, 14:53:10 UTC
d847448 Fix the load_and_check_config() function to support the "postgresql" cls value and replace usage of the "local" scheduler cls with "postgresql" everywhere. 08 July 2022, 12:23:46 UTC
7f60f11 Updated backport on buster-swh from debian/1.2.0-1_swh1 (unstable-swh) 03 June 2022, 13:52:19 UTC
d7d7aeb Merge tag 'debian/1.2.0-1_swh1' into debian/buster-swh 03 June 2022, 13:52:14 UTC
d8bc426 Updated debian changelog for version 1.2.0 03 June 2022, 13:47:47 UTC
8ebdc1a Update upstream source from tag 'debian/upstream/1.2.0' Update to upstream version '1.2.0' with Debian dir 5308e2d13ddbbe929fe3a61aa2a483e605787857 03 June 2022, 13:47:45 UTC
ad7ca47 New upstream version 1.2.0 03 June 2022, 13:47:44 UTC
0496c39 Remove unused get_current_version method Attribute current_version is already set and directly used by swh db [version|init|upgrade] clis. Related to T4305 03 June 2022, 12:44:56 UTC
ef15385 tests: use stock pytest_postgresql factory function instead of (soon-to-be-deprecated) swh-core's postgresql_fact one. 31 May 2022, 14:46:05 UTC
1e34659 Updated backport on buster-swh from debian/1.1.2-1_swh1 (unstable-swh) 12 May 2022, 11:58:56 UTC
bc1a6cb Merge tag 'debian/1.1.2-1_swh1' into debian/buster-swh 12 May 2022, 11:58:55 UTC
4e04ccf Updated debian changelog for version 1.1.2 12 May 2022, 11:55:14 UTC
b2e342f Update upstream source from tag 'debian/upstream/1.1.2' Update to upstream version '1.1.2' with Debian dir 77f8815b707fded0e03cae23a4909d7f281a2e97 12 May 2022, 11:55:13 UTC
407dd3d New upstream version 1.1.2 12 May 2022, 11:55:12 UTC
e56fc4d interface: Return enabled origins only by default in get_listed_origins Add a new enabled_only parameter set to True by default in get_listed_origins scheduler method. It enables to filter out by default disabled listed origins when requesting the result of a listing and avoid possible errors in listers implementation. 12 May 2022, 10:07:17 UTC
c7c53ea add strict asyncio_mode in pytest.ini 09 May 2022, 10:13:54 UTC
1d50b2e cli/task: Fix sphinx >= 4.4 warning Fix "more than one target found for cross-reference 'Origin'" sphinx warning. 06 May 2022, 15:06:23 UTC
881b521 Add missing sentry captures 28 April 2022, 13:59:44 UTC
5c68ac1 Updated backport on buster-swh from debian/1.1.1-1_swh1 (unstable-swh) 28 April 2022, 09:40:07 UTC
3b23f58 Merge tag 'debian/1.1.1-1_swh1' into debian/buster-swh 28 April 2022, 09:40:06 UTC
f092ed3 Updated debian changelog for version 1.1.1 28 April 2022, 09:36:24 UTC
23ce0d9 Update upstream source from tag 'debian/upstream/1.1.1' Update to upstream version '1.1.1' with Debian dir 51b9198d0925a58c5f477ee300095bd0c9e8f9b6 28 April 2022, 09:36:23 UTC
d9e982e New upstream version 1.1.1 28 April 2022, 09:36:23 UTC
82274c1 cli/utils: Fix parsing of empty strings 27 April 2022, 13:15:28 UTC
353cf2a Bump mypy to v0.942 26 April 2022, 11:05:15 UTC
a79109d Updated backport on buster-swh from debian/1.1.0-1_swh1 (unstable-swh) 26 April 2022, 10:39:25 UTC
053d425 Merge tag 'debian/1.1.0-1_swh1' into debian/buster-swh 26 April 2022, 10:39:24 UTC
f642da4 Updated debian changelog for version 1.1.0 26 April 2022, 10:35:52 UTC
d912c65 Update upstream source from tag 'debian/upstream/1.1.0' Update to upstream version '1.1.0' with Debian dir 728c35186bf7d46bb2e39efbe69cf3e4981c7311 26 April 2022, 10:35:51 UTC
442fcdb New upstream version 1.1.0 26 April 2022, 10:35:50 UTC
0365b85 Add a 'lister_instance_name' argument to all tasks created from ListedOrigin This will allow loaders to use the right API credentials to fetch extrinsic metadata for the origin from the forge. 26 April 2022, 10:28:37 UTC
42e362d Add a 'lister_name' argument to all tasks created from ListedOrigin This will allow loaders to guess the forge type, and use the right API to fetch extrinsic metadata for the origin from the forge. 26 April 2022, 10:28:33 UTC
back to top