e9d8a5f | Antoine R. Dumont (@ardumont) | 17 December 2019, 12:28:42 UTC | scheduler.backend: Rename appropriately module elasticsearch_memory | 17 December 2019, 12:33:43 UTC |
2cbfb78 | Antoine R. Dumont (@ardumont) | 17 December 2019, 11:51:28 UTC | Add tests to in memory elasticsearch implementation | 17 December 2019, 12:33:43 UTC |
ba5920d | Antoine R. Dumont (@ardumont) | 17 December 2019, 11:51:13 UTC | backend_es: Add tests around elasticsearch client instantiation | 17 December 2019, 12:33:43 UTC |
38d17de | Antoine R. Dumont (@ardumont) | 17 December 2019, 11:50:13 UTC | tests/common: Remove uneeded behavior | 17 December 2019, 12:33:43 UTC |
ac32b5e | Antoine R. Dumont (@ardumont) | 17 December 2019, 09:59:19 UTC | backend: Add alternate memory elasticsearch implem to allow testing | 17 December 2019, 12:33:43 UTC |
7b1c2d5 | Antoine R. Dumont (@ardumont) | 17 December 2019, 09:57:31 UTC | scheduler.backend_es: Allow using different elasticsearch clients For the moment, only 1 official es client exists | 17 December 2019, 12:33:43 UTC |
ec207fb | Antoine R. Dumont (@ardumont) | 17 December 2019, 09:51:20 UTC | scheduler.backend: Make the returned result a dict | 17 December 2019, 12:33:42 UTC |
f97bff6 | Antoine R. Dumont (@ardumont) | 17 December 2019, 09:50:27 UTC | cli.task: Make page_token actually a string even from the cli That actually make it consistent with the api | 17 December 2019, 12:33:42 UTC |
d8859d7 | Antoine R. Dumont (@ardumont) | 16 December 2019, 16:15:42 UTC | backend_es: Add initialization endpoint | 17 December 2019, 12:33:42 UTC |
d5cea20 | Antoine R. Dumont (@ardumont) | 16 December 2019, 16:15:24 UTC | backend_es: Remove unused endpoint | 17 December 2019, 12:33:42 UTC |
18df124 | Antoine R. Dumont (@ardumont) | 16 December 2019, 16:14:54 UTC | cli.tasks: Unify logging instruction | 17 December 2019, 12:33:42 UTC |
c5e189b | Antoine R. Dumont (@ardumont) | 16 December 2019, 16:14:08 UTC | test: Allow status definition during task template generation | 17 December 2019, 12:33:42 UTC |
844f3e0 | Antoine R. Dumont (@ardumont) | 16 December 2019, 10:07:27 UTC | tests.scheduler: Extract common utility function and test it | 17 December 2019, 12:33:42 UTC |
2d56669 | Antoine R. Dumont (@ardumont) | 16 December 2019, 09:07:01 UTC | scheduler.cli.task: Rename appropriately backend variable | 17 December 2019, 12:33:42 UTC |
793c233 | Antoine R. Dumont (@ardumont) | 16 December 2019, 09:06:10 UTC | scheduler.backend_es: Rename backend class appropriately | 17 December 2019, 12:33:42 UTC |
d5bf6b1 | Antoine R. Dumont (@ardumont) | 14 December 2019, 17:44:57 UTC | cli.task: Rename internal method appropriately | 17 December 2019, 12:33:42 UTC |
eb1c3d3 | Antoine R. Dumont (@ardumont) | 14 December 2019, 17:43:21 UTC | backend_es: Use consistent logging instruction | 17 December 2019, 12:33:42 UTC |
b376eb9 | Antoine R. Dumont (@ardumont) | 14 December 2019, 17:42:14 UTC | backend_es: Enclose close instruction within finally | 17 December 2019, 12:33:42 UTC |
f6726e9 | Antoine R. Dumont (@ardumont) | 14 December 2019, 10:10:49 UTC | backend_es: Create index when it does not exist | 17 December 2019, 12:33:41 UTC |
ad54c6b | Antoine R. Dumont (@ardumont) | 14 December 2019, 09:49:59 UTC | backend_es: Open indices prior to indexing method calls | 17 December 2019, 12:33:41 UTC |
305422b | Antoine R. Dumont (@ardumont) | 14 December 2019, 09:49:35 UTC | cli.task: Tasks needs to be sorted prior to group by call | 17 December 2019, 12:33:41 UTC |
d603608 | Antoine R. Dumont (@ardumont) | 13 December 2019, 14:35:38 UTC | cli.task: Use the configuration provided by the cli | 17 December 2019, 12:33:41 UTC |
285a27c | Jenkins for Software Heritage | 17 December 2019, 11:10:15 UTC | Updated backport on buster-swh from debian/0.0.66-1_swh1 (unstable-swh) | 17 December 2019, 11:10:15 UTC |
9e4ad26 | Jenkins for Software Heritage | 17 December 2019, 11:10:14 UTC | Merge tag 'debian/0.0.66-1_swh1' into debian/buster-swh swh-scheduler Debian release 0.0.66-1~swh1 # Conflicts: # debian/changelog | 17 December 2019, 11:10:14 UTC |
6be249d | Jenkins for Software Heritage | 17 December 2019, 11:08:25 UTC | Updated debian changelog for version 0.0.66 | 17 December 2019, 11:08:25 UTC |
6c1ca59 | Jenkins for Software Heritage | 17 December 2019, 11:08:25 UTC | Update upstream source from tag 'debian/upstream/0.0.66' Update to upstream version '0.0.66' with Debian dir ea0d7bb4ca79706a12cf077ce51b65347b3869d9 | 17 December 2019, 11:08:25 UTC |
32b5d68 | Jenkins for Software Heritage | 17 December 2019, 11:08:24 UTC | New upstream version 0.0.66 | 17 December 2019, 11:08:24 UTC |
e0dd669 | Valentin Lorentz | 10 December 2019, 15:44:47 UTC | Initialize Sentry on worker startup. | 16 December 2019, 17:55:11 UTC |
f1b3f49 | Valentin Lorentz | 10 December 2019, 15:44:02 UTC | Print a traceback in case a signal callback crashes. Celery silently eats errors happening in these functions. | 16 December 2019, 17:54:54 UTC |
dbd4a2f | Antoine R. Dumont (@ardumont) | 14 December 2019, 17:25:07 UTC | backend: Align paginated endpoint consistently with others | 16 December 2019, 15:34:27 UTC |
3ab0348 | Antoine R. Dumont (@ardumont) | 14 December 2019, 13:25:17 UTC | backend: Filter properly archive within the defined range Prior to this commit, we could list tasks whose started date was null. Now we fallback on the scheduled task which is the next best date we have. | 14 December 2019, 13:25:17 UTC |
080db58 | Antoine R. Dumont (@ardumont) | 13 December 2019, 15:33:48 UTC | test_scheduler: Add some more check on filtering test | 13 December 2019, 15:34:08 UTC |
b8b171d | Antoine R. Dumont (@ardumont) | 13 December 2019, 14:07:28 UTC | backend: Make filter_task_to_archive a paginated endpoint Related to T1931 | 13 December 2019, 14:08:20 UTC |
2b93efb | Antoine R. Dumont (@ardumont) | 13 December 2019, 14:05:55 UTC | tox: Add ipdb dependency on py3-dev env | 13 December 2019, 14:08:10 UTC |
75269d4 | Jenkins for Software Heritage | 13 December 2019, 11:02:14 UTC | Updated backport on buster-swh from debian/0.0.65-1_swh2 (unstable-swh) | 13 December 2019, 11:02:14 UTC |
e03e4ab | Jenkins for Software Heritage | 13 December 2019, 11:02:14 UTC | Merge tag 'debian/0.0.65-1_swh2' into debian/buster-swh swh-scheduler Debian release 0.0.65-1~swh2 # gpg: Signature made Fri Dec 13 11:00:17 2019 UTC # gpg: using RSA key 3291900A8E20E2EF43F8F9D8DCAE90522E7122BF # gpg: Can't check signature: No public key # Conflicts: # debian/changelog | 13 December 2019, 11:02:14 UTC |
51cf8d2 | Nicolas Dandrimont | 13 December 2019, 10:58:10 UTC | Add pytest-mock build-dependency. | 13 December 2019, 10:58:10 UTC |
5d2169b | Jenkins for Software Heritage | 13 December 2019, 10:54:31 UTC | Updated debian changelog for version 0.0.65 | 13 December 2019, 10:54:31 UTC |
9ca8ebf | Jenkins for Software Heritage | 13 December 2019, 10:54:31 UTC | Update upstream source from tag 'debian/upstream/0.0.65' Update to upstream version '0.0.65' with Debian dir cb72769857093221b8c0f70696b92e5fd5f0c859 | 13 December 2019, 10:54:31 UTC |
25f03c2 | Jenkins for Software Heritage | 13 December 2019, 10:54:30 UTC | New upstream version 0.0.65 | 13 December 2019, 10:54:30 UTC |
ee162fe | Nicolas Dandrimont | 13 December 2019, 10:29:22 UTC | Use a btree of (task_type, md5(arguments)) to match task arguments The former index on hash(arguments->'args') has lost relevance as about half the tasks (the ones for the loader) have the same value (an empty list) for this field. This index is more universal, faster, and also easier to convince the planner of using. If we want more specific indexes (e.g. on specific keyword arguments) we'll be able to add that separately. | 13 December 2019, 10:32:33 UTC |
0b04220 | David Douard | 12 December 2019, 16:37:39 UTC | Remove the creation of the 'load-deposit' task type it's now managed by swh-loader-core directly. | 12 December 2019, 16:37:39 UTC |
4071d71 | David Douard | 12 December 2019, 11:27:13 UTC | Make --status option of 'swh scheduler task list' a click.Choice | 12 December 2019, 11:27:13 UTC |
a18f562 | David Douard | 04 December 2019, 09:55:25 UTC | celery: add 2 statsd probes for the runner and listener - runner: counting the number of scheduled tasks, - listener: counting the number of processed events. | 04 December 2019, 15:36:19 UTC |
f206076 | David Douard | 04 December 2019, 09:25:37 UTC | celery: make SWHTask send start/end of execution statsd gauges with timestamps Closes T2119. | 04 December 2019, 09:28:26 UTC |
08243bb | David Douard | 04 December 2019, 09:23:15 UTC | tests: fix celery_task's test_multiping kwargs were not passed correctly. Also add a test_ping_with_kw test. | 04 December 2019, 09:25:05 UTC |
9bdefcc | Antoine R. Dumont (@ardumont) | 26 November 2019, 15:18:04 UTC | d/*: Drop python3-swh.scheduler.updater package | 26 November 2019, 15:18:04 UTC |
8c1e051 | Antoine R. Dumont (@ardumont) | 26 November 2019, 11:29:47 UTC | scheduler.updater: Remove dead code | 26 November 2019, 11:29:47 UTC |
95940a8 | Nicolas Dandrimont | 21 November 2019, 12:57:25 UTC | Migrate tox.ini to extras = xxx instead of deps = .[testing] | 21 November 2019, 12:57:25 UTC |
7c40132 | Nicolas Dandrimont | 21 November 2019, 12:50:03 UTC | Merge tox test environment configurations | 21 November 2019, 12:54:47 UTC |
101a131 | David Douard | 21 November 2019, 12:50:21 UTC | Add a pre-commit config file | 21 November 2019, 12:50:21 UTC |
104fee0 | Nicolas Dandrimont | 21 November 2019, 11:04:54 UTC | Drop version constraint on pytest < 4 | 21 November 2019, 11:04:54 UTC |
56e4a12 | Nicolas Dandrimont | 20 November 2019, 18:56:38 UTC | Include all requirements in MANIFEST.in | 20 November 2019, 18:56:38 UTC |
6c46f86 | Jenkins for Software Heritage | 20 November 2019, 13:31:50 UTC | Updated backport on buster-swh from debian/0.0.64-1_swh1 (unstable-swh) | 20 November 2019, 13:31:50 UTC |
e74f3d4 | Jenkins for Software Heritage | 20 November 2019, 13:31:49 UTC | Merge tag 'debian/0.0.64-1_swh1' into debian/buster-swh swh-scheduler Debian release 0.0.64-1~swh1 # Conflicts: # debian/changelog | 20 November 2019, 13:31:49 UTC |
f7efc21 | Jenkins for Software Heritage | 20 November 2019, 13:29:37 UTC | Updated debian changelog for version 0.0.64 | 20 November 2019, 13:29:37 UTC |
08b7968 | Jenkins for Software Heritage | 20 November 2019, 13:29:37 UTC | Update upstream source from tag 'debian/upstream/0.0.64' Update to upstream version '0.0.64' with Debian dir 72863d086cb3767dc0ad0295ee5aa9f8d7d9a2c9 | 20 November 2019, 13:29:37 UTC |
64b771c | Jenkins for Software Heritage | 20 November 2019, 13:29:36 UTC | New upstream version 0.0.64 | 20 November 2019, 13:29:36 UTC |
ae5c10e | Jenkins for Software Heritage | 19 November 2019, 16:11:39 UTC | Updated backport on buster-swh from debian/0.0.63-1_swh2 (unstable-swh) | 19 November 2019, 16:11:39 UTC |
8e7f60a | Jenkins for Software Heritage | 19 November 2019, 16:11:38 UTC | Merge tag 'debian/0.0.63-1_swh2' into debian/buster-swh swh-scheduler Debian release 0.0.63-1~swh2 # gpg: Signature made Tue Nov 19 16:09:16 2019 UTC # gpg: using RSA key BF00203D741AC9D546A8BE0752E2E9840D10C3B8 # gpg: Can't check signature: No public key # Conflicts: # debian/changelog | 19 November 2019, 16:11:38 UTC |
f535d1e | Antoine R. Dumont (@ardumont) | 19 November 2019, 16:08:50 UTC | d/changelog: Bump new release | 19 November 2019, 16:08:50 UTC |
9af6ca0 | Antoine R. Dumont (@ardumont) | 19 November 2019, 15:55:02 UTC | d/control: Add missing build dependency For tests, the build depends on the python3-swh.lister | 19 November 2019, 15:55:02 UTC |
c973ec0 | Antoine R. Dumont (@ardumont) | 19 November 2019, 14:12:27 UTC | req-swh*: Remove old package loader backend names Related to T1389 T2098 Related to D2306 D2305 D2304 | 19 November 2019, 14:12:27 UTC |
74dde4d | Jenkins for Software Heritage | 19 November 2019, 13:14:59 UTC | Updated debian changelog for version 0.0.63 | 19 November 2019, 13:14:59 UTC |
2d3903e | Jenkins for Software Heritage | 19 November 2019, 13:14:59 UTC | Update upstream source from tag 'debian/upstream/0.0.63' Update to upstream version '0.0.63' with Debian dir 2bda427c1a4cb586293c806adbbead175610bebb | 19 November 2019, 13:14:59 UTC |
4cb289c | Jenkins for Software Heritage | 19 November 2019, 13:14:58 UTC | New upstream version 0.0.63 | 19 November 2019, 13:14:58 UTC |
9358572 | Antoine R. Dumont (@ardumont) | 15 November 2019, 15:02:12 UTC | swh.scheduler.cli: Add `swh scheduler task-type register` cli This allos registering of worker's task types to the scheduler through setuptools' mechanism. | 19 November 2019, 11:11:33 UTC |
5767bd5 | Jenkins for Software Heritage | 30 October 2019, 16:30:59 UTC | Updated backport on buster-swh from debian/0.0.62-1_swh1 (unstable-swh) | 30 October 2019, 16:30:59 UTC |
6ccf889 | Jenkins for Software Heritage | 30 October 2019, 16:30:59 UTC | Merge tag 'debian/0.0.62-1_swh1' into debian/buster-swh swh-scheduler Debian release 0.0.62-1~swh1 # Conflicts: # debian/changelog | 30 October 2019, 16:30:59 UTC |
8ec34fe | Nicolas Dandrimont | 23 October 2019, 08:40:37 UTC | Remove collect_ignore from conftest.py This got solved when we started using the shared_task decorator instead of instantiating our own app. | 23 October 2019, 08:40:42 UTC |
4df2406 | Nicolas Dandrimont | 18 October 2019, 15:54:04 UTC | Use the shared_task decorator instead of binding to a specific celery app | 23 October 2019, 08:32:26 UTC |
ecf38eb | David Douard | 18 October 2019, 14:37:28 UTC | celery/tests: mostly revert e770eb30 to fix celery app initialization in tests This revision did fix tests for the scheduler itself, but broke all other tests of scheduler dependent swh packages. In this fix, we ensure we override the `app` in swh.scheduler.celery_backend.config, since it is used by all celery task declarations (via the @app.task() decorator). | 18 October 2019, 14:37:28 UTC |
5185ff4 | Jenkins for Software Heritage | 18 October 2019, 11:46:26 UTC | Updated debian changelog for version 0.0.62 | 18 October 2019, 11:46:26 UTC |
9485932 | Jenkins for Software Heritage | 18 October 2019, 11:46:25 UTC | New upstream version 0.0.62 | 18 October 2019, 11:46:25 UTC |
b8be489 | Jenkins for Software Heritage | 18 October 2019, 11:46:25 UTC | Update upstream source from tag 'debian/upstream/0.0.62' Update to upstream version '0.0.62' with Debian dir c16c5c73fe26f74a2326573d98d759a1c191f90b | 18 October 2019, 11:46:25 UTC |
787c7a9 | Antoine R. Dumont (@ardumont) | 18 October 2019, 09:49:39 UTC | celery_backend.config: Make JournalHandler import optional swh-core no longer comes with JournalHandler by default. | 18 October 2019, 11:33:25 UTC |
c2a020d | David Douard | 16 October 2019, 08:50:06 UTC | tests: rewrite tests using pytest and the new rpc fixtures from swh.core | 16 October 2019, 11:20:42 UTC |
a7e15bf | David Douard | 16 October 2019, 08:46:11 UTC | add a new get_priority_ratios endpoint to the scheduler this is necessary to make it much easier to write tests so they do not need to execute SQL statements, which makes possible to run exactly the same tests with the SchedulerBackend as the RemoteScheduler one (see the following revision). | 16 October 2019, 11:20:42 UTC |
c2ccf46 | David Douard | 16 October 2019, 08:43:03 UTC | updater/tests: rewrite updater's tests as pytest functions The way the scheduler_db and updater_db fixtures are built is not very straighforward nor satisfying, but it works. | 16 October 2019, 11:20:42 UTC |
37b909e | David Douard | 16 October 2019, 08:40:28 UTC | conftest: simplify the swh_scheduler() fixture simply use the postgresql.dsn as connection string. | 16 October 2019, 08:42:04 UTC |
e770eb3 | Antoine R. Dumont (@ardumont) | 10 October 2019, 13:14:25 UTC | tests: Explicit registering test tasks step for the swh_app Prior to this commit, the celery "app" import changed. Making the runtime application load prior to the tests "swh_app". In effect, making the tasks not being consumed by workers. This explicitely forces the tests tasks registering to "swh_app". In effect clarifying code and fixing the current tests. Related D2082 Related 8eafc70 | 10 October 2019, 13:26:54 UTC |
349d23e | Antoine R. Dumont (@ardumont) | 10 October 2019, 10:05:38 UTC | scheduler: Use directly the package's server module to start server Related D2109 Related D2110 | 10 October 2019, 10:05:38 UTC |
8eafc70 | Antoine R. Dumont (@ardumont) | 10 October 2019, 09:51:27 UTC | tox.ini: Use tests installed files instead of working directory Related D2082 | 10 October 2019, 09:51:27 UTC |
2a6e2a9 | Jenkins for Software Heritage | 07 October 2019, 14:39:30 UTC | Updated backport on buster-swh from debian/0.0.61-1_swh1 (unstable-swh) | 07 October 2019, 14:39:30 UTC |
00099f4 | Jenkins for Software Heritage | 07 October 2019, 14:39:30 UTC | Merge tag 'debian/0.0.61-1_swh1' into debian/buster-swh swh-scheduler Debian release 0.0.61-1~swh1 # Conflicts: # debian/changelog | 07 October 2019, 14:39:30 UTC |
bd3b53d | Jenkins for Software Heritage | 07 October 2019, 14:37:37 UTC | Updated debian changelog for version 0.0.61 | 07 October 2019, 14:37:37 UTC |
ecefa55 | Jenkins for Software Heritage | 07 October 2019, 14:37:37 UTC | Update upstream source from tag 'debian/upstream/0.0.61' Update to upstream version '0.0.61' with Debian dir b27725dd029d671bceb2495ca4559e488038392f | 07 October 2019, 14:37:37 UTC |
7f9b26f | Jenkins for Software Heritage | 07 October 2019, 14:37:36 UTC | New upstream version 0.0.61 | 07 October 2019, 14:37:36 UTC |
5955c8d | Antoine R. Dumont (@ardumont) | 02 October 2019, 04:53:21 UTC | celery_backend/config: Fix wrong statement Dict's get method does not take keyword argument. Related D2033#47672 | 02 October 2019, 04:56:13 UTC |
f9b1484 | Jenkins for Software Heritage | 01 October 2019, 16:28:54 UTC | Updated backport on buster-swh from debian/0.0.60-1_swh2 (unstable-swh) | 01 October 2019, 16:28:54 UTC |
c27cd3a | Jenkins for Software Heritage | 01 October 2019, 16:28:53 UTC | Merge tag 'debian/0.0.60-1_swh2' into debian/buster-swh swh-scheduler Debian release 0.0.60-1~swh2 # gpg: Signature made Tue Oct 1 16:26:46 2019 UTC # gpg: using RSA key 3291900A8E20E2EF43F8F9D8DCAE90522E7122BF # gpg: Can't check signature: No public key # Conflicts: # debian/changelog | 01 October 2019, 16:28:53 UTC |
6dad1f6 | Nicolas Dandrimont | 01 October 2019, 16:15:19 UTC | Force postgresql executable to a pg_ctl that exists when running tests. | 01 October 2019, 16:26:39 UTC |
873d3c5 | Jenkins for Software Heritage | 01 October 2019, 11:22:44 UTC | Updated debian changelog for version 0.0.60 | 01 October 2019, 11:22:44 UTC |
a0fb3fa | Jenkins for Software Heritage | 01 October 2019, 11:22:43 UTC | Updated version 0.0.60 from 'debian/upstream/0.0.60' with Debian dir 87a64a11ce0817864c7788160e7dcb0664aa318b | 01 October 2019, 11:22:43 UTC |
2984a95 | Jenkins for Software Heritage | 01 October 2019, 11:22:42 UTC | New upstream version 0.0.60 | 01 October 2019, 11:22:42 UTC |
06137f0 | Stefano Zacchiroli | 01 October 2019, 11:08:23 UTC | tox: anticipate mypy run to just after flake8 | 01 October 2019, 11:08:23 UTC |
c78b846 | Stefano Zacchiroli | 27 September 2019, 08:38:31 UTC | init.py: switch to documented way of extending path make mypy 0.730 pass cleanly again | 27 September 2019, 08:38:31 UTC |
1a691b5 | Stefano Zacchiroli | 24 September 2019, 11:55:04 UTC | tox.ini: add mypy section | 24 September 2019, 11:55:04 UTC |
c4fa353 | Stefano Zacchiroli | 24 September 2019, 11:43:34 UTC | typing: minimal changes to make a no-op mypy run pass | 24 September 2019, 11:44:50 UTC |
3cd5697 | Stefano Zacchiroli | 24 September 2019, 11:42:23 UTC | fix typo in docstring and sample file name courtesy of codespell | 24 September 2019, 11:42:23 UTC |