| bd3da9b | Antoine R. Dumont (@ardumont) | 26 April 2018, 15:34:07 UTC | New upstream version 0.0.26 | 26 April 2018, 15:34:07 UTC |
| ffd2dda | Antoine R. Dumont (@ardumont) | 26 April 2018, 15:33:52 UTC | d/rules: Fix package build | 26 April 2018, 15:33:52 UTC |
| 950a7d5 | Antoine R. Dumont (@ardumont) | 25 April 2018, 14:10:26 UTC | swh.scheduler.tests: Test remote scheduler api as well Related T1036 | 25 April 2018, 16:37:43 UTC |
| 6ef0a88 | Antoine R. Dumont (@ardumont) | 25 April 2018, 11:03:44 UTC | swh.scheduler: Add tests around removing archivable tasks Related T986 Related T1034 | 25 April 2018, 16:36:34 UTC |
| 7afd050 | Antoine R. Dumont (@ardumont) | 25 April 2018, 09:57:17 UTC | swh.scheduler: Add tests around filtering archivable tasks Related T986 Related T1034 | 25 April 2018, 16:36:34 UTC |
| 5ccfa8b | Antoine R. Dumont (@ardumont) | 25 April 2018, 09:02:02 UTC | swh-scheduler-schema: Fix unneeded drop instructions Should not have been committed in that file, only in the migration part. | 25 April 2018, 16:36:34 UTC |
| b7490ee | Antoine R. Dumont (@ardumont) | 24 April 2018, 14:48:05 UTC | swh.scheduler.cli: Improve docstring | 25 April 2018, 16:36:34 UTC |
| 8c3910a | Antoine R. Dumont (@ardumont) | 24 April 2018, 14:26:13 UTC | swh.scheduler.cli: Permit to specify the backend to use in cli Related T1034 | 25 April 2018, 16:36:34 UTC |
| 9e13fd2 | Antoine R. Dumont (@ardumont) | 24 April 2018, 13:40:33 UTC | swh.scheduler.api: Bootstrap scheduler's remote api Related T1034 | 25 April 2018, 16:36:34 UTC |
| 5349689 | Antoine R. Dumont (@ardumont) | 24 April 2018, 12:08:28 UTC | swh.scheduler: Use `get_scheduler` api to instantiate a scheduler Related T1034 | 24 April 2018, 14:55:43 UTC |
| ef4eb14 | Antoine R. Dumont (@ardumont) | 24 April 2018, 12:07:48 UTC | swh.scheduler.backend: Fix docstring | 24 April 2018, 12:07:48 UTC |
| 7d95da2 | Antoine R. Dumont (@ardumont) | 18 April 2018, 10:34:43 UTC | New upstream version 0.0.25 | 18 April 2018, 10:34:43 UTC |
| 8124229 | Antoine R. Dumont (@ardumont) | 18 April 2018, 09:33:45 UTC | swh.scheduler.cli.archive: Index arguments.kwargs as text Related T1023 | 18 April 2018, 09:33:45 UTC |
| b67f570 | Antoine R. Dumont (@ardumont) | 13 April 2018, 12:55:32 UTC | New upstream version 0.0.24 | 13 April 2018, 12:55:32 UTC |
| f4587a3 | Antoine R. Dumont (@ardumont) | 13 April 2018, 09:45:32 UTC | data/template: Do not index the arguments field (it's in _source) As this field is randomly large depending on the task at end, this triggers a limit (index.mapping.total_fields.limit to 1000). We do not really need this in the index as the data will still be in the _source. [1] https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html#mapping-limit-settings Related T1023 | 13 April 2018, 09:59:37 UTC |
| 915db34 | Antoine R. Dumont (@ardumont) | 13 April 2018, 09:10:30 UTC | data/README: Add a small readme to explain es install step | 13 April 2018, 09:59:36 UTC |
| 547eb89 | Antoine R. Dumont (@ardumont) | 13 April 2018, 09:07:51 UTC | swh.scheduler.cli: Add a bulk index flag to separate read from index Try to reduce the number of connection timeout error. There are 3 solutions: - either increase the default timeout of 10 per request - same on server configuration basis (no) - either reduce the number of data to bulk index, this is the chosen solution Related T1023 | 13 April 2018, 09:59:27 UTC |
| 532a340 | Antoine R. Dumont (@ardumont) | 10 April 2018, 15:43:07 UTC | New upstream version 0.0.23 | 10 April 2018, 15:43:07 UTC |
| e972b6a | Antoine R. Dumont (@ardumont) | 10 April 2018, 14:21:57 UTC | swh.scheduler.cli.archive: Simplify task and task_run ids extraction Related T986 | 10 April 2018, 15:43:03 UTC |
| 6c11eb6 | Antoine R. Dumont (@ardumont) | 10 April 2018, 14:06:38 UTC | swh.sched.cli.archive: Improve logging Related T986 | 10 April 2018, 15:43:03 UTC |
| 1da2d71 | Antoine R. Dumont (@ardumont) | 10 April 2018, 13:53:05 UTC | swh.scheduler.cli.archive: Delete only completely indexed tasks Prior to this commit, it could happen that we removed tasks even though we did not yet index associated task_run. Related T986 | 10 April 2018, 15:42:57 UTC |
| 04ccc2d | Antoine R. Dumont (@ardumont) | 10 April 2018, 12:44:15 UTC | swh.sched.cli.archive: Use interval period to filter archival tasks Related T986 | 10 April 2018, 14:00:12 UTC |
| 962fd8b | Antoine R. Dumont (@ardumont) | 10 April 2018, 12:42:35 UTC | swh.scheduler.backend_es: Return operation failure instead of raising Prior to this commit, an error would raise and stop all indexation. As the code is already waiting for a tuple (operation-status, item), we instead leverage this and continue working on indexation. All items whose `operation-status` is False (meaning failure to index, whatever the reason) is not indexed. Another run would then pickup the leftover and index it. Related T986 | 10 April 2018, 13:55:38 UTC |
| a04fb85 | Antoine R. Dumont (@ardumont) | 09 April 2018, 14:09:16 UTC | New upstream version 0.0.22 | 09 April 2018, 14:09:16 UTC |
| 9b9b88c | Antoine R. Dumont (@ardumont) | 09 April 2018, 13:59:16 UTC | d/control: Update to recent python3-elasticsearch client | 09 April 2018, 14:09:12 UTC |
| 9e5bf35 | Antoine R. Dumont (@ardumont) | 30 March 2018, 13:02:55 UTC | New upstream version 0.0.21 | 30 March 2018, 13:02:55 UTC |
| ffd00cb | Antoine R. Dumont (@ardumont) | 30 March 2018, 12:54:26 UTC | swh.scheduler.backend_es: Fix config base filename variable name | 30 March 2018, 12:55:25 UTC |
| ad0f0e2 | Antoine R. Dumont (@ardumont) | 30 March 2018, 10:22:28 UTC | data/elastic-template.json: Use elasticsearch's default conf Defaults to 5 shards and 1 replica | 30 March 2018, 12:54:54 UTC |
| 6cc6cb7 | Antoine R. Dumont (@ardumont) | 30 March 2018, 09:44:18 UTC | New upstream version 0.0.20 | 30 March 2018, 09:44:18 UTC |
| d6b393d | Antoine R. Dumont (@ardumont) | 29 March 2018, 15:11:37 UTC | swh.scheduler.cli.archive: By default, archive last month's data Related T986 | 29 March 2018, 15:32:25 UTC |
| e785252 | Antoine R. Dumont (@ardumont) | 29 March 2018, 08:47:08 UTC | swh.scheduler.cli: Use bulk api to index tasks Unfortunately, the bulk api does not return the full indexed data's original source [1] (which we need to identify the data clean up in the db). So we leverage elasticsearch's multi-get api to read back the original source. Related T986 | 29 March 2018, 10:32:37 UTC |
| 4d13f5d | Antoine R. Dumont (@ardumont) | 26 March 2018, 15:51:33 UTC | swh.scheduler.cli.archive: Improve dry-run behavior Related T986 | 26 March 2018, 15:56:00 UTC |
| af4c9b8 | Antoine R. Dumont (@ardumont) | 26 March 2018, 15:32:57 UTC | swh.scheduler.cli.archive: Optimize reading task lookup query Related T986 | 26 March 2018, 15:32:57 UTC |
| d83da86 | Antoine R. Dumont (@ardumont) | 22 March 2018, 16:16:12 UTC | swh.scheduler.cli.archive: Remove unnecessary order by condition Related T986 | 22 March 2018, 16:39:13 UTC |
| bcb4f74 | Antoine R. Dumont (@ardumont) | 22 March 2018, 15:57:35 UTC | swh.scheduler.cli.archive: Open batch size option for index/cleanup Related T986 | 22 March 2018, 16:39:13 UTC |
| 9e9f7ef | Antoine R. Dumont (@ardumont) | 22 March 2018, 15:51:03 UTC | swh.scheduler.cli.archive: Stream indexed tasks removal Related T986 | 22 March 2018, 15:51:03 UTC |
| e3867ed | Antoine R. Dumont (@ardumont) | 22 March 2018, 15:50:07 UTC | swh.scheduler.cli.archive: Fix edge case when reading tasks Related T986 | 22 March 2018, 15:50:07 UTC |
| b76384b | Antoine R. Dumont (@ardumont) | 22 March 2018, 12:51:32 UTC | swh.scheduler.cli: Open endpoint to archive tasks in elasticsearch Related T986 | 22 March 2018, 12:51:48 UTC |
| 8435efd | Antoine R. Dumont (@ardumont) | 21 March 2018, 18:58:45 UTC | swh.scheduler.cli: Add sql function to list archivable tasks Related T986 Related P240 | 21 March 2018, 18:58:45 UTC |
| 0324fb2 | Antoine R. Dumont (@ardumont) | 15 March 2018, 10:21:06 UTC | sql/data: Fix missing num_retries in deposit task types | 15 March 2018, 10:21:06 UTC |
| 367ab15 | Antoine R. Dumont (@ardumont) | 14 March 2018, 12:31:59 UTC | swh.core.serializer: Move to msgpack serialization format This permits to serialize bytes which was an existing behavior in celery 3.1 (when pickle was the default format). Related T782 Related D300 | 14 March 2018, 12:31:59 UTC |
| a474ed5 | Antoine R. Dumont (@ardumont) | 09 March 2018, 15:56:03 UTC | swh.scheduler.cli: Unify pretty print output | 09 March 2018, 15:56:03 UTC |
| 0e8e237 | Antoine R. Dumont (@ardumont) | 09 March 2018, 15:51:04 UTC | sql/data: Add new task type for loading mercurial dump | 09 March 2018, 15:54:32 UTC |
| 5a70627 | Antoine R. Dumont (@ardumont) | 09 March 2018, 15:50:41 UTC | swh.scheduler.cli: Add sample use case for the scheduling cli | 09 March 2018, 15:54:32 UTC |
| 3d68f20 | Antoine R. Dumont (@ardumont) | 09 March 2018, 15:50:11 UTC | swh.scheduler.cli: Open policy column to the scheduling cli | 09 March 2018, 15:54:31 UTC |
| 185f4b3 | Antoine R. Dumont (@ardumont) | 09 March 2018, 15:44:19 UTC | swh.scheduler.cli: Open the delimiter option as cli argument When providing json values (args|kwargs column), csv's default comma separator and json's default separator values clash. So the cli fails to schedule task. This commit permits to overcome that issue by providing the delimiter ';' to the cli call when json is required. | 09 March 2018, 15:52:49 UTC |
| 4b918af | Antoine R. Dumont (@ardumont) | 08 March 2018, 10:27:47 UTC | Fix issue when updating task-type without any retry delay defined Related P231 Related T902 | 08 March 2018, 10:27:47 UTC |
| f8ddae1 | Antoine R. Dumont (@ardumont) | 08 March 2018, 10:27:26 UTC | Fix pep8 violation regarding bare except | 08 March 2018, 10:27:26 UTC |
| 46329da | Antoine R. Dumont (@ardumont) | 19 December 2017, 13:35:40 UTC | swh-scheduler/data: Add new oneshot scheduling load-mercurial task Related T908 | 19 December 2017, 13:37:59 UTC |
| b74c89f | Antoine Pietri | 14 December 2017, 17:26:59 UTC | backend: fix default scheduling_db value for consistency | 14 December 2017, 17:26:59 UTC |
| 5b17566 | Antoine Pietri | 14 December 2017, 16:53:30 UTC | backend: doc: fix return value of create_tasks | 14 December 2017, 16:53:30 UTC |
| f08d9e7 | Antoine R. Dumont (@ardumont) | 07 December 2017, 14:18:31 UTC | Reference new task_type in scheduler | 07 December 2017, 14:18:31 UTC |
| 11ec4ba | Antoine R. Dumont (@ardumont) | 29 November 2017, 11:51:15 UTC | New upstream version 0.0.19 | 29 November 2017, 11:51:15 UTC |
| 255d851 | Antoine R. Dumont (@ardumont) | 29 November 2017, 11:09:32 UTC | swh.scheduler.utils: Open utility function to create oneshot task | 29 November 2017, 11:30:35 UTC |
| cb7d5fa | Nicolas Dandrimont | 08 November 2017, 16:06:22 UTC | New upstream version 0.0.18 | 08 November 2017, 16:06:22 UTC |
| 723221e | Nicolas Dandrimont | 08 November 2017, 15:38:52 UTC | Update for celery 4 | 08 November 2017, 16:00:50 UTC |
| 09b82c9 | Stefano Zacchiroli | 02 November 2017, 10:09:25 UTC | docs: add absolute anchor to documentation index | 02 November 2017, 10:09:25 UTC |
| 926cb2e | Nicolas Dandrimont | 12 October 2017, 16:49:02 UTC | New upstream version 0.0.17 | 12 October 2017, 16:49:02 UTC |
| cd827b6 | Nicolas Dandrimont | 12 October 2017, 16:47:31 UTC | Add missing __init__.py files | 12 October 2017, 16:48:15 UTC |
| 0a3d8ec | Nicolas Dandrimont | 12 October 2017, 15:16:58 UTC | Cleanup packaging | 12 October 2017, 15:17:55 UTC |
| 79b0ccd | Antoine R. Dumont (@ardumont) | 11 October 2017, 08:43:27 UTC | swh.scheduler.cli: Fix missing task type option | 11 October 2017, 08:57:49 UTC |
| d5f2d51 | Nicolas Dandrimont | 11 September 2017, 12:10:41 UTC | sql/updates/06: add name to check constraint | 11 September 2017, 12:10:41 UTC |
| 73f4aee | Nicolas Dandrimont | 11 September 2017, 12:01:31 UTC | sql: one-shot tasks don't really have an interval | 11 September 2017, 12:06:56 UTC |
| b4bbc05 | Stefano Zacchiroli | 06 September 2017, 18:45:30 UTC | sanitize docstrings for sphinx | 06 September 2017, 18:45:30 UTC |
| ca0230d | Nicolas Dandrimont | 01 September 2017, 18:20:42 UTC | sql: clean up schema syntax issues | 01 September 2017, 18:21:29 UTC |
| f785a81 | Stefano Zacchiroli | 30 August 2017, 10:26:02 UTC | docs/: add sphinx apidoc generation skeleton change cherry picked from python module template commit 71b117ba0cf9f1251b1cac26d0994df03a4c787d | 30 August 2017, 10:26:02 UTC |
| 89625c1 | Nicolas Dandrimont | 20 August 2017, 02:58:55 UTC | updates/05: fix SQL syntax to let the update run | 20 August 2017, 02:58:55 UTC |
| 017c885 | Nicolas Dandrimont | 07 August 2017, 16:44:03 UTC | New upstream version 0.0.16 | 07 August 2017, 16:44:03 UTC |
| ee545d4 | Nicolas Dandrimont | 07 August 2017, 16:43:57 UTC | test_scheduler: add missing db attribute to database tests | 07 August 2017, 16:43:57 UTC |
| deb40f2 | Nicolas Dandrimont | 07 August 2017, 16:33:40 UTC | requirements: bump swh.core to 0.0.34 | 07 August 2017, 16:33:54 UTC |
| 8348905 | Nicolas Dandrimont | 07 August 2017, 16:07:30 UTC | sql: add update script from 04 to 05 | 07 August 2017, 16:07:30 UTC |
| 3194c2b | Nicolas Dandrimont | 25 July 2017, 16:49:17 UTC | sql: new logic for recurrence of one-shot tasks | 25 July 2017, 16:49:17 UTC |
| 030a51f | Nicolas Dandrimont | 25 July 2017, 15:42:31 UTC | sql: add the recurrence policy to tasks and related fields to task_type This is the groundwork for the implementation of T720: - it adds the minimal set of fields required to allow for a task to be retried on transient failures; - it adds knowledge of whether a task is one-shot or recurring | 25 July 2017, 15:44:40 UTC |
| adb5afe | Nicolas Dandrimont | 25 July 2017, 13:30:48 UTC | test_scheduler: more generic tests wrt task types | 25 July 2017, 13:30:48 UTC |
| 31a6697 | Nicolas Dandrimont | 25 July 2017, 12:39:26 UTC | celery_backend.runner: respect the task's maximum queue length Close T726 | 25 July 2017, 12:39:26 UTC |
| a0420b0 | Nicolas Dandrimont | 21 July 2017, 16:57:18 UTC | backend: Add max queue length to task types One more stepping stone towards T726 | 21 July 2017, 16:57:18 UTC |
| 4fe37f7 | Nicolas Dandrimont | 21 July 2017, 14:09:55 UTC | backend: add get_tasks method | 21 July 2017, 14:09:55 UTC |
| 5ccc579 | Nicolas Dandrimont | 21 July 2017, 13:58:35 UTC | backend: add task_type argument to peek/grab_ready_tasks | 21 July 2017, 13:58:35 UTC |
| af220e2 | Nicolas Dandrimont | 19 July 2017, 18:02:11 UTC | backend: add tests for grab_ready_tasks | 19 July 2017, 18:02:11 UTC |
| d8945ae | Nicolas Dandrimont | 19 July 2017, 14:39:42 UTC | backend: add peek_ready_tasks tests | 19 July 2017, 14:39:42 UTC |
| d74d79f | Nicolas Dandrimont | 19 July 2017, 12:04:54 UTC | tests: add create_tasks test | 19 July 2017, 12:10:21 UTC |
| 0b88b14 | Nicolas Dandrimont | 19 July 2017, 10:08:30 UTC | backend: add get_task_types | 19 July 2017, 10:08:30 UTC |
| 2e2205c | Nicolas Dandrimont | 19 July 2017, 10:06:03 UTC | tests: refactor and extend | 19 July 2017, 10:07:01 UTC |
| d4eb07e | Nicolas Dandrimont | 17 July 2017, 17:28:43 UTC | Add test infrastructure for the scheduler backend | 17 July 2017, 17:31:19 UTC |
| 285e7b5 | Nicolas Dandrimont | 17 July 2017, 17:04:32 UTC | Add sql Makefile | 17 July 2017, 17:04:32 UTC |
| 5fa3510 | Nicolas Dandrimont | 19 June 2017, 17:44:56 UTC | New upstream version 0.0.15 | 19 June 2017, 17:44:56 UTC |
| 0e14eff | Nicolas Dandrimont | 19 June 2017, 17:00:18 UTC | celery_backend.config: add a shortcut to get a queue length | 19 June 2017, 17:00:18 UTC |
| 7caf543 | Nicolas Dandrimont | 19 June 2017, 14:03:19 UTC | celery_backend.config: add a method to get queue statistics from RabbitMQ First part of implementing T726 | 19 June 2017, 14:03:19 UTC |
| 54bcdda | Nicolas Dandrimont | 19 June 2017, 09:51:39 UTC | swh-worker-control: remove spurious print | 19 June 2017, 09:51:39 UTC |
| 80851f5 | Antoine Pietri | 15 June 2017, 15:13:22 UTC | celery: enable console logging on debug loglevel | 15 June 2017, 15:13:22 UTC |
| 2d8f636 | Nicolas Dandrimont | 12 June 2017, 15:50:31 UTC | New upstream version 0.0.14 | 12 June 2017, 15:50:31 UTC |
| 7c1c041 | Nicolas Dandrimont | 12 June 2017, 15:49:58 UTC | listener: update for the new task-result event | 12 June 2017, 15:49:58 UTC |
| d3abfa4 | Nicolas Dandrimont | 12 June 2017, 15:49:12 UTC | listener: derive the reliable events receiver from the base celery class | 12 June 2017, 15:49:12 UTC |
| 671a2c0 | Nicolas Dandrimont | 12 June 2017, 13:30:02 UTC | task: move run to run_task to allow overriding the default behavior of tasks This is an API break, but it's just a matter of renaming all existing tasks to run_task instead of run. Close T536 | 12 June 2017, 13:30:02 UTC |
| 07ea766 | Nicolas Dandrimont | 31 May 2017, 16:27:53 UTC | swh-worker-control: read the remote monotonic time to get task start times | 31 May 2017, 16:27:53 UTC |
| c02757c | Nicolas Dandrimont | 31 May 2017, 16:27:00 UTC | celery_backend: add a remote-control command to get the monotonic clock | 31 May 2017, 16:27:00 UTC |
| 324a3cc | Nicolas Dandrimont | 31 May 2017, 15:49:06 UTC | bin/swh-worker-control: add command-line interface for worker remote control | 31 May 2017, 15:51:04 UTC |
| 4aaf546 | Nicolas Dandrimont | 07 April 2017, 09:57:50 UTC | New upstream version 0.0.13 | 07 April 2017, 09:57:50 UTC |
| f758291 | Nicolas Dandrimont | 07 April 2017, 09:57:36 UTC | requirements-swh.txt: use proper syntax | 07 April 2017, 09:57:36 UTC |
| a760866 | Nicolas Dandrimont | 07 April 2017, 09:53:08 UTC | d/copyright: cleanup | 07 April 2017, 09:53:08 UTC |
