255d851 | Antoine R. Dumont (@ardumont) | 29 November 2017, 11:09:32 UTC | swh.scheduler.utils: Open utility function to create oneshot task | 29 November 2017, 11:30:35 UTC |
723221e | Nicolas Dandrimont | 08 November 2017, 15:38:52 UTC | Update for celery 4 | 08 November 2017, 16:00:50 UTC |
09b82c9 | Stefano Zacchiroli | 02 November 2017, 10:09:25 UTC | docs: add absolute anchor to documentation index | 02 November 2017, 10:09:25 UTC |
cd827b6 | Nicolas Dandrimont | 12 October 2017, 16:47:31 UTC | Add missing __init__.py files | 12 October 2017, 16:48:15 UTC |
0a3d8ec | Nicolas Dandrimont | 12 October 2017, 15:16:58 UTC | Cleanup packaging | 12 October 2017, 15:17:55 UTC |
79b0ccd | Antoine R. Dumont (@ardumont) | 11 October 2017, 08:43:27 UTC | swh.scheduler.cli: Fix missing task type option | 11 October 2017, 08:57:49 UTC |
d5f2d51 | Nicolas Dandrimont | 11 September 2017, 12:10:41 UTC | sql/updates/06: add name to check constraint | 11 September 2017, 12:10:41 UTC |
73f4aee | Nicolas Dandrimont | 11 September 2017, 12:01:31 UTC | sql: one-shot tasks don't really have an interval | 11 September 2017, 12:06:56 UTC |
b4bbc05 | Stefano Zacchiroli | 06 September 2017, 18:45:30 UTC | sanitize docstrings for sphinx | 06 September 2017, 18:45:30 UTC |
ca0230d | Nicolas Dandrimont | 01 September 2017, 18:20:42 UTC | sql: clean up schema syntax issues | 01 September 2017, 18:21:29 UTC |
f785a81 | Stefano Zacchiroli | 30 August 2017, 10:26:02 UTC | docs/: add sphinx apidoc generation skeleton change cherry picked from python module template commit 71b117ba0cf9f1251b1cac26d0994df03a4c787d | 30 August 2017, 10:26:02 UTC |
89625c1 | Nicolas Dandrimont | 20 August 2017, 02:58:55 UTC | updates/05: fix SQL syntax to let the update run | 20 August 2017, 02:58:55 UTC |
ee545d4 | Nicolas Dandrimont | 07 August 2017, 16:43:57 UTC | test_scheduler: add missing db attribute to database tests | 07 August 2017, 16:43:57 UTC |
deb40f2 | Nicolas Dandrimont | 07 August 2017, 16:33:40 UTC | requirements: bump swh.core to 0.0.34 | 07 August 2017, 16:33:54 UTC |
8348905 | Nicolas Dandrimont | 07 August 2017, 16:07:30 UTC | sql: add update script from 04 to 05 | 07 August 2017, 16:07:30 UTC |
3194c2b | Nicolas Dandrimont | 25 July 2017, 16:49:17 UTC | sql: new logic for recurrence of one-shot tasks | 25 July 2017, 16:49:17 UTC |
030a51f | Nicolas Dandrimont | 25 July 2017, 15:42:31 UTC | sql: add the recurrence policy to tasks and related fields to task_type This is the groundwork for the implementation of T720: - it adds the minimal set of fields required to allow for a task to be retried on transient failures; - it adds knowledge of whether a task is one-shot or recurring | 25 July 2017, 15:44:40 UTC |
adb5afe | Nicolas Dandrimont | 25 July 2017, 13:30:48 UTC | test_scheduler: more generic tests wrt task types | 25 July 2017, 13:30:48 UTC |
31a6697 | Nicolas Dandrimont | 25 July 2017, 12:39:26 UTC | celery_backend.runner: respect the task's maximum queue length Close T726 | 25 July 2017, 12:39:26 UTC |
a0420b0 | Nicolas Dandrimont | 21 July 2017, 16:57:18 UTC | backend: Add max queue length to task types One more stepping stone towards T726 | 21 July 2017, 16:57:18 UTC |
4fe37f7 | Nicolas Dandrimont | 21 July 2017, 14:09:55 UTC | backend: add get_tasks method | 21 July 2017, 14:09:55 UTC |
5ccc579 | Nicolas Dandrimont | 21 July 2017, 13:58:35 UTC | backend: add task_type argument to peek/grab_ready_tasks | 21 July 2017, 13:58:35 UTC |
af220e2 | Nicolas Dandrimont | 19 July 2017, 18:02:11 UTC | backend: add tests for grab_ready_tasks | 19 July 2017, 18:02:11 UTC |
d8945ae | Nicolas Dandrimont | 19 July 2017, 14:39:42 UTC | backend: add peek_ready_tasks tests | 19 July 2017, 14:39:42 UTC |
d74d79f | Nicolas Dandrimont | 19 July 2017, 12:04:54 UTC | tests: add create_tasks test | 19 July 2017, 12:10:21 UTC |
0b88b14 | Nicolas Dandrimont | 19 July 2017, 10:08:30 UTC | backend: add get_task_types | 19 July 2017, 10:08:30 UTC |
2e2205c | Nicolas Dandrimont | 19 July 2017, 10:06:03 UTC | tests: refactor and extend | 19 July 2017, 10:07:01 UTC |
d4eb07e | Nicolas Dandrimont | 17 July 2017, 17:28:43 UTC | Add test infrastructure for the scheduler backend | 17 July 2017, 17:31:19 UTC |
285e7b5 | Nicolas Dandrimont | 17 July 2017, 17:04:32 UTC | Add sql Makefile | 17 July 2017, 17:04:32 UTC |
0e14eff | Nicolas Dandrimont | 19 June 2017, 17:00:18 UTC | celery_backend.config: add a shortcut to get a queue length | 19 June 2017, 17:00:18 UTC |
7caf543 | Nicolas Dandrimont | 19 June 2017, 14:03:19 UTC | celery_backend.config: add a method to get queue statistics from RabbitMQ First part of implementing T726 | 19 June 2017, 14:03:19 UTC |
54bcdda | Nicolas Dandrimont | 19 June 2017, 09:51:39 UTC | swh-worker-control: remove spurious print | 19 June 2017, 09:51:39 UTC |
80851f5 | Antoine Pietri | 15 June 2017, 15:13:22 UTC | celery: enable console logging on debug loglevel | 15 June 2017, 15:13:22 UTC |
7c1c041 | Nicolas Dandrimont | 12 June 2017, 15:49:58 UTC | listener: update for the new task-result event | 12 June 2017, 15:49:58 UTC |
d3abfa4 | Nicolas Dandrimont | 12 June 2017, 15:49:12 UTC | listener: derive the reliable events receiver from the base celery class | 12 June 2017, 15:49:12 UTC |
671a2c0 | Nicolas Dandrimont | 12 June 2017, 13:30:02 UTC | task: move run to run_task to allow overriding the default behavior of tasks This is an API break, but it's just a matter of renaming all existing tasks to run_task instead of run. Close T536 | 12 June 2017, 13:30:02 UTC |
07ea766 | Nicolas Dandrimont | 31 May 2017, 16:27:53 UTC | swh-worker-control: read the remote monotonic time to get task start times | 31 May 2017, 16:27:53 UTC |
c02757c | Nicolas Dandrimont | 31 May 2017, 16:27:00 UTC | celery_backend: add a remote-control command to get the monotonic clock | 31 May 2017, 16:27:00 UTC |
324a3cc | Nicolas Dandrimont | 31 May 2017, 15:49:06 UTC | bin/swh-worker-control: add command-line interface for worker remote control | 31 May 2017, 15:51:04 UTC |
f758291 | Nicolas Dandrimont | 07 April 2017, 09:57:36 UTC | requirements-swh.txt: use proper syntax | 07 April 2017, 09:57:36 UTC |
a760866 | Nicolas Dandrimont | 07 April 2017, 09:53:08 UTC | d/copyright: cleanup | 07 April 2017, 09:53:08 UTC |
35997f8 | Nicolas Dandrimont | 07 April 2017, 09:52:27 UTC | celery_backend.config: use the JournalHandler for logging | 07 April 2017, 09:52:27 UTC |
aaa5dfd | Nicolas Dandrimont | 09 March 2017, 10:12:09 UTC | celery_backend.config: only log to database if log_db is present | 09 March 2017, 10:12:09 UTC |
0767b4c | Antoine Pietri | 15 February 2017, 14:23:52 UTC | requirements.txt: ensure we use celery<4 (we don't support other versions for now) | 15 February 2017, 14:23:52 UTC |
6192ac1 | Antoine R. Dumont (@ardumont) | 10 February 2017, 14:03:02 UTC | Add utils.get_task fn to simplify contextual task instance retrieval Permits to unify the task instance retrieval, which then permits to send messages to the associated task's queue. | 10 February 2017, 14:11:54 UTC |
1aedc67 | Antoine Pietri | 09 February 2017, 11:21:06 UTC | requirements: split internal and external requirements in two separate files | 09 February 2017, 13:32:06 UTC |
d65742e | Antoine Pietri | 09 February 2017, 11:19:06 UTC | celery_backend: pep8: fix missing blank lines | 09 February 2017, 11:21:23 UTC |
05f41c2 | Nicolas Dandrimont | 21 October 2016, 14:09:11 UTC | sql: fix default_interval selection for new tasks | 21 October 2016, 14:09:11 UTC |
b048926 | Nicolas Dandrimont | 20 October 2016, 14:02:57 UTC | backend: the status of a task is task.status, not task.task_status... | 20 October 2016, 14:02:57 UTC |
da7b1b7 | Nicolas Dandrimont | 19 October 2016, 13:39:05 UTC | backend: allow disabling tasks | 19 October 2016, 13:39:05 UTC |
0fb8eef | Antoine R. Dumont (@ardumont) | 23 August 2016, 16:44:15 UTC | Fix docstring typo | 05 September 2016, 12:27:28 UTC |
51cb666 | Nicolas Dandrimont | 01 September 2016, 15:23:15 UTC | celery_backend.config: add SWH_WORKER_INSTANCE environment variable This allows to run several workers with different configuration files. | 01 September 2016, 15:23:15 UTC |
dced5d5 | Nicolas Dandrimont | 24 February 2016, 15:36:52 UTC | celery_backend.config: Put pickle last in the accepted encodings | 24 February 2016, 15:36:52 UTC |
7d6e331 | Nicolas Dandrimont | 24 February 2016, 15:36:02 UTC | backend: allow for mass-scheduling of task runs This allows the runner to be (~10 times) more efficient when running tasks | 24 February 2016, 15:36:02 UTC |
15c3d8f | Nicolas Dandrimont | 24 February 2016, 11:10:51 UTC | backend: use copy for mass-scheduling of tasks | 24 February 2016, 11:10:51 UTC |
4886edf | Nicolas Dandrimont | 23 February 2016, 16:08:46 UTC | Add cli to setup.py | 23 February 2016, 16:08:46 UTC |
f390dec | Nicolas Dandrimont | 23 February 2016, 16:05:27 UTC | cli: New command-line interface based on Click | 23 February 2016, 16:06:13 UTC |
77bcb2a | Nicolas Dandrimont | 23 February 2016, 16:05:44 UTC | backend: allow overriding the config at instantiation | 23 February 2016, 16:06:13 UTC |
35edc1a | Nicolas Dandrimont | 23 February 2016, 16:04:33 UTC | backend: use arrow instead of dateutil | 23 February 2016, 16:04:33 UTC |
e9c4442 | Nicolas Dandrimont | 23 February 2016, 11:07:03 UTC | d/copyright: update for Celery license | 23 February 2016, 11:07:36 UTC |
d89860f | Nicolas Dandrimont | 23 February 2016, 10:58:01 UTC | Add missing Celery license file | 23 February 2016, 10:58:01 UTC |
314e4dd | Nicolas Dandrimont | 23 February 2016, 10:56:41 UTC | celery_backend.listener: Remove useless arguments to event functions | 23 February 2016, 10:56:41 UTC |
606fd39 | Nicolas Dandrimont | 23 February 2016, 10:56:06 UTC | celery_backend.listener: force sending the events at startup | 23 February 2016, 10:56:06 UTC |
e88c731 | Nicolas Dandrimont | 22 February 2016, 14:07:52 UTC | celery_backend.listener: make the event listener reliable We reimplement the event listener using kombu directly instead of Celery, which allows us to create persistent queues and manually ack the messages once we have received them. We bundle messages by packets of 1000 or 1/second when sending them to the database. We only ack the messages once the database transcation has ended. | 22 February 2016, 14:07:52 UTC |
c4e1b63 | Nicolas Dandrimont | 22 February 2016, 14:06:55 UTC | backend: implement the timestamp argument for the task_run functions | 22 February 2016, 14:07:22 UTC |
911df24 | Nicolas Dandrimont | 22 February 2016, 14:06:03 UTC | celery_backend.runner: only commit once per packet | 22 February 2016, 14:06:03 UTC |
f3c8cad | Nicolas Dandrimont | 19 February 2016, 17:49:45 UTC | celery: move to celery_backend to avoid clashing with system celery | 19 February 2016, 17:50:40 UTC |
0df7211 | Nicolas Dandrimont | 19 February 2016, 17:46:40 UTC | d/control: add missing B-D on Celery | 19 February 2016, 17:46:40 UTC |
d389ce2 | Nicolas Dandrimont | 19 February 2016, 17:35:27 UTC | setup.py: install all packages | 19 February 2016, 17:43:53 UTC |
06d7095 | Nicolas Dandrimont | 19 February 2016, 17:34:53 UTC | tests: Add basic tests for task | 19 February 2016, 17:34:53 UTC |
482d414 | Nicolas Dandrimont | 19 February 2016, 17:27:03 UTC | schema: replace the eventful boolean with a task run status | 19 February 2016, 17:27:03 UTC |
655a9d0 | Nicolas Dandrimont | 19 February 2016, 11:47:49 UTC | scheduler.celery: add listener for events and runner to generate tasks | 19 February 2016, 11:47:49 UTC |
8d512e3 | Nicolas Dandrimont | 18 February 2016, 16:47:49 UTC | backend: wrap all the DB methods | 18 February 2016, 16:47:49 UTC |
c70627e | Nicolas Dandrimont | 16 February 2016, 18:24:55 UTC | backend: initialize a simple backend for swh.scheduler | 16 February 2016, 18:24:55 UTC |
196e506 | Nicolas Dandrimont | 16 February 2016, 18:24:25 UTC | celery: move worker to a "celery.config" class, add listener | 16 February 2016, 18:24:25 UTC |
efc7ff8 | Nicolas Dandrimont | 16 February 2016, 10:27:40 UTC | sql: update testdata and functions for new schema | 16 February 2016, 10:27:40 UTC |
86f5e28 | Nicolas Dandrimont | 16 February 2016, 09:51:17 UTC | sql/schema: Clean up formatting | 16 February 2016, 09:51:17 UTC |
8ba5475 | Nicolas Dandrimont | 15 February 2016, 15:24:51 UTC | sql: add schema for swh-scheduler, and some test data | 15 February 2016, 15:24:51 UTC |
a1b0a98 | Nicolas Dandrimont | 15 February 2016, 10:07:20 UTC | d/rules: test proper module | 15 February 2016, 10:07:20 UTC |
533dece | Nicolas Dandrimont | 15 February 2016, 10:06:55 UTC | .gitignore: ignore vcversioner file | 15 February 2016, 10:06:55 UTC |
759c5f1 | Nicolas Dandrimont | 15 February 2016, 10:01:52 UTC | Move worker and task setup to swh.scheduler | 15 February 2016, 10:01:52 UTC |
efd4735 | Nicolas Dandrimont | 15 February 2016, 09:38:13 UTC | Update template data for swh.scheduler | 15 February 2016, 09:38:13 UTC |
2083b9a | Nicolas Dandrimont | 15 February 2016, 09:34:23 UTC | import template from swh-py-template (init-py-repo) | 15 February 2016, 09:34:23 UTC |