5913718 | Valentin Lorentz | 04 April 2019, 13:52:07 UTC | Refactor content_add's code into smaller functions. | 04 April 2019, 15:10:50 UTC |
4c8a0ac | Valentin Lorentz | 04 April 2019, 13:14:57 UTC | Make origin_add copy objects before changing them. hypothesis seems to reuse objects across tests now. | 04 April 2019, 13:14:57 UTC |
81a9954 | Valentin Lorentz | 03 April 2019, 11:38:58 UTC | Don't leak origin ids to the journal. | 04 April 2019, 09:47:18 UTC |
e9a3198 | Antoine R. Dumont (@ardumont) | 03 April 2019, 14:35:55 UTC | api.server: Add tests around send_metric function | 03 April 2019, 15:56:49 UTC |
1a401c0 | Antoine R. Dumont (@ardumont) | 03 April 2019, 14:09:16 UTC | api.server: Make send_metric be in charge of the metric parsing | 03 April 2019, 14:20:12 UTC |
7d6d894 | Antoine R. Dumont (@ardumont) | 03 April 2019, 13:47:06 UTC | api.server: Use constants for all metrics | 03 April 2019, 13:50:26 UTC |
1da7b50 | Antoine R. Dumont (@ardumont) | 03 April 2019, 13:42:03 UTC | api.server: Rename send_metric as we really send only 1 metric | 03 April 2019, 13:50:26 UTC |
ff05590 | Antoine R. Dumont (@ardumont) | 03 April 2019, 13:37:43 UTC | api.server: Rename constant | 03 April 2019, 13:38:18 UTC |
2f1e763 | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:59:55 UTC | api.server: Refactor repetition into a send_metrics function | 03 April 2019, 13:06:02 UTC |
6dee5f5 | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:50:37 UTC | api.server: Fix wrong api call | 03 April 2019, 13:06:02 UTC |
04792f8 | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:43:23 UTC | server: Add metrics on tool_add endpoint | 03 April 2019, 13:06:02 UTC |
ee934b6 | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:42:43 UTC | storage: Make tool_add endpoint return a list instead of a generator This aligns with other endpoints and allow to extract metrics easily | 03 April 2019, 13:06:02 UTC |
9b35b2d | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:39:25 UTC | server: Add metrics on origin_metadata_add & metadata_provider_add | 03 April 2019, 13:06:02 UTC |
6064a31 | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:28:34 UTC | api.{client,server}: Remove dead code about entity | 03 April 2019, 13:06:02 UTC |
907c4ad | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:27:30 UTC | api.server: Add metrics on origin_add and origin_add_one | 03 April 2019, 13:06:02 UTC |
b7d9896 | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:22:06 UTC | api.server: Add metrics to origin_visit_add | 03 April 2019, 13:06:02 UTC |
e357a38 | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:13:40 UTC | api.server: Include the encoding step when timing | 03 April 2019, 13:06:02 UTC |
bb19ce3 | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:00:10 UTC | server.api: Fix metric names respecting prometheus conventions [1] [1] https://prometheus.io/docs/practices/naming/ | 03 April 2019, 13:06:02 UTC |
4518363 | Antoine R. Dumont (@ardumont) | 03 April 2019, 08:49:41 UTC | api.server.process_metrics: Skip processing when value is 0 | 03 April 2019, 13:06:02 UTC |
cb7b9ee | Antoine R. Dumont (@ardumont) | 03 April 2019, 08:48:59 UTC | api.server: Rename to process_metrics function name | 03 April 2019, 13:06:02 UTC |
529c4be | Antoine R. Dumont (@ardumont) | 03 April 2019, 08:48:35 UTC | storage: Fix typo in variable name | 03 April 2019, 13:06:02 UTC |
ce999b5 | Antoine R. Dumont (@ardumont) | 02 April 2019, 16:45:52 UTC | api.server: Exploit new endpoint to compute metrics | 03 April 2019, 13:06:01 UTC |
f18af30 | Antoine R. Dumont (@ardumont) | 02 April 2019, 16:44:24 UTC | swh.storage: Adapt and unify _add endpoints | 03 April 2019, 13:06:01 UTC |
ddd1835 | Antoine R. Dumont (@ardumont) | 01 April 2019, 14:27:23 UTC | api.server: Send computed metrics | 03 April 2019, 12:57:30 UTC |
007f3de | Antoine R. Dumont (@ardumont) | 01 April 2019, 14:04:25 UTC | storage*.snapshot_add: Return summary | 03 April 2019, 12:57:30 UTC |
ab7f008 | Antoine R. Dumont (@ardumont) | 01 April 2019, 13:55:07 UTC | storage*.release_add: Return summary | 03 April 2019, 12:53:50 UTC |
01b8a9a | Antoine R. Dumont (@ardumont) | 01 April 2019, 13:54:42 UTC | storage.in_memory: Fix missing filtering on release_add | 03 April 2019, 12:53:50 UTC |
f366446 | Antoine R. Dumont (@ardumont) | 01 April 2019, 13:48:36 UTC | storage*.revision_add: Return summary | 03 April 2019, 12:53:50 UTC |
fb4be03 | Antoine R. Dumont (@ardumont) | 01 April 2019, 13:40:44 UTC | storage: Align implementations to compute correct metrics It was simpler to return the correct number instead of making the in-memory returns wrong results. | 03 April 2019, 12:53:50 UTC |
25da407 | Antoine R. Dumont (@ardumont) | 01 April 2019, 13:11:16 UTC | directory_add: Normalize metrics keys | 03 April 2019, 12:53:50 UTC |
cd78364 | Antoine R. Dumont (@ardumont) | 01 April 2019, 11:59:43 UTC | content_add: Normalize the summary keys | 03 April 2019, 12:53:50 UTC |
b341446 | Antoine R. Dumont (@ardumont) | 01 April 2019, 10:27:57 UTC | api.server.directory_add: Add counter metrics | 03 April 2019, 12:53:50 UTC |
ecbb611 | Antoine R. Dumont (@ardumont) | 01 April 2019, 10:27:42 UTC | storage*.directory_add: Return summary | 03 April 2019, 12:53:50 UTC |
71bd5bf | Antoine R. Dumont (@ardumont) | 01 April 2019, 10:05:07 UTC | storage.in_memory.content_add: Adapt implementation to send summary | 03 April 2019, 12:53:50 UTC |
a284ec0 | Antoine R. Dumont (@ardumont) | 01 April 2019, 09:58:06 UTC | storage: Update tests to check the summary result act as expected | 03 April 2019, 12:53:50 UTC |
ee62f46 | Antoine R. Dumont (@ardumont) | 01 April 2019, 09:53:50 UTC | api.server: Remove encode step from metrics | 03 April 2019, 12:53:49 UTC |
7f798bb | Antoine R. Dumont (@ardumont) | 01 April 2019, 09:53:30 UTC | api.server: Update docstrings on content_add to explicit the output | 03 April 2019, 12:53:49 UTC |
3e45179 | Antoine R. Dumont (@ardumont) | 29 March 2019, 16:13:58 UTC | Install counter metrics | 03 April 2019, 12:53:49 UTC |
2539858 | Antoine R. Dumont (@ardumont) | 29 March 2019, 09:39:12 UTC | api.server: Remove timed metric on the get_storage function | 03 April 2019, 12:53:49 UTC |
86308f6 | Valentin Lorentz | 01 April 2019, 14:11:44 UTC | Make snapshot_add take an iterable of snapshot. For uniformity with other _add() endpoints (except origin_visit_add). Noone uses the new signature of snapshot_add except the tests yet, so it's the perfect time to make this breaking change. | 03 April 2019, 12:47:02 UTC |
1ddb145 | Valentin Lorentz | 01 April 2019, 10:00:51 UTC | Use sha1 instead of bigint as FK from origin_visit to snapshot (part 3: cleanup) | 01 April 2019, 13:21:24 UTC |
7b56701 | Valentin Lorentz | 01 April 2019, 09:56:43 UTC | Use sha1 instead of bigint as FK from origin_visit to snapshot (part 2: backfill + start using the new colum for reading) | 01 April 2019, 13:21:24 UTC |
a150fdf | Valentin Lorentz | 01 April 2019, 09:50:30 UTC | Use sha1 instead of bigint as FK from origin_visit to snapshot (part 1: add new column) | 01 April 2019, 13:17:34 UTC |
480e098 | Valentin Lorentz | 01 April 2019, 13:03:53 UTC | Fix import. | 01 April 2019, 13:03:53 UTC |
3c849b5 | Nicolas Dandrimont | 28 March 2019, 17:41:19 UTC | Remove lingering debug print | 28 March 2019, 17:41:19 UTC |
742dd4f | Valentin Lorentz | 28 March 2019, 12:28:16 UTC | Split Storage.snapshot_add (add the snapshot + update the origin_visit) `snapshot_add(origin, visit, snap)` becomes `snapshot_add(origin)` + `origin_visit_update(origin, visit, snapshot_id=snap['id'])` | 28 March 2019, 15:01:49 UTC |
3457207 | Valentin Lorentz | 28 March 2019, 12:27:33 UTC | Add a new argument snapshot_id to origin_visit_update, and make all its arguments optional. | 28 March 2019, 12:27:33 UTC |
246855c | Valentin Lorentz | 26 March 2019, 16:11:06 UTC | Add a new JournalWriter interface, which is notified by swh-storage before writing to pgsql. | 28 March 2019, 10:42:27 UTC |
f734c4e | Antoine R. Dumont (@ardumont) | 28 March 2019, 08:58:54 UTC | api.server: Remove unneeded stats.increment call as this is already done further down the stack | 28 March 2019, 08:58:54 UTC |
3c2dfc0 | Antoine R. Dumont (@ardumont) | 27 March 2019, 18:35:23 UTC | swh.storage.api.server: Add metrics to storage endpoints | 27 March 2019, 19:24:56 UTC |
3a7f8a9 | Antoine R. Dumont (@ardumont) | 26 March 2019, 17:49:33 UTC | Add @timed decorator around all api.server endpoints | 26 March 2019, 17:49:33 UTC |
c47f92a | Antoine R. Dumont (@ardumont) | 26 March 2019, 13:46:45 UTC | storage: Add a timed decoration to count calls and time it | 26 March 2019, 14:19:43 UTC |
aaab31c | Nicolas Dandrimont | 25 March 2019, 15:26:40 UTC | Mark the node id columns as not null for directory, revision and snapshot This is an oversight from the migration to split indexes. | 25 March 2019, 15:31:28 UTC |
4ed7d0a | David Douard | 06 March 2019, 11:27:41 UTC | API: refactor the /directory/ls endpoint use the generic POST based implementation. | 18 March 2019, 15:58:11 UTC |
97981b2 | David Douard | 06 March 2019, 11:24:46 UTC | API: simplify a bit the RemoteStorage class set the exception class as class attribute wich allows to get rid of the __init__ overloading. | 18 March 2019, 15:57:14 UTC |
8bbbf9f | Valentin Lorentz | 26 February 2019, 09:50:44 UTC | Add an helper function to list all origins in the storage. | 11 March 2019, 13:20:52 UTC |
c533d6e | Valentin Lorentz | 27 February 2019, 09:42:29 UTC | Double the timeout of revision_get. Metadata indexers often hit the limit. | 27 February 2019, 09:42:29 UTC |
a1b8387 | Antoine R. Dumont (@ardumont) | 21 February 2019, 13:59:02 UTC | api.server: Fix wrong exception type | 21 February 2019, 13:59:02 UTC |
b4f2042 | Antoine R. Dumont (@ardumont) | 21 February 2019, 12:43:53 UTC | storage.cli: Fix cli entry point name to the expected name (setup.py) | 21 February 2019, 12:43:53 UTC |
95b0b42 | Antoine R. Dumont (@ardumont) | 21 February 2019, 12:28:51 UTC | api.server: Use the right Exception for the job | 21 February 2019, 12:28:51 UTC |
247ab97 | Antoine R. Dumont (@ardumont) | 21 February 2019, 12:23:36 UTC | test_server: Refactor and fix issue for pytest on python3.5 Prior to this commit, the build failed with: TypeError: argument should be string, bytes or integer, not LocalPath | 21 February 2019, 12:23:36 UTC |
a2db5ef | Antoine R. Dumont (@ardumont) | 21 February 2019, 11:21:42 UTC | api.server: Remove unused import | 21 February 2019, 11:21:42 UTC |
b38ecc0 | Antoine R. Dumont (@ardumont) | 21 February 2019, 11:16:32 UTC | swh.storage.cli: Migrate the api server startup in swh.storage.cli | 21 February 2019, 11:21:11 UTC |
2d500d1 | Antoine R. Dumont (@ardumont) | 21 February 2019, 10:50:52 UTC | api.wsgi: Open wsgi entrypoint and check config at startup time Related T1533 | 21 February 2019, 11:21:10 UTC |
750e292 | Antoine R. Dumont (@ardumont) | 21 February 2019, 10:49:35 UTC | api.server: Remove implicit config from Also: - add tests to the loading checks - rename run_from_webserver to make_app_from_configfile Related T1533 | 21 February 2019, 11:21:10 UTC |
2696b47 | Antoine R. Dumont (@ardumont) | 20 February 2019, 17:21:12 UTC | api.server: Make the api server load and check its configuration Related T1533 | 21 February 2019, 11:21:10 UTC |
006d102 | Valentin Lorentz | 21 February 2019, 09:18:26 UTC | Double the timeout of snapshot_get_latest. Metadata indexers often hit the limit. | 21 February 2019, 09:18:50 UTC |
0b10c28 | Antoine R. Dumont (@ardumont) | 19 February 2019, 11:05:46 UTC | Remove debian packaging from master branch Related T1532 | 19 February 2019, 11:05:46 UTC |
152cb52 | Valentin Lorentz | 15 February 2019, 14:10:09 UTC | Move origin_get_range's right after origin_get. | 15 February 2019, 14:10:09 UTC |
a94451a | Antoine R. Dumont (@ardumont) | 13 February 2019, 22:55:12 UTC | api/server: Do not read configuration at each request | 13 February 2019, 22:55:12 UTC |
e2c4d38 | Antoine Lambert | 01 February 2019, 15:50:18 UTC | storage: Add origin_count method Related T1463 | 12 February 2019, 13:25:54 UTC |
1efbe5c | Valentin Lorentz | 12 February 2019, 13:14:14 UTC | Fix test to work with the latest version of swh.core. | 12 February 2019, 13:14:14 UTC |
e59eb38 | Valentin Lorentz | 06 February 2019, 15:45:13 UTC | Make Storage.origin_get support a list of origins, like other Storage.*_get methods. Backward compatibility is preserved. | 07 February 2019, 14:38:01 UTC |
eeecf6f | Valentin Lorentz | 01 February 2019, 12:30:40 UTC | Stop using _to_bytes functions. | 01 February 2019, 14:02:21 UTC |
8cb3ca2 | David Douard | 24 January 2019, 14:57:58 UTC | Use the BaseDb (and friends) from swh-core since it has migrated there. | 28 January 2019, 16:31:19 UTC |
4c49b3f | Antoine Lambert | 25 January 2019, 15:14:58 UTC | storage.in_memory: Fix unknown revision handling in revision_log method | 25 January 2019, 15:37:03 UTC |
17b13af | Antoine Lambert | 25 January 2019, 15:13:49 UTC | storage.in_memory: Fix unknown origin handling in origin*_get methods | 25 January 2019, 15:15:13 UTC |
f308c10 | Antoine Lambert | 25 January 2019, 13:55:19 UTC | debian/rules: Exclude hypothesis generated data from package | 25 January 2019, 13:55:19 UTC |
76d7838 | Antoine Lambert | 15 January 2019, 15:55:56 UTC | storage: Add origin_get_range method to list all archived origins Related T1399 | 17 January 2019, 10:50:39 UTC |
69f48be | Antoine R. Dumont (@ardumont) | 11 January 2019, 15:18:36 UTC | sql/upgrades/129: Add missing trigger for origin visit update event Also no need for a new function notify_changed_origin_visit since the new function does the same as the original one. Related T1466 | 11 January 2019, 15:19:45 UTC |
d19392c | Valentin Lorentz | 10 January 2019, 15:17:18 UTC | Notify Kafka when an origin visit is update. | 10 January 2019, 15:51:24 UTC |
c5cee88 | Antoine Lambert | 09 January 2019, 15:24:32 UTC | storage.db: Remove unused person_add method Related D897 | 09 January 2019, 15:25:00 UTC |
1f73266 | Antoine Lambert | 09 January 2019, 15:20:24 UTC | storage.in_memory: Return copies of origin visit dicts | 09 January 2019, 15:20:24 UTC |
12a2dd6 | Antoine Lambert | 08 January 2019, 13:44:52 UTC | Add in_memory.person_get implementation and improve person tests Related T1377 | 09 January 2019, 15:16:16 UTC |
fc7c534 | Antoine R. Dumont (@ardumont) | 20 December 2018, 12:11:04 UTC | listener: Make kafka client be less verbose | 20 December 2018, 12:11:24 UTC |
d124710 | Antoine R. Dumont (@ardumont) | 20 December 2018, 12:10:53 UTC | listener: Use swh.model.hashutil.hash_to_bytes | 20 December 2018, 12:10:53 UTC |
a229a07 | Antoine R. Dumont (@ardumont) | 20 December 2018, 09:55:37 UTC | listener: Add logging behavior | 20 December 2018, 10:48:35 UTC |
2a03e1c | Antoine R. Dumont (@ardumont) | 20 December 2018, 09:54:53 UTC | listener: Adapt decoding behavior depending on the object type This fixes issues regarding origin/origin_visit | 20 December 2018, 10:46:20 UTC |
d19a0d1 | Antoine R. Dumont (@ardumont) | 14 December 2018, 14:43:53 UTC | Update requirements to latest swh.core Related T1444 | 14 December 2018, 14:45:23 UTC |
02e8c76 | Antoine Lambert | 11 December 2018, 09:53:19 UTC | storage.in_memory: Use in-memory objstorage to store contents | 11 December 2018, 09:53:19 UTC |
d5e8229 | Antoine Lambert | 04 December 2018, 12:33:55 UTC | storage.in_memory: Use integer ids for origin and visit instead of tuples Related T1421 | 04 December 2018, 14:58:15 UTC |
bda7cf1 | Valentin Lorentz | 04 December 2018, 13:00:25 UTC | snapshot_add: Check the origin visit exists. Summary: Instead of failing silently (PG storage) or raising an un-documented exception (in-mem storage). Reviewers: #reviewers, anlambert Reviewed By: #reviewers, anlambert Subscribers: swh-public-ci Differential Revision: https://forge.softwareheritage.org/D769 | 04 December 2018, 14:56:12 UTC |
b6b004b | Valentin Lorentz | 04 December 2018, 13:00:25 UTC | snapshot_add: Check the origin visit exists. Instead of failing silently (PG storage) or raising an un-documented exception (in-mem storage). | 04 December 2018, 13:00:25 UTC |
12f5c5a | Valentin Lorentz | 29 November 2018, 14:58:30 UTC | Add 'recursive' argument to the in-mem storage. Summary: Also add test coverage for that feature and improve doc. Reviewers: #reviewers, ardumont Reviewed By: #reviewers, ardumont Subscribers: swh-public-ci Differential Revision: https://forge.softwareheritage.org/D737 | 30 November 2018, 10:28:16 UTC |
df27d11 | Valentin Lorentz | 29 November 2018, 14:58:30 UTC | Add 'recursive' argument to the in-mem storage. Summary: Also add test coverage for that feature and improve doc. Reviewers: #reviewers Differential Revision: https://forge.softwareheritage.org/D737 | 29 November 2018, 14:59:03 UTC |
d8463b1 | Valentin Lorentz | 22 November 2018, 13:38:23 UTC | Use the new doc style for {revision,release}_add. Summary: Also fix the 'committer_date' on the PG Storage's revision_add, that slipped through D676. Reviewers: #reviewers, douardda Reviewed By: #reviewers, douardda Subscribers: douardda, ardumont, swh-public-ci Differential Revision: https://forge.softwareheritage.org/D698 | 29 November 2018, 13:40:49 UTC |
f203821 | Valentin Lorentz | 29 November 2018, 09:23:30 UTC | Fix column name in metadata_provider_get. Test Plan: Test coverage will come in the next diff Reviewers: #reviewers, ardumont Reviewed By: #reviewers, ardumont Subscribers: ardumont, swh-public-ci Differential Revision: https://forge.softwareheritage.org/D716 | 29 November 2018, 13:40:40 UTC |
7af40f6 | Valentin Lorentz | 27 November 2018, 17:00:57 UTC | Fix documented return type of metadata_provider_add. Summary: As the in-mem storage was written based on this documentation, fix it as well. Add a test for this behavior. Depends on D716. Reviewers: #reviewers, ardumont Reviewed By: #reviewers, ardumont Subscribers: ardumont, swh-public-ci Differential Revision: https://forge.softwareheritage.org/D717 | 29 November 2018, 09:21:43 UTC |
abee27c | Antoine Lambert | 27 November 2018, 14:39:54 UTC | Storage.release_get: Yield None for non-existent release Similar to Storage.revision_get, yields None when querying a non-existent release. | 27 November 2018, 16:31:51 UTC |
cd2db9b | Valentin Lorentz | 22 November 2018, 10:24:09 UTC | Document edge cases of content_get_range. Reviewers: #reviewers, douardda, ardumont Reviewed By: #reviewers, douardda, ardumont Subscribers: ardumont, douardda, swh-public-ci Differential Revision: https://forge.softwareheritage.org/D694 | 27 November 2018, 10:44:12 UTC |