75ac20f | Valentin Lorentz | 05 June 2019, 11:25:05 UTC | Add test for content_get on missing content. | 05 June 2019, 11:45:23 UTC |
909c69b | Valentin Lorentz | 04 June 2019, 13:14:40 UTC | Fix recursive entry names in in_memory.py's directory_entry_get_by_path | 04 June 2019, 13:14:40 UTC |
efda98e | Valentin Lorentz | 29 May 2019, 12:52:46 UTC | Fix origin_search in the in-mem storage to search for sub-strings. | 04 June 2019, 11:29:24 UTC |
3556b74 | Valentin Lorentz | 29 May 2019, 12:54:47 UTC | Unify argument names of the pg and in-mem storage. Argument names matter for the remote API. | 29 May 2019, 16:13:03 UTC |
17d4112 | David Douard | 17 May 2019, 12:43:09 UTC | Update the README file especially for latest CLI modifications. Add a slightly more detailed step-by-step 'quick start' section. | 17 May 2019, 13:08:59 UTC |
289b233 | David Douard | 17 May 2019, 12:42:26 UTC | cli: small improvements of 'rpc-serve --help' message | 17 May 2019, 13:07:24 UTC |
39a1c45 | Valentin Lorentz | 17 May 2019, 12:53:46 UTC | Make content_find return an empty list instead of None. | 17 May 2019, 12:53:46 UTC |
ea3f2d3 | Valentin Lorentz | 17 May 2019, 12:51:17 UTC | Update import of swh.core.db.tests.db_testing. | 17 May 2019, 12:51:17 UTC |
01ed375 | Valentin Lorentz | 17 May 2019, 11:32:55 UTC | Update import of swh.core.api.tests.server_testing. | 17 May 2019, 11:32:59 UTC |
fa02fa4 | David Douard | 15 May 2019, 15:08:04 UTC | cli: add a 'storage' cli group with a 'rpc-serve' subcommand we do add this 'storage' group even if it contains only one subcommand (serve) for UI consistency: swh storage rpc-serve We do keep the main swh-storage console_script entry point as is for bw compat, however. | 16 May 2019, 15:28:29 UTC |
02134a7 | mihir(faux__) | 24 March 2019, 16:17:31 UTC | Changes the output of content_find method to a list in case of hash collisions and makes the sql query on python side and added test duplicate input, colliding sha256 and colliding blake2s256 | 16 May 2019, 09:06:06 UTC |
85b3da9 | David Douard | 07 May 2019, 15:12:11 UTC | bump dependency to swh.core[db,http] >= 0.0.60 so tests are not broken when next swh.core version is published. | 07 May 2019, 15:12:11 UTC |
2e70516 | Antoine Lambert | 23 April 2019, 14:56:29 UTC | algos.revisions_walker: Add methods to query produced history state Add methods is_history_truncated and missing_revisions to check if the produced history got truncated or not. Closes T1677 | 06 May 2019, 10:02:21 UTC |
6510b5e | Antoine Lambert | 18 April 2019, 14:17:16 UTC | algos.revisions_walker: Handle truncated/shallow histories It might exist cases where a revision's parent is referenced but doesn't exist in the archive resulting in a truncated/shallow history when walking on the revisions graph. That commit ensures the revision walkers will not end up in errors when encountering such cases. Closes T1675 | 18 April 2019, 15:30:17 UTC |
685b601 | Valentin Lorentz | 15 April 2019, 10:43:31 UTC | Make revision_add and directory_add call `_missing` using the same transaction. | 16 April 2019, 08:27:07 UTC |
7baf562 | Nicolas Dandrimont | 12 April 2019, 10:03:09 UTC | Update coverage gitignore | 12 April 2019, 10:03:09 UTC |
58b49f9 | Nicolas Dandrimont | 11 April 2019, 09:42:19 UTC | Use the proper order to record content byte addition The current way the value is returned creates a metric whose name is backwards. | 11 April 2019, 12:55:43 UTC |
63f50a5 | Nicolas Dandrimont | 11 April 2019, 10:15:49 UTC | Fix backwards-compatibility for snapshot_add in the server | 11 April 2019, 12:55:43 UTC |
cf99702 | Valentin Lorentz | 09 April 2019, 14:40:49 UTC | Use the db_transaction decorator on all _add() methods. So they gracefully release the connection on error instead of relying on reference-counting to call the Db's `__del__` (which does not happen in Hypothesis tests) because a ref to it is kept via the traceback object. | 09 April 2019, 16:06:24 UTC |
dab5bf8 | Valentin Lorentz | 09 April 2019, 14:36:31 UTC | Rewrite skipped_content_missing to not use a temporary table. | 09 April 2019, 16:05:46 UTC |
5aacb79 | Valentin Lorentz | 09 April 2019, 14:44:03 UTC | Add a test that adds arbitrary objects to the storage. | 09 April 2019, 15:58:28 UTC |
cd0d309 | Valentin Lorentz | 09 April 2019, 14:24:36 UTC | Fix person name clashes in revision_add and release_add. When two revisions/releases have authors with the same fullname but different names (which should never happen in practice, but since name/email parsing is on a best-effort basis, you never know...). Bug found by Hypothesis. | 09 April 2019, 15:49:38 UTC |
70c43ba | Valentin Lorentz | 09 April 2019, 14:42:19 UTC | Fix order of results of person_get. | 09 April 2019, 15:31:07 UTC |
8672a27 | Valentin Lorentz | 09 April 2019, 14:28:38 UTC | Add test for skipped_content_missing. | 09 April 2019, 15:27:12 UTC |
95c0b27 | Valentin Lorentz | 09 April 2019, 14:12:00 UTC | Fix support of releases with no author or date. Bug found by Hypothesis. | 09 April 2019, 15:24:12 UTC |
ba3a9af | Antoine R. Dumont (@ardumont) | 09 April 2019, 09:48:49 UTC | listener: Remove | 09 April 2019, 15:19:54 UTC |
681070f | Valentin Lorentz | 09 April 2019, 14:09:06 UTC | Make test_origin_get_range run faster (less objects). | 09 April 2019, 14:09:06 UTC |
2aa256b | Valentin Lorentz | 08 April 2019, 20:24:10 UTC | Fix hypothesis tests. Why does test_origin_get_range even work? | 08 April 2019, 20:24:10 UTC |
00e5f33 | Valentin Lorentz | 08 April 2019, 13:40:24 UTC | Make test_origin_get_range run faster. | 08 April 2019, 13:40:24 UTC |
92a575f | Valentin Lorentz | 05 April 2019, 18:00:42 UTC | Move Hypothesis strategies to swh-model. | 08 April 2019, 13:23:51 UTC |
a574fe5 | Valentin Lorentz | 05 April 2019, 14:19:38 UTC | Add a new endpoint origin_visit_upsert to add an origin_visit with a specific id. Also allows skipping an extra origin_visit_update query to set snapshot/metadata/status. | 05 April 2019, 15:40:28 UTC |
d110b58 | Valentin Lorentz | 04 April 2019, 18:42:32 UTC | Make content_add_metadata require a ctime argument. This makes Python set the ctime instead of pgsql. | 05 April 2019, 09:15:18 UTC |
a62b1ad | Valentin Lorentz | 04 April 2019, 18:44:56 UTC | Prevent the 'snapshots' hypothesis strategy from generating loops. | 04 April 2019, 18:44:56 UTC |
ee62e05 | Valentin Lorentz | 04 April 2019, 13:54:03 UTC | Add a new endpoint content_add_metadata. | 04 April 2019, 15:15:51 UTC |
5913718 | Valentin Lorentz | 04 April 2019, 13:52:07 UTC | Refactor content_add's code into smaller functions. | 04 April 2019, 15:10:50 UTC |
4c8a0ac | Valentin Lorentz | 04 April 2019, 13:14:57 UTC | Make origin_add copy objects before changing them. hypothesis seems to reuse objects across tests now. | 04 April 2019, 13:14:57 UTC |
81a9954 | Valentin Lorentz | 03 April 2019, 11:38:58 UTC | Don't leak origin ids to the journal. | 04 April 2019, 09:47:18 UTC |
e9a3198 | Antoine R. Dumont (@ardumont) | 03 April 2019, 14:35:55 UTC | api.server: Add tests around send_metric function | 03 April 2019, 15:56:49 UTC |
1a401c0 | Antoine R. Dumont (@ardumont) | 03 April 2019, 14:09:16 UTC | api.server: Make send_metric be in charge of the metric parsing | 03 April 2019, 14:20:12 UTC |
7d6d894 | Antoine R. Dumont (@ardumont) | 03 April 2019, 13:47:06 UTC | api.server: Use constants for all metrics | 03 April 2019, 13:50:26 UTC |
1da7b50 | Antoine R. Dumont (@ardumont) | 03 April 2019, 13:42:03 UTC | api.server: Rename send_metric as we really send only 1 metric | 03 April 2019, 13:50:26 UTC |
ff05590 | Antoine R. Dumont (@ardumont) | 03 April 2019, 13:37:43 UTC | api.server: Rename constant | 03 April 2019, 13:38:18 UTC |
2f1e763 | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:59:55 UTC | api.server: Refactor repetition into a send_metrics function | 03 April 2019, 13:06:02 UTC |
6dee5f5 | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:50:37 UTC | api.server: Fix wrong api call | 03 April 2019, 13:06:02 UTC |
04792f8 | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:43:23 UTC | server: Add metrics on tool_add endpoint | 03 April 2019, 13:06:02 UTC |
ee934b6 | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:42:43 UTC | storage: Make tool_add endpoint return a list instead of a generator This aligns with other endpoints and allow to extract metrics easily | 03 April 2019, 13:06:02 UTC |
9b35b2d | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:39:25 UTC | server: Add metrics on origin_metadata_add & metadata_provider_add | 03 April 2019, 13:06:02 UTC |
6064a31 | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:28:34 UTC | api.{client,server}: Remove dead code about entity | 03 April 2019, 13:06:02 UTC |
907c4ad | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:27:30 UTC | api.server: Add metrics on origin_add and origin_add_one | 03 April 2019, 13:06:02 UTC |
b7d9896 | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:22:06 UTC | api.server: Add metrics to origin_visit_add | 03 April 2019, 13:06:02 UTC |
e357a38 | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:13:40 UTC | api.server: Include the encoding step when timing | 03 April 2019, 13:06:02 UTC |
bb19ce3 | Antoine R. Dumont (@ardumont) | 03 April 2019, 09:00:10 UTC | server.api: Fix metric names respecting prometheus conventions [1] [1] https://prometheus.io/docs/practices/naming/ | 03 April 2019, 13:06:02 UTC |
4518363 | Antoine R. Dumont (@ardumont) | 03 April 2019, 08:49:41 UTC | api.server.process_metrics: Skip processing when value is 0 | 03 April 2019, 13:06:02 UTC |
cb7b9ee | Antoine R. Dumont (@ardumont) | 03 April 2019, 08:48:59 UTC | api.server: Rename to process_metrics function name | 03 April 2019, 13:06:02 UTC |
529c4be | Antoine R. Dumont (@ardumont) | 03 April 2019, 08:48:35 UTC | storage: Fix typo in variable name | 03 April 2019, 13:06:02 UTC |
ce999b5 | Antoine R. Dumont (@ardumont) | 02 April 2019, 16:45:52 UTC | api.server: Exploit new endpoint to compute metrics | 03 April 2019, 13:06:01 UTC |
f18af30 | Antoine R. Dumont (@ardumont) | 02 April 2019, 16:44:24 UTC | swh.storage: Adapt and unify _add endpoints | 03 April 2019, 13:06:01 UTC |
ddd1835 | Antoine R. Dumont (@ardumont) | 01 April 2019, 14:27:23 UTC | api.server: Send computed metrics | 03 April 2019, 12:57:30 UTC |
007f3de | Antoine R. Dumont (@ardumont) | 01 April 2019, 14:04:25 UTC | storage*.snapshot_add: Return summary | 03 April 2019, 12:57:30 UTC |
ab7f008 | Antoine R. Dumont (@ardumont) | 01 April 2019, 13:55:07 UTC | storage*.release_add: Return summary | 03 April 2019, 12:53:50 UTC |
01b8a9a | Antoine R. Dumont (@ardumont) | 01 April 2019, 13:54:42 UTC | storage.in_memory: Fix missing filtering on release_add | 03 April 2019, 12:53:50 UTC |
f366446 | Antoine R. Dumont (@ardumont) | 01 April 2019, 13:48:36 UTC | storage*.revision_add: Return summary | 03 April 2019, 12:53:50 UTC |
fb4be03 | Antoine R. Dumont (@ardumont) | 01 April 2019, 13:40:44 UTC | storage: Align implementations to compute correct metrics It was simpler to return the correct number instead of making the in-memory returns wrong results. | 03 April 2019, 12:53:50 UTC |
25da407 | Antoine R. Dumont (@ardumont) | 01 April 2019, 13:11:16 UTC | directory_add: Normalize metrics keys | 03 April 2019, 12:53:50 UTC |
cd78364 | Antoine R. Dumont (@ardumont) | 01 April 2019, 11:59:43 UTC | content_add: Normalize the summary keys | 03 April 2019, 12:53:50 UTC |
b341446 | Antoine R. Dumont (@ardumont) | 01 April 2019, 10:27:57 UTC | api.server.directory_add: Add counter metrics | 03 April 2019, 12:53:50 UTC |
ecbb611 | Antoine R. Dumont (@ardumont) | 01 April 2019, 10:27:42 UTC | storage*.directory_add: Return summary | 03 April 2019, 12:53:50 UTC |
71bd5bf | Antoine R. Dumont (@ardumont) | 01 April 2019, 10:05:07 UTC | storage.in_memory.content_add: Adapt implementation to send summary | 03 April 2019, 12:53:50 UTC |
a284ec0 | Antoine R. Dumont (@ardumont) | 01 April 2019, 09:58:06 UTC | storage: Update tests to check the summary result act as expected | 03 April 2019, 12:53:50 UTC |
ee62f46 | Antoine R. Dumont (@ardumont) | 01 April 2019, 09:53:50 UTC | api.server: Remove encode step from metrics | 03 April 2019, 12:53:49 UTC |
7f798bb | Antoine R. Dumont (@ardumont) | 01 April 2019, 09:53:30 UTC | api.server: Update docstrings on content_add to explicit the output | 03 April 2019, 12:53:49 UTC |
3e45179 | Antoine R. Dumont (@ardumont) | 29 March 2019, 16:13:58 UTC | Install counter metrics | 03 April 2019, 12:53:49 UTC |
2539858 | Antoine R. Dumont (@ardumont) | 29 March 2019, 09:39:12 UTC | api.server: Remove timed metric on the get_storage function | 03 April 2019, 12:53:49 UTC |
86308f6 | Valentin Lorentz | 01 April 2019, 14:11:44 UTC | Make snapshot_add take an iterable of snapshot. For uniformity with other _add() endpoints (except origin_visit_add). Noone uses the new signature of snapshot_add except the tests yet, so it's the perfect time to make this breaking change. | 03 April 2019, 12:47:02 UTC |
1ddb145 | Valentin Lorentz | 01 April 2019, 10:00:51 UTC | Use sha1 instead of bigint as FK from origin_visit to snapshot (part 3: cleanup) | 01 April 2019, 13:21:24 UTC |
7b56701 | Valentin Lorentz | 01 April 2019, 09:56:43 UTC | Use sha1 instead of bigint as FK from origin_visit to snapshot (part 2: backfill + start using the new colum for reading) | 01 April 2019, 13:21:24 UTC |
a150fdf | Valentin Lorentz | 01 April 2019, 09:50:30 UTC | Use sha1 instead of bigint as FK from origin_visit to snapshot (part 1: add new column) | 01 April 2019, 13:17:34 UTC |
480e098 | Valentin Lorentz | 01 April 2019, 13:03:53 UTC | Fix import. | 01 April 2019, 13:03:53 UTC |
3c849b5 | Nicolas Dandrimont | 28 March 2019, 17:41:19 UTC | Remove lingering debug print | 28 March 2019, 17:41:19 UTC |
742dd4f | Valentin Lorentz | 28 March 2019, 12:28:16 UTC | Split Storage.snapshot_add (add the snapshot + update the origin_visit) `snapshot_add(origin, visit, snap)` becomes `snapshot_add(origin)` + `origin_visit_update(origin, visit, snapshot_id=snap['id'])` | 28 March 2019, 15:01:49 UTC |
3457207 | Valentin Lorentz | 28 March 2019, 12:27:33 UTC | Add a new argument snapshot_id to origin_visit_update, and make all its arguments optional. | 28 March 2019, 12:27:33 UTC |
246855c | Valentin Lorentz | 26 March 2019, 16:11:06 UTC | Add a new JournalWriter interface, which is notified by swh-storage before writing to pgsql. | 28 March 2019, 10:42:27 UTC |
f734c4e | Antoine R. Dumont (@ardumont) | 28 March 2019, 08:58:54 UTC | api.server: Remove unneeded stats.increment call as this is already done further down the stack | 28 March 2019, 08:58:54 UTC |
3c2dfc0 | Antoine R. Dumont (@ardumont) | 27 March 2019, 18:35:23 UTC | swh.storage.api.server: Add metrics to storage endpoints | 27 March 2019, 19:24:56 UTC |
3a7f8a9 | Antoine R. Dumont (@ardumont) | 26 March 2019, 17:49:33 UTC | Add @timed decorator around all api.server endpoints | 26 March 2019, 17:49:33 UTC |
c47f92a | Antoine R. Dumont (@ardumont) | 26 March 2019, 13:46:45 UTC | storage: Add a timed decoration to count calls and time it | 26 March 2019, 14:19:43 UTC |
aaab31c | Nicolas Dandrimont | 25 March 2019, 15:26:40 UTC | Mark the node id columns as not null for directory, revision and snapshot This is an oversight from the migration to split indexes. | 25 March 2019, 15:31:28 UTC |
4ed7d0a | David Douard | 06 March 2019, 11:27:41 UTC | API: refactor the /directory/ls endpoint use the generic POST based implementation. | 18 March 2019, 15:58:11 UTC |
97981b2 | David Douard | 06 March 2019, 11:24:46 UTC | API: simplify a bit the RemoteStorage class set the exception class as class attribute wich allows to get rid of the __init__ overloading. | 18 March 2019, 15:57:14 UTC |
8bbbf9f | Valentin Lorentz | 26 February 2019, 09:50:44 UTC | Add an helper function to list all origins in the storage. | 11 March 2019, 13:20:52 UTC |
c533d6e | Valentin Lorentz | 27 February 2019, 09:42:29 UTC | Double the timeout of revision_get. Metadata indexers often hit the limit. | 27 February 2019, 09:42:29 UTC |
a1b8387 | Antoine R. Dumont (@ardumont) | 21 February 2019, 13:59:02 UTC | api.server: Fix wrong exception type | 21 February 2019, 13:59:02 UTC |
b4f2042 | Antoine R. Dumont (@ardumont) | 21 February 2019, 12:43:53 UTC | storage.cli: Fix cli entry point name to the expected name (setup.py) | 21 February 2019, 12:43:53 UTC |
95b0b42 | Antoine R. Dumont (@ardumont) | 21 February 2019, 12:28:51 UTC | api.server: Use the right Exception for the job | 21 February 2019, 12:28:51 UTC |
247ab97 | Antoine R. Dumont (@ardumont) | 21 February 2019, 12:23:36 UTC | test_server: Refactor and fix issue for pytest on python3.5 Prior to this commit, the build failed with: TypeError: argument should be string, bytes or integer, not LocalPath | 21 February 2019, 12:23:36 UTC |
a2db5ef | Antoine R. Dumont (@ardumont) | 21 February 2019, 11:21:42 UTC | api.server: Remove unused import | 21 February 2019, 11:21:42 UTC |
b38ecc0 | Antoine R. Dumont (@ardumont) | 21 February 2019, 11:16:32 UTC | swh.storage.cli: Migrate the api server startup in swh.storage.cli | 21 February 2019, 11:21:11 UTC |
2d500d1 | Antoine R. Dumont (@ardumont) | 21 February 2019, 10:50:52 UTC | api.wsgi: Open wsgi entrypoint and check config at startup time Related T1533 | 21 February 2019, 11:21:10 UTC |
750e292 | Antoine R. Dumont (@ardumont) | 21 February 2019, 10:49:35 UTC | api.server: Remove implicit config from Also: - add tests to the loading checks - rename run_from_webserver to make_app_from_configfile Related T1533 | 21 February 2019, 11:21:10 UTC |
2696b47 | Antoine R. Dumont (@ardumont) | 20 February 2019, 17:21:12 UTC | api.server: Make the api server load and check its configuration Related T1533 | 21 February 2019, 11:21:10 UTC |