| e1a460c | Antoine R. Dumont (@ardumont) | 09 November 2018, 17:40:49 UTC | tests: Move property_based marker to toplevel test class | 14 November 2018, 15:07:33 UTC |
| 3829885 | Antoine R. Dumont (@ardumont) | 14 November 2018, 14:08:35 UTC | tests: Use sorted | 14 November 2018, 15:07:33 UTC |
| c876d86 | Antoine R. Dumont (@ardumont) | 09 November 2018, 15:27:19 UTC | storage: Open content_get_range endpoint Related T991 | 14 November 2018, 15:07:33 UTC |
| a429574 | Antoine R. Dumont (@ardumont) | 14 November 2018, 13:58:33 UTC | tests: Use sorted on list directly | 14 November 2018, 15:07:33 UTC |
| c7a3266 | Antoine R. Dumont (@ardumont) | 14 November 2018, 13:58:17 UTC | tests: Build content generated in less steps | 14 November 2018, 15:07:33 UTC |
| f87808c | Antoine R. Dumont (@ardumont) | 14 November 2018, 13:58:05 UTC | tests: Clarify gen_content docstring | 14 November 2018, 15:07:33 UTC |
| 0c51e4d | Antoine R. Dumont (@ardumont) | 09 November 2018, 13:53:11 UTC | tests: Add test content generation util function and use it Test back `content_get` and `content_get_metadata` as a first step. Plan to use those for the new and upcoming `content_get_range` function. | 14 November 2018, 15:07:32 UTC |
| 823465f | Valentin Lorentz | 14 November 2018, 14:55:56 UTC | Remove SQLisms from the tests and API. Reviewers: #reviewers, olasd Reviewed By: #reviewers, olasd Subscribers: olasd, swh-public-ci Differential Revision: https://forge.softwareheritage.org/D642 | 14 November 2018, 14:58:30 UTC |
| 64e67aa | David Douard | 12 November 2018, 13:05:25 UTC | make-doc: add an 'assets' make target for the doc used by the swh-docs to generate images and schemas when building the whole documentation. | 12 November 2018, 15:19:22 UTC |
| 1031dc5 | Antoine Lambert | 12 November 2018, 12:56:28 UTC | debian: Remove nose and migrate to pytest | 12 November 2018, 13:10:07 UTC |
| 1a1d214 | Antoine Lambert | 12 November 2018, 10:12:36 UTC | algos.revisions_walker: Fix get_revisions_walker doc | 12 November 2018, 10:12:36 UTC |
| 14e17d1 | Valentin Lorentz | 09 November 2018, 11:08:01 UTC | Fix storage doc. Reviewers: #reviewers, olasd Reviewed By: #reviewers, olasd Subscribers: olasd, swh-public-ci Differential Revision: https://forge.softwareheritage.org/D643 | 09 November 2018, 17:38:49 UTC |
| b944f00 | Valentin Lorentz | 09 November 2018, 11:10:35 UTC | Restructure tests to prepare for tests with no DB. Reviewers: #reviewers, olasd Reviewed By: #reviewers, olasd Subscribers: swh-public-ci Differential Revision: https://forge.softwareheritage.org/D644 | 09 November 2018, 17:38:38 UTC |
| 7934ded | David Douard | 09 November 2018, 12:23:30 UTC | Fix rst syntax in docstrings | 09 November 2018, 14:15:17 UTC |
| 81b8c9b | David Douard | 09 November 2018, 12:21:24 UTC | Add :orphan: metadata to archive-copies and sql-storage doc files to prevent sphinx from complaining about these not being included. | 09 November 2018, 14:15:17 UTC |
| 88c33cc | Antoine Lambert | 06 November 2018, 21:19:01 UTC | algos: Add iterators to walk across revisions history This commit adds iterators to walk across the history of revisions heading to a given one. The following types of iteration are offered: - committer_date: revisions are returned in reverse chronological order of their commit date (same as git log) - dfs: revisions are returned in the same order they are visited when performing a depth-first search in pre order on the revisions DAG - dfs_post: revisions are returned in the same order they are visited when performing a depth-first search in post order on the revisions DAG - bfs: revisions are returned in the same order they are visited when performing a breadth-first search on the revisions DAG Another iterator of type path, returning only revisions that modify a specific path in reverse chronological order of their commit date, is also introduced. Nevertheless, due to client-side implementation, its performances are far from optimal when walking across a really large history. Related T1026 Related T1284 | 09 November 2018, 13:04:04 UTC |
| 19d4019 | Valentin Lorentz | 08 November 2018, 13:32:28 UTC | Rename the 'ts' argument of 'origin_visit_add' to 'date'. Summary: To be consistent with the name used in the DB and the dict key returned by 'origin_visit_get'. Note that this makes the client incompatible with older versions of the server. Reviewers: #reviewers, olasd Reviewed By: #reviewers, olasd Subscribers: olasd, swh-public-ci Differential Revision: https://forge.softwareheritage.org/D636 | 09 November 2018, 10:45:41 UTC |
| 884d9e7 | Antoine Lambert | 08 November 2018, 13:42:42 UTC | sql/Makefile: Fix pifpaf test as it was always considered used | 08 November 2018, 13:43:03 UTC |
| 465192e | Antoine R. Dumont (@ardumont) | 18 October 2018, 14:23:59 UTC | storage.listener: Subscribe listener to snapshot insertion Related T1275 | 26 October 2018, 10:21:52 UTC |
| 1ad38af | Antoine R. Dumont (@ardumont) | 18 October 2018, 13:24:09 UTC | sql: Add snapshot trigger event on insertion Related T1275 | 26 October 2018, 10:21:52 UTC |
| fba230d | Stefano Zacchiroli | 26 October 2018, 07:57:04 UTC | DB schema: manual fix for some arrow source/dest to avoid degenerate angles in rendering | 26 October 2018, 07:57:04 UTC |
| a579a1a | Stefano Zacchiroli | 26 October 2018, 07:51:57 UTC | DB schema: add light foreign key arrows from snapshot to * | 26 October 2018, 07:51:57 UTC |
| b032d2a | Stefano Zacchiroli | 26 October 2018, 07:51:25 UTC | DB schema: remove left-over occurrence stuff from snapshot cluster | 26 October 2018, 07:51:25 UTC |
| 1c74f6d | Antoine R. Dumont (@ardumont) | 24 October 2018, 13:42:12 UTC | tox.ini: Align configuration file with other Also, without this, this failed because somehow, something is not finding requirements-test.txt. ``` Could not open requirements file: [Errno 2] No such file or directory: ' requirements-test.txt' ``` | 24 October 2018, 13:42:12 UTC |
| 7628710 | Stefano Zacchiroli | 24 October 2018, 12:35:02 UTC | SQL schema: update to reflect recent changes recent changes include most notably the removal of occurrence-related tables, but also the addition of table columns accumulated over time Closes T1283 | 24 October 2018, 12:35:02 UTC |
| a6742b7 | Stefano Zacchiroli | 24 October 2018, 12:34:52 UTC | DB schema doc: error out if SQL schema cannot be found | 24 October 2018, 12:34:52 UTC |
| 582d72d | Stefano Zacchiroli | 24 October 2018, 12:22:42 UTC | docs Makefile: only echo SQL queries on error | 24 October 2018, 12:22:48 UTC |
| 5fe9221 | Stefano Zacchiroli | 24 October 2018, 12:20:01 UTC | docs Makefile: forward "distclean" to sql dir | 24 October 2018, 12:20:01 UTC |
| b91482a | Stefano Zacchiroli | 24 October 2018, 12:00:01 UTC | gitignore and remove generate archive copies diagram Closes T1269 | 24 October 2018, 12:00:01 UTC |
| 242f985 | Nicolas Dandrimont | 23 October 2018, 14:48:15 UTC | Don't run pytest in the docs directory | 23 October 2018, 14:48:15 UTC |
| e77d809 | David Douard | 22 October 2018, 16:06:59 UTC | tests: increase deadline for CI seems to run slower on there. | 22 October 2018, 16:06:59 UTC |
| 076144e | David Douard | 18 October 2018, 15:17:09 UTC | tests: assertEquals -> assertEqual | 22 October 2018, 14:46:34 UTC |
| 32ef182 | David Douard | 18 October 2018, 15:11:35 UTC | tests: use pytest and add a simple tox file related to T1261 | 22 October 2018, 14:45:08 UTC |
| 5b38656 | David Douard | 18 October 2018, 15:07:19 UTC | flake8 | 22 October 2018, 14:32:18 UTC |
| a8fbce5 | Valentin Lorentz | 22 October 2018, 07:27:41 UTC | Warn when using db_transaction/db_transaction_generator where you shouldn't. Summary: I did it, it caused closed cursors that hang the process, that's hard to debug. Reviewers: #reviewers, ardumont Reviewed By: #reviewers, ardumont Subscribers: ardumont Differential Revision: https://forge.softwareheritage.org/D556 | 22 October 2018, 09:03:36 UTC |
| d211615 | Nicolas Dandrimont | 18 October 2018, 16:30:44 UTC | Mark snapshot tests as needing the database | 18 October 2018, 16:30:44 UTC |
| b2d4921 | Nicolas Dandrimont | 18 October 2018, 16:27:56 UTC | Mark minimum hypothesis version for datetimes | 18 October 2018, 16:27:56 UTC |
| ec7a119 | Nicolas Dandrimont | 18 October 2018, 16:25:07 UTC | Add hypothesis to debian build-dependencies | 18 October 2018, 16:25:07 UTC |
| ad92248 | Nicolas Dandrimont | 17 October 2018, 19:25:42 UTC | Add a function to retrieve a full snapshot Summary: This function wraps the paginating branch listing API to do its work. It's in swh.storage.algos because it should be done client-side rather than server-side. The tests are fairly simple, and mostly add a bunch of tooling around hypothesis to generate data that conforms to the swh data schema. Those things could probably be adequately moved to a new submodule of swh.model. Test Plan: new tests added Reviewers: anlambert Differential Revision: https://forge.softwareheritage.org/D551 | 18 October 2018, 14:57:55 UTC |
| 7cd7597 | Nicolas Dandrimont | 17 October 2018, 14:54:14 UTC | Rewrite snapshot_get in terms of snapshot_get_branches Summary: This increases uniformity and reduces code duplication between the two functions. The addition of the 'next_branch' return value in snapshot_get_branches allows callers to get branches incrementally in a loop which is friendlier. Depends on D545 Test Plan: tests have been updated to add the new next_branch field Reviewers: anlambert Differential Revision: https://forge.softwareheritage.org/D546 | 17 October 2018, 14:56:42 UTC |
| 0d52bf4 | Nicolas Dandrimont | 17 October 2018, 14:51:59 UTC | Rewrite snapshot pagination tests declaratively Summary: This looks less confusing than removing a bunch of values from the test data. Test Plan: make test still passes Reviewers: anlambert Differential Revision: https://forge.softwareheritage.org/D545 | 17 October 2018, 14:56:07 UTC |
| 80cd420 | David Douard | 17 October 2018, 13:56:52 UTC | packaging: bump swh-core dependency to 0.0.44 | 17 October 2018, 13:56:52 UTC |
| 9ce5f78 | David Douard | 17 October 2018, 13:56:13 UTC | README: swh-storage-testdata is not required any more | 17 October 2018, 13:56:13 UTC |
| a9c32d2 | David Douard | 16 October 2018, 08:59:14 UTC | sql: adapt Makefile for sql file renames and make it pifpaf compliant ie. delete stamps when executed via pifpaf for the db setup. | 17 October 2018, 13:46:36 UTC |
| ee27736 | David Douard | 16 October 2018, 08:58:09 UTC | kill sql/db-init in favor of swh-core's new swh-db-init tool | 17 October 2018, 13:46:36 UTC |
| c4cdb64 | David Douard | 12 October 2018, 11:27:47 UTC | tests: refactor the StorageTestFixture class and move sql files in the package so we can build the db setup sql file on the fly: - make it inherit from SingleDbTestFixture, so we can - move sql files in swh/storage/sql/ - get rid og the setUpClass static method to reuse the one from SingleDbTestFixture, - apply the required modifications to tests based on this StorageTestFixture. Warnings: - this requires D538, - this will break tests in swh-vault. Also add README.md to the MANIFEST.in file so it does not require a too recent setuptool. | 17 October 2018, 13:46:36 UTC |
| 8f9af9b | Nicolas Dandrimont | 17 October 2018, 13:16:31 UTC | Remove some leftover compatibility code | 17 October 2018, 13:16:31 UTC |
| 2d5efac | Nicolas Dandrimont | 17 October 2018, 12:44:42 UTC | Reorder upgrade statements so that they actually pass | 17 October 2018, 12:44:42 UTC |
| 4f05900 | Nicolas Dandrimont | 17 October 2018, 12:22:08 UTC | Remove redundant removals from the 122->123 upgrade | 17 October 2018, 12:22:08 UTC |
| 435ebcd | Nicolas Dandrimont | 15 October 2018, 08:09:29 UTC | Drop table occurrence_history Summary: The migration to snapshots has been completed, and all users have been migrated. Depends on D534. Close T830. Test Plan: make test after a testdata rebuild is ok Reviewers: zack Maniphest Tasks: T830 Differential Revision: https://forge.softwareheritage.org/D535 | 15 October 2018, 14:22:17 UTC |
| b486063 | Nicolas Dandrimont | 15 October 2018, 13:49:18 UTC | Drop confusing functions revision_get_by/revision_log_by Summary: The combination of timestamp search for an occurrence with branch name filtering never got ported to the snapshot model. As this function only has a single user, which is a single endpoint deep in the public-facing API, let's just drop it. revision_log_by uses the same filter but gives a log of parent revisions, and has never been used by any client. This will break this API endpoint, which is tracked in T1221, but iff we want to keep it, I think it can be easily reimplemented by composing origin_visit_get_by and the new snapshot_get_branches function. Test Plan: It's dead, Jim. Reviewers: zack Differential Revision: https://forge.softwareheritage.org/D534 | 15 October 2018, 13:58:52 UTC |
| 3a59229 | Nicolas Dandrimont | 12 October 2018, 09:54:00 UTC | Rename test methods to test_* to allow py.test collection Summary: Part of T1261 Test Plan: Make test runs the same number of tests before and after Reviewers: #reviewers! Differential Revision: https://forge.softwareheritage.org/D522 | 12 October 2018, 09:57:14 UTC |
| 65e6b69 | Nicolas Dandrimont | 10 October 2018, 14:23:33 UTC | Drop unused entity tables Summary: Close T838 Test Plan: Remove all the tests! Reviewers: #reviewers! Maniphest Tasks: T838 Differential Revision: https://forge.softwareheritage.org/D509 | 10 October 2018, 14:25:54 UTC |
| ee40117 | Nicolas Dandrimont | 10 October 2018, 13:55:36 UTC | Merge remote-tracking branch 'origin/master' | 10 October 2018, 13:55:36 UTC |
| e9e5e73 | Antoine Lambert | 03 October 2018, 09:33:32 UTC | storage: Enable to paginate, filter and count snapshot content Related T1207 | 09 October 2018, 13:28:34 UTC |
| 80b4644 | Nicolas Dandrimont | 08 October 2018, 15:03:46 UTC | Update version requirements for swh.model | 08 October 2018, 15:03:46 UTC |
| 7ae19db | David Douard | 04 October 2018, 15:16:56 UTC | add build/ to gitignore | 08 October 2018, 10:22:39 UTC |
| 63c924d | David Douard | 04 October 2018, 15:15:31 UTC | setup: prepare for pypi upload related to T1242 | 08 October 2018, 10:22:32 UTC |
| b1eaa9a | David Douard | 04 October 2018, 15:14:20 UTC | docs: merge both README files in a single README.md also documement the usage of pifpaf to run pg tests | 05 October 2018, 14:38:22 UTC |
| fb794bf | Antoine Pietri | 05 October 2018, 10:56:14 UTC | Add requirements-test.txt | 05 October 2018, 10:56:16 UTC |
| 3d1b95e | Nicolas Dandrimont | 03 October 2018, 16:46:17 UTC | Remove the occurrence table Summary: This table is obsolete, and was never really used in the first place. This solves the first half of T839. Depends on D471. Test Plan: rebuild-testdata and make test Reviewers: #reviewers! Differential Revision: https://forge.softwareheritage.org/D472 | 03 October 2018, 16:55:46 UTC |
| 842b7a3 | Nicolas Dandrimont | 03 October 2018, 16:32:20 UTC | Remove some unused SQL functions Summary: These functions aren't used by any known clients Test Plan: rebuild-testdata + make test Reviewers: #reviewers! Differential Revision: https://forge.softwareheritage.org/D471 | 03 October 2018, 16:34:31 UTC |
| 5193e6d | Nicolas Dandrimont | 03 October 2018, 15:15:56 UTC | Drop backwards-compatibility features between snapshots and occurrences Summary: Old occurrences have been fully migrated to snapshots now, so backwards compatibility is not needed anymore. That's one more step towards the removal of occurrences (T830). This changes the origin_visit_get_by method to stop returning a list of occurrences. The only user of this method is the api/1/origin/{x}/visit/{y} view of the web API, which will need to be updated for T1207 anyway. Test Plan: integration tests updated for origin_visit_get_by; Running the web app with the updated storage only yields the expected change in behavior for the origin/visit API view, navigation is unaffected. Reviewers: #reviewers! Differential Revision: https://forge.softwareheritage.org/D466 | 03 October 2018, 16:18:42 UTC |
| 4b1db40 | David Douard | 03 October 2018, 13:39:42 UTC | doc: use inkscape instead of dia to generate pdf and svg image files | 03 October 2018, 13:44:09 UTC |
| 8e545c7 | Stefano Zacchiroli | 27 September 2018, 08:11:51 UTC | SQL schema: uniformize comments Summary: Partial cherry-pick of ddb37a5896d04a95cafee413d048bea8df265379 Test Plan: rebuild-testdata and make test report no changes Reviewers: #reviewers! Subscribers: zack Differential Revision: https://forge.softwareheritage.org/D463 | 03 October 2018, 13:14:04 UTC |
| 1751385 | Nicolas Dandrimont | 03 October 2018, 10:24:03 UTC | Remove unused function release_get_by Summary: release_get_by has never been used except by its own tests, and it still uses the legacy occurrence_* tables that we want to remove (T830). Test Plan: - make test in swh-environment passes - grep -r release_get_by swh-* yields no result Reviewers: #reviewers! Differential Revision: https://forge.softwareheritage.org/D460 | 03 October 2018, 12:32:27 UTC |
| 16ffb6e | Stefano Zacchiroli | 03 October 2018, 09:11:19 UTC | docs: make sure images are built/cleaned on sphinx targets | 03 October 2018, 09:11:19 UTC |
| 7fa1a34 | Stefano Zacchiroli | 03 October 2018, 09:10:46 UTC | images/Makefile: fix filenames they have been broken since the recent rename | 03 October 2018, 09:10:46 UTC |
| f5dd8ac | Stefano Zacchiroli | 02 October 2018, 08:53:49 UTC | nicer landing page, pointing to homepage and API doc Closes T1194 | 02 October 2018, 08:53:49 UTC |
| 003e1a7 | Stefano Zacchiroli | 01 October 2018, 13:47:54 UTC | docs: add title and brief module description | 01 October 2018, 13:47:54 UTC |
| 2eced78 | Antoine Lambert | 26 September 2018, 12:14:36 UTC | algos.dir_iterators: Implement iterator protocol in DirectoryIterator - add Python iterator protocol support in the DirectoryIterator class in order to easily visit in a recursive way any directory stored in the archive. - add convenient function dir_iterator wrapping the instantiation of the DirectoryIterator class - add tests Related T1177 | 28 September 2018, 12:32:17 UTC |
| 9adaed7 | Antoine Lambert | 25 September 2018, 17:18:10 UTC | algos.dir_iterators: Fix handling of empty directory Two issues found regarding the way empty directories were handled: - _empty_dir_hash variable did not have correct type (str instead of bytes) so empty directory test based on hash comparison was always failing - in the step method of DirectoryIterator, no need to push a new frame for an empty directory as this will stop the iteration | 28 September 2018, 12:32:17 UTC |
| 3a86982 | Antoine Pietri | 25 September 2018, 10:56:12 UTC | sql/bin/db-init: remove createuser and privileges (we now assume a peer superuser) | 25 September 2018, 11:05:21 UTC |
| 1286dfa | Antoine Pietri | 24 September 2018, 15:56:54 UTC | sql/bin/db-init: simpler usage with sensible defaults | 25 September 2018, 10:52:10 UTC |
| 332a1e1 | Stefano Zacchiroli | 19 September 2018, 15:19:44 UTC | use /usr/bin/env python3 as shebang, to be nice to virtualenv | 19 September 2018, 15:19:44 UTC |
| e01f26c | Antoine Lambert | 19 September 2018, 09:31:40 UTC | swh.storage.db: Fix origin_visit_get_latest_snapshot query Prior to this fix, storage.snapshot_get_latest(origin) was returning the first snapshot instead of the last one. | 19 September 2018, 09:32:04 UTC |
| 384b120 | Stefano Zacchiroli | 07 September 2018, 11:28:56 UTC | docs: add description of the archive copies layout diagram | 07 September 2018, 11:28:56 UTC |
| a66745a | Stefano Zacchiroli | 07 September 2018, 09:38:45 UTC | docs: add layout diagram describing existing archive copies | 07 September 2018, 09:39:44 UTC |
| 46822cb | Antoine R. Dumont (@ardumont) | 06 September 2018, 16:42:33 UTC | Add snapshot to the statistics endpoint counter Related T1180 | 06 September 2018, 18:41:18 UTC |
| 8ccac07 | Stefano Zacchiroli | 06 September 2018, 10:19:41 UTC | docs: add general intro, drop bogus reference to archiver blueprint | 06 September 2018, 10:19:41 UTC |
| 19c0344 | Stefano Zacchiroli | 06 September 2018, 10:19:20 UTC | algos/diff.py: fix typo in comment brought to you by codespell | 06 September 2018, 10:19:20 UTC |
| 329d0f9 | Antoine Pietri | 05 September 2018, 12:30:37 UTC | storage: set timeout to 20s for directory_ls | 05 September 2018, 12:30:46 UTC |
| 9c39449 | Antoine Lambert | 02 August 2018, 13:01:24 UTC | origin_search: Add option to filter out origins with no visit | 02 August 2018, 13:01:24 UTC |
| 9e5a8b7 | Antoine R. Dumont (@ardumont) | 27 July 2018, 13:52:30 UTC | swh.storage.storage: origin_add returns updated list of dict with id Prior to this commit, this returned only the list of new ids. This is currently not used anywhere in our stack. Would it have been, this would have force the client to try and be smart for dealing with ids. | 27 July 2018, 13:57:13 UTC |
| a167960 | Nicolas Dandrimont | 31 May 2018, 15:06:28 UTC | Migrate away from temporary tables for read queries Summary: To do so, we import a function from a recent version of psycopg2, execute_values, which can execute queries efficiently with a list of values. We also scale the cursor back from having things in SQL functions towards having things inside the db.py database "backend". This will make it easier to iterate, as we won't have to deploy function changes to twenty different databases. After these changes, testing the web UI on a physical replica works. Close T1073 Test Plan: Local integration tests are happy; Navigating the frontend backed by a physical replica seems to be okay now. Reviewers: #reviewers! Maniphest Tasks: T1073 Differential Revision: https://forge.softwareheritage.org/D340 | 05 June 2018, 11:56:09 UTC |
| 7c85120 | Nicolas Dandrimont | 04 June 2018, 14:33:24 UTC | Remove now unused function directory_get Summary: As of D337, directory_get has no users left; Time to remove it. Test Plan: make test in the toplevel environment still works Reviewers: #reviewers! Differential Revision: https://forge.softwareheritage.org/D339 | 04 June 2018, 15:36:44 UTC |
| 9f2b266 | Nicolas Dandrimont | 30 May 2018, 12:31:36 UTC | Make sure there are no pagination issues for origin_visit_get Although most visits should be sequential, and the date should be monotonic, re-importing an old snapshot as a new visit would break the pagination. | 30 May 2018, 12:31:36 UTC |
| c4494aa | Nicolas Dandrimont | 29 May 2018, 15:30:18 UTC | Fix codespell issues | 29 May 2018, 15:30:18 UTC |
| 779cfb5 | Nicolas Dandrimont | 29 May 2018, 15:26:24 UTC | Don't assign the objstorage result to a variable | 29 May 2018, 15:26:24 UTC |
| 2c17350 | Nicolas Dandrimont | 29 May 2018, 15:22:31 UTC | Rewrite concurrent.futures import to shorten lines | 29 May 2018, 15:22:31 UTC |
| 79cc3f6 | Nicolas Dandrimont | 29 May 2018, 14:38:50 UTC | Better test coverage for content_missing | 29 May 2018, 14:38:50 UTC |
| 70574c5 | Nicolas Dandrimont | 28 May 2018, 16:04:44 UTC | Don't use a temporary table to fetch info about contents | 28 May 2018, 16:04:44 UTC |
| 6d5d999 | Nicolas Dandrimont | 28 May 2018, 12:37:52 UTC | Make the db_transaction{,_generator} decorators support client options Summary: Allow adding server-side statement timeouts for database operations Test Plan: make test still works Reviewers: #reviewers! Differential Revision: https://forge.softwareheritage.org/D334 | 28 May 2018, 13:18:26 UTC |
| d826f75 | Antoine R. Dumont (@ardumont) | 24 May 2018, 10:02:11 UTC | swh.storage.api.client: Permit to specify the query timeout option Related T1061 | 24 May 2018, 10:02:11 UTC |
| 77e69ec | Nicolas Dandrimont | 12 May 2018, 15:49:44 UTC | Use a concurrent.future to parallelize objstorage and storage addition | 12 May 2018, 15:49:44 UTC |
| 4a9b623 | Nicolas Dandrimont | 11 May 2018, 13:40:03 UTC | test_api_client: stop leaking directories Summary: When mkdtemp is called, shutil.rmtree must be called as well Test Plan: Look at /tmp before and after running tests, notice no new directories instead of 60. Reviewers: #reviewers! Differential Revision: https://forge.softwareheritage.org/D331 | 11 May 2018, 13:42:09 UTC |
| 721d088 | Nicolas Dandrimont | 11 May 2018, 10:10:18 UTC | Add test to ensure storage/objstorage consistency is kept at all times Summary: The behavior of storage when the underlying objstorage had an exception was never actually tested. This new test weeded out a bug in the threaded implementation for copy_to. Test Plan: the new test passes Reviewers: #reviewers! Differential Revision: https://forge.softwareheritage.org/D330 | 11 May 2018, 10:13:44 UTC |
| 18c9dad | Nicolas Dandrimont | 07 May 2018, 13:19:34 UTC | Only instantiate the storage backend once per import This allows connection reuse for postgresql and potential remote backends such as for the object storage, rather than reinitiating all connections on every request. | 09 May 2018, 14:43:32 UTC |
| 5a2de1c | Nicolas Dandrimont | 04 May 2018, 17:19:55 UTC | Use thread-aware psycopg2 connection pooling for database access Summary: This allows to use swh.storage with a modicum of concurrency Test Plan: clearly, make test should still pass Reviewers: #reviewers! Differential Revision: https://forge.softwareheritage.org/D325 | 07 May 2018, 13:52:35 UTC |
| 6c93693 | Nicolas Dandrimont | 04 May 2018, 15:03:40 UTC | Stop using the storage.db attribute directly Summary: Add a level of indirection to allow swapping out the implementation of the db attribute Test Plan: once again, make test keeps on working Reviewers: #reviewers! Differential Revision: https://forge.softwareheritage.org/D324 | 07 May 2018, 13:46:21 UTC |
