swh:1:snp:eb70f1f85391e4b077c211bec36af0061c4bf937

sort by:
Revision Author Date Message Commit Date
006d102 Double the timeout of snapshot_get_latest. Metadata indexers often hit the limit. 21 February 2019, 09:18:50 UTC
0b10c28 Remove debian packaging from master branch Related T1532 19 February 2019, 11:05:46 UTC
152cb52 Move origin_get_range's right after origin_get. 15 February 2019, 14:10:09 UTC
a94451a api/server: Do not read configuration at each request 13 February 2019, 22:55:12 UTC
e2c4d38 storage: Add origin_count method Related T1463 12 February 2019, 13:25:54 UTC
1efbe5c Fix test to work with the latest version of swh.core. 12 February 2019, 13:14:14 UTC
e59eb38 Make Storage.origin_get support a list of origins, like other Storage.*_get methods. Backward compatibility is preserved. 07 February 2019, 14:38:01 UTC
eeecf6f Stop using _to_bytes functions. 01 February 2019, 14:02:21 UTC
8cb3ca2 Use the BaseDb (and friends) from swh-core since it has migrated there. 28 January 2019, 16:31:19 UTC
4c49b3f storage.in_memory: Fix unknown revision handling in revision_log method 25 January 2019, 15:37:03 UTC
17b13af storage.in_memory: Fix unknown origin handling in origin*_get methods 25 January 2019, 15:15:13 UTC
f308c10 debian/rules: Exclude hypothesis generated data from package 25 January 2019, 13:55:19 UTC
76d7838 storage: Add origin_get_range method to list all archived origins Related T1399 17 January 2019, 10:50:39 UTC
69f48be sql/upgrades/129: Add missing trigger for origin visit update event Also no need for a new function notify_changed_origin_visit since the new function does the same as the original one. Related T1466 11 January 2019, 15:19:45 UTC
d19392c Notify Kafka when an origin visit is update. 10 January 2019, 15:51:24 UTC
c5cee88 storage.db: Remove unused person_add method Related D897 09 January 2019, 15:25:00 UTC
1f73266 storage.in_memory: Return copies of origin visit dicts 09 January 2019, 15:20:24 UTC
12a2dd6 Add in_memory.person_get implementation and improve person tests Related T1377 09 January 2019, 15:16:16 UTC
fc7c534 listener: Make kafka client be less verbose 20 December 2018, 12:11:24 UTC
d124710 listener: Use swh.model.hashutil.hash_to_bytes 20 December 2018, 12:10:53 UTC
a229a07 listener: Add logging behavior 20 December 2018, 10:48:35 UTC
2a03e1c listener: Adapt decoding behavior depending on the object type This fixes issues regarding origin/origin_visit 20 December 2018, 10:46:20 UTC
d19a0d1 Update requirements to latest swh.core Related T1444 14 December 2018, 14:45:23 UTC
02e8c76 storage.in_memory: Use in-memory objstorage to store contents 11 December 2018, 09:53:19 UTC
d5e8229 storage.in_memory: Use integer ids for origin and visit instead of tuples Related T1421 04 December 2018, 14:58:15 UTC
bda7cf1 snapshot_add: Check the origin visit exists. Summary: Instead of failing silently (PG storage) or raising an un-documented exception (in-mem storage). Reviewers: #reviewers, anlambert Reviewed By: #reviewers, anlambert Subscribers: swh-public-ci Differential Revision: https://forge.softwareheritage.org/D769 04 December 2018, 14:56:12 UTC
b6b004b snapshot_add: Check the origin visit exists. Instead of failing silently (PG storage) or raising an un-documented exception (in-mem storage). 04 December 2018, 13:00:25 UTC
12f5c5a Add 'recursive' argument to the in-mem storage. Summary: Also add test coverage for that feature and improve doc. Reviewers: #reviewers, ardumont Reviewed By: #reviewers, ardumont Subscribers: swh-public-ci Differential Revision: https://forge.softwareheritage.org/D737 30 November 2018, 10:28:16 UTC
df27d11 Add 'recursive' argument to the in-mem storage. Summary: Also add test coverage for that feature and improve doc. Reviewers: #reviewers Differential Revision: https://forge.softwareheritage.org/D737 29 November 2018, 14:59:03 UTC
d8463b1 Use the new doc style for {revision,release}_add. Summary: Also fix the 'committer_date' on the PG Storage's revision_add, that slipped through D676. Reviewers: #reviewers, douardda Reviewed By: #reviewers, douardda Subscribers: douardda, ardumont, swh-public-ci Differential Revision: https://forge.softwareheritage.org/D698 29 November 2018, 13:40:49 UTC
f203821 Fix column name in metadata_provider_get. Test Plan: Test coverage will come in the next diff Reviewers: #reviewers, ardumont Reviewed By: #reviewers, ardumont Subscribers: ardumont, swh-public-ci Differential Revision: https://forge.softwareheritage.org/D716 29 November 2018, 13:40:40 UTC
7af40f6 Fix documented return type of metadata_provider_add. Summary: As the in-mem storage was written based on this documentation, fix it as well. Add a test for this behavior. Depends on D716. Reviewers: #reviewers, ardumont Reviewed By: #reviewers, ardumont Subscribers: ardumont, swh-public-ci Differential Revision: https://forge.softwareheritage.org/D717 29 November 2018, 09:21:43 UTC
abee27c Storage.release_get: Yield None for non-existent release Similar to Storage.revision_get, yields None when querying a non-existent release. 27 November 2018, 16:31:51 UTC
cd2db9b Document edge cases of content_get_range. Reviewers: #reviewers, douardda, ardumont Reviewed By: #reviewers, douardda, ardumont Subscribers: ardumont, douardda, swh-public-ci Differential Revision: https://forge.softwareheritage.org/D694 27 November 2018, 10:44:12 UTC
479ad67 sql: Update origin trigger to provide full origin information 26 November 2018, 14:28:28 UTC
d190fda Implement content_get and content_get_range for the in-mem storage. Summary: Needed to work on T1307 for the indexers. Reviewers: #reviewers, ardumont, douardda Reviewed By: #reviewers, ardumont, douardda Subscribers: douardda, ardumont, zack, swh-public-ci Differential Revision: https://forge.softwareheritage.org/D695 26 November 2018, 14:24:36 UTC
e894064 doc: update index to match new swh-doc format related to T1330 23 November 2018, 13:48:09 UTC
90da2a9 Fix docstrings for Storage.add_{release,revision}. Reviewers: #reviewers, ardumont Reviewed By: #reviewers, ardumont Subscribers: ardumont, swh-public-ci Differential Revision: https://forge.softwareheritage.org/D676 22 November 2018, 12:10:31 UTC
dd62eae Increase test coverage for stat_counters + fix its bugs. Reviewers: #reviewers, anlambert Reviewed By: #reviewers, anlambert Subscribers: anlambert, swh-public-ci Differential Revision: https://forge.softwareheritage.org/D679 21 November 2018, 16:09:34 UTC
5d822c9 Add 'inmemory' storage class in get_storage. Summary: https://forge.softwareheritage.org/D684#inline-3666 Reviewers: #reviewers, anlambert Reviewed By: #reviewers, anlambert Subscribers: swh-public-ci, ardumont, anlambert Differential Revision: https://forge.softwareheritage.org/D689 21 November 2018, 14:50:48 UTC
d1ce244 Run property-based tests on RemoteStorage and in_memory.Storage. Reviewers: #reviewers, ardumont Reviewed By: #reviewers, ardumont Subscribers: ardumont, swh-public-ci Differential Revision: https://forge.softwareheritage.org/D678 21 November 2018, 14:33:19 UTC
37d70d6 Make it official that content_missing accepts missing keys. Summary: The PG Storage provides that feature, and the Git loader uses it. This commit documents that feature, adds it to the Mem Storage, and changes test_content_missing to test that behavior. Reviewers: #reviewers, ardumont Reviewed By: #reviewers, ardumont Subscribers: swh-public-ci Differential Revision: https://forge.softwareheritage.org/D680 21 November 2018, 14:32:35 UTC
97b356a Add empty fetch_history_{start,end} endpoints to the in-mem storage. Summary: The Git loader calls them; but we're going to remove them (T829). Reviewers: #reviewers, ardumont Reviewed By: #reviewers, ardumont Subscribers: swh-public-ci Differential Revision: https://forge.softwareheritage.org/D681 21 November 2018, 14:32:21 UTC
8ea06fa tests: Move generation tests fixture in a dedicated module So that tests that requires tests fixture without hypothesis does not break. Related T1354 19 November 2018, 09:28:26 UTC
b61cf3d Add in-memory storage. Test Plan: Land the two parent diffs, then rebase this one on top of them. Reviewers: #reviewers, douardda Reviewed By: #reviewers, douardda Subscribers: olasd, douardda, swh-public-ci Differential Revision: https://forge.softwareheritage.org/D645 19 November 2018, 09:03:52 UTC
fc3b9dc Document metadata providers. Reviewers: #reviewers, douardda, moranegg Reviewed By: #reviewers, douardda, moranegg Subscribers: douardda, moranegg, swh-public-ci Differential Revision: https://forge.softwareheritage.org/D637 15 November 2018, 09:02:57 UTC
12edfac tox: Define a py3-slow environment Use: tox -e py3-slow 14 November 2018, 15:07:33 UTC
880aa9c tests: Use fast/slow profile names 14 November 2018, 15:07:33 UTC
3f05ada Makefile: Adapt makefile to use the dev profile 14 November 2018, 15:07:33 UTC
076a392 tests: Use hypothesis profile mechanism to configure samples size 14 November 2018, 15:07:33 UTC
04323a6 tests: Decrease generation sample to make tests faster Currently investigating hypothesis settings [1] [1] https://hypothesis.readthedocs.io/en/latest/settings.html#settings-profiles 14 November 2018, 15:07:33 UTC
8aa990f tests: Add docstring to test to detail what's tested 14 November 2018, 15:07:33 UTC
e1a460c tests: Move property_based marker to toplevel test class 14 November 2018, 15:07:33 UTC
3829885 tests: Use sorted 14 November 2018, 15:07:33 UTC
c876d86 storage: Open content_get_range endpoint Related T991 14 November 2018, 15:07:33 UTC
a429574 tests: Use sorted on list directly 14 November 2018, 15:07:33 UTC
c7a3266 tests: Build content generated in less steps 14 November 2018, 15:07:33 UTC
f87808c tests: Clarify gen_content docstring 14 November 2018, 15:07:33 UTC
0c51e4d tests: Add test content generation util function and use it Test back `content_get` and `content_get_metadata` as a first step. Plan to use those for the new and upcoming `content_get_range` function. 14 November 2018, 15:07:32 UTC
823465f Remove SQLisms from the tests and API. Reviewers: #reviewers, olasd Reviewed By: #reviewers, olasd Subscribers: olasd, swh-public-ci Differential Revision: https://forge.softwareheritage.org/D642 14 November 2018, 14:58:30 UTC
64e67aa make-doc: add an 'assets' make target for the doc used by the swh-docs to generate images and schemas when building the whole documentation. 12 November 2018, 15:19:22 UTC
1031dc5 debian: Remove nose and migrate to pytest 12 November 2018, 13:10:07 UTC
1a1d214 algos.revisions_walker: Fix get_revisions_walker doc 12 November 2018, 10:12:36 UTC
14e17d1 Fix storage doc. Reviewers: #reviewers, olasd Reviewed By: #reviewers, olasd Subscribers: olasd, swh-public-ci Differential Revision: https://forge.softwareheritage.org/D643 09 November 2018, 17:38:49 UTC
b944f00 Restructure tests to prepare for tests with no DB. Reviewers: #reviewers, olasd Reviewed By: #reviewers, olasd Subscribers: swh-public-ci Differential Revision: https://forge.softwareheritage.org/D644 09 November 2018, 17:38:38 UTC
7934ded Fix rst syntax in docstrings 09 November 2018, 14:15:17 UTC
81b8c9b Add :orphan: metadata to archive-copies and sql-storage doc files to prevent sphinx from complaining about these not being included. 09 November 2018, 14:15:17 UTC
88c33cc algos: Add iterators to walk across revisions history This commit adds iterators to walk across the history of revisions heading to a given one. The following types of iteration are offered: - committer_date: revisions are returned in reverse chronological order of their commit date (same as git log) - dfs: revisions are returned in the same order they are visited when performing a depth-first search in pre order on the revisions DAG - dfs_post: revisions are returned in the same order they are visited when performing a depth-first search in post order on the revisions DAG - bfs: revisions are returned in the same order they are visited when performing a breadth-first search on the revisions DAG Another iterator of type path, returning only revisions that modify a specific path in reverse chronological order of their commit date, is also introduced. Nevertheless, due to client-side implementation, its performances are far from optimal when walking across a really large history. Related T1026 Related T1284 09 November 2018, 13:04:04 UTC
19d4019 Rename the 'ts' argument of 'origin_visit_add' to 'date'. Summary: To be consistent with the name used in the DB and the dict key returned by 'origin_visit_get'. Note that this makes the client incompatible with older versions of the server. Reviewers: #reviewers, olasd Reviewed By: #reviewers, olasd Subscribers: olasd, swh-public-ci Differential Revision: https://forge.softwareheritage.org/D636 09 November 2018, 10:45:41 UTC
884d9e7 sql/Makefile: Fix pifpaf test as it was always considered used 08 November 2018, 13:43:03 UTC
465192e storage.listener: Subscribe listener to snapshot insertion Related T1275 26 October 2018, 10:21:52 UTC
1ad38af sql: Add snapshot trigger event on insertion Related T1275 26 October 2018, 10:21:52 UTC
fba230d DB schema: manual fix for some arrow source/dest to avoid degenerate angles in rendering 26 October 2018, 07:57:04 UTC
a579a1a DB schema: add light foreign key arrows from snapshot to * 26 October 2018, 07:51:57 UTC
b032d2a DB schema: remove left-over occurrence stuff from snapshot cluster 26 October 2018, 07:51:25 UTC
1c74f6d tox.ini: Align configuration file with other Also, without this, this failed because somehow, something is not finding requirements-test.txt. ``` Could not open requirements file: [Errno 2] No such file or directory: ' requirements-test.txt' ``` 24 October 2018, 13:42:12 UTC
7628710 SQL schema: update to reflect recent changes recent changes include most notably the removal of occurrence-related tables, but also the addition of table columns accumulated over time Closes T1283 24 October 2018, 12:35:02 UTC
a6742b7 DB schema doc: error out if SQL schema cannot be found 24 October 2018, 12:34:52 UTC
582d72d docs Makefile: only echo SQL queries on error 24 October 2018, 12:22:48 UTC
5fe9221 docs Makefile: forward "distclean" to sql dir 24 October 2018, 12:20:01 UTC
b91482a gitignore and remove generate archive copies diagram Closes T1269 24 October 2018, 12:00:01 UTC
242f985 Don't run pytest in the docs directory 23 October 2018, 14:48:15 UTC
e77d809 tests: increase deadline for CI seems to run slower on there. 22 October 2018, 16:06:59 UTC
076144e tests: assertEquals -> assertEqual 22 October 2018, 14:46:34 UTC
32ef182 tests: use pytest and add a simple tox file related to T1261 22 October 2018, 14:45:08 UTC
5b38656 flake8 22 October 2018, 14:32:18 UTC
a8fbce5 Warn when using db_transaction/db_transaction_generator where you shouldn't. Summary: I did it, it caused closed cursors that hang the process, that's hard to debug. Reviewers: #reviewers, ardumont Reviewed By: #reviewers, ardumont Subscribers: ardumont Differential Revision: https://forge.softwareheritage.org/D556 22 October 2018, 09:03:36 UTC
d211615 Mark snapshot tests as needing the database 18 October 2018, 16:30:44 UTC
b2d4921 Mark minimum hypothesis version for datetimes 18 October 2018, 16:27:56 UTC
ec7a119 Add hypothesis to debian build-dependencies 18 October 2018, 16:25:07 UTC
ad92248 Add a function to retrieve a full snapshot Summary: This function wraps the paginating branch listing API to do its work. It's in swh.storage.algos because it should be done client-side rather than server-side. The tests are fairly simple, and mostly add a bunch of tooling around hypothesis to generate data that conforms to the swh data schema. Those things could probably be adequately moved to a new submodule of swh.model. Test Plan: new tests added Reviewers: anlambert Differential Revision: https://forge.softwareheritage.org/D551 18 October 2018, 14:57:55 UTC
7cd7597 Rewrite snapshot_get in terms of snapshot_get_branches Summary: This increases uniformity and reduces code duplication between the two functions. The addition of the 'next_branch' return value in snapshot_get_branches allows callers to get branches incrementally in a loop which is friendlier. Depends on D545 Test Plan: tests have been updated to add the new next_branch field Reviewers: anlambert Differential Revision: https://forge.softwareheritage.org/D546 17 October 2018, 14:56:42 UTC
0d52bf4 Rewrite snapshot pagination tests declaratively Summary: This looks less confusing than removing a bunch of values from the test data. Test Plan: make test still passes Reviewers: anlambert Differential Revision: https://forge.softwareheritage.org/D545 17 October 2018, 14:56:07 UTC
80cd420 packaging: bump swh-core dependency to 0.0.44 17 October 2018, 13:56:52 UTC
9ce5f78 README: swh-storage-testdata is not required any more 17 October 2018, 13:56:13 UTC
a9c32d2 sql: adapt Makefile for sql file renames and make it pifpaf compliant ie. delete stamps when executed via pifpaf for the db setup. 17 October 2018, 13:46:36 UTC
ee27736 kill sql/db-init in favor of swh-core's new swh-db-init tool 17 October 2018, 13:46:36 UTC
c4cdb64 tests: refactor the StorageTestFixture class and move sql files in the package so we can build the db setup sql file on the fly: - make it inherit from SingleDbTestFixture, so we can - move sql files in swh/storage/sql/ - get rid og the setUpClass static method to reuse the one from SingleDbTestFixture, - apply the required modifications to tests based on this StorageTestFixture. Warnings: - this requires D538, - this will break tests in swh-vault. Also add README.md to the MANIFEST.in file so it does not require a too recent setuptool. 17 October 2018, 13:46:36 UTC
8f9af9b Remove some leftover compatibility code 17 October 2018, 13:16:31 UTC
2d5efac Reorder upgrade statements so that they actually pass 17 October 2018, 12:44:42 UTC
back to top