37e7281 | Stefano Zacchiroli | 21 September 2015, 15:22:44 UTC | db.py: use single quotes consistently (style) | 21 September 2015, 15:22:44 UTC |
16aca48 | Stefano Zacchiroli | 21 September 2015, 15:21:16 UTC | db: uniform case-ing in SQL query (cosmetic) | 21 September 2015, 15:21:16 UTC |
ba5443a | Nicolas Dandrimont | 21 September 2015, 14:02:29 UTC | sql/swh-schema.sql: add indexes on directory lists | 21 September 2015, 14:02:29 UTC |
b8aa5c4 | Nicolas Dandrimont | 18 September 2015, 13:10:35 UTC | Merge branch 'master' into directory-listing-arrays | 18 September 2015, 13:10:35 UTC |
e9f08de | Nicolas Dandrimont | 18 September 2015, 10:58:11 UTC | swh.storage.storage: allow listing missing contents per hash | 18 September 2015, 10:58:11 UTC |
662c8ee | Nicolas Dandrimont | 18 September 2015, 10:57:03 UTC | swh.storage.storage: add the contents that _are_ missing | 18 September 2015, 10:57:03 UTC |
04c1db8 | Nicolas Dandrimont | 18 September 2015, 10:42:14 UTC | swh.storage.storage: content ids are sha1, not sha1_git | 18 September 2015, 10:42:14 UTC |
8f97c01 | Nicolas Dandrimont | 18 September 2015, 10:31:25 UTC | Merge branch 'master' into directory-listing-arrays | 18 September 2015, 10:31:25 UTC |
9308a0a | Nicolas Dandrimont | 18 September 2015, 10:30:54 UTC | swh.storage.storage: skip already written content | 18 September 2015, 10:30:54 UTC |
41abf75 | Nicolas Dandrimont | 18 September 2015, 08:01:12 UTC | Merge branch 'master' into directory-listing-arrays | 18 September 2015, 08:01:12 UTC |
2ccb3a5 | Nicolas Dandrimont | 17 September 2015, 15:58:08 UTC | sql/swh-schema.sql: allow storing dangling occurrences and releases | 17 September 2015, 15:58:08 UTC |
af41d20 | Nicolas Dandrimont | 17 September 2015, 15:10:49 UTC | swh.storage.storage: implement occurrence_add | 17 September 2015, 15:10:49 UTC |
cc82413 | Nicolas Dandrimont | 17 September 2015, 14:49:32 UTC | swh.storage.storage: add origin_get and origin_add_one | 17 September 2015, 14:49:32 UTC |
0128716 | Nicolas Dandrimont | 17 September 2015, 14:37:25 UTC | swh.storage.db: Hand-roll CSV to avoid issues with None vs. empty strings | 17 September 2015, 14:37:25 UTC |
01c0a1b | Nicolas Dandrimont | 17 September 2015, 12:17:37 UTC | sql/swh-schema.sql: Bump schema version | 17 September 2015, 12:17:37 UTC |
6f5d47d | Nicolas Dandrimont | 17 September 2015, 12:15:44 UTC | swh.storage.storage: Implement release_add | 17 September 2015, 12:15:44 UTC |
3126a95 | Nicolas Dandrimont | 17 September 2015, 12:15:07 UTC | swh.storage.storage: Implement release_missing | 17 September 2015, 12:15:07 UTC |
0266474 | Nicolas Dandrimont | 17 September 2015, 12:14:19 UTC | swh.storage.storage: shortcut when there are no missing revisions | 17 September 2015, 12:14:19 UTC |
a891d5f | Nicolas Dandrimont | 17 September 2015, 12:13:23 UTC | swh.storage.db: Add more quoting to csv output | 17 September 2015, 12:13:23 UTC |
5cacce7 | Nicolas Dandrimont | 17 September 2015, 12:12:38 UTC | sql/swh-schema.sql: revisions can have the same parent several times | 17 September 2015, 12:12:38 UTC |
3c1e11e | Nicolas Dandrimont | 17 September 2015, 11:52:31 UTC | sql/swh-schema.sql: messages/comments as byteas | 17 September 2015, 11:52:31 UTC |
160f679 | Nicolas Dandrimont | 17 September 2015, 08:57:01 UTC | sql/swh-schema.sql: Add unique indexes on directory_entry for null times | 17 September 2015, 08:57:21 UTC |
ed8032b | Nicolas Dandrimont | 17 September 2015, 08:05:22 UTC | sql/swh-func,schema.sql: Directory listings as arrays | 17 September 2015, 08:05:22 UTC |
b2bcde3 | Nicolas Dandrimont | 17 September 2015, 08:02:16 UTC | swh.storage.db: Use CSV format for copy_to | 17 September 2015, 08:02:16 UTC |
3503aeb | Nicolas Dandrimont | 17 September 2015, 08:01:47 UTC | Bulk addition of releases | 17 September 2015, 08:01:47 UTC |
cc312eb | Nicolas Dandrimont | 17 September 2015, 07:59:54 UTC | Only insert distinct rows in directory_entry_* | 17 September 2015, 07:59:54 UTC |
dc9b73a | Nicolas Dandrimont | 16 September 2015, 14:50:51 UTC | sql/swh-func.sql: Add date offsets to bulk insertion of revisions | 16 September 2015, 14:50:51 UTC |
3c32d31 | Nicolas Dandrimont | 16 September 2015, 14:40:39 UTC | sql/swh-func.sql: add swh_mktemp_release | 16 September 2015, 14:40:39 UTC |
9ce0124 | Nicolas Dandrimont | 16 September 2015, 13:06:11 UTC | Add offsets to the dates that need them in the schema timestamptz doesn't store timezone information, and we need the offset information to be able to recreate git objects | 16 September 2015, 13:06:11 UTC |
6428e6b | Nicolas Dandrimont | 16 September 2015, 10:55:19 UTC | Merge branch 'tmp-directory-add' | 16 September 2015, 10:55:19 UTC |
7eca197 | Nicolas Dandrimont | 16 September 2015, 10:53:20 UTC | Implement revision_add for swh.storage.Storage | 16 September 2015, 10:53:20 UTC |
4042c3b | Nicolas Dandrimont | 16 September 2015, 10:52:42 UTC | swh.storage.storage: implement revision_missing | 16 September 2015, 10:52:42 UTC |
ec33fca | Nicolas Dandrimont | 16 September 2015, 10:51:12 UTC | swh.storage.db: update copy_to to not fail on missing keys | 16 September 2015, 10:51:12 UTC |
06c888b | Nicolas Dandrimont | 16 September 2015, 10:49:32 UTC | sql/swh-schema.sql: Remove historical comment on revision | 16 September 2015, 10:49:32 UTC |
5f404f1 | Nicolas Dandrimont | 16 September 2015, 10:48:41 UTC | sql/swh-schema.sql: add default values and indexes on person | 16 September 2015, 10:48:41 UTC |
b1d6dc1 | Nicolas Dandrimont | 16 September 2015, 10:47:56 UTC | Add a revision-missing stored procedure | 16 September 2015, 10:47:56 UTC |
07fb39c | Nicolas Dandrimont | 16 September 2015, 10:47:02 UTC | Add a revision-specific temporary table creation function | 16 September 2015, 10:47:02 UTC |
cb4f413 | Stefano Zacchiroli | 16 September 2015, 09:24:36 UTC | SQL: better rendeding of PDF version of the schema | 16 September 2015, 09:24:36 UTC |
b8ba308 | Stefano Zacchiroli | 16 September 2015, 08:50:50 UTC | sql/Makefile: properly handle pdf dep on clusters.dot | 16 September 2015, 08:50:55 UTC |
e1f846c | Stefano Zacchiroli | 15 September 2015, 16:38:13 UTC | add back needed __init__.py to non-namespace packages | 15 September 2015, 16:38:13 UTC |
78ea4c3 | Nicolas Dandrimont | 15 September 2015, 16:14:31 UTC | swh.storage.storage: only COPY missing directories | 15 September 2015, 16:14:31 UTC |
42b7a68 | Stefano Zacchiroli | 15 September 2015, 16:08:44 UTC | get rid of __init__.py | 15 September 2015, 16:08:44 UTC |
e4e2cb9 | Nicolas Dandrimont | 15 September 2015, 14:54:47 UTC | swh.storage.db/storage: Allow listing one directory's contents | 15 September 2015, 14:54:47 UTC |
098f66d | Nicolas Dandrimont | 15 September 2015, 14:53:31 UTC | swh.storage.storage: flake8 on directory_missing | 15 September 2015, 14:53:31 UTC |
030afa9 | Nicolas Dandrimont | 15 September 2015, 14:52:52 UTC | swh.storage.storage Update db_transaction to avoid side-effects | 15 September 2015, 14:52:52 UTC |
99ab9d5 | Nicolas Dandrimont | 15 September 2015, 14:51:30 UTC | swh.storage.db: Unwrap byteas directly in the Db layer | 15 September 2015, 14:51:30 UTC |
f4df49b | Nicolas Dandrimont | 15 September 2015, 14:49:58 UTC | sql/swh-func.sql: Add directory listing function | 15 September 2015, 14:49:58 UTC |
8a202df | Nicolas Dandrimont | 15 September 2015, 14:49:19 UTC | sql/swh-schema.sql: remove foreign key on directory_entry_dir | 15 September 2015, 14:49:19 UTC |
f1dadf5 | Nicolas Dandrimont | 15 September 2015, 14:46:47 UTC | sql/swh-schema.sql: Add indexes on directory_entry_* | 15 September 2015, 14:49:05 UTC |
0db9be6 | Nicolas Dandrimont | 15 September 2015, 11:52:46 UTC | sql/swh-func.sql: Cleanup directory_entry-related functions | 15 September 2015, 14:46:17 UTC |
80029af | Nicolas Dandrimont | 14 September 2015, 18:47:45 UTC | Commit first implementation of directory_add | 14 September 2015, 18:47:45 UTC |
3bc7312 | Stefano Zacchiroli | 14 September 2015, 17:15:20 UTC | SQL doc: refactor and typeset non-fkey arrows | 14 September 2015, 17:15:20 UTC |
0caeedc | Stefano Zacchiroli | 14 September 2015, 17:09:28 UTC | SQL doc: add revision entry dir to dot clusters | 14 September 2015, 17:09:28 UTC |
cd46701 | Stefano Zacchiroli | 14 September 2015, 17:01:40 UTC | DB testing: update doc and tests to use swh-storage-testdata | 14 September 2015, 17:01:40 UTC |
1123290 | Stefano Zacchiroli | 14 September 2015, 16:48:17 UTC | sql/Makefile: ignore dropdb errors | 14 September 2015, 16:48:17 UTC |
1c44963 | Nicolas Dandrimont | 14 September 2015, 16:39:16 UTC | swh.storage.storage: Add method for directory_missing | 14 September 2015, 16:40:13 UTC |
5e72c51 | Stefano Zacchiroli | 14 September 2015, 16:14:01 UTC | swh.storage: cosmetic fix in content_missing docstring | 14 September 2015, 16:14:01 UTC |
069c812 | Nicolas Dandrimont | 14 September 2015, 15:45:11 UTC | sql/swh-schema.sql: Add length constraints on checksum domains | 14 September 2015, 15:45:11 UTC |
603eec5 | Nicolas Dandrimont | 14 September 2015, 15:12:35 UTC | swh.storage.storage: Add content_missing method to Storage | 14 September 2015, 15:12:35 UTC |
ee0c895 | Nicolas Dandrimont | 14 September 2015, 15:08:47 UTC | swh.storage.tests: refactor storage test | 14 September 2015, 15:08:47 UTC |
f3053e1 | Nicolas Dandrimont | 14 September 2015, 15:05:59 UTC | swh.storage.storage: allow returning data from db_transaction | 14 September 2015, 15:05:59 UTC |
bf9829e | Nicolas Dandrimont | 14 September 2015, 13:48:12 UTC | Update swh.storage.db for the new hash API | 14 September 2015, 13:48:12 UTC |
900c496 | Nicolas Dandrimont | 14 September 2015, 13:43:25 UTC | Update ObjStorage API for the new hashes ObjStorage is now using bytearrays for its keys instead of hex strings | 14 September 2015, 13:43:25 UTC |
a9a8a09 | Nicolas Dandrimont | 14 September 2015, 09:23:47 UTC | Use functools.wraps on decorators functools.wraps preserves the wrapped function's metadata which makes navigation and interactive use nicer | 14 September 2015, 09:23:47 UTC |
f3f2a7f | Nicolas Dandrimont | 14 September 2015, 08:57:24 UTC | Remove origin.parent_id Nested revisions can now be handled through directory_entry_rev | 14 September 2015, 08:57:24 UTC |
4beae32 | Nicolas Dandrimont | 14 September 2015, 08:56:02 UTC | Add foreign key to directory(id) on directory_list_rev | 14 September 2015, 08:56:02 UTC |
e95ab80 | Nicolas Dandrimont | 14 September 2015, 08:49:55 UTC | Add directory_entry_rev and directory_list_rev tables Those tables are useful to let a directory point to another revision, for instance when using submodules or for tarball in tarball. | 14 September 2015, 08:49:55 UTC |
6c54578 | Nicolas Dandrimont | 14 September 2015, 08:38:41 UTC | Add documentation for all the Storage.<>_add methods | 14 September 2015, 08:38:41 UTC |
59420a8 | Nicolas Dandrimont | 14 September 2015, 08:37:59 UTC | Fix spelling errors on occurrence | 14 September 2015, 08:37:59 UTC |
741d8ac | Nicolas Dandrimont | 14 September 2015, 08:36:59 UTC | Update signature for revision_missing | 14 September 2015, 08:36:59 UTC |
6f53bdb | Stefano Zacchiroli | 13 September 2015, 13:32:24 UTC | SQL: add logical clusters to graphviz version of the DB schema | 13 September 2015, 13:32:24 UTC |
cd302ad | Stefano Zacchiroli | 13 September 2015, 13:20:25 UTC | SQL: create domain unix_path for path and path components | 13 September 2015, 13:20:25 UTC |
46f2e02 | Stefano Zacchiroli | 13 September 2015, 13:14:50 UTC | SQL: bump db version SQL: use bytea for checksum types | 13 September 2015, 13:19:45 UTC |
1439e03 | Stefano Zacchiroli | 12 September 2015, 11:30:26 UTC | storage.py: flake8 fixes (cosmetic) | 12 September 2015, 11:30:26 UTC |
48fe79a | Nicolas Dandrimont | 11 September 2015, 14:26:24 UTC | Add skeleton API methods for Storage | 11 September 2015, 14:26:52 UTC |
f5cecc9 | Nicolas Dandrimont | 11 September 2015, 14:25:20 UTC | Fix Storage.content_add reference to HASH_ALGORITHMS | 11 September 2015, 14:26:52 UTC |
60db4f3 | Nicolas Dandrimont | 11 September 2015, 14:24:56 UTC | Rename Storage.add_content to content_add | 11 September 2015, 14:26:46 UTC |
3abed1d | Antoine R. Dumont | 10 September 2015, 15:11:22 UTC | Fix typo | 10 September 2015, 15:25:11 UTC |
d824587 | Antoine R. Dumont | 10 September 2015, 15:10:54 UTC | Add committer_date on revision table | 10 September 2015, 15:25:11 UTC |
b98d564 | Stefano Zacchiroli | 09 September 2015, 15:06:30 UTC | db.Db: refactor copy_to() function to COPY to arbitrary tables as a side effect, change the API of storage.add_content to require more complete dictionaries. In particular, dictionaries passed there need to have an explicit "length" key | 09 September 2015, 15:06:32 UTC |
2916ac2 | Stefano Zacchiroli | 09 September 2015, 13:54:38 UTC | stored proc: generalize mktemp function it can create temporary tables that mimick any existing table | 09 September 2015, 13:54:38 UTC |
f9e05ff | Stefano Zacchiroli | 09 September 2015, 13:11:01 UTC | SQL schema: add default 'visible' for content.status | 09 September 2015, 13:11:01 UTC |
06628dc | Stefano Zacchiroli | 09 September 2015, 13:10:36 UTC | SQL schema: ensure content.ctime is not NULL we had a default, but that is not enough to ensure an (explicit) NULL will not be inserted | 09 September 2015, 13:10:36 UTC |
813fdf7 | Stefano Zacchiroli | 08 September 2015, 20:34:54 UTC | db.py: add @stored_procedure decorator to define in a declarative way Python end points for argument-less stored procedures | 08 September 2015, 20:34:56 UTC |
f5bf0e1 | Stefano Zacchiroli | 08 September 2015, 20:12:39 UTC | swh.storage: add and use @db_transaction decorator | 08 September 2015, 20:12:39 UTC |
d6b946e | Stefano Zacchiroli | 08 September 2015, 20:00:05 UTC | db_testing: do not try to rollback closed connections | 08 September 2015, 20:00:05 UTC |
fbc1ed5 | Stefano Zacchiroli | 08 September 2015, 19:58:30 UTC | refactor db.Db to be a proxy with storage procedure wrappers This way, the only code that actually call into remote stored procedures will be in this module. Other modules will just call into db.Db (in pure Python) and will not need to touch raw SQL. | 08 September 2015, 19:58:30 UTC |
c5449b2 | Stefano Zacchiroli | 08 September 2015, 13:17:54 UTC | db_testing: bug fix: call specific super() methods to *actually* be cooperative with other classes | 08 September 2015, 13:17:56 UTC |
30d75b5 | Stefano Zacchiroli | 08 September 2015, 11:45:48 UTC | document how to run tests that need a working DB this include adding convenience make "test-{db,nodb}" targets to Makefile.local | 08 September 2015, 11:45:50 UTC |
a7c7bdc | Stefano Zacchiroli | 08 September 2015, 11:36:44 UTC | rename swh.storage.bundle -> swh.storage.storage the apparent name duplication is not a big deal, as it is a hidden module anyhow, that is proxied in the swh.storage.Storage class | 08 September 2015, 11:36:46 UTC |
5ae50a5 | Stefano Zacchiroli | 08 September 2015, 11:34:34 UTC | test_storage.py: test add_content() | 08 September 2015, 11:34:34 UTC |
7c55543 | Stefano Zacchiroli | 08 September 2015, 11:33:55 UTC | test_objstorage.py: use assert(Not)In where possible (style) | 08 September 2015, 11:33:59 UTC |
5494a4a | Stefano Zacchiroli | 08 September 2015, 11:33:25 UTC | swh.storage(.db): allow to pass an established connection object to constructor | 08 September 2015, 11:33:25 UTC |
90b50b4 | Stefano Zacchiroli | 08 September 2015, 10:38:07 UTC | test_db.py: (trivial) tests for swh.storage.db | 08 September 2015, 10:38:07 UTC |
d01a389 | Stefano Zacchiroli | 08 September 2015, 10:37:54 UTC | db_testing.py: fixture to ease testing using the DB | 08 September 2015, 10:37:54 UTC |
e8f6b63 | Stefano Zacchiroli | 08 September 2015, 08:27:23 UTC | make swh.storage.tests a proper python module this allows to have (importable) testing helper modules in there | 08 September 2015, 08:27:23 UTC |
d6599ba | Stefano Zacchiroli | 08 September 2015, 08:26:50 UTC | sql/Makefile: add dumpdb target | 08 September 2015, 08:26:50 UTC |
e1bd7d8 | Stefano Zacchiroli | 07 September 2015, 14:48:51 UTC | bin/swh-storage-add-dir: test/bench script to inject a full dir note that this script inject into both the DB and the object storage, whereas bin/swh-objstorage-add-dir only injects into the latter | 07 September 2015, 14:48:51 UTC |
cb62421 | Stefano Zacchiroli | 07 September 2015, 14:47:54 UTC | add_content(): be resilient to duplicates in the input | 07 September 2015, 14:47:54 UTC |
ad9363e | Stefano Zacchiroli | 07 September 2015, 14:43:30 UTC | add_content(): fix the name "tmp_content" for the temp add table This simplifies client code, and also allows to use non-quoted SQL in the stored procedures. To avoid losing generality, this commit also adds "ON COMMIT DROP" to the temporary table creation, so that different transactions can coexist. | 07 September 2015, 14:43:30 UTC |