Skip to main content

Browse the archive

https://github.com/SoftwareHeritage/swh-storage
22 December 2022, 21:01:20 UTC
sort by:
Revision Author Date Message Commit Date
78ea4c3 Nicolas Dandrimont 15 September 2015, 16:14:31 UTC swh.storage.storage: only COPY missing directories 15 September 2015, 16:14:31 UTC
e4e2cb9 Nicolas Dandrimont 15 September 2015, 14:54:47 UTC swh.storage.db/storage: Allow listing one directory's contents 15 September 2015, 14:54:47 UTC
098f66d Nicolas Dandrimont 15 September 2015, 14:53:31 UTC swh.storage.storage: flake8 on directory_missing 15 September 2015, 14:53:31 UTC
030afa9 Nicolas Dandrimont 15 September 2015, 14:52:52 UTC swh.storage.storage Update db_transaction to avoid side-effects 15 September 2015, 14:52:52 UTC
99ab9d5 Nicolas Dandrimont 15 September 2015, 14:51:30 UTC swh.storage.db: Unwrap byteas directly in the Db layer 15 September 2015, 14:51:30 UTC
f4df49b Nicolas Dandrimont 15 September 2015, 14:49:58 UTC sql/swh-func.sql: Add directory listing function 15 September 2015, 14:49:58 UTC
8a202df Nicolas Dandrimont 15 September 2015, 14:49:19 UTC sql/swh-schema.sql: remove foreign key on directory_entry_dir 15 September 2015, 14:49:19 UTC
f1dadf5 Nicolas Dandrimont 15 September 2015, 14:46:47 UTC sql/swh-schema.sql: Add indexes on directory_entry_* 15 September 2015, 14:49:05 UTC
0db9be6 Nicolas Dandrimont 15 September 2015, 11:52:46 UTC sql/swh-func.sql: Cleanup directory_entry-related functions 15 September 2015, 14:46:17 UTC
80029af Nicolas Dandrimont 14 September 2015, 18:47:45 UTC Commit first implementation of directory_add 14 September 2015, 18:47:45 UTC
cd46701 Stefano Zacchiroli 14 September 2015, 17:01:40 UTC DB testing: update doc and tests to use swh-storage-testdata 14 September 2015, 17:01:40 UTC
1123290 Stefano Zacchiroli 14 September 2015, 16:48:17 UTC sql/Makefile: ignore dropdb errors 14 September 2015, 16:48:17 UTC
1c44963 Nicolas Dandrimont 14 September 2015, 16:39:16 UTC swh.storage.storage: Add method for directory_missing 14 September 2015, 16:40:13 UTC
5e72c51 Stefano Zacchiroli 14 September 2015, 16:14:01 UTC swh.storage: cosmetic fix in content_missing docstring 14 September 2015, 16:14:01 UTC
069c812 Nicolas Dandrimont 14 September 2015, 15:45:11 UTC sql/swh-schema.sql: Add length constraints on checksum domains 14 September 2015, 15:45:11 UTC
603eec5 Nicolas Dandrimont 14 September 2015, 15:12:35 UTC swh.storage.storage: Add content_missing method to Storage 14 September 2015, 15:12:35 UTC
ee0c895 Nicolas Dandrimont 14 September 2015, 15:08:47 UTC swh.storage.tests: refactor storage test 14 September 2015, 15:08:47 UTC
f3053e1 Nicolas Dandrimont 14 September 2015, 15:05:59 UTC swh.storage.storage: allow returning data from db_transaction 14 September 2015, 15:05:59 UTC
bf9829e Nicolas Dandrimont 14 September 2015, 13:48:12 UTC Update swh.storage.db for the new hash API 14 September 2015, 13:48:12 UTC
900c496 Nicolas Dandrimont 14 September 2015, 13:43:25 UTC Update ObjStorage API for the new hashes ObjStorage is now using bytearrays for its keys instead of hex strings 14 September 2015, 13:43:25 UTC
a9a8a09 Nicolas Dandrimont 14 September 2015, 09:23:47 UTC Use functools.wraps on decorators functools.wraps preserves the wrapped function's metadata which makes navigation and interactive use nicer 14 September 2015, 09:23:47 UTC
f3f2a7f Nicolas Dandrimont 14 September 2015, 08:57:24 UTC Remove origin.parent_id Nested revisions can now be handled through directory_entry_rev 14 September 2015, 08:57:24 UTC
4beae32 Nicolas Dandrimont 14 September 2015, 08:56:02 UTC Add foreign key to directory(id) on directory_list_rev 14 September 2015, 08:56:02 UTC
e95ab80 Nicolas Dandrimont 14 September 2015, 08:49:55 UTC Add directory_entry_rev and directory_list_rev tables Those tables are useful to let a directory point to another revision, for instance when using submodules or for tarball in tarball. 14 September 2015, 08:49:55 UTC
6c54578 Nicolas Dandrimont 14 September 2015, 08:38:41 UTC Add documentation for all the Storage.<>_add methods 14 September 2015, 08:38:41 UTC
59420a8 Nicolas Dandrimont 14 September 2015, 08:37:59 UTC Fix spelling errors on occurrence 14 September 2015, 08:37:59 UTC
741d8ac Nicolas Dandrimont 14 September 2015, 08:36:59 UTC Update signature for revision_missing 14 September 2015, 08:36:59 UTC
6f53bdb Stefano Zacchiroli 13 September 2015, 13:32:24 UTC SQL: add logical clusters to graphviz version of the DB schema 13 September 2015, 13:32:24 UTC
cd302ad Stefano Zacchiroli 13 September 2015, 13:20:25 UTC SQL: create domain unix_path for path and path components 13 September 2015, 13:20:25 UTC
46f2e02 Stefano Zacchiroli 13 September 2015, 13:14:50 UTC SQL: bump db version SQL: use bytea for checksum types 13 September 2015, 13:19:45 UTC
1439e03 Stefano Zacchiroli 12 September 2015, 11:30:26 UTC storage.py: flake8 fixes (cosmetic) 12 September 2015, 11:30:26 UTC
48fe79a Nicolas Dandrimont 11 September 2015, 14:26:24 UTC Add skeleton API methods for Storage 11 September 2015, 14:26:52 UTC
f5cecc9 Nicolas Dandrimont 11 September 2015, 14:25:20 UTC Fix Storage.content_add reference to HASH_ALGORITHMS 11 September 2015, 14:26:52 UTC
60db4f3 Nicolas Dandrimont 11 September 2015, 14:24:56 UTC Rename Storage.add_content to content_add 11 September 2015, 14:26:46 UTC
3abed1d Antoine R. Dumont 10 September 2015, 15:11:22 UTC Fix typo 10 September 2015, 15:25:11 UTC
d824587 Antoine R. Dumont 10 September 2015, 15:10:54 UTC Add committer_date on revision table 10 September 2015, 15:25:11 UTC
b98d564 Stefano Zacchiroli 09 September 2015, 15:06:30 UTC db.Db: refactor copy_to() function to COPY to arbitrary tables as a side effect, change the API of storage.add_content to require more complete dictionaries. In particular, dictionaries passed there need to have an explicit "length" key 09 September 2015, 15:06:32 UTC
2916ac2 Stefano Zacchiroli 09 September 2015, 13:54:38 UTC stored proc: generalize mktemp function it can create temporary tables that mimick any existing table 09 September 2015, 13:54:38 UTC
f9e05ff Stefano Zacchiroli 09 September 2015, 13:11:01 UTC SQL schema: add default 'visible' for content.status 09 September 2015, 13:11:01 UTC
06628dc Stefano Zacchiroli 09 September 2015, 13:10:36 UTC SQL schema: ensure content.ctime is not NULL we had a default, but that is not enough to ensure an (explicit) NULL will not be inserted 09 September 2015, 13:10:36 UTC
813fdf7 Stefano Zacchiroli 08 September 2015, 20:34:54 UTC db.py: add @stored_procedure decorator to define in a declarative way Python end points for argument-less stored procedures 08 September 2015, 20:34:56 UTC
f5bf0e1 Stefano Zacchiroli 08 September 2015, 20:12:39 UTC swh.storage: add and use @db_transaction decorator 08 September 2015, 20:12:39 UTC
d6b946e Stefano Zacchiroli 08 September 2015, 20:00:05 UTC db_testing: do not try to rollback closed connections 08 September 2015, 20:00:05 UTC
fbc1ed5 Stefano Zacchiroli 08 September 2015, 19:58:30 UTC refactor db.Db to be a proxy with storage procedure wrappers This way, the only code that actually call into remote stored procedures will be in this module. Other modules will just call into db.Db (in pure Python) and will not need to touch raw SQL. 08 September 2015, 19:58:30 UTC
c5449b2 Stefano Zacchiroli 08 September 2015, 13:17:54 UTC db_testing: bug fix: call specific super() methods to *actually* be cooperative with other classes 08 September 2015, 13:17:56 UTC
30d75b5 Stefano Zacchiroli 08 September 2015, 11:45:48 UTC document how to run tests that need a working DB this include adding convenience make "test-{db,nodb}" targets to Makefile.local 08 September 2015, 11:45:50 UTC
a7c7bdc Stefano Zacchiroli 08 September 2015, 11:36:44 UTC rename swh.storage.bundle -> swh.storage.storage the apparent name duplication is not a big deal, as it is a hidden module anyhow, that is proxied in the swh.storage.Storage class 08 September 2015, 11:36:46 UTC
5ae50a5 Stefano Zacchiroli 08 September 2015, 11:34:34 UTC test_storage.py: test add_content() 08 September 2015, 11:34:34 UTC
7c55543 Stefano Zacchiroli 08 September 2015, 11:33:55 UTC test_objstorage.py: use assert(Not)In where possible (style) 08 September 2015, 11:33:59 UTC
5494a4a Stefano Zacchiroli 08 September 2015, 11:33:25 UTC swh.storage(.db): allow to pass an established connection object to constructor 08 September 2015, 11:33:25 UTC
90b50b4 Stefano Zacchiroli 08 September 2015, 10:38:07 UTC test_db.py: (trivial) tests for swh.storage.db 08 September 2015, 10:38:07 UTC
d01a389 Stefano Zacchiroli 08 September 2015, 10:37:54 UTC db_testing.py: fixture to ease testing using the DB 08 September 2015, 10:37:54 UTC
e8f6b63 Stefano Zacchiroli 08 September 2015, 08:27:23 UTC make swh.storage.tests a proper python module this allows to have (importable) testing helper modules in there 08 September 2015, 08:27:23 UTC
d6599ba Stefano Zacchiroli 08 September 2015, 08:26:50 UTC sql/Makefile: add dumpdb target 08 September 2015, 08:26:50 UTC
e1bd7d8 Stefano Zacchiroli 07 September 2015, 14:48:51 UTC bin/swh-storage-add-dir: test/bench script to inject a full dir note that this script inject into both the DB and the object storage, whereas bin/swh-objstorage-add-dir only injects into the latter 07 September 2015, 14:48:51 UTC
cb62421 Stefano Zacchiroli 07 September 2015, 14:47:54 UTC add_content(): be resilient to duplicates in the input 07 September 2015, 14:47:54 UTC
ad9363e Stefano Zacchiroli 07 September 2015, 14:43:30 UTC add_content(): fix the name "tmp_content" for the temp add table This simplifies client code, and also allows to use non-quoted SQL in the stored procedures. To avoid losing generality, this commit also adds "ON COMMIT DROP" to the temporary table creation, so that different transactions can coexist. 07 September 2015, 14:43:30 UTC
f19bdd9 Stefano Zacchiroli 07 September 2015, 13:57:17 UTC SQL: add (commented) sha256 index 07 September 2015, 13:57:46 UTC
df7b743 Stefano Zacchiroli 07 September 2015, 13:56:35 UTC swh.storage.Storage: proxy for DB+FS with add_content() support 07 September 2015, 13:57:46 UTC
d8b04cb Stefano Zacchiroli 07 September 2015, 13:56:03 UTC db.Db: simple proxy for psycopg2 connection 07 September 2015, 13:57:46 UTC
8178e97 Stefano Zacchiroli 07 September 2015, 13:55:34 UTC objstorage: idempotent addition, ignore dups get rid of clobber behaviour, too complex for our needs having an idempotent semantic is simpler and cleaner anyhow 07 September 2015, 13:57:46 UTC
6c30b8d Stefano Zacchiroli 07 September 2015, 13:54:37 UTC SQL: server-side function to add content entries 07 September 2015, 13:57:42 UTC
a742bcb Stefano Zacchiroli 06 September 2015, 08:47:52 UTC SQL: replace left-over TAB characters with spaces (cosmetic) 06 September 2015, 08:47:52 UTC
719157c Stefano Zacchiroli 06 September 2015, 08:47:05 UTC SQL: add creation time to content table 06 September 2015, 08:47:05 UTC
a2939c6 Stefano Zacchiroli 04 September 2015, 14:20:02 UTC SQL: move gist extension creation to *-init.sql 04 September 2015, 14:20:02 UTC
e686c60 Stefano Zacchiroli 04 September 2015, 14:16:24 UTC SQL: reorganize *.sql files and Makefile separate SQL files into: - swh-init: DB wide changes, e.g., language loading, extensions, etc. - swh-schema: DB schema, e.g., tables, views, etc. - swh-data: DB static data, e.g., predefined rows 04 September 2015, 14:16:24 UTC
d7268e8 Stefano Zacchiroli 04 September 2015, 12:10:39 UTC objstorage: add support for len() and __iter__() 04 September 2015, 12:10:39 UTC
8ab2913 Stefano Zacchiroli 04 September 2015, 09:50:43 UTC objstorage: return object id upon add() note: this was already documented properly in the API doc, but not actually implemented by add() methods 04 September 2015, 09:50:45 UTC
006b4c8 Stefano Zacchiroli 04 September 2015, 07:45:09 UTC objstorage: use __contains__ for membership test 04 September 2015, 07:45:09 UTC
abb162c Stefano Zacchiroli 03 September 2015, 15:33:03 UTC SQL: rename "references" to "branches" in the occurrence table 03 September 2015, 15:34:43 UTC
441fa4d Stefano Zacchiroli 03 September 2015, 15:32:44 UTC SQL: remove out-of-date TODOs 03 September 2015, 15:32:44 UTC
d415446 Stefano Zacchiroli 03 September 2015, 15:24:59 UTC SQL: add type of revision (git, tarball, etc.) 03 September 2015, 15:24:59 UTC
75768b7 Stefano Zacchiroli 03 September 2015, 15:22:15 UTC SQL: add parent_rank to revision_history this is needed to be able to deterministically rebuild merge commits 03 September 2015, 15:22:15 UTC
7bfe3b0 Stefano Zacchiroli 03 September 2015, 15:21:45 UTC SQL: drop foreign key reference from revision to directories rationale: allow to store commits whose tree has not been fetched 03 September 2015, 15:21:45 UTC
b0a8a20 Stefano Zacchiroli 03 September 2015, 15:05:32 UTC SQL: add primary key to occurrence_history 03 September 2015, 15:05:32 UTC
c606d62 Stefano Zacchiroli 03 September 2015, 15:05:17 UTC SQL: add authority to occurence_history 03 September 2015, 15:05:17 UTC
a1f5164 Stefano Zacchiroli 03 September 2015, 14:47:40 UTC SQL: let directory entries reference child dir deferredly 03 September 2015, 14:47:40 UTC
9f48e14 Stefano Zacchiroli 03 September 2015, 14:46:26 UTC SQL: switch content primary key to sha1 (from sha1_git) 03 September 2015, 14:46:26 UTC
2102d91 Stefano Zacchiroli 03 September 2015, 14:31:19 UTC SQL: rename git_object_id -> sha1_git note: keep 2 different types --- sha1, sha1_git --- as they represent different types of information 03 September 2015, 14:31:19 UTC
c8ced16 Stefano Zacchiroli 03 September 2015, 14:28:02 UTC SQL: add status (AKA visibility) to content table 03 September 2015, 14:28:02 UTC
9bd8c2c Stefano Zacchiroli 03 September 2015, 14:15:38 UTC SQL: new modeling of file/directories (joint work with Nicolas and Antoine) 03 September 2015, 14:21:26 UTC
4f9dd68 Stefano Zacchiroli 03 September 2015, 12:03:24 UTC objstorage: switch docstring conventions to google style see: https://google-styleguide.googlecode.com/svn/trunk/pyguide.html?showone=Comments#Comments 03 September 2015, 12:03:24 UTC
b56253a Stefano Zacchiroli 03 September 2015, 11:51:02 UTC objstorage: simplify exc hierarchy to have a top-level Error exc 03 September 2015, 11:51:02 UTC
bed8079 Stefano Zacchiroli 03 September 2015, 10:21:12 UTC objstorage: switch to sha1 object ids (from sha1_git) 03 September 2015, 10:21:17 UTC
a25d5ab Stefano Zacchiroli 03 September 2015, 10:02:33 UTC import DB schema under sql/ 03 September 2015, 10:02:33 UTC
3b493ba Stefano Zacchiroli 03 September 2015, 08:51:52 UTC sync Makefile with py-template 03 September 2015, 08:51:52 UTC
c7aeeab Stefano Zacchiroli 03 September 2015, 08:35:29 UTC Makefile: sync with py template 03 September 2015, 08:35:29 UTC
3e2b044 Stefano Zacchiroli 02 September 2015, 16:46:16 UTC ObjStorage: add a high-level description of the API 02 September 2015, 16:46:16 UTC
798709b Stefano Zacchiroli 02 September 2015, 16:40:50 UTC ObjStorage doc string: refer to ID_HASH_ALGO, instead of hard-coding sha1_git 02 September 2015, 16:40:50 UTC
b88abfa Stefano Zacchiroli 02 September 2015, 16:34:55 UTC document in house exceptions 02 September 2015, 16:34:55 UTC
f224581 Stefano Zacchiroli 02 September 2015, 16:27:45 UTC bin/swh-objstorage-fsck: fsck-like script for entire dirs 02 September 2015, 16:27:45 UTC
7e3d88b Stefano Zacchiroli 02 September 2015, 16:23:11 UTC add get() method(s) 02 September 2015, 16:23:11 UTC
df2c667 Stefano Zacchiroli 02 September 2015, 16:22:57 UTC tests: better organization for setUp values also: add an explicit missing_obj_id value 02 September 2015, 16:22:57 UTC
b22ba36 Stefano Zacchiroli 02 September 2015, 16:14:27 UTC rename write context manager to _write_obj_file it is a more appropriate name (since it is used also when clobbering), and it also makes clear that it is not supposed to be used from the outside 02 September 2015, 16:14:35 UTC
cf5ff76 Stefano Zacchiroli 02 September 2015, 16:13:18 UTC tests: preseed obj_id on add wherever possible 02 September 2015, 16:13:18 UTC
15fd4c2 Stefano Zacchiroli 02 September 2015, 15:51:04 UTC ensure we can easily switch to sha1 algo instead of sha1_git, if we need to check() will be more efficient with sha1, and avoid an extra pass on the gz content to compute the length 02 September 2015, 15:51:06 UTC
c428f77 Stefano Zacchiroli 02 September 2015, 15:43:00 UTC swh-objstorage-add-dir: deal with duplicates 02 September 2015, 15:43:00 UTC
75fb365 Stefano Zacchiroli 02 September 2015, 15:24:44 UTC add check() method it does fsck-like integrity checks on the object storage 02 September 2015, 15:24:44 UTC
3d3ac64 Stefano Zacchiroli 02 September 2015, 14:29:55 UTC Makefile: sync with py-template 02 September 2015, 14:29:55 UTC
359571d Stefano Zacchiroli 02 September 2015, 14:25:30 UTC add has() method and clobbering support 02 September 2015, 14:25:30 UTC
back to top