https://github.com/voldemort/voldemort
Revision db6de769eb8df2c3c66b55fefa3dea4305566574 authored by Arunachalam Thirupathi on 07 April 2015, 21:00:38 UTC, committed by Arunachalam Thirupathi on 07 April 2015, 21:52:46 UTC
1) Move some code into a method 2) Allocate the buffer per fetch instead of per file. Tested by fetching 2 directories on HDFS and verified the output.
1 parent 9b417b6
- HEAD
- refs/heads/adminClientFixes
- refs/heads/autoexp
- refs/heads/bdb-417
- refs/heads/bdb-split
- refs/heads/bindExceptionRetry
- refs/heads/clientBuffer3
- refs/heads/coadmin
- refs/heads/consis
- refs/heads/cooordinator-netty-port-fix
- refs/heads/directWrite3
- refs/heads/dps-voldemort-release-0.0.2
- refs/heads/fd_integration_test
- refs/heads/fetcher_per_file_checksum
- refs/heads/finalGradleDeps
- refs/heads/final_webhdfs
- refs/heads/gh-pages
- refs/heads/hadoop2
- refs/heads/hdfsfetchertest_npe_bug_fix
- refs/heads/li-907
- refs/heads/li-911
- refs/heads/li-918
- refs/heads/li-928-2
- refs/heads/li-r1006
- refs/heads/li-r1008
- refs/heads/li-r1012
- refs/heads/li-r1024
- refs/heads/li-r1024-readonly
- refs/heads/li-r1028
- refs/heads/li-r1028-bugfix
- refs/heads/li-r1044
- refs/heads/li-r1046
- refs/heads/li-r1104
- refs/heads/li-r1106
- refs/heads/li-r1114
- refs/heads/li-r1124
- refs/heads/li-r1132
- refs/heads/li-r1140
- refs/heads/li-r1142
- refs/heads/li-r940
- refs/heads/li-r942
- refs/heads/li-r946
- refs/heads/li-rel-926
- refs/heads/li10
- refs/heads/li11
- refs/heads/li9
- refs/heads/master
- refs/heads/nioFixes
- refs/heads/node0Issue
- refs/heads/origin/zone-expansion
- refs/heads/persistency-check
- refs/heads/playing_around_with_rocksdb
- refs/heads/proxy-put-metadata-listener
- refs/heads/queryKeyCleanup
- refs/heads/reducer_checksumfile_bug_fix
- refs/heads/release-051
- refs/heads/release-057
- refs/heads/release-060
- refs/heads/release-070
- refs/heads/release-080
- refs/heads/release-0801
- refs/heads/release-0802
- refs/heads/release-081
- refs/heads/release-090
- refs/heads/release-0901
- refs/heads/release-095li1
- refs/heads/release-095li2
- refs/heads/release-095li3
- refs/heads/release-095li4
- refs/heads/release-095li5
- refs/heads/release-095li8-SNAPSHOT
- refs/heads/release-095li9
- refs/heads/release-096
- refs/heads/release-096li1
- refs/heads/release-096li2
- refs/heads/release-096li3
- refs/heads/release-096li4
- refs/heads/release-096li5
- refs/heads/release-096li6
- refs/heads/release-096li7
- refs/heads/release-096li8
- refs/heads/release-1.0.0
- refs/heads/release-1.0.24
- refs/heads/release-1.1.1
- refs/heads/release-1.1.2
- refs/heads/release-1.1.4
- refs/heads/release-1.1.5
- refs/heads/release-1.1.6
- refs/heads/release-1.1.7
- refs/heads/release-1.1.8
- refs/heads/release-1.1.9
- refs/heads/release-1.10.0
- refs/heads/release-1.10.1
- refs/heads/release-1.10.10
- refs/heads/release-1.10.11
- refs/heads/release-1.10.12
- refs/heads/release-1.10.13
- refs/heads/release-1.10.14
- refs/heads/release-1.10.15
- refs/heads/release-1.10.16
- refs/heads/release-1.10.17
- refs/heads/release-1.10.18
- refs/heads/release-1.10.19
- refs/heads/release-1.10.19.1
- refs/heads/release-1.10.2
- refs/heads/release-1.10.20
- refs/heads/release-1.10.21
- refs/heads/release-1.10.22
- refs/heads/release-1.10.23
- refs/heads/release-1.10.24
- refs/heads/release-1.10.26
- refs/heads/release-1.10.3
- refs/heads/release-1.10.4
- refs/heads/release-1.10.5
- refs/heads/release-1.10.6
- refs/heads/release-1.10.7
- refs/heads/release-1.10.8
- refs/heads/release-1.10.9
- refs/heads/release-1.2.0
- refs/heads/release-1.2.3
- refs/heads/release-1.3.0
- refs/heads/release-1.3.1
- refs/heads/release-1.3.3
- refs/heads/release-1.3.4
- refs/heads/release-1.4.2
- refs/heads/release-1.4.5
- refs/heads/release-1.4.6
- refs/heads/release-1.5.0
- refs/heads/release-1.5.1
- refs/heads/release-1.5.2
- refs/heads/release-1.5.3
- refs/heads/release-1.5.4
- refs/heads/release-1.5.5
- refs/heads/release-1.5.6
- refs/heads/release-1.5.7
- refs/heads/release-1.5.8
- refs/heads/release-1.5.9
- refs/heads/release-1.6.0
- refs/heads/release-1.6.1
- refs/heads/release-1.6.2
- refs/heads/release-1.6.3
- refs/heads/release-1.6.4
- refs/heads/release-1.6.6
- refs/heads/release-1.6.8
- refs/heads/release-1.6.9
- refs/heads/release-1.7.0
- refs/heads/release-1.7.1
- refs/heads/release-1.7.2
- refs/heads/release-1.7.3
- refs/heads/release-1.8.0
- refs/heads/release-1.8.1
- refs/heads/release-1.8.10
- refs/heads/release-1.8.11
- refs/heads/release-1.8.12
- refs/heads/release-1.8.13
- refs/heads/release-1.8.14
- refs/heads/release-1.8.15
- refs/heads/release-1.8.16
- refs/heads/release-1.8.2
- refs/heads/release-1.8.3
- refs/heads/release-1.8.4
- refs/heads/release-1.8.5
- refs/heads/release-1.8.6
- refs/heads/release-1.8.7
- refs/heads/release-1.8.8
- refs/heads/release-1.8.9
- refs/heads/release-1.9.0
- refs/heads/release-1.9.1
- refs/heads/release-1.9.10
- refs/heads/release-1.9.11
- refs/heads/release-1.9.12
- refs/heads/release-1.9.13
- refs/heads/release-1.9.14
- refs/heads/release-1.9.15
- refs/heads/release-1.9.16
- refs/heads/release-1.9.17
- refs/heads/release-1.9.18
- refs/heads/release-1.9.19
- refs/heads/release-1.9.2
- refs/heads/release-1.9.20
- refs/heads/release-1.9.21
- refs/heads/release-1.9.22
- refs/heads/release-1.9.3
- refs/heads/release-1.9.4
- refs/heads/release-1.9.5
- refs/heads/release-1.9.6
- refs/heads/release-1.9.7
- refs/heads/release-1.9.8
- refs/heads/release-1.9.9
- refs/heads/release-114
- refs/heads/reportSystemVersion
- refs/heads/rest_service
- refs/heads/rest_service_branch
- refs/heads/revert-223-master
- refs/heads/roMoreLog
- refs/heads/slop
- refs/heads/slopPusher
- refs/heads/socketpool
- refs/heads/sslbnp
- refs/heads/tehuti_metrics
- refs/heads/topush2
- refs/heads/venice
- refs/tags/release-1.10.24-cutoff
- refs/tags/release-1.10.25-cutoff
- refs/tags/release-1.5.9-cutoff
- refs/tags/release-1.6.0-cutoff
- refs/tags/release-1.6.3-cutoff
- refs/tags/release-1.6.4
- refs/tags/release-1.6.4-cutoff
- refs/tags/release-1.6.6-cutoff
- refs/tags/release-1.8.14-cutoff
- db6de769eb8df2c3c66b55fefa3dea4305566574
- vector-clock-fixes
- v1.4.0
- release-1.9.9-cutoff
- release-1.9.8-cutoff
- release-1.9.7-cutoff
- release-1.9.6-cutoff
- release-1.9.5-cutoff
- release-1.9.4-cutoff
- release-1.9.3-cutoff
- release-1.9.22-cutoff
- release-1.9.21-cutoff
- release-1.9.20-cutoff
- release-1.9.2-cutoff
- release-1.9.19-cutoff
- release-1.9.18-cutoff
- release-1.9.17-cutoff
- release-1.9.16-cutoff
- release-1.9.15-cutoff
- release-1.9.14-cutoff
- release-1.9.13-cutoff
- release-1.9.12-cutoff
- release-1.9.11-cutoff
- release-1.9.10-cutoff
- release-1.9.1-cutoff
- release-1.9.0-cutoff
- release-1.8.9-cutoff
- release-1.8.8-cutoff
- release-1.8.5-cutoff
- release-1.8.4-cutoff
- release-1.8.3-cutoff
- release-1.8.16-cutoff
- release-1.8.15-cutoff
- release-1.8.13-cutoff
- release-1.8.12-cutoff
- release-1.8.11-cutoff
- release-1.8.10-cutoff
- release-1.8.1-cutoff
- release-1.7.3-cutoff
- release-1.7.2-cutoff
- release-1.7.1-cutoff
- release-1.7.0-cutoff
- release-1.6.8-cutoff
- release-1.6.2-cutoff
- release-1.6.1-cutoff
- release-1.5.8-cutoff
- release-1.5.7-cutoff
- release-1.5.4-cutoff
- release-1.5.2-cutoff
- release-1.5.1-cutoff
- release-1.3.0-cutoff
- release-1.10.9-cutoff
- release-1.10.8-cutoff
- release-1.10.7-cutoff
- release-1.10.6-cutoff
- release-1.10.5-cutoff
- release-1.10.4-cutoff
- release-1.10.3-cutoff
- release-1.10.26-cutoff
- release-1.10.23-cutoff
- release-1.10.22-cutoff
- release-1.10.21-cutoff
- release-1.10.20-cutoff
- release-1.10.2-cutoff
- release-1.10.19.1-cutoff
- release-1.10.19-cutoff
- release-1.10.18-cutoff
- release-1.10.17-cutoff
- release-1.10.16-cutoff
- release-1.10.15-cutoff
- release-1.10.14-cutoff
- release-1.10.13-cutoff
- release-1.10.12-cutoff
- release-1.10.11-cutoff
- release-1.10.10-cutoff
- release-1.10.1-cutoff
- release-1.10.0-cutoff
- before-replicatype-was-removed
- before-donorbased-was-removed
History 
Cook and download a directory from the Software Heritage Vault
You have requested the cooking of the directory with identifier swh:1:dir:4f7f2ac261cc0ed7010b345f2f5cff5a805b1346 into a standard tar.gz archive.
Are you sure you want to continue ?
Download a directory from the Software Heritage Vault
You have requested the download of the directory with identifier swh:1:dir:4f7f2ac261cc0ed7010b345f2f5cff5a805b1346 as a standard tar.gz archive.
Are you sure you want to continue ?
Cook and download a revision from the Software Heritage Vault
You have requested the cooking of the history heading to revision with identifier swh:1:rev:db6de769eb8df2c3c66b55fefa3dea4305566574 into a bare git archive.
Are you sure you want to continue ?
Download a revision from the Software Heritage Vault
You have requested the download of the history heading to revision with identifier swh:1:rev:db6de769eb8df2c3c66b55fefa3dea4305566574 as a bare git archive.
Are you sure you want to continue ?
Invalid Email !
The provided email is not well-formed.
Download link has expired
The requested archive is no longer available for download from the Software Heritage Vault.
Do you want to cook it again ?
Take a new snapshot of a software origin
If the archived software origin currently browsed is not synchronized with its upstream version (for instance when new commits have been issued), you can explicitly request Software Heritage to take a new snapshot of it.
Use the form below to proceed. Once a request has been submitted and accepted, it will be processed as soon as possible. You can then check its processing state by visiting this dedicated page. Processing "take a new snapshot" request ...
Permalinks
To reference or cite the objects present in the Software Heritage archive, permalinks based on SoftWare Hash IDentifiers (SWHIDs) must be used.
Select below a type of object currently browsed in order to display its associated SWHID and permalink.
Tip revision: db6de769eb8df2c3c66b55fefa3dea4305566574 authored by Arunachalam Thirupathi on 07 April 2015, 21:00:38 UTC
Refactor the HDFS fetcher
Refactor the HDFS fetcher
Tip revision: db6de76
| File | Mode | Size |
|---|---|---|
| .settings | ||
| bin | ||
| clients | ||
| config | ||
| contrib | ||
| docs | ||
| example | ||
| gradle | ||
| private-lib | ||
| public-lib | ||
| src | ||
| test | ||
| voldemort-contrib | ||
| .gitignore | -rw-r--r-- | 242 bytes |
| CONTRIBUTORS | -rw-r--r-- | 659 bytes |
| LICENSE | -rw-r--r-- | 11.1 KB |
| NOTES | -rw-r--r-- | 2.5 KB |
| NOTICE | -rw-r--r-- | 8.1 KB |
| README.md | -rw-r--r-- | 4.4 KB |
| build.gradle | -rw-r--r-- | 19.7 KB |
| build.xml | -rw-r--r-- | 22.1 KB |
| gradle.properties | -rw-r--r-- | 1.2 KB |
| gradlew | -rwxr-xr-x | 5.0 KB |
| gradlew.bat | -rw-r--r-- | 2.3 KB |
| release_notes.txt | -rw-r--r-- | 40.4 KB |
| settings.gradle | -rw-r--r-- | 149 bytes |
| tomcat-tasks.properties | -rw-r--r-- | 420 bytes |
| web.xml | -rw-r--r-- | 1.1 KB |
The diff you're trying to view is too large. Only the first 1000 changed files have been loaded.
Computing file changes ...
