| 22be8bc | Arunachalam Thirupathi | 09 December 2015, 03:21:01 UTC | Releasing Voldemort 1.10.6 | 09 December 2015, 03:21:01 UTC |
| 36d3655 | Arunachalam Thirupathi | 08 December 2015, 22:32:12 UTC | Swap Only on IOException,UnreachableStoreException Currently Swap is attempted for any failure. Now the swap will be attempted only if the failure is an IO or UnreachableStoreException. Other exceptions will cause the push to fail. | 08 December 2015, 23:36:33 UTC |
| 33f48ed | ARUNACHALAM THIRUPATHI | 08 December 2015, 07:51:08 UTC | Create JMX mbeans for tracking multiple server states Following states are tracked Server 0->normal, 1->offline, 2->rebalancing SlopStreaming 0->enabled,1->disabled PartitionStreaming 0->enabled,1->disabled ReadOnlyFetching 0->enabled,1->disabled QuotaEnforcing 0->enabled,1->disabled | 08 December 2015, 07:51:08 UTC |
| aa11fd1 | Yan | 08 December 2015, 02:05:18 UTC | Merge pull request #352 from squarY/bouncycastle Let read only server use bouncy castle as JCE provider. | 08 December 2015, 02:05:18 UTC |
| 695ae80 | Yan Yan(Data Infrastructure) | 03 December 2015, 19:42:00 UTC | Let read only server use bouncy castle as JCE provider. Initialize BouncyCastleProvider in VoldemortServer constructor if it's enabled. Fix minor format issue. Fix unessary format changing. Fix unessary format issue. Remove useless parameter. | 05 December 2015, 00:17:47 UTC |
| c927688 | Felix GV | 04 December 2015, 01:24:36 UTC | BnP logging improvements: - BnP job now emits config properties in one entry per line. - Bumped up some useful logs to INFO level in the HttpHook. | 04 December 2015, 19:08:41 UTC |
| 5b09c8e | Yan | 25 November 2015, 23:14:16 UTC | Merge pull request #350 from squarY/master Releasing Voldemort 1.10.5 | 25 November 2015, 23:14:16 UTC |
| fb23553 | Yan Yan(Data Infrastructure) | 25 November 2015, 02:39:22 UTC | Releasing Voldemort 1.10.5 Fix typos. | 25 November 2015, 23:12:37 UTC |
| d53d534 | Yan | 23 November 2015, 22:32:48 UTC | Merge pull request #349 from squarY/sslbnpmerged Squash commits. Let Voldemort node can modify URL before fetching file. | 23 November 2015, 22:32:48 UTC |
| f88adc4 | Yan Yan | 19 November 2015, 07:47:05 UTC | Add 3 properties in Voldemort configuration. Let voldemort node can turn on or turn off SSL when fetching file from HDFS. Fix the issue when url dose not contain protocl(Eg. local file path), parsing url will cause String index out of range exception. Rollback the format change to original codes. Make modify URL feature more generic and use java.net.URL instead of parsing URL mannually. Move modify URL feature to Utils class. And let VoldemortSwapJob to invode this method to replace url. Let voldemort node modify URL separately before fetching file. | 21 November 2015, 01:03:33 UTC |
| 71cc55c | Arunachalam Thirupathi | 17 November 2015, 02:57:09 UTC | Update metadata version cluster, stores At times metadata version on a cluster.xml and stores.xml drifts The next metadata update instead of consolidating these versions updates at few places and ignores at few other places. This causes the client to not re-bootstrap correctly when the cluster.xml or stores.xml is changed. Now when a cluster.xml is changed, the version is synchronized across the cluster to let the clients auto rebootstrap. This fix merges the VectorClock on all the nodes to be updated so that the Stores version will be updated correctly. The old methods which does not take nodes as parameters are removed and the public method exposes the nodes as parameters. | 18 November 2015, 02:50:45 UTC |
| b30fdfe | Greg Banks | 17 November 2015, 06:27:25 UTC | Merge pull request #345 from gnb/VOLDENG-2171bis Extend shell "preflist" command to show partitions, v2 | 17 November 2015, 06:27:25 UTC |
| 24c8290 | Greg Banks | 17 November 2015, 01:07:51 UTC | Extend shell "preflist" command to show partitions, v2 After valuable feedback from athirupthi | 17 November 2015, 01:07:51 UTC |
| 64c90eb | Greg Banks | 16 November 2015, 22:10:09 UTC | Merge pull request #343 from gnb/VOLDENG-2170 Fix shell "preflist" command key parsing | 16 November 2015, 22:10:09 UTC |
| 7970985 | Greg Banks | 16 November 2015, 19:40:45 UTC | Fix shell "preflist" command key parsing | 16 November 2015, 19:40:45 UTC |
| d8068f3 | Greg Banks | 13 November 2015, 04:24:00 UTC | Merge pull request #339 from stotch/generate_cluster_xml_fixes Small fixes to generate_cluster_xml.py | 13 November 2015, 04:24:00 UTC |
| d1926e5 | Greg Banks | 13 November 2015, 02:49:19 UTC | Merge pull request #342 from gnb/rorep2 ReadOnlyReplicationHelperCLI: do not rely on down node | 13 November 2015, 02:49:19 UTC |
| bc679b7 | Greg Banks | 13 November 2015, 01:46:17 UTC | ReadOnlyReplicationHelperCLI: do not rely on down node When building a list of partitions to be copied between nodes to restore a down node, don't expect to be able to fetch any useful metadata from the down node. v2: better method naming per feedback from athirupathi | 13 November 2015, 02:46:22 UTC |
| 91b1f09 | Brendan Harris | 12 November 2015, 02:49:15 UTC | Removed trailing whitespace, made list of partitions sorted by integer value and marked file as executable | 12 November 2015, 02:49:19 UTC |
| c3a8360 | Felix GV | 12 November 2015, 00:12:32 UTC | Releasing Voldemort 1.10.4 | 12 November 2015, 00:12:32 UTC |
| 9c66806 | singhsiddharth | 11 November 2015, 22:10:15 UTC | Merge pull request #338 from FelixGV/ro_fix_handleDisableStoreVersion Bug fix for BnP HA | 11 November 2015, 22:10:15 UTC |
| 975459c | Felix GV | 11 November 2015, 21:57:05 UTC | Bug fix for BnP HA: - StoreVersionManager.getDisabledMarkerFile() did not look for the right file name. - Also added unit tests and better log messages for this. Debuggability improvements in AdminServiceRequestHandler: - More exceptions now logged with their full stacktrace on the server-side. - The request info is now printed in a more readable format. BnP job will now fail properly when clusters are inconsistent, rather than just exiting. | 11 November 2015, 22:01:14 UTC |
| 640fbc7 | Felix GV | 09 November 2015, 23:04:14 UTC | HadoopUtils clean up: - Clarified JSON-specific error messages. - Removed dead code. | 09 November 2015, 23:04:14 UTC |
| 678ba81 | singhsiddharth | 06 November 2015, 07:33:48 UTC | Merge pull request #335 from stotch/voldemort-shell-properties-squashed Make VoldemortClientShell adminClient use user-defined properties file | 06 November 2015, 07:33:48 UTC |
| 6ebecb0 | Brendan Harris | 06 November 2015, 05:20:59 UTC | adminClient now uses the same ClientConfig as the SocketStoreClient so that it too can get the overrides passed in by a user defined config file | 06 November 2015, 05:22:27 UTC |
| 9fa20d4 | Felix GV | 04 November 2015, 18:47:40 UTC | Releasing Voldemort 1.10.3 | 04 November 2015, 18:47:40 UTC |
| a0e19c6 | James Lent | 16 October 2015, 21:07:53 UTC | Implement the truncate function in the RocksDB storage engine. This functionality is required by the delete store command. It is implemented using a non default Column Family named after the store. This approach could be extended to allow multiple stores to be supported by a single RocksDB database. It is, however, not backwards compatible with the existing code. A data migration would be required to support existing stores. | 04 November 2015, 18:28:32 UTC |
| 2b6015e | Felix GV | 03 November 2015, 01:25:21 UTC | Moved BnP's verifyOrAddStore() to the AdminClient. This makes it easier to leverage this more resilient/idempotent Add Store operation from other processes. Also included minor refactorings. | 03 November 2015, 21:23:20 UTC |
| 6faa899 | singhsiddharth | 03 November 2015, 19:54:19 UTC | Merge pull request #333 from stotch/voldemort-shell-properties-squashed voldemort-shell support for properties file overrides | 03 November 2015, 19:54:19 UTC |
| 8c6dcf8 | Brendan Harris | 03 November 2015, 01:43:27 UTC | Removing the redundant argument check so that the VoldemortClientShell.java option parser can be used to query help Removing the redundant argument check so that the VoldemortClientShell.java option parser can be used to query help Augmented help and added an option for passing in a properties file The shell now passes in an option that tells VoldemortClientShell that it is being called from voldemort-shell.sh so that it can properly format the help output The shell now passes in an option that tells VoldemortClientShell that it is being called from voldemort-shell.sh so that it can properly format the help output Made help for --voldemort-shell option more 'helpful' Mistakenly bumped up a greaterthan check on the positional arguments ... Rolling that back Rewording --help helper text to be more helpful | 03 November 2015, 06:52:41 UTC |
| 1ac3a94 | Arunachalam Thirupathi | 03 November 2015, 00:36:26 UTC | Forklift tool schema comparison 1) ignore-schema-check was handled but not exposed as a parameter. Seems like joptSimpleParser will error out, if the parameters are not set up correctly. 2) Serializers are compared using string which resulted in lot more error. Using the Schema comparison based on Avro and JSON now. Most of the code here is auto generated (except for AVroGenericVersioned) where I modified some code by hand. 3) Fixed invalid stores xml. | 03 November 2015, 02:32:46 UTC |
| 0311bc6 | Felix GV | 30 October 2015, 22:54:20 UTC | Releasing Voldemort 1.10.2 | 30 October 2015, 22:56:18 UTC |
| bc10986 | singhsiddharth | 30 October 2015, 22:47:50 UTC | Merge pull request #329 from FelixGV/bnp_rename_fix Fixed rename operation in HdfsFailedFetchLock | 30 October 2015, 22:47:50 UTC |
| c7bce4d | Felix GV | 30 October 2015, 22:31:42 UTC | Fixed rename operation in HdfsFailedFetchLock Regression introduced in 831b80955ed609b7012bb85e8ce50d119170f635. | 30 October 2015, 22:31:42 UTC |
| 5f51537 | Felix GV | 29 October 2015, 01:51:20 UTC | Added a validator for Read-Only files. Given a data directory, the ReadOnlyFileValidator class checks a few things: - Index files are well formed: - Well sorted - No negative offset - Data files are well formed: - Number of records for a given key hash is > 0 - Key and value sizes are > 0 - Index and data files match together: - No gaps in data files, when traversing them with the offsets from index files - Raw keys in data files match the hashes found in their index files It then prints a summary of the amount and kinds of problems found. Disclaimers: - This tool only works with Voldemort Read-Only V2 formatted files. - This tool does NOT rely on the checksum in the .metadata file. - This tool only inspects the structural integrity of index and data files, and whether they match together. - If a problem is detected, you're in trouble for sure, but if no problems are detected, that doesn't mean an unsupported problem didn't slip through undetected (i.e.: false negatives are possible but false positives are not). Also added a bash run script in bin/read-only-file-validator.sh | 29 October 2015, 23:35:10 UTC |
| ec50a64 | Felix GV | 27 October 2015, 22:40:39 UTC | Releasing Voldemort 1.10.1 | 27 October 2015, 22:40:39 UTC |
| abf6353 | Felix GV | 27 October 2015, 01:23:37 UTC | Fixed regression introduced in d25d169862cc12e599bb1c916d684d89b5b4bfd4. The Read-Only servers used to percolate exception messages back to Build and Push jobs, for easy debugging of Hadoop-related exceptions (i.e.: AccessControlException). As part of the commit above, the server begun swallowing the underlying exceptions' message, which makes debugging BnP jobs harder. This commit fixes the regression. Also included some minor clean ups, such as JavaDoc improvements and @Deprecated annotations for the test-only functions. | 27 October 2015, 22:34:04 UTC |
| 10d94a3 | Arunachalam Thirupathi | 27 October 2015, 21:46:30 UTC | CopyStats Regression MessageDigest.digest() is read once, after it is read the contents of the digest are reset. Without knowing this, now started outputting checksum to stats file which broke the normal flow. Passing Around MessageDigest is dangerous because of this, so modified the method to return byte arrays which are then passed around. | 27 October 2015, 22:03:37 UTC |
| f83442d | Arunachalam Thirupathi | 24 October 2015, 01:55:18 UTC | Client do not remove dead connections When Server grace fully closes a connection, client ignores the connection close. After this an operation that gets this connection fails with EOFException as read/write returns -1 which causes the operation to fail. Mostly this is fine as the backup operation generaly succeeds, but when a cluster is bounced due to upgrade or other reasons, and if a client ends up caching all dead connections, it causes the operation to fail. After a client operation is done, previously the socket is registered with the selector for no-intention. For the next operation we start with write. With this change, the socket will be left in read mode. So FIN packet from the server will be processed and the socket will be closed. Any connection checkout from the pool needs to validate the connection. There is a race condition as the returned connection from the cache could still be killed later, that is handled as well. Stream processors must be closed after the socket is closed. Client will not be able to handle these close messages correctly. Added more comments to the Client Request Executor. Tests for dead connection clearing | 27 October 2015, 21:56:42 UTC |
| 9c87ad8 | ARUNACHALAM THIRUPATHI | 27 October 2015, 01:19:29 UTC | Merge pull request #325 from arunthirupathi/checkSumToStatsFile Output Checksum to Stats File | 27 October 2015, 01:19:29 UTC |
| 1e08ab1 | Arunachalam Thirupathi | 26 October 2015, 19:24:26 UTC | Output Checksum to Stats File 1) Output Checksum to stats file, can be used for individual file validation 2) Open the stats file in append only mode, so when a version is pushed deleted and re-pushed we have some history in the stats file. 3) Add a version to the stats file. So now the stats file will have the suffix -v<Version_number> which can be used by the script for parsing of the columns | 27 October 2015, 01:17:29 UTC |
| 95f60e8 | Felix GV | 22 October 2015, 04:09:24 UTC | Removed the set quota admin op from BnP. The default quota for new stores is now controlled by a server-side config, so this is redundant (and harmful to users who do not wish to use quotas in this manner). | 26 October 2015, 18:26:05 UTC |
| 49beabb | Felix GV | 22 October 2015, 21:25:39 UTC | Read-Only server now avoids needlessly recreating symlinks. There are cases (i.e.: emergency situations) where we may want to chmod -w a data directory and recreating the symlinks systematically then prevents the server from being able to start up. This change inspects the symlink which is already present and avoids recreating it if it's already pointing at the right destination. | 22 October 2015, 21:25:39 UTC |
| d780f55 | Felix GV | 19 October 2015, 19:58:17 UTC | Releasing Voldemort 1.10.0 | 19 October 2015, 19:58:17 UTC |
| f7303df | Felix GV | 16 October 2015, 22:47:38 UTC | Protection against data corruption. If index or data files are somehow corrupted, or mismatched (meaning that the index file from one store/version gets used in combination with a data file for another store/version), then the server gets into bad buffer allocation problems, causing IllegalArgumentExceptions and potentially OOMing in the process. This commit makes it easier to debug the issues and protect against the OOMs. Changes include: - Disallow allocation of negative size buffers. - Disallow allocation of excessively large value buffers (configurable, max 25 MB by default). - Disallow allocation of a buffer which would go past the data file's length. - Server-side logs now print out the file name and key (in hex) which triggered the problem. - Server-side logs now print out the socket name and stacktrace when SelectorManagerWorker catches a Throwable. - Client-side will report an "internal server error" when any of the bad allocation scenarios described above happen. - Slight optimization in the allocation strategy while reading the key/values in a data file. | 19 October 2015, 19:23:46 UTC |
| d4e27d9 | Arunachalam Thirupathi | 17 October 2015, 02:01:42 UTC | Test for ChunkedFileSet Collission Test for ChunkedFileSet collission case. | 17 October 2015, 02:02:22 UTC |
| 8b7b4cc | James Lent | 24 September 2015, 16:33:40 UTC | Make some modifications to how the benchmarking tool works in local mode as part of an evaluation of the RocksDB strorage engine: 1) Ensure that all the warm up records requested are created to eliminate some random errors. 2) Modify the mixed operation in local mode to increment the VectorClock to prevent all iterations from failing due to ObsoleteVersion exceptions. 3) Modify the mixed operation to perform the write even if the read returns nothing rather than silently passing without performing a write (I considered making this an error, but, preferred this approach). 4) Add a new Warning count similar to the ReturnCode count to report how often a mixed operation reads nothing (could be used for other situations). 5) Move the RecordCode report outside the summaryOnly check so that the error counts are always reported. 6) Make it possible to configure the storage engine in local mode. 7) Make several of the RocksDB parameters configurable. | 16 October 2015, 02:08:09 UTC |
| 13d055a | Bhavani Sudha Saktheeswaran | 08 October 2015, 18:04:02 UTC | Releasing Voldemort 1.9.22 | 08 October 2015, 18:04:02 UTC |
| d708069 | Bhavani Sudha Saktheeswaran | 06 October 2015, 23:31:44 UTC | fix tests | 06 October 2015, 23:31:44 UTC |
| 26d9582 | Bhavani Sudha Saktheeswaran | 05 October 2015, 17:43:48 UTC | Use IOUtils.closeQuietly() to close AdminClient Add more tests - add more tests for disk-quota - reorganize HDFSFetcher code | 05 October 2015, 22:42:53 UTC |
| b21a46a | Bhavani Sudha Saktheeswaran | 28 September 2015, 23:48:00 UTC | Fix default quota value for new stores - Make default quota value for new stores created via BnP to be configurable - -1 for default quota value indicates no quota restriction - fix unit tests fix tests minor fix Fix bug - delete quota call for, setQuoataForNode() now deletes quota from the quota store instead of the actual store name | 03 October 2015, 00:14:26 UTC |
| 831b809 | Felix GV | 02 October 2015, 14:58:10 UTC | BnP HA debuggability improvements. | 02 October 2015, 14:59:30 UTC |
| 985de33 | ARUNACHALAM THIRUPATHI | 07 June 2015, 22:59:44 UTC | Add heart beat for NIO Selector On Voldemort Server, when there is a disk crash the selector gets hung. Acceptor still keeps on accepting sockets and after a while the file descriptor limit is reached and everything is in a hung state. No JVM stack or heap dump. Now a selector heart beat is added and if the selector is past the max heart beat time ( defaults to 3 minutes) the acceptor will stop assigning connections to this selector. If all selectors are unhealthy ( past max heart beat time) acceptor will close the socket so that client can recover from these errors faster. | 02 October 2015, 01:52:56 UTC |
| 4cc1c14 | ARUNACHALAM THIRUPATHI | 01 October 2015, 05:57:42 UTC | Merge pull request #309 from jalkjaer/master add additional keepalive settings to avoid client connection leaks | 01 October 2015, 05:57:42 UTC |
| dc50968 | singhsiddharth | 29 September 2015, 19:23:21 UTC | Merge pull request #315 from FelixGV/hadoop_utils_clean_ups Trimmed the fat in HadoopUtils. | 29 September 2015, 19:23:21 UTC |
| 810129c | Felix GV | 28 September 2015, 23:19:09 UTC | Trimmed the fat in HadoopUtils. Half the functions were not used anywhere, and it's doubtful we would ever need them in the future. | 28 September 2015, 23:19:09 UTC |
| 6050ab9 | Bhavani Sudha Saktheeswaran | 23 September 2015, 23:58:54 UTC | Remove unnecessary special handling of QuotaExceededExceptions Add new Exception type for invalid stores Add log message when adding quota minor fix - set quota to 0 only if the store definition needs to be created. minor fix: - change exception type to be VoldemortException instead of QuotaExceededException | 25 September 2015, 22:36:11 UTC |
| 2f72ffb | Bhavani Sudha Saktheeswaran | 05 September 2015, 00:44:41 UTC | Iteration 1: Disk Quota - Add Exception communication from server to client Iteration 2: Disk Quota - Random function that generates QuotaException. For testing end to end exception flow. Later this will be replaced by appropriate quota checks Iteration 3: Disk Quota - compute the bytes written to index and data files in each reducer. Add this information to the filename - estimate the total disk size in bytes needed for a <node, store> pair. Done by iterating through all the index and data files and getting theie sizes from the file names, after the job is run. Iteration 4: Disk Quota - In HDFSFetcher, fetch estimated disk size from metadata file - In HDFSFetcher, fetch disk quota limit for <node, store> pair using admin client Minor fix: Add log message Minor fix: Fix exception handling minor fix: Handle Quota Exceptions appropriately minor fix: Fix more exceptions More fixes for Exception logs More exception handling fixes Handling Quota Exception in AdminStoreSwapper Iteration 5: Fix disk estimation logic - Instead of renaing the .data and .index files, add the file size info to corresponding '.checksum' files - Introduce new CheckSumMetadata file that holds the following fields - checksum, data file size, index file size - CheckSumMetadata is similar to ReadOnlyStorageMetadata file in terms of serialization and deserialization minor fix: exclude unnecessary "_" in file names Cleanup debug logs Add quota checks - Determine if quota needs to be checked for the incoming push. For pre existing stores quota is not checked. They will be quota-ed in future. - Check if there is sufficient quota left for a new push. - Filter out stores that push data mistakenly to the Voldemort cluster. This is based on the assumption that for invalid stores, during build phase, a store definition is with 0 quota. All other stores that are onboarded through Nuage will have proper quota set already. More fixes: - Fix Non Quota Exception Handling. Propogates actual error message to client side. - Change getQuota() to getQuotaForNode() in HDFSFetcher. - Add debug log messages - Set zero Quota for stores that are created during Build and Push. We want all new stores to be pre created with appropriate quota via Nuage or via admin client. | 25 September 2015, 22:36:11 UTC |
| 0af020f | Felix GV | 25 September 2015, 15:46:35 UTC | Proper interruption of BnP hooks. | 25 September 2015, 15:46:35 UTC |
| 4622df9 | Felix GV | 24 September 2015, 23:19:32 UTC | Releasing Voldemort 1.9.21 | 24 September 2015, 23:19:32 UTC |
| 26e1f48 | Felix GV | 24 September 2015, 07:32:28 UTC | New server config to determine HDFS login interval. Default: fetcher.login.interval.ms=-1 (re-login every time) | 24 September 2015, 21:59:21 UTC |
| df73cd4 | Johannes Alkjaer | 21 September 2015, 22:19:47 UTC | added settings nio.connector.keepalive and nio.admin.connector.keepalive for inbound NIO Connections | 22 September 2015, 20:29:43 UTC |
| 711cc40 | Felix GV | 15 September 2015, 01:37:00 UTC | Releasing Voldemort 1.9.20 | 15 September 2015, 01:40:58 UTC |
| ebbc44f | James Lent | 14 September 2015, 15:22:52 UTC | Fix the RocksDB iteration logic for keys and entries. The problems addressed include: 1) Key and entry iterators failed to strip prefix from keys (if required). 2) RocksdbStorageEngineTest failed to test BdbStorageEngine (all tests). 3) Both Key iterators would miss first entry and then iterate off the end. With these changes I was able to remove the Ignore annotation from 2 tests. | 15 September 2015, 00:18:45 UTC |
| 2738b7c | Felix GV | 14 September 2015, 22:53:34 UTC | Fixed HdfsFetcher test code and trimmed some fat. | 14 September 2015, 22:54:40 UTC |
| 2aec46a | Arunachalam Thirupathi | 11 September 2015, 01:59:30 UTC | Avoid duplicate JMX for Store and Pipeline stats Current code creates duplicate JMX for storeClient , pipeline stats when the getStoreClient method is called more than once. This fix avoids the duplicate jmx registration by caching the first creation and registering only once. Read only stroage engine registers the last swapped with node id in the name, this creates multiple counters which clutters the log and tracking across multiple nodes last swap time difficult to visualize. | 14 September 2015, 18:35:28 UTC |
| 7975559 | Felix GV | 11 September 2015, 23:13:05 UTC | BnP HA improvements. - HdfsFailedFetchLock now happens on the server-side. - BnP no longer relies on any specific node (the node.id config parameter is eliminated). - Moved some authentication-related code out of HdfsFetcher and into HadoopUtils. | 14 September 2015, 04:22:10 UTC |
| 5c3d158 | Felix GV | 08 September 2015, 22:33:46 UTC | Graceful recovery from incomplete BnP store creation. | 08 September 2015, 22:33:46 UTC |
| 8563062 | Felix GV | 31 August 2015, 20:59:24 UTC | Added long retries to ServerTestUtils.startVoldemortServer It will now retry for up to 120 attempts, and up to 5 minutes before giving up. This is a workaround for the flaky non-deterministic issues we see in a lot of tests. | 02 September 2015, 17:33:25 UTC |
| 4b0fc58 | Felix GV | 31 August 2015, 20:58:35 UTC | Marked some RocksDB tests with @Ignore. The corresponding functionality is not implemented, so these failures are expected. | 31 August 2015, 20:58:35 UTC |
| 54164ff | Felix GV | 28 August 2015, 21:23:54 UTC | Changed routing strategy for system stores to "all-routing". Previously, the metadata_version_persistence and store_quotas stores were using the "local-pref-all-routing" strategy which leads to undeterministic behavior. | 28 August 2015, 21:23:54 UTC |
| 4280c14 | Felix GV | 25 August 2015, 00:37:19 UTC | Fixed broken test and marked HTTP Service as deprecated. | 25 August 2015, 00:37:19 UTC |
| 9e06265 | Felix GV | 24 August 2015, 23:22:16 UTC | Added warning messages on the stream admin commands. These commands are not considered production-ready. They are intended for debugging purposes. Also cleaned up a bit of dead code in admin commands. | 24 August 2015, 23:22:16 UTC |
| 8997512 | James Lent | 20 August 2015, 17:44:40 UTC | Add a 'binary' format to the fetch-entries admin command so that it will create a file comaptible with the update-entries command. For consistency sake add this same format to the fetch-keys command. | 24 August 2015, 22:38:28 UTC |
| ef29e5d | Arunachalam Thirupathi | 22 August 2015, 01:09:35 UTC | Fork lift corrupts the data on schema mismatch 1) If the source and destination schema does not match, currently forklift corrupts the destination data by streaming in bytes from the source data. After this commit, the forklift will fail when a schema mismatch is detected. The old behavior if required can be achieved by undocumented parameter ignore-schema-mismatch Default of fork lift which forklifts all stores is changed to fail when store name is not specified. I can't imagine a situation where you want to forklift form one cluster to other often. If an admin forgets to specify this parameter, they are forklifting the entire cluster which is not definitely intended default. Added 5 unit tests ( 3 for key mismatch and 2 for value mismatch). Added pretty print functions to Compression and SerializerDefintion. | 24 August 2015, 22:35:59 UTC |
| fce83e7 | Arunachalam Thirupathi | 22 August 2015, 01:07:25 UTC | Make avro utf8 and bytes readable in shell output 1) fetch-keys and fetch-entries streaming option for utf8 and bytes are not human readable. This is a problem if you want to sample and read them using the shell. 2) voldemort-shell.sh does not output the avro bytes in a readable format. The previous output was some internal state and it does not convey what is the output. | 24 August 2015, 22:35:59 UTC |
| eec454a | Felix GV | 20 August 2015, 01:09:14 UTC | Improving debuggability of read-only fetches. - All SchedulerService threads now have a unique name, instead of being all called "java.util.concurrent.ThreadPoolExecutor$Worker". - AsyncOperation instances will now override their current thread's name to provide even more detail about what's running, and then restore the original thread name. - The fetcher loop in the server will report slightly more useful info to the BnP job via the AsyncOperation's status message. - The fetcher loop in the server will also print local logs which are more similar to the BnP-side log, for easier log correlation. - HdfsCopyStats flush lines as they happen, in order to improve debuggability of stalled or abrupbtly interrupted fetches... | 24 August 2015, 22:27:48 UTC |
| fd345f7 | Felix GV | 15 August 2015, 00:20:02 UTC | Releasing Voldemort 1.9.19 | 15 August 2015, 00:20:02 UTC |
| 07f0f72 | Felix GV | 14 August 2015, 22:33:10 UTC | Removed MemLock class and related configs. It turns out our mlock call always failed and we've been running fine without it anyway, so there's no point in keeping that code around. | 14 August 2015, 22:33:10 UTC |
| 9e5fe19 | Greg Banks | 13 August 2015, 14:45:24 UTC | Cleanup formatting of system store schema. | 13 August 2015, 14:45:24 UTC |
| a6294ce | Arunachalam Thirupathi | 12 August 2015, 22:26:15 UTC | Revert "Clean Set Quota Fix" This reverts commit 3ff1cbe47a2206834c2cd6b77878dcccb3f0be67. The unit tests are broken. On a deeper look, file backed caching storage engine, does not have a version per key, but at a file level. Any updates to the file with lower version will be rejected. So we have to go with super clocks for now. I am reverting the commit to unblock the release of a new version. | 12 August 2015, 22:27:04 UTC |
| fa0acf6 | James Lent | 22 July 2015, 19:19:38 UTC | Ensure that all the AbstractStorageEngineTest tests get run by all the subclasses. Add the @Test annotation to several tests. Without that annotation it appears that those subclasses that are parameterized do not run these specific test cases. This is most likely a base gradle issue. Perhaps somewhat related to: https://issues.gradle.org/browse/GRADLE-3112 | 12 August 2015, 19:03:44 UTC |
| da0ec6d | Felix GV | 12 August 2015, 01:35:40 UTC | Removed HadoopStoreJobRunner and related scripts. This is to avoid confusion. There is no point in running those scripts. - For building and pushing a read-only store, VoldemortBuildAndPushJobRunner and ./bin/run-bnp.sh should be used. If someone wants to just build without pushing, that can be done by passing push=false to the BnP config. - For swapping a store version, it can be done via the vadmin.sh script. | 12 August 2015, 17:54:11 UTC |
| 00ad250 | Felix GV | 12 August 2015, 00:09:29 UTC | Changes to @elad's shadowJar build, to ensure all dependencies are bundled up. | 12 August 2015, 17:54:11 UTC |
| a4bc78c | Elad Efrat | 04 August 2015, 13:22:17 UTC | Minimal config for read-only store, with Hadoop (BnP) hints. | 12 August 2015, 17:54:11 UTC |
| 3add1aa | Elad Efrat | 04 August 2015, 13:16:06 UTC | Print number of partitions, useful for debugging. | 12 August 2015, 17:54:11 UTC |
| e5eee78 | Elad Efrat | 04 August 2015, 13:12:45 UTC | Add BnP job and script from @FelixGV with slight changes. | 12 August 2015, 17:54:11 UTC |
| c974058 | Elad Efrat | 04 August 2015, 13:05:25 UTC | Make this work on Hadoop 2.x by shading a few dependencies. Shade avro (to 1.4.0) and protobuf (to 2.3.0) and always include jdom (1.1). Mostly from #274, also see #284. | 12 August 2015, 17:54:11 UTC |
| e3113e3 | Felix GV | 12 August 2015, 01:56:04 UTC | Addressing stylistic comment. | 12 August 2015, 17:41:35 UTC |
| 98cd411 | ARUNACHALAM THIRUPATHI | 12 August 2015, 01:40:33 UTC | Merge pull request #296 from arunthirupathi/setQuota Clean Set Quota Fix | 12 August 2015, 01:40:33 UTC |
| 8c068ac | Felix GV | 12 August 2015, 00:46:46 UTC | Allow RO servers to run without Kerberos enabled. | 12 August 2015, 00:46:46 UTC |
| 3ff1cbe | Arunachalam Thirupathi | 11 August 2015, 19:35:49 UTC | Clean Set Quota Fix Clean up the Set quota to not generate super clocks. Wait for all the nodes to complete, before returning success. | 11 August 2015, 19:35:49 UTC |
| 2a0c81e | Greg Banks | 06 August 2015, 15:49:56 UTC | Remove unused members and methods from MemLock public setFile() is unused, and also dangerous because there is no point in the lifetime of the object when it can have a useful effect. The descriptor member is not needed after the ctor. | 11 August 2015, 18:30:03 UTC |
| dc91e04 | Greg Banks | 06 August 2015, 15:41:08 UTC | Factor out some common code in ChunkedFileSet into a new mapAndRememberIndexFile() method. | 11 August 2015, 18:30:03 UTC |
| faa2ca0 | Greg Banks | 06 August 2015, 15:32:14 UTC | Remove unused mapFile() method in ChunkedFileSet. | 11 August 2015, 18:30:03 UTC |
| 8647f82 | Greg Banks | 06 August 2015, 15:30:11 UTC | Simplify MappedFileReader, close Unix fd early Remove most of the members and make them local variables in the map() function which is the only place that uses them. This simplifies the c'tor and means it cannot throw IOException anymore. It also means we close the Unix file descriptor used to create the mapping much earlier. This will help reduce the load on file descriptors. | 11 August 2015, 18:30:03 UTC |
| 1eafb88 | Greg Banks | 06 August 2015, 07:41:41 UTC | Remove unused fields from MappedFileReader - fadvise was an unused leftover from previous code - offset was used but it's value was always zero, stop pretending otherwise | 11 August 2015, 18:30:03 UTC |
| a4fe60a | Greg Banks | 06 August 2015, 07:38:03 UTC | Merge BaseMappedFile into its only subclass. It had precisely one subclass, did not define any semantically meaningful behavior, and nobody used the base class. It will not be missed. | 11 August 2015, 18:30:03 UTC |
| 4b74809 | Greg Banks | 06 August 2015, 06:59:32 UTC | Report mmap/munmap errors using IOException which the signature claims can be thrown but never was. Ensure that the only callchain (MappedFileReader -> MemLock -> mman) will actually handle and log IOException correctly. Also, stop claiming that mlock() and munlock() throw IOException. They never did and it would not be helpful, in fact we want to ignore all errors from those functions because their normal behavior in unprivileged processes is to fail. | 11 August 2015, 18:30:03 UTC |
| 9fb2689 | Greg Banks | 06 August 2015, 05:33:50 UTC | Remove all traces of MAP_ALIGN which doesn't exist on Linux (the binary value we were passing into the kernel was some other harmless option) and even if it did exist it has no effect because we're hardcoding the alignment to 0 which means "let the kernel choose" i.e. the default behavior. | 11 August 2015, 18:30:03 UTC |
