https://github.com/voldemort/voldemort

sort by:
Revision Author Date Message Commit Date
3b6c282 Bug fixes in HdfsFetcher revealed by HdfsFetcherTest (NPE and hiding the VoldemortSerializationException) 14 January 2013, 02:52:29 UTC
f97699b Minor debug fixes 12 January 2013, 00:25:55 UTC
a18d91b Fixing .classpath which had the wrong hadoop-core version. Also changed the Mlock related info messages to debug. 11 January 2013, 23:43:00 UTC
c611a8c Updated the hadoop-core jar to 1.0.4-p2 11 January 2013, 23:42:06 UTC
4f5ce5b Added the ability to delete old checksum files in the Build and Push reducer. Also updated the hadoop core library to version 1.0.4-p2-rc2 containing the fix to the User ID command usage. 11 January 2013, 23:42:06 UTC
ec3b37e Improve batch modifications on BDB-JE 11 January 2013, 23:36:41 UTC
903e749 Monitoring for streaming operations 11 January 2013, 23:36:41 UTC
4ae1336 Fix tests for testStartVoldemortCluster to not consume so much memory test/common/voldemort/ServerTestUtils.java - remove stress test from normal testing - test the method startVoldemortCluster once and confirm it returns a non-null Cluster object test/common/voldemort/ServerTestUtils.java - clarified a comment 10 January 2013, 16:28:34 UTC
db493a1 Updated release_notes and release version to 1.1.7 including Kerberos related changes 04 January 2013, 00:09:05 UTC
5aa3716 NIO + BDB stats - Code review comments 02 January 2013, 21:37:43 UTC
dd29d0e Stats to understand NIO layer performance + BDB exception counts et al 02 January 2013, 21:37:43 UTC
fbe6718 Creating constants for the default kerberos principal and keytab path 07 December 2012, 23:34:43 UTC
ef5420f Removing the additional JVM args from the Voldemort scripts 07 December 2012, 01:15:20 UTC
7d93e6d Adding KDC info and other JVM arguments to the voldemort scripts 07 December 2012, 01:06:16 UTC
e7b93ba Adding a hack to bypass Kerberos authentication for hftp protocol. TODO: remove this bypass once the libhadoop on Solaris is well tested 06 December 2012, 19:54:19 UTC
22b99c3 Finalized changes to HdfsFetcher to make it work with a Kerberized Hadoop cluster over webhdfs 06 December 2012, 19:54:19 UTC
44c4667 Standardizing the Kerberos login phase : explicitly specify the Hadoop config and keytab path. Also assumes that extra kerberos related config parameters are passed to the Java process 06 December 2012, 19:54:19 UTC
577378e Correcting usage: proxyUser to kerberosUser 06 December 2012, 19:54:19 UTC
2965762 Fixing a bug where we dont need to have hadoop conf in the classpath 06 December 2012, 19:54:19 UTC
c8d7a6c Fix in FsPermission constructor to maintain hadoop jar backwards compatibility 06 December 2012, 19:54:19 UTC
0f2f85e Fixed a jmx unregister bug in hdfs fetcher 06 December 2012, 19:54:19 UTC
e6b24c6 Doing authentication in a synchronized block for the Hdfs fetcher, setting correct permission for the hadoop files 06 December 2012, 19:54:19 UTC
7693d6b Fixed the main method params in HdfsFetcher 06 December 2012, 19:54:19 UTC
c745f80 Added configurable Kerberos support to HdfsFetcher and upgraded hadoop jars to 1.0.2 06 December 2012, 19:54:19 UTC
daa49bf Add support for kerberized grids in the job by supporting protocols 06 December 2012, 19:12:57 UTC
374d02a Added release notes 29 November 2012, 22:43:43 UTC
99dc97f Adding mlock for RO stores 29 November 2012, 21:57:23 UTC
7c49c64 Fixed constructor to actually take the mlock parameter made it true by default 29 November 2012, 21:57:23 UTC
2f5e776 Fixed bug with typesetting of the native args by wrapping it into a native wrapper 29 November 2012, 21:57:23 UTC
220c5f1 debug msgs# Explicit paths specified without -i nor -o; assuming --only paths... 29 November 2012, 21:57:23 UTC
763ff89 Hardcoding indexmlock to true 29 November 2012, 21:57:23 UTC
32fb76a Code cleanup 29 November 2012, 21:57:23 UTC
4db8cc3 Implemented a jna mlock to map and ping index files for RO stores in memory 29 November 2012, 21:57:23 UTC
349f852 Changed release number 27 November 2012, 23:56:22 UTC
d56a614 fixed avro mapper 26 November 2012, 18:28:01 UTC
60a987c Fixed mapper issue 21 November 2012, 23:46:02 UTC
84eda3a Prepared release 1.1.2 31 October 2012, 18:43:11 UTC
ba8c6fb Revert return type of Versioned.getVersion() to be Version rather than VectorClock 31 October 2012, 17:52:09 UTC
3c99c01 Bumped curr.release to 1.1.1 30 October 2012, 15:34:36 UTC
6246d4e Fixes for connection leak and ZenStoreClient config - Applied fix for socketChannel leak in ClientRequestExecutorFactory.create() - Added comments to document other code paths at risk of leaking socketDestinations - changed ClientConfig default from ZenStoreClient to DefaultStoreClient - updated release notes 29 October 2012, 23:32:32 UTC
c227e34 Updating release notes and version 19 October 2012, 19:10:48 UTC
3f1ec39 Resolving Conflict: Adding Imports back in test 19 October 2012, 04:13:49 UTC
cd16456 Add RetentionEnforcingStore, with support for online retention on reads 19 October 2012, 04:09:44 UTC
08e5258 Added parameters to control retention job 1. day of the week the retention job starts 2. if the retention job starts at the same hour each day 19 October 2012, 04:07:25 UTC
3d51f85 Fixing SlowStorageEngine and FileBackedCachingStorageEngine build issues 19 October 2012, 03:56:09 UTC
5e48136 BDB data conversion utility 19 October 2012, 03:50:44 UTC
7c9e2b0 Add BDB params -- background_proactive_migration, level based eviction 19 October 2012, 03:50:44 UTC
68b31b9 Adding Partition Scan support for rebalancing 19 October 2012, 03:50:44 UTC
98182b6 Code review - partition scan 19 October 2012, 03:50:44 UTC
fbe56d2 Partition scans - more tests, typo fixes 19 October 2012, 03:50:44 UTC
4c1064e Implementing partition scans 19 October 2012, 03:47:16 UTC
07e509d Upgrading to JE 4.1.17 19 October 2012, 03:45:40 UTC
0d449a7 Code review changes 19 October 2012, 03:45:40 UTC
692b63f initial commit - new duplicate handling 19 October 2012, 03:42:51 UTC
5a021db Revised NOTE in the release_notes about version numbering. 18 October 2012, 14:59:42 UTC
922ec3a Explained the versioning number change. 17 October 2012, 22:47:43 UTC
b6447a5 Tweaked release notes 17 October 2012, 22:36:48 UTC
c6a6e21 Bumped release to 1.0.0 and added release notes. 17 October 2012, 21:34:53 UTC
e1796c4 Hardened junit long test RebalanceLongTest.java. Other test hardening. build.xml - bumped all maxmemory settings to 2048m - Placed a 90 minute timeout on the long test at ant level. test/long/voldemort/client/rebalance/RebalanceLongTest.java - null out some objects in the hopes of reducing the overall memory footprint of these tests. We are truly abusing junit with a long-running, multi-threaded test, that has 10 sub tests and 4 distinct parameter settings. test/unit/voldemort/client/rebalance/AbstractRebalanceTest.java - start of tests is not clear in junit log output. Added logger.info to start of tests to make grepping through the log when tests have failed badly and/or are running in an infitie loop easier. - Bumped each test timeout up to 10 minutes. Again, note abuse of junit: tests should not be defined at the abstract class level. This makes it hard to set appropriate limits (such as timeout) for each specific test. Long tests should have a different timeout than short tests... 17 October 2012, 16:03:40 UTC
34ab463 Added two tools for repeatedly running junit tests. Either specific tests, or all of junit. The benefit of these scripts is that the results from each run are archived toa temp directory. This allows you to stress test big changes, find intermittent failures, and so on. 16 October 2012, 21:34:09 UTC
61c11c7 Cleaned up comments and TODOs from prior commits. 16 October 2012, 21:34:09 UTC
2411377 Fixed two small errors I introduced while fixing tests (or merging with master). 16 October 2012, 21:34:08 UTC
7d60d6f updated copyrights on touched files. 16 October 2012, 21:34:08 UTC
7c793f6 Additional hardening of tests to reduce the number of intermittent BindException errors due to a TOCTOU issue with getLocalCluster. test/integration/voldemort/performance/RoutedStoreParallelismTest.java - switched to startVoldemortCluster test/unit/voldemort/server/gossip/GossiperTest.java - hand-coded test-specific startParallelVoldemortCluster. Not pretty. Not pretty at all. But, should retry in the face of such exceptions. Switched TODO to comment about possible susceptability to BindExceptions: - test/unit/voldemort/client/rebalance/RebalanceTest.java - test/unit/voldemort/scheduled/StreamingSlopPusherTest.java 16 October 2012, 21:34:08 UTC
de66f98 Hardening test utils and tests to reduce the number of intermittent BindException errors due to a TOCTOU issue with getLocalCluster. The main improvement is the addition of ServerTestUtils.startVoldemortCluster that wraps getLocalCluster and a bunch of startVoldemortServer calls in a retry loop based on wether a BindException occurs. This is suitable to ~75% of our test cases that use getLocalCluster. test/common/voldemort/ServerTestUtils.java - Added startVoldemortCluster test/unit/voldemort/utils/ServerTestUtilsTest.java - Tests for ServerTestUtils to reproduce intermittent failures src/java/voldemort/server/VoldemortServer.java - TODO note about method that plays a role in another intermittent test failure invovling cluster.xml Switched test to use startVoldemortCluster test/unit/voldemort/client/AdminServiceBasicTest.java test/unit/voldemort/client/rebalance/AdminRebalanceTest.java test/unit/voldemort/cluster/failuredetector/ServerStoreVerifierTest.java test/unit/voldemort/server/EndToEndTest.java test/unit/voldemort/server/socket/NioStatsJmxTest.java test/unit/voldemort/store/readonly/swapper/StoreSwapperTest.java test/unit/voldemort/store/rebalancing/RebootstrappingStoreTest.java test/unit/voldemort/versioning/ChainedInconsistencyResolverTest.java Junit3 -> Junit 4 test/unit/voldemort/client/AdminServiceBasicTest.java test/unit/voldemort/client/rebalance/AdminRebalanceTest.java test/unit/voldemort/server/gossip/GossiperTest.java test/unit/voldemort/server/socket/NioStatsJmxTest.java test/unit/voldemort/store/readonly/swapper/StoreSwapperTest.java test/unit/voldemort/versioning/ChainedInconsistencyResolverTest.java Annotated with a TODO about the test still needing to be hardened against TOCTOU issue with getLocalCluster: test/integration/voldemort/performance/RoutedStoreParallelismTest.java test/unit/voldemort/client/rebalance/RebalanceTest.java test/unit/voldemort/scheduled/StreamingSlopPusherTest.java test/unit/voldemort/server/gossip/GossiperTest.java 16 October 2012, 21:34:08 UTC
03c60ce Adding a specific mini test that exercises ServerTestUtils.startVoldemortServer. This ~15 line program that simply starts some Voldemort servers using test utils can tickle two different intermittent failures: (1) ObsoleteVersionException when loading cluster.xml Testcase: startMultipleVoldemortServers took 0.385 sec Caused an ERROR A successor version version() to this version() exists for key cluster.xml voldemort.versioning.ObsoleteVersionException: A successor version version() to this version() exists for key cluster.xml and (2) A bind issue characterized as follows: Testcase: startMultipleVoldemortServers took 2.066 sec Caused an ERROR java.net.BindException: Address already in use voldemort.VoldemortException: java.net.BindException: Address already in use at voldemort.server.niosocket.NioSocketService.startInner(NioSocketService.java:131) at voldemort.server.AbstractService.start(AbstractService.java:62) at voldemort.server.VoldemortServer.startInner(VoldemortServer.java:255) at voldemort.server.AbstractService.start(AbstractService.java:62) at voldemort.ServerTestUtils.startVoldemortServer(ServerTestUtils.java:666) at voldemort.utils.ServerTestUtilsTest.setUp(ServerTestUtilsTest.java:50) Caused by: java.net.BindException: Address already in use at sun.nio.ch.Net.bind(Native Method) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:126) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:59) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:52) at voldemort.server.niosocket.NioSocketService.startInner(NioSocketService.java:125) 16 October 2012, 21:34:08 UTC
d685b67 fixed try catch in versioned avro serializer 16 October 2012, 21:29:06 UTC
1524721 Added testcase for schema evolution check 16 October 2012, 03:10:09 UTC
5aacdc7 Added comments to the new functions 16 October 2012, 02:52:20 UTC
20261b9 - Refactored schema check method - added fix to the verioned serializer to support writing of objects created using the old schema 15 October 2012, 20:27:03 UTC
f2bfbc0 Added check in serializer if writer's schema greater than reader raise an exception 15 October 2012, 20:27:03 UTC
1ba1779 Added support for Avro schema evolution in RO Stores 15 October 2012, 20:27:03 UTC
0c4bef5 Updated AsyncMetadataVersionManagerTest for checking individual store definition updates. Added mechanism in VoldemortAdminTool to update individual store metadata version 15 October 2012, 18:24:31 UTC
8804f79 Added the ability to auto-bootstrap on store definition changes 12 October 2012, 19:02:40 UTC
5e00c76 code cleanup 12 October 2012, 02:50:30 UTC
d919558 fixed test case 11 October 2012, 23:40:48 UTC
dc4d7b6 Changed test case to Junit 4 11 October 2012, 22:29:12 UTC
f66c20d Refactored checkcompatibility into Validator class as a static method 11 October 2012, 22:29:12 UTC
46f42fe Added check for schema backwards compatibility for Avro in Admin Client tool and server startup 11 October 2012, 22:21:02 UTC
9656114 Fixed comment Please enter the commit message for your changes. Lines starting 11 October 2012, 21:59:54 UTC
6aaecc8 Added unit test 11 October 2012, 21:59:54 UTC
4b83abd Added support for schema evolution for the Avro generic serializer 11 October 2012, 21:59:54 UTC
79d9163 Added comment to explain why NPEs can end up in the log during shutdown if there async requests are queued up. 11 October 2012, 15:29:46 UTC
a34e62a Changed both serial (sync) operations and parallel (async) operations to deduct the elapsed checkout time from the operation (routing) timeout for specific requests. 11 October 2012, 03:38:52 UTC
bb73de1 Fixed up ClientSocketStatsTest to match change in return code from -1 to 0. 10 October 2012, 20:42:47 UTC
0df53c0 Minor fixes to various comments to clarify some implementation/usage details. 10 October 2012, 16:52:30 UTC
31bc42e Added @override to some methods as Eclipse asked me to. Fixed missing include in test. 09 October 2012, 21:07:05 UTC
1890420 Changes to make tight timing tests for QueuedKeyedResourcePool and SlowStorageEngine less sensitive on slower machines. 09 October 2012, 20:26:14 UTC
e070703 Clean up get stats methods in (Queued)KeyedResourcePool. Fix error in test case. 09 October 2012, 20:26:13 UTC
0626f60 Switched SlowStorageEngine to take a StorageEngine<K,V,T> in the constructor to be more flexible. 09 October 2012, 20:26:13 UTC
e372717 Fixed E2E non blocking checkout test to actually check for non-blocking checkouts. (Addresses review feedback from Chinmay Soman.) 09 October 2012, 20:26:13 UTC
4e6e8b4 synchronize the reset of a specific keyed pool to avoid invoking destroyResource on the same resource multiple times. 09 October 2012, 20:26:13 UTC
a3ca980 bumped up test timeouts since Hudson seems slower than local machine for contention experiments. 09 October 2012, 20:26:13 UTC
f2466fa Copyright statement cleanup. 09 October 2012, 20:26:13 UTC
b4c3bff Additional Jmx Getters so that we can better understand stats sample sizes. 09 October 2012, 20:26:13 UTC
133b9c7 Minor changes to deal with remaining TODOs in this change. I still believe there are some ugly code paths that fire off too many exceptions when we tear down a connection. Hopefully, the connection re-write that is starting off will clean up these ugly code paths. 09 October 2012, 20:26:13 UTC
a28bcd9 Minor cleanup --- changed some todos to documentation and comments. 09 October 2012, 20:26:13 UTC
e1fc74c Added Jmx interfaces for all queue stats we now track. Updated ClientSocketStatsTest as well. Added a big TODO expressing concern over how statistics are tracked with suggestions for improvements. 09 October 2012, 20:26:13 UTC
233ceb1 Removed the commented out implementations of distinct policies for async socket checkout. 09 October 2012, 20:26:13 UTC
f1cd8ef Implementations of various async vs sync queueing policies for socket checkout. Two commented out policies are included in this commit. src/java/voldemort/store/socket/clientrequest/ClientRequestExecutorPool.java - a few TODOs to be investigated before completing work on async checkouts /src/java/voldemort/utils/pool/AsyncResourceRequest.java - a couple helper methods for implementing/debugging queueing policies src/java/voldemort/utils/pool/KeyedResourcePool.java - refactor to clean up checkin method - TODOs for further code cleanup - cleaned up all methods for tracking stats, added stats tracking of length of synchronous queue - various aspects of (commented out) socket checkout queuing policies src/java/voldemort/utils/pool/QueuedKeyedResourcePool.java - fixes to async socket checkout - various aspects of (commented out) socket checkout queuing policies - TODOs for further code cleanup - cleaned up stats tracking for async queue length *Test.java - minor tweaks/cleanup 09 October 2012, 20:26:13 UTC
back to top