| bf52cd1 | Jay J Wylie | 12 July 2013, 16:07:59 UTC | Added 'details' option to KeyVersionFetcherCLI The 'details' option results in the inclusion of the partition ID, node ID, and host for each fetched key-version. This should help relate specific changes in the key-versions fetched before and after rebalance to specific steps in the rebalancing plan. | 12 July 2013, 16:07:59 UTC |
| 616f704 | Jay J Wylie | 10 July 2013, 17:10:30 UTC | Removed maxTries option from Rebalancing maxTries was a hack to work around problems with the scheduling of rebalancing tasks. The new scheduler (is supposed to) ensure that no node is doing more than one rebalancing thing (stealing or donating) at time. Therefore, not being able to schedule an asynchronous rebalancing task (stealer-based) on a node is a big problem and should not be retried. | 10 July 2013, 17:10:30 UTC |
| 111a378 | Jay J Wylie | 09 July 2013, 21:04:59 UTC | Drop server-side rebalancing optimization, fix proxy put config The RebalancePlan object now produces "optimized" plans. I.e., data is not moved to a server that already hosts that data. This means that server-side optimizations to not fetch data the server already has are not needed. The voldemort config had a parameter to turn proxy puts on/off. Now that they are tested, they should always be on for the sake of making sure rebalances are abortable. Also, the proxy put default number of threads was one. It is now set in the same manner as nio selectors. Sorry for all the unnecessary white space changes. I need to reset my eclipse settings. | 09 July 2013, 21:04:59 UTC |
| 279c6c3 | Jay J Wylie | 09 July 2013, 17:54:53 UTC | Complete dropping KeyDistributionGenerator | 09 July 2013, 17:54:53 UTC |
| 3b022ae | Jay J Wylie | 09 July 2013, 17:51:46 UTC | Drop KeyDistributionGenerator and Entropy The intended usage of KeyDistributionGenerator and Entropy has been replaced. The PartitionAnalysisCLI and PartitionBalance tools replace the 'skew' estimation functionality of these tools. The KeySampler and KeyVerionFetcher replace the verification functionality of the EntropyChecker. The helper method KeyDistributionGenerator.getUniqueStoreDefinitionsWithCounts has moved to StoreDefinitionUtils.getUniqueStoreDefinitionsWithCounts. | 09 July 2013, 17:51:46 UTC |
| c0d1599 | Siddharth Singh | 08 July 2013, 23:08:10 UTC | Fix NPE in the test code Pass an empty list instead of a null. | 08 July 2013, 23:42:03 UTC |
| 399bd28 | Siddharth Singh | 04 July 2013, 00:29:55 UTC | Copy analysis file to output_dir | 04 July 2013, 00:37:54 UTC |
| 87691c6 | Siddharth Singh | 03 July 2013, 22:54:34 UTC | Rename new cluster script | 04 July 2013, 00:37:23 UTC |
| 61c15f6 | Siddharth Singh | 03 July 2013, 19:29:32 UTC | More cleanup to repartitioner Fix varible names, typos add comments | 03 July 2013, 22:49:13 UTC |
| c93bc85 | Siddharth Singh | 03 July 2013, 17:26:04 UTC | Make repartitioner interface consistent + cleanup | 03 July 2013, 17:36:00 UTC |
| dbb8acf | Jay J Wylie | 02 July 2013, 23:55:40 UTC | Merge branch 'zone-expansion' of github.com:voldemort/voldemort into zone-expansion | 02 July 2013, 23:55:40 UTC |
| e6fee03 | Siddharth Singh | 01 July 2013, 21:56:26 UTC | Expose max contiguous partitons option to script | 01 July 2013, 23:41:41 UTC |
| a114f7e | Siddharth Singh | 01 July 2013, 18:11:32 UTC | Cleanup minor typos and comments | 01 July 2013, 23:41:41 UTC |
| 9592791 | Siddharth Singh | 01 July 2013, 20:06:32 UTC | Copy plan outout to the final directory | 01 July 2013, 23:41:41 UTC |
| baefd00 | Siddharth Singh | 29 June 2013, 00:51:13 UTC | Remove vold_home parameter from scripts These scripts are being shipped as part of vold repo and so no need for this extra parameter. | 01 July 2013, 23:41:41 UTC |
| d678308 | Siddharth Singh | 29 June 2013, 00:01:56 UTC | Get rid of tee-ing the plan to a separate file The rebalance planner writes the output to a file named plan.out so this is not needed anymore. | 01 July 2013, 23:41:41 UTC |
| b404fe7 | Siddharth Singh | 28 June 2013, 23:54:55 UTC | Fix comments | 01 July 2013, 23:41:41 UTC |
| a2d7f73 | Siddharth Singh | 28 June 2013, 23:51:48 UTC | Add cluster expansion script | 01 July 2013, 23:41:41 UTC |
| 7111739 | Siddharth Singh | 28 June 2013, 22:42:34 UTC | Fix location of example stores | 01 July 2013, 23:41:41 UTC |
| 326ed4f | Siddharth Singh | 28 June 2013, 22:24:24 UTC | Add rebalance shuffle script | 01 July 2013, 23:41:41 UTC |
| 078da6d | Jay J Wylie | 01 July 2013, 19:45:26 UTC | Change Repartioner to swap among nodes from all specified zones. PartitionBalance - Added comment about possible weakness of our utility method. Repartitoiner.java - Changed random shuffle implementation to swap among all nodes in specified zones. - Left TODOs and 'dead code' implemntation of intra-zone swapping vs inter-zone swapping. | 01 July 2013, 19:45:26 UTC |
| 5459942 | Jay J Wylie | 28 June 2013, 01:41:55 UTC | Making StoreRoutingPlan even lighter-weight | 28 June 2013, 04:43:12 UTC |
| bf7daad | Jay J Wylie | 28 June 2013, 01:27:02 UTC | Adding BaseStoreRoutingPlan. | 28 June 2013, 04:43:12 UTC |
| 606f6fa | Jay J Wylie | 28 June 2013, 01:25:44 UTC | Refactor StoreRoutingPlan. Break Base from rest to expedite construction. | 28 June 2013, 04:43:12 UTC |
| f8765a3 | Jay J Wylie | 25 June 2013, 16:37:13 UTC | Addressed review feedback on rebalance scheduler RebalanceController - Changed default parallelism to "infinite" since scheduler throttles parallelism - Added TODOs for cleanup of scheduler - Added javadoc to document scheduler and its methods - Catch exceptions, log them, re-throw as VoldemmortRebalancingException A ton of white space changes due to futzing with eclipse code formatter preferences. Sorry. | 28 June 2013, 04:43:12 UTC |
| d7356df | Jay J Wylie | 24 June 2013, 15:47:07 UTC | Tweak new rebalance scheduler Randomized the order of rebalance tasks in each stealer's list. This will avoid biasing the rebalance based on the order tasks were generated. | 28 June 2013, 04:43:12 UTC |
| 34e849e | Jay J Wylie | 23 June 2013, 23:04:00 UTC | Initial hack at new rebalance scheduler. Added RebalanceController.scheduler - limits each node to participating in a single task as either a stealer or donor. - randomizes the order in which tasks are attempted to be scheduled - not a clean implementation, but enough to evaluate. | 28 June 2013, 04:43:12 UTC |
| a26a558 | Jay J Wylie | 23 June 2013, 16:11:25 UTC | Tweaks to PartitionBalance and RebalancePlan PartitionBalance - calculate partition-stores per zone. - This measure provides more context to evaluate the size of any plans to rebalance the cluster. RebalancePlanCLI - fix typo in verbose usage message | 28 June 2013, 04:43:12 UTC |
| 33ea454 | Zhongjie Wu | 24 June 2013, 19:59:11 UTC | bug fixes in coordinator | 24 June 2013, 23:51:34 UTC |
| 93b9495 | Siddharth Singh | 21 June 2013, 19:41:49 UTC | Bump release version | 21 June 2013, 19:41:49 UTC |
| 4d72ef4 | Siddharth Singh | 21 June 2013, 19:35:16 UTC | Add 1.3.4 and 1.4.0 release notes | 21 June 2013, 19:35:16 UTC |
| 155b934 | Siddharth Singh | 19 June 2013, 00:17:53 UTC | Fix comments and typos | 21 June 2013, 15:48:56 UTC |
| 3d8eb81 | Siddharth Singh | 19 June 2013, 00:15:40 UTC | Add separate dummy stores for 2 and 3 zone cluster | 21 June 2013, 15:48:47 UTC |
| 3944a3c | Siddharth Singh | 19 June 2013, 00:15:11 UTC | Cleanup the arguments and add sanity check | 21 June 2013, 15:48:39 UTC |
| 5d1e8e3 | Siddharth Singh | 18 June 2013, 01:00:07 UTC | Step 2: Add logic to handle zoned clusters | 21 June 2013, 15:48:29 UTC |
| d2452a9 | Siddharth Singh | 15 June 2013, 01:19:57 UTC | Step 1 : Generate cluster script cleanup | 21 June 2013, 15:48:14 UTC |
| 882c01f | Jay J Wylie | 20 June 2013, 22:33:40 UTC | Fix RebalanceUtils.getLatestCluster. RebalanceUtils - changed getLatestCluster to examine contents of cluster.xml on different nodes rather than timestamps since timestamps are incomparable across servers. AbstractZonedRebalanceTest - added a specific rebalance test that goes from current -> shuffle and then shuffle -> current. This confirms that rebalance can be invoked repeated times (if need be). | 20 June 2013, 23:11:05 UTC |
| 0c8f0fa | Abhinay Nagpal | 13 June 2013, 23:22:51 UTC | Fixed systemstore constants and got rid of the superfluous zone rep factor | 20 June 2013, 23:11:05 UTC |
| a758c2a | Abhinay Nagpal | 13 June 2013, 21:49:19 UTC | Tests for system store with a 3 zone cluster | 20 June 2013, 23:11:05 UTC |
| 3512cff | Vinoth Chandar | 14 June 2013, 00:35:07 UTC | Modifying RoutedStoreTest to use Consistent Strategy | 20 June 2013, 23:11:05 UTC |
| 9a90156 | Abhinay Nagpal | 13 June 2013, 17:58:27 UTC | Addressed code review comments added more assertions check for getZoneNary and getNodeIdForZoneNary in zone 1 and 2 Added a test for method 'storeRoutingPlan.zoneNAryExists() | 20 June 2013, 23:09:23 UTC |
| 4f135d5 | Abhinay Nagpal | 12 June 2013, 20:38:24 UTC | Test for 3 zones | 20 June 2013, 23:09:23 UTC |
| d2848a1 | Siddharth Singh | 12 June 2013, 21:51:14 UTC | Add dummy-stores.xml + Update the path in script | 20 June 2013, 23:09:23 UTC |
| 803d7dd | Siddharth Singh | 12 June 2013, 21:45:15 UTC | Move rebalance scripts to bin directory Moving the script from top level to the bin directory | 20 June 2013, 23:09:23 UTC |
| 1807fae | Siddharth Singh | 12 June 2013, 21:23:49 UTC | Copy final cluster xml file to output dir When the final cluster.xml is generated it is placed in outputdir/step3 folder. Copy it to outputdir as well. | 20 June 2013, 23:09:23 UTC |
| f9936c1 | Siddharth Singh | 12 June 2013, 03:53:08 UTC | Add rebalance new cluster script | 20 June 2013, 23:09:23 UTC |
| 9d9b47e | Siddharth Singh | 11 June 2013, 22:30:56 UTC | Address code review comments | 20 June 2013, 23:09:23 UTC |
| 407d4e4 | Siddharth Singh | 10 June 2013, 23:47:07 UTC | Edit script to use the new option | 20 June 2013, 23:09:22 UTC |
| ba371fc | Siddharth Singh | 10 June 2013, 23:42:46 UTC | Add an option for choosing zone id for shuffle Add a new option that lets the user choose if they want the shuffle to operate only on one of the zone ids. | 20 June 2013, 23:09:22 UTC |
| c1fb3aa | Siddharth Singh | 10 June 2013, 20:03:21 UTC | Add zone expansion script | 20 June 2013, 23:09:22 UTC |
| aa96bbf | Siddharth Singh | 07 June 2013, 22:18:05 UTC | Write plan to a file inside the output dir | 20 June 2013, 23:09:22 UTC |
| 9ee4ebe | Siddharth Singh | 03 June 2013, 23:27:10 UTC | Print the utility value | 20 June 2013, 23:09:22 UTC |
| f55aace | Jay J Wylie | 07 June 2013, 21:23:36 UTC | Fix final-cluster to be required arg of RebalancePlanCLI | 20 June 2013, 23:09:22 UTC |
| 77f3b69 | Jay J Wylie | 06 June 2013, 20:54:24 UTC | Correctly print RebalanceTask at creation time. | 20 June 2013, 23:09:22 UTC |
| ec9cd1e | Jay J Wylie | 04 June 2013, 16:58:29 UTC | Minor fix to KeyVersionFetcherCLI | 20 June 2013, 23:09:22 UTC |
| 0ef8faf | Jay J Wylie | 04 June 2013, 15:56:24 UTC | Make KeyVersionFetcherCLI have generous timeouts | 20 June 2013, 23:09:22 UTC |
| 3122693 | Siddharth Singh | 30 May 2013, 22:41:44 UTC | Add comments to the tools | 20 June 2013, 23:09:22 UTC |
| e40f793 | Jay J Wylie | 29 May 2013, 22:52:46 UTC | Fix to print one timestamp per-line from KeyVersionFetcherCLI | 20 June 2013, 23:09:22 UTC |
| ffc9b68 | Jay J Wylie | 29 May 2013, 22:35:50 UTC | Make KeyVersionFetcherCLI ZoneNAry aware | 20 June 2013, 23:09:22 UTC |
| 412088d | Jay J Wylie | 29 May 2013, 15:23:26 UTC | Address minor review feedback Mostly, made variable names more clear AdminClient - cleaned up comments & TODO RebalanceController - fix subtle bug in when proxyPause invoked RepartitoinerTest - changed zone expansion test to cover two use cases: when invoked from RepartitoinerCLI with a current cluster versus with an interim cluster. | 20 June 2013, 23:09:22 UTC |
| f816900 | Jay J Wylie | 24 May 2013, 22:55:45 UTC | Fixed typo. | 20 June 2013, 23:09:22 UTC |
| e876dd1 | Jay J Wylie | 24 May 2013, 22:21:43 UTC | Change variable name in RouteToAllStrategy. | 20 June 2013, 23:09:22 UTC |
| 89969de | Jay J Wylie | 24 May 2013, 22:05:19 UTC | Fixed cluster nomenclature: current, interim, final Touched a lot of code to normalize variable names, method names, and comments about clusters: - current : current cluster in prod - interim : current cluster + new nodes/zones w empty partitions - final : final cluster w shuffled partitions and/or populated new nodes/zones Added TODO about voldemort server excepting on invalid cluster xml Dropped TODOs that were unnecessary | 20 June 2013, 23:09:22 UTC |
| 4c276b8 | Jay J Wylie | 24 May 2013, 19:48:30 UTC | Added "progress bar" for rebalance batch plan Added RebalanceBatchPlanProgressBar - progress tracking object for each rebalancing batch - integrated with RebalanceController, AsyncRebalanceTask, and (Donor|Stealer)BasedRebalanceTask Did other general clean up of logging during rebalance to make it appropriately verbose (more verbose in some places, less verbose in others...) | 20 June 2013, 23:09:22 UTC |
| c9b370e | Jay J Wylie | 23 May 2013, 23:04:44 UTC | Add 'proxy pause' between cluster change & rebalance The proxy pause is a window during which clients can pick up the new cluster metadata, and servers can establish proxy bridges, before servers start moving data around for rebalancing. This allows us to observe the cost of proxying separate from rebalance and allows clients to pick up new metadata before any data is moved. Also added some more TODOs about documenting the various rebalanceState methods... | 20 June 2013, 23:09:21 UTC |
| d7f81d7 | Jay J Wylie | 23 May 2013, 18:41:26 UTC | Fix arg checking typo in RebalanceControllerCLI. | 20 June 2013, 23:09:21 UTC |
| 2bf1543 | Jay J Wylie | 23 May 2013, 16:24:26 UTC | Fix potential overflow in AdminClient.waitForCompletion | 20 June 2013, 23:09:21 UTC |
| 81f0a16 | Jay J Wylie | 23 May 2013, 15:58:22 UTC | Fix some bugs I introduced and added more TODOs Fixed overflow introduced in AdminClient.waitForCompletion by passing in Long.MAX_VALUE for duration. Verify cluster store definition in StoreRoutingPlan. This requires working around existing problems with how system stores are handled (the store definition is hard-coded for two zones). Left some TODOs about testing and fixing all of this. Added TODOs about currentCluster vs interimCluster. Need to tweak interface to RebalanceController and RebalancePlan to be consistent with recommended usage (i.e., deploying interimCluster before starting rebalance). Minor tweaks to tests based on above changes. | 20 June 2013, 23:09:21 UTC |
| c794504 | Jay J Wylie | 22 May 2013, 20:43:53 UTC | Minor TODO and copyright cleanup. | 20 June 2013, 23:09:21 UTC |
| 6a283ae | Jay J Wylie | 22 May 2013, 20:22:01 UTC | Fix invalid metadata rate calculation Fixed the calculation of invalid metadata rate in RebalanceUtils. Also added more/different pretty printing of a rebalance plan. | 20 June 2013, 23:09:21 UTC |
| 45903e0 | Jay J Wylie | 22 May 2013, 17:33:05 UTC | Address review of initial RebalanceController work - Removed 'timeout' option for rebalancing since always setting timeouts to infintie (Long.MAX_VALUE) is the recommended practice. - Cleaned up some TODOs and added some javadoc in response to code review from Lei | 20 June 2013, 23:09:21 UTC |
| 92faebd | Jay J Wylie | 22 May 2013, 16:28:06 UTC | Added junit-rebalance and junit-long-rebalance build targets | 20 June 2013, 23:09:21 UTC |
| 10901f7 | Jay J Wylie | 22 May 2013, 16:08:02 UTC | Completed unit tests for zone expansion Added zone expansion unit tests to AbstractZonedRebalanceTest. This required tweaking the helper methods in RebalanceUtils and ClusterTestUtils. | 20 June 2013, 23:09:21 UTC |
| d344725 | Jay J Wylie | 21 May 2013, 23:25:29 UTC | Added more zone expansion test for RebalanceBatchPlan | 20 June 2013, 23:09:21 UTC |
| d4c314f | Jay J Wylie | 21 May 2013, 22:56:13 UTC | RebalanceController uses atomic cluster/store RebalancePlan RebalanceBatchPlan - split storeDefs into currentStoreDefs & finalStoreDefs - clean up some variable naming *RoutingStrategy - marked all override methods with @Override RouteToAllStrategy - Implemented getPartitionList/getReplicatingPartitionList/getMasterPartition - Use a single imaginary parition id ("0") to implement these methods - This allows StoreRoutingPlan to be constructed for this strategy AbstractZonedRebalanceTest - fix proxy put test | 20 June 2013, 23:09:21 UTC |
| 8bbf529 | Jay J Wylie | 21 May 2013, 17:17:43 UTC | Fixes after rebase/dropped propagateCluster method After the rebase with the atomic update of cluster/stores branch, had some clean up to do. Got code running and unit tests passing again. Note that VAdminProto.java had to be regenerated after the merge. Noted that RebalanceUtils.propagate cluster was unnecessary (and potentially dangerous) and so dropped it from the code. | 20 June 2013, 23:09:21 UTC |
| c228e5f | Jay J Wylie | 17 May 2013, 22:26:19 UTC | Added unit tests for RebalancePlan, RebalanceBatchPlan, and AbstractZonedRebalanceTest. Added following rebalance test case coverage: - RebalancePlanTest / RebalanceBatchPlanTest / AbstractZonedRebalanceTest - no-op / shuffle / cluster expansion / zone expansion - 2-zone / 3-zone Also did some clean up of rebalance test utils and rebalance tests in general. | 20 June 2013, 23:09:05 UTC |
| 91c0b18 | Jay J Wylie | 16 May 2013, 15:21:09 UTC | Addressed review feedback on new RebalanceController. | 20 June 2013, 23:09:05 UTC |
| ad4b726 | Jay J Wylie | 15 May 2013, 22:02:03 UTC | Fix to store routing plan to correctly track zone-primaries per-node. | 20 June 2013, 23:09:05 UTC |
| 8bc5e17 | Jay J Wylie | 15 May 2013, 16:53:55 UTC | Minor changes to address second round of review feedback on rebalance plan. | 20 June 2013, 23:09:05 UTC |
| 5244216 | Jay J Wylie | 14 May 2013, 23:08:06 UTC | Address review comments for RebalancePlan RebalanceBatchPlan - factored logic to decide which donor to steal from out of constructBatchPlan. - huge header comment about other policies to consider RebalanceController - dropped comment about historic (arbitrary) sleep command RebalancePlan - added helper hacky method to "clone" cluster RebalanceCluster - TODO to add a .clone() method StoreRoutingPlan - cleaned up javadoc, TODOs, and method names DonorBasedRebalancePusherSlave - set sleep to 30 seconds. Made comments and logger messages consistent with code. StorageEngine - cleaned up javadoc for isPartitionAware() RebalanceUtils - fix validation method to use safe(r) comparison. RebalanceBatchPlanTest - fixed tests to (mostly) only test plan invariants. Prior test code focussed on exact plan details and so was hard-coded to the implementation, rather than the interface. | 20 June 2013, 23:09:05 UTC |
| 4fdeaf2 | Jay J Wylie | 13 May 2013, 18:31:16 UTC | Remove no-longer needed test from RebalancePartitionsInfoTest; annotated another test as mostly failing.. | 20 June 2013, 23:09:05 UTC |
| 8a9a791 | Jay J Wylie | 13 May 2013, 18:09:33 UTC | Clean up Rebalance(Plan|Controller)CLI RebalancePlan(CLI) - set batch size default to INTEGER.MAX_VALUE RebalanceControllerCLI - Cleaned up handling of all optional arguments | 20 June 2013, 23:09:05 UTC |
| 039b7fc | Jay J Wylie | 13 May 2013, 17:43:34 UTC | Renamed Rebalance(Long)Test to NonZonedRebalanceLongTest.java. Added ZonedRebalanceLongTest.java. | 20 June 2013, 23:09:05 UTC |
| a2b950c | Jay J Wylie | 13 May 2013, 17:24:58 UTC | Cleaned up existing TODOs and added some javadoc comments. | 20 June 2013, 23:09:04 UTC |
| 8121c78 | Jay J Wylie | 13 May 2013, 17:02:02 UTC | Renamed RebalanceClusterPlan(Test).java to RebalanceBatchPlan(Test).java. Cleaned up TODOs and comments and variable names in RebalanceBatchPlan. | 20 June 2013, 23:09:04 UTC |
| 1219576 | Jay J Wylie | 13 May 2013, 16:33:12 UTC | Fixed RebalanceClusterPlanTest to be junit 4 and to work as well as possible at this time. Test needs to be re-written to test new planning rather than old planning. | 20 June 2013, 23:09:04 UTC |
| 083a3e8 | Jay J Wylie | 13 May 2013, 16:13:30 UTC | Fix on-wire protocol for RebalancePartitionInfoMap Completed deprecation of member 'attempt' of RebalancePArtitionInfoMap by making it optional and prefixing name with OBSOLETE. This leads to big changes to auto generated file VAdminProto.java. Some other minor tweaks to RebalancePArtitionsInfo and comments as well. | 20 June 2013, 23:09:04 UTC |
| 2b69609 | Jay J Wylie | 10 May 2013, 19:38:10 UTC | Dropping RebalanceNodePlan, the class that sorted lists of RebalancePartitionsInfo by node id (either stealer or donor node), and then flattened the sorted list out and returned it again. AKA the identity operation on a list of RebalancePartitionsInfo. | 20 June 2013, 23:08:23 UTC |
| 42d3868 | Jay J Wylie | 10 May 2013, 19:34:20 UTC | Dropped Rebalance*BatchPlan.java, the classes I thought we needed to encapsulate an executable plan. | 20 June 2013, 23:08:23 UTC |
| d6cd930 | Jay J Wylie | 10 May 2013, 19:31:33 UTC | Interim commit before dropping ExecutableBatch objects RebalanceClusterPlan - removed TODOs about trying to "intelligently" order rebalance tasks RebalanceController - switched to just using REbalancePlan until tasks are actually constructed that are either stealer- or donor-based. RebalancePartitionsInfo - static method for pretty printing a list thereof *Test - clean up to deal with other type & interface changes... | 20 June 2013, 23:08:23 UTC |
| 91be6db | Jay J Wylie | 10 May 2013, 17:59:57 UTC | Interim commit: OrderedClusterTransition has been removed from the code base. | 20 June 2013, 23:08:23 UTC |
| d9e2623 | Jay J Wylie | 10 May 2013, 17:30:19 UTC | interim commit. Some work on taking logic of OrderedClusterTransition and moving it into ExecutableBatches. Logic is broken though, and, more importantly, unnecessary. Given proxy-gets server from local zone and there can only be nominal perf gain from ordering rebalancing tasks at node level, OrderedClusterTransition needs to be removed from teh code, not incorporated into new planning/execution code. | 20 June 2013, 23:08:23 UTC |
| c1c367e | Jay J Wylie | 10 May 2013, 16:09:16 UTC | Continued refactoring and clearing out deprecated code. OrderedClusterTransition - removed unnecessary members: id currentCluster targetCluster rebalanceclusterPlan storeDefs RebalanceController - renamed some member variables for sake of clarity - switched from orderedClusterTransition.getId to batchCount to uniquely ID batches of work... RebalanceNodePlan - added TODOs - removed unnecessary members: attempt storeToReplicaToDeletePartitionList Patched up all tests to not have 'compile time' errors. Have not run tests though... | 20 June 2013, 23:08:23 UTC |
| 70f8259 | Jay J Wylie | 09 May 2013, 22:54:06 UTC | Switched to new RebalanceController based on RebalancePlan and dropped tons of now deprecated code. Notes: - delete of partition-stores during rebalancing has been removed and is no longer an option. - the old RebalanceCLI now only offers the entropy tool features. cannot be dropped until SREs are OK with KeySampler/KeyVersionFetcher tool chain for verifying rebalances. - Dropped RebalanceClientConfig. Configuration parameters are now explicit in pertinent constructors. - Tests are failing or disabled: - donor-based rebalancing is disabled in test suites. Does not currently work with new rebalancing. - RebalanceClusterPlanTest tests mostly fail at this time. - RebalanceTest.serverSideRouting[1] may timeout - ZonedRebalanceTest.testProxyGetDuringRebalancing[1] may timeout OrderedClusterTransition - many small changes to work with all other changes - this class is to-be-deprecated regardless RebalanceClusterPlan - removed tons of deprecated code that permitted old controller to continue to work RebalanceController - Now uses RebalancePlan - No longer offers delete-after-rebalancing feature - Dropped tons of deprecated code... RebalanceTypedBatchPlan RebalanceDonorBasedBatchPlan RebalanceStealerBasedBatchPlan - no longer extends RebalanceClusterPlan - the classes that make a generic plan executable as either stealer- or donor-based - still need to be renamed so purpose of these key classes is more clear RebalancePlanCLI RebalanceControllerCLI - moderate clean up RebalanceUtils - dropped unnecessary helper mehods *Test - Signficant clean up: - use Rebalance*BatchPlan and RebalancePlan - don't check keys deleted since no longer supported | 20 June 2013, 23:08:23 UTC |
| 79ca1e0 | Jay J Wylie | 09 May 2013, 20:04:44 UTC | Unit tests work with new rebalance controller. RebalanceController - helper method to construct RebalancePlan based on cluster read from servers - divide by zero fix - many TODOs and Deprecate annotations... .*RebalanceTask - drop RebalanceclientConfig from construction and direclty pass in the config parameters needed. *Rebalance*Test - do not test donor-based rebalancing. (TODOs to add it back.) - do not test that keys are actually deleted when delete is true in rebalancing. - use the new controller/plan code path in all rebalancing unit tests. | 20 June 2013, 23:08:23 UTC |
| 464df36 | Jay J Wylie | 09 May 2013, 15:39:43 UTC | Initial commit of new rebalance controller. This is an interim commit. The new RebalanceControllerCLI has been added. Portions of the RebalanceController have been updated to use the RebalancePlan. All the tests that should pass, still pass for the old RebalanceController. About to switch tests to use new RebalanceController that works with a RebalancePlan. | 20 June 2013, 23:08:23 UTC |
| 0515ae4 | Jay J Wylie | 08 May 2013, 21:10:52 UTC | Minor changes inr esponse to review feedback and documenting a set of tests that are known to fail at this time. | 20 June 2013, 23:08:23 UTC |
| 8d5d076 | Jay J Wylie | 08 May 2013, 16:24:52 UTC | Refactoring: More class renaming and file reorganization ClusterInstance has been dropped. ClusterInstanceTest is now PartitionBalanceTest and is in tools. ClusterTestUtils is a new util class. All of the getFOOClusterBar and getFOOStoreDef(s)BAR methods have been moved into this test helper. | 20 June 2013, 23:08:23 UTC |
| 54e48c5 | Jay J Wylie | 08 May 2013, 16:04:34 UTC | Refactoring: Renamed some classes and reorganized their location RepartitionCLI -> RepartitionerCLI RepartitionUtils -> Repartitioner PartitionBalance & Repartitioner : voldemort/utils -> voldemort/tools Test classes & files updated to correspond with src/java. | 20 June 2013, 23:08:23 UTC |
