| 9f9f689 | Travis Abbott | 04 March 2013, 20:40:20 UTC | pass lsf resources on to flow | 04 March 2013, 20:40:47 UTC |
| 26cded3 | Thomas Mooney | 04 March 2013, 16:49:35 UTC | Wire up the WF_EXCLUDE_JOB_GROUP variable to bring back job groups for LSF. | 04 March 2013, 16:49:35 UTC |
| 945d332 | Mark Burnett | 27 February 2013, 21:59:00 UTC | add WF_USE_FLOW environment variable if that variable is set, try to load flow and use that instead | 27 February 2013, 22:00:30 UTC |
| e1248d4 | Scott Smith | 28 January 2013, 21:31:18 UTC | Added WF_JOB_EXTRA_PARAMS environment variable to append overrides to all bsub commands (currently only LSF is supported. | 28 January 2013, 21:31:18 UTC |
| 32a3a01 | Scott Smith | 13 December 2012, 23:51:07 UTC | added WF_EXCLUDE_JOB_GROUP variable for use with OpenLava | 13 December 2012, 23:51:07 UTC |
| 9252020 | Scott Smith | 13 December 2012, 23:44:21 UTC | Updated workflow to work with openlava as well as commercial LSF. | 13 December 2012, 23:44:21 UTC |
| ca05ba8 | Nathan Nutter | 03 December 2012, 19:37:38 UTC | updated UR submodule | 03 December 2012, 19:37:38 UTC |
| 568385f | Scott Smith | 30 November 2012, 19:52:32 UTC | re-enabled support for passing is_param to Command objects in workflows | 30 November 2012, 19:52:32 UTC |
| cd65039 | Scott Smith | 30 November 2012, 16:32:48 UTC | disabled support for is_param properties in workflow until some test cases are updated | 30 November 2012, 16:32:48 UTC |
| 9415270 | Scott Smith | 30 November 2012, 15:11:37 UTC | Fixed bug recognizing Command is_param properties. The recent addition to handle params & inputs from Commands more smartly had a bug, causing it to still not recognize params. This is now fixed. | 30 November 2012, 15:11:37 UTC |
| 51beeae | Scott Smith | 30 November 2012, 00:42:06 UTC | Handle command inputs and calls better. Previously, if you had a command with an OO input, you had to make the id_by property an input, but make the object accessor NOT an input for Workflow to dispatch the command. This is now fixed. Now any case where a command has properties which are id_by, or via, etc., will work. In cases where an output is a single value but the recipient can take a list of values, the conversion into a one-item-list happens automatically. When a command is created but has errors this was not previously checked before executing. Now commands with errors will fail before execution, like they do on the command-line. Exception throwing inside of Workflow::OperationType::Command::call did not clean up after itself, and now it does. Error logging on the command object is now set correctly, instead of just being set on the class twice. | 30 November 2012, 00:42:06 UTC |
| bdf89b4 | Scott Smith | 29 November 2012, 23:06:09 UTC | updated Workflow::Link::Instance to handle checking both inputs and outputs for the raw_value question | 29 November 2012, 23:06:09 UTC |
| 25a4c35 | Scott Smith | 29 November 2012, 22:51:09 UTC | allow any input to be used in a subsequent link as well as any output | 29 November 2012, 22:51:09 UTC |
| e6808bf | Scott Smith | 29 November 2012, 22:36:58 UTC | capture $@ immediately after eval when executing a step, since the code in-between the eval and the error logging will stomp on $@ | 29 November 2012, 22:36:58 UTC |
| f3a7235 | Scott Smith | 29 November 2012, 22:36:06 UTC | removed hard-coded /gsc path, eliminating check to ensure the log directory is on network disk | 29 November 2012, 22:36:06 UTC |
| 28e0c1d | Nathan Nutter | 31 October 2012, 03:01:28 UTC | fix typo | 31 October 2012, 03:01:28 UTC |
| 43fcb3c | Nathan Nutter | 31 October 2012, 01:45:32 UTC | fix uninitialized warnings | 31 October 2012, 01:45:32 UTC |
| 186c1fe | dmorton | 25 October 2012, 22:45:24 UTC | Start timing UR::Contex->commit in urd | 25 October 2012, 22:58:16 UTC |
| 18e9cab | dmorton | 25 October 2012, 22:44:54 UTC | Start timing loading of libraries when shortcutting | 25 October 2012, 22:58:16 UTC |
| a7f08cd | dmorton | 25 October 2012, 22:44:26 UTC | start timing shortcuts | 25 October 2012, 22:44:26 UTC |
| 6023229 | dmorton | 25 October 2012, 22:43:17 UTC | Start timing shortcuts | 25 October 2012, 22:43:17 UTC |
| 1bb7935 | dmorton | 25 October 2012, 21:44:49 UTC | Add ability to add instrumentation via statsd | 25 October 2012, 21:44:49 UTC |
| cd937bf | dmorton | 24 October 2012, 20:34:25 UTC | Reduce fork_limit to 1 to stop workflow servers from running out of memory. When jobs try to shortcut they're using about 80-100Mb per fork and that's making us top out over the 500Mb limit per job. LSF doesn't poll really fast so most of the time we're skirting by, but sometimes we get caught. | 24 October 2012, 20:34:25 UTC |
| cba0eaa | David Morton | 23 October 2012, 16:16:49 UTC | Disable lsftail to stop workflow servers from TERM_MEMLIMITing | 23 October 2012, 16:18:10 UTC |
| dc6b73c | Ben Oberkfell | 23 October 2012, 02:30:12 UTC | File::Temp was not loaded always, and was breaking workflow syncs to postgres in some cases | 23 October 2012, 02:30:15 UTC |
| 301757e | Ben Oberkfell | 16 October 2012, 20:07:29 UTC | update ur submodule | 16 October 2012, 20:07:29 UTC |
| c3b7c03 | dmorton | 12 October 2012, 19:10:04 UTC | Add extra commit after 'UR/workflow/finalize_instance' | 12 October 2012, 19:10:04 UTC |
| f8d15b1 | dmorton | 12 October 2012, 02:09:28 UTC | Fix typos Embarassing | 12 October 2012, 02:09:28 UTC |
| ef71731 | dmorton | 11 October 2012, 22:58:52 UTC | Make lsftail's Wheel work again It was getting garbage-collected. | 11 October 2012, 22:58:52 UTC |
| 6d2e351 | dmorton | 11 October 2012, 15:26:30 UTC | Stop logfile spam | 11 October 2012, 15:26:30 UTC |
| f6acdea | dmorton | 08 October 2012, 23:09:03 UTC | Ensure hubd and urd processes have meaningful names. | 08 October 2012, 23:09:03 UTC |
| e382ef3 | dmorton | 08 October 2012, 14:27:21 UTC | Remove old test. The bug this test was designed to catch has been fixed for some time and in fact doesn't even make sense now. I think that we DO want to shut down if we get SIGCHLD in the middle of starting up a workflow. | 08 October 2012, 20:41:37 UTC |
| a9e9070 | dmorton | 08 October 2012, 01:58:41 UTC | Remove locking from workflow server. Requires upgrade to POE::Component::IKC >= 0.2200-1 The version requirement on IKC is to allow the hub server to obtain it's own random port instead of the parent process obtaining it and then later the hub obtaining it from the os (which led to race conditions). The main purpose of the locking was to block other workflow servers from attempting to get the same ports by only letting one Server::Remote::launch instance run at a time. The locking code was itself vulnerable to race conditions though. Now, instead of locking out other workflows from launching, we just let each workflow get it's own ports and make them communicate back to us what port they are on. For this communication we use a named-pipe (fifo) that allows the hub server to communicate it's location (hostname:port) to the parent process that can then tell the ur process. The ur process no longer is a server and communicates only with the hub. The remaining locking (which appears to have been put in place to ensure that the hub and ur processes were fully up and operational before they started to communicate) was removed and replaced by letting the ur process announce it's readyness to the hub server. All messages that were intended to be sent to the ur process (formerly a server) are sent instead through a passthru on the hub server. The passthru keeps the messages in an inventory until the ur process has announced that it is ready to recieve messages. | 08 October 2012, 20:41:37 UTC |
| 7eedac5 | dmorton | 08 October 2012, 19:58:08 UTC | Upgrade version of POE::Component::IKC to 0.2200-1 This is done in a non-standard way, and is a temporary solution. In order to have code 'in the wild' that relies on the old version of IKC we're putting it into the workflow lib directly. Later, when all code running 'in the wild' is using this version of IKC we'll upgrade the system installed version and remove this installation. | 08 October 2012, 20:41:37 UTC |
| 19819ae | dmorton | 07 October 2012, 23:16:59 UTC | Remove unused orphan related code | 08 October 2012, 19:30:35 UTC |
| 86ed019 | dmorton | 07 October 2012, 23:13:11 UTC | Minor readability changes | 08 October 2012, 19:30:35 UTC |
| 1dad48a | dmorton | 05 October 2012, 14:45:29 UTC | Rename opdata to op_instance for clarity No functional changes | 08 October 2012, 19:30:35 UTC |
| 420f133 | dmorton | 08 October 2012, 14:41:24 UTC | Prepare Workflow for POE::Component::IKC upgrade With version 0.2200-1 we need to explicitly create our Responders. They used to be made in Client->spawn. | 08 October 2012, 18:25:13 UTC |
| d382a3e | dmorton | 08 October 2012, 16:19:21 UTC | Add quotes around hostname. Accidentally left out when when I refactored earlier. Seems to lead to errors that look like: 2012-10-08 10:40:13-0500 blade13-4-15: Bareword found where operator expected at -e line 1, near "15.gsc" 2012-10-08 10:40:13-0500 blade13-4-15: (Missing operator before gsc?) 2012-10-08 10:40:13-0500 blade13-4-15: syntax error at -e line 1, near "15.gsc" 2012-10-08 10:40:13-0500 blade13-4-15: Execution of -e aborted due to compilation errors. | 08 October 2012, 16:19:21 UTC |
| 422ee19 | dmorton | 04 October 2012, 15:35:32 UTC | Remove cruft from Workflow::Server::Hub Update worker to use cleaned-up interface too. | 04 October 2012, 15:35:32 UTC |
| 2df3c45 | dmorton | 04 October 2012, 14:05:42 UTC | Refactor Workflow::Server::Remote | 04 October 2012, 15:15:51 UTC |
| 590fca2 | dmorton | 04 October 2012, 13:15:33 UTC | Remove unused heap variable from the UR POE kernel | 04 October 2012, 13:15:33 UTC |
| 6380275 | Nathan Nutter | 03 October 2012, 13:56:36 UTC | typo | 03 October 2012, 13:56:36 UTC |
| c9c7838 | dmorton | 02 October 2012, 21:48:39 UTC | Clean up Server code No functional changes made, (except logging) just layout and naming | 02 October 2012, 21:48:39 UTC |
| b00730b | dmorton | 02 October 2012, 21:37:27 UTC | Make runq slightly faster is_ready is SLOW so we put it at the end of the boolean && chain. | 02 October 2012, 21:37:27 UTC |
| bede632 | dmorton | 02 October 2012, 21:33:14 UTC | Cleanup Command::Server Add clarifying comments and remove commented out stuff. | 02 October 2012, 21:33:14 UTC |
| aef6dc2 | dmorton | 02 October 2012, 21:27:54 UTC | Add POE_flow_diagrams | 02 October 2012, 21:27:54 UTC |
| 4895a84 | Ben Oberkfell | 26 September 2012, 18:46:11 UTC | update ur submodule | 26 September 2012, 18:46:11 UTC |
| 4c33d20 | Ben Oberkfell | 25 September 2012, 21:39:07 UTC | update submodule for UR to remove debug msg | 25 September 2012, 21:39:07 UTC |
| f8cc708 | Ben Oberkfell | 25 September 2012, 21:27:46 UTC | update UR submodule | 25 September 2012, 21:27:46 UTC |
| 547bb03 | Ben Oberkfell | 25 September 2012, 20:29:07 UTC | rewrite DS name to avoid going to oracles data dictionary | 25 September 2012, 20:29:07 UTC |
| a7cd661 | Ben Oberkfell | 25 September 2012, 20:28:48 UTC | update UR | 25 September 2012, 20:28:48 UTC |
| e8da079 | Ben Oberkfell | 25 September 2012, 19:12:20 UTC | update submodule | 25 September 2012, 19:12:20 UTC |
| 9a167f2 | Ben Oberkfell | 25 September 2012, 18:11:43 UTC | stat correct lsf acct file | 25 September 2012, 18:11:43 UTC |
| 7a02562 | Ben Oberkfell | 25 September 2012, 16:57:40 UTC | suppress warning | 25 September 2012, 16:57:40 UTC |
| 0a9c457 | Ben Oberkfell | 25 September 2012, 16:55:43 UTC | fix get user id | 25 September 2012, 16:55:43 UTC |
| 08faa3e | Ben Oberkfell | 25 September 2012, 16:52:55 UTC | this should not lean on Genome | 25 September 2012, 16:52:55 UTC |
| 3a47061 | Ben Oberkfell | 25 September 2012, 16:42:27 UTC | update submodule | 25 September 2012, 16:42:27 UTC |
| 311ad68 | Ben Oberkfell | 21 September 2012, 20:44:37 UTC | changes to permit double-syncing to postgres | 21 September 2012, 20:44:37 UTC |
| 00b02e1 | Thomas Mooney | 29 August 2012, 20:25:00 UTC | Add a subroutine to get the parent execution's log directory. | 29 August 2012, 20:25:00 UTC |
| fbbb2f3 | Thomas Mooney | 28 August 2012, 21:49:03 UTC | Don't let the "tmp" case match "gtmp", which was already handled. All gtmp requests were being squashed to 1 (since nobody asked for >1TB of temp space). This version would still overwrite the gtmp request with the tmp request if both were supplied. | 28 August 2012, 21:49:03 UTC |
| 4fbd2eb | Thomas Mooney | 23 August 2012, 18:43:04 UTC | Further optimization: promote ops hash to dedicated parameter. Use by ref. | 23 August 2012, 18:43:04 UTC |
| 799eb24 | Thomas Mooney | 22 August 2012, 19:43:37 UTC | Re-use the template for creating links. | 22 August 2012, 19:43:37 UTC |
| 9ac40d5 | Thomas Mooney | 22 August 2012, 17:24:18 UTC | Reuse the has of operations by name. This change reduced the runtime of a workflow with about 600 operations from 125s to 9s by not recomputing the hash each time. | 22 August 2012, 17:24:18 UTC |
| 64433f6 | Thomas Mooney | 06 August 2012, 20:10:25 UTC | Create a non-functioning OperationType if the class can't be instantiated. This allows a workflow to be viewed even if its implementing commands aren't found. The workflow produced will probably be invalid and thus cannot actually be run. (The step would certainly fail if it were attempted.) | 06 August 2012, 20:10:25 UTC |
| 6074ea0 | Thomas Mooney | 01 August 2012, 13:43:22 UTC | Set shortcut parallelization to 2. | 01 August 2012, 13:43:22 UTC |
| 5b9979f | Thomas Mooney | 31 July 2012, 15:04:09 UTC | Reduce the parallelization to four ways. | 31 July 2012, 15:04:09 UTC |
| 1ec4e79 | Thomas Mooney | 30 July 2012, 22:57:45 UTC | Allow server to fork up to 15 ways to facilitate parallel shortcutting. | 30 July 2012, 22:57:45 UTC |
| 13e9b45 | Anthony Brummett | 26 July 2012, 21:13:40 UTC | Fix for sigChild handling with newer POE | 26 July 2012, 21:13:40 UTC |
| 3ecf8f6 | Anthony Brummett | 26 July 2012, 21:12:58 UTC | Fix when running with newer POE.pm We can't rely on the return value from POE::Kernel->run() | 26 July 2012, 21:12:58 UTC |
| e055a42 | Gabriel Sanderson | 16 July 2012, 21:22:45 UTC | Remove breakpoints | 16 July 2012, 21:22:45 UTC |
| 5b092c9 | Scott Smith | 25 June 2012, 06:51:07 UTC | fix typo in DB::enable_profile() | 25 June 2012, 06:51:07 UTC |
| b3a4fb3 | Scott Smith | 25 June 2012, 06:30:41 UTC | Removed Workflow::Operation::Instance set_input_links() and merged the logic into the one place it is called in Workflow::Model::Instance create(). Elminated n**2 logic. | 25 June 2012, 06:30:41 UTC |
| 1ab120f | Nathan Nutter | 13 June 2012, 21:31:38 UTC | increase the POE IKC call/post_respond timeout Increase the POE::Component::IKC::ClientLite timeout (for call and post_respond) to four weeks. | 13 June 2012, 21:31:38 UTC |
| eb6298a | Nathan Nutter | 26 April 2012, 22:08:05 UTC | Revert "Remove breakpoints." This reverts commit 3ab461a5791fc3e29154e222f18d9d38d5a73638. | 08 June 2012, 15:40:52 UTC |
| 8fe4da0 | Joshua McMichael | 06 June 2012, 21:18:21 UTC | added start_time_nice field to InstanceExecution for displaying start_time w/o the trailing sub-second zeros | 06 June 2012, 21:18:21 UTC |
| 8dfdc7d | Thomas Mooney | 16 May 2012, 22:51:52 UTC | Make input/output connectors indirect properties instead of transient. | 16 May 2012, 22:51:52 UTC |
| c2afd26 | Thomas Mooney | 16 May 2012, 21:54:19 UTC | Make input and output connectors properties on the model. | 16 May 2012, 21:54:19 UTC |
| 1ebedab | Scott Smith | 16 May 2012, 21:46:02 UTC | refactored root_id on Workflow::Operation::Instance to have root_id be calculated, instead of a transient property set during create() and load() | 16 May 2012, 21:46:02 UTC |
| 0463457 | Thomas Mooney | 16 May 2012, 18:06:14 UTC | Store input and output in canonical order to preven unneccessary db updates. Additionally don't reserialize input and output upon thawing. | 16 May 2012, 18:06:14 UTC |
| 9885763 | apregier | 11 May 2012, 15:59:34 UTC | CHANGELOG Workflow UR::Objects can be passed between operations When a UR::Object is passed to a Workflow operation, get is called on the id so that the operation gets the actual object rather than a copy that has been serialized and de-serialized. | 11 May 2012, 15:59:34 UTC |
| e8bba9f | apregier | 11 May 2012, 15:55:12 UTC | CHANGELOG Enable debug of command in lsf workflow If the environment variable WF_DEBUG_MATCH is set, the graphical debugger ptkdb will be displayed when that command is launched. The interface will be displayed as specified by the DISPLAY environment variable, so be sure that is set correctly. WF_DEBUG_MATCH should be set to a regex that will match the command being bsubbed by the workflow operation, e.g. the name of the command module. | 11 May 2012, 15:55:12 UTC |
| 0a37d07 | Adam Coffman | 07 May 2012, 19:54:36 UTC | update job group from workflow-worker2 to workflow-worker | 07 May 2012, 19:54:36 UTC |
| 3ab461a | Thomas Mooney | 25 April 2012, 17:28:42 UTC | Remove breakpoints. | 25 April 2012, 17:28:42 UTC |
| 96ac692 | Nathan Nutter | 20 March 2012, 23:33:56 UTC | update ur submodule | 20 March 2012, 23:33:56 UTC |
| 33b38e0 | Anthony Brummett | 19 March 2012, 14:32:27 UTC | Partially back out an earlier change with Workflow::Model::create() Had to back out a change to UR::BoolExpr::Template::get_rule_for_values() because hard refs weren't being handled properly in another part of the code. That change means we have to create the object and assign the listrefs afterward instead of doing it in create() | 19 March 2012, 14:32:27 UTC |
| e17b4aa | Anthony Brummett | 16 March 2012, 21:18:20 UTC | Improve validate() speed for the dangling links check Build a hash of properties for optypes as we go instead of retrieving them through the indirect accessor method every time | 16 March 2012, 21:18:20 UTC |
| 13f4ae7 | Anthony Brummett | 16 March 2012, 21:15:37 UTC | Fix an incompatability introduced by a prior commit with operations_in_series() The new operations_in_series might complain about more than one link, so the check in validate() needs to use looser whitespace matchin | 16 March 2012, 21:15:37 UTC |
| e1820e9 | Anthony Brummett | 16 March 2012, 21:15:05 UTC | Speed up validate() orphans check by using the already retrieved list of operations | 16 March 2012, 21:15:05 UTC |
| f102972 | Anthony Brummett | 16 March 2012, 21:12:52 UTC | Improve speed of validate() in the unique names check use a template+rule to get matching Operations instead of a params list | 16 March 2012, 21:12:52 UTC |
| ca4d1d7 | Anthony Brummett | 16 March 2012, 21:11:41 UTC | Speed things up by using template+rule for get() instead of a params list | 16 March 2012, 21:11:41 UTC |
| 1113b6c | Anthony Brummett | 16 March 2012, 21:01:19 UTC | Allow the log_path param to be a relative path as long as its absolute equivalent still starts with /gsc | 16 March 2012, 21:01:19 UTC |
| 5ceb00b | Anthony Brummett | 16 March 2012, 20:56:54 UTC | Move the order of execution around to make it more readable | 16 March 2012, 20:56:54 UTC |
| eef64e2 | Anthony Brummett | 16 March 2012, 20:56:18 UTC | Refactor overridden create() to break out the non-create part That way others can call the non-create code | 16 March 2012, 20:56:18 UTC |
| d9f7f73 | Anthony Brummett | 16 March 2012, 15:58:59 UTC | Switch to new algorithm for operations_in_series() The old one looked like it was O(n^3) on the number of nodes in the workflow. The new one is linear on the number of nodes+edges. Basically taken from wikipedia's page on Topological Sorting | 16 March 2012, 15:58:59 UTC |
| 32e1149 | Anthony Brummett | 14 March 2012, 20:48:46 UTC | Improve the speed of Workflow::Model::create() by using templates and create_entity() instead of add_operation It does the same work, but is optimized to do jsut the work necessary instead of going through the generid add_* method in the AccessorWriter | 14 March 2012, 20:48:46 UTC |
| f8ab9f6 | Scott Smith | 14 February 2012, 19:32:50 UTC | Merge branch 'master' of ssh://git/srv/git/workflow | 14 February 2012, 19:32:50 UTC |
| e740152 | Scott Smith | 14 February 2012, 19:32:38 UTC | updated error output | 14 February 2012, 19:32:38 UTC |
| 3687764 | Jason Walker | 07 February 2012, 16:48:32 UTC | Improve error message when two operations have the same name. Dump both of them | 07 February 2012, 16:48:32 UTC |
| e6c8bb8 | Nathan Nutter | 06 February 2012, 23:00:12 UTC | show lock info if lock time exceeded | 06 February 2012, 23:00:12 UTC |
