| 3c31477 | Thomas B. Mooney | 22 April 2019, 15:02:26 UTC | Merge pull request #1931 from tmooney/trusted_importer New command to import a directory of instrument data. | 22 April 2019, 15:02:26 UTC |
| 626fe46 | Thomas Mooney | 19 April 2019, 21:41:28 UTC | Make read_count into a required property. | 19 April 2019, 21:41:28 UTC |
| b4e740f | Thomas B. Mooney | 19 April 2019, 20:05:13 UTC | Only pass the description along if we have one. No setting of descriptions to `0` or the empty string ;) | 19 April 2019, 20:05:13 UTC |
| 62ca245 | Thomas B. Mooney | 16 April 2019, 21:53:02 UTC | Skip the `rsync` test on protocol version 30. Probably older versions don't work either, but we don't currently have to deal with those :) | 16 April 2019, 21:53:02 UTC |
| 74af554 | Thomas B. Mooney | 16 April 2019, 21:02:26 UTC | New command to import a directory of instrument data. No validation is performed whatsoever... it trusts you and the data! If it happens to conform to an expected standard, it'll work with existing tools. If it doesn't, it won't! But in the age of CWL pipelines, someone could make one that handles it. | 16 April 2019, 21:02:26 UTC |
| 283a878 | Thomas B. Mooney | 02 April 2019, 18:02:54 UTC | Merge pull request #1930 from tmooney/dont_load_disk_group_in_lims_importer Don't bother loading up the disk group. | 02 April 2019, 18:02:54 UTC |
| 465b342 | Thomas B. Mooney | 02 April 2019, 15:28:23 UTC | Don't bother loading up the disk group. The allocation system will handle that! Also... sometimes the config specifies a specific volume instead of a group, in which case this extra code would crash. | 02 April 2019, 15:28:23 UTC |
| 7c89f25 | Thomas B. Mooney | 19 March 2019, 14:34:46 UTC | Merge pull request #1924 from tmooney/cqid_imports_data Site Feature: Import instrument data from LIMS in CQID. | 19 March 2019, 14:34:46 UTC |
| 02c6795 | Thomas B. Mooney | 07 March 2019, 16:05:59 UTC | Fixup CQID test so that instrument data already looks "imported". | 11 March 2019, 22:41:20 UTC |
| bcc19c1 | Thomas B. Mooney | 11 March 2019, 17:59:07 UTC | Merge pull request #1926 from tmooney/note_only_the_lsf_job_id Ensure only the LSF Job ID gets into the process note. | 11 March 2019, 17:59:07 UTC |
| 994a93d | Thomas B. Mooney | 11 March 2019, 17:58:42 UTC | Merge pull request #1925 from tmooney/dont_mail_build_reports Stop e-mailing build reports. | 11 March 2019, 17:58:42 UTC |
| 9875147 | Thomas B. Mooney | 11 March 2019, 17:58:10 UTC | Merge pull request #1923 from tmooney/cromwell_in_workflow_view Display Cromwell workflows in `genome model build view`. | 11 March 2019, 17:58:10 UTC |
| 6dce239 | Thomas B. Mooney | 11 March 2019, 17:56:22 UTC | Merge pull request #1922 from tmooney/simple_backend_fixes A few fixes for the "simple" backend. | 11 March 2019, 17:56:22 UTC |
| b32d835 | Thomas B. Mooney | 11 March 2019, 17:56:01 UTC | Merge pull request #1916 from johnegarza/import_basic_picard_fix Using updated picard version to fix a bug in the old version | 11 March 2019, 17:56:01 UTC |
| f5f810f | Thomas B. Mooney | 11 March 2019, 14:24:40 UTC | Ensure only the LSF Job ID gets into the process note. There's logic to prepend any notes with a sudo message when running as another user, but that's not helpful here. (Really, storing this as a note is not great--it should just be a column on the table.) | 11 March 2019, 14:24:40 UTC |
| 902e058 | Thomas B. Mooney | 07 March 2019, 21:04:39 UTC | Stop e-mailing build reports. We'd already turned these off for production users as they're noisy. | 07 March 2019, 21:04:39 UTC |
| cc589ec | Thomas B. Mooney | 05 March 2019, 23:01:59 UTC | Site Feature: Import instrument data from LIMS in CQID. This will cause CQID failures when the data could not be imported. But that's probably fine, since the builds would themselves fail later! There is an additional complication in that this would need to be run as a user with access to import the data. | 05 March 2019, 23:01:59 UTC |
| dd3f895 | Alex Paul | 04 March 2019, 15:25:01 UTC | Typo fix: s/thsi/this. Co-Authored-By: tmooney <mooney@wustl.edu> | 04 March 2019, 15:25:01 UTC |
| 298f3b5 | Thomas B. Mooney | 28 February 2019, 20:07:17 UTC | Implement the summary threshold. | 28 February 2019, 20:07:17 UTC |
| 3bf04c1 | Thomas B. Mooney | 28 February 2019, 19:59:12 UTC | Display Cromwell workflows in `genome model build view`. For CWL, this will be a lot more informative than the Ptero workflow (if one even exists)! | 28 February 2019, 19:59:12 UTC |
| 1d2fce3 | Thomas B. Mooney | 28 February 2019, 16:56:33 UTC | A few fixes for the "simple" backend. * The bsub module expects "log_dir" not "out_dir". * The DAG should process a simple workflow like an inline one. | 28 February 2019, 16:56:33 UTC |
| 385fe08 | Thomas B. Mooney | 27 February 2019, 17:46:32 UTC | Merge pull request #1921 from tmooney/cromwell_set_default_memory_request Set default memory request of 4096MB. | 27 February 2019, 17:46:32 UTC |
| daff014 | Thomas B. Mooney | 27 February 2019, 17:21:50 UTC | Merge pull request #1920 from tmooney/cromwell_config_span Add "span[hosts=1]" to our LSF requests. | 27 February 2019, 17:21:50 UTC |
| 5e5cc79 | Thomas B. Mooney | 27 February 2019, 16:28:30 UTC | Merge pull request #1919 from tmooney/everything_old_is_new_again-simple_backend New WorkflowBuilder Backend: `simple` | 27 February 2019, 16:28:30 UTC |
| 7a4e2f3 | Thomas B. Mooney | 26 February 2019, 19:22:33 UTC | Set default memory request of 4096MB. | 26 February 2019, 19:22:33 UTC |
| 09a575f | Thomas B. Mooney | 26 February 2019, 15:33:47 UTC | Add "span[hosts=1]" to our LSF requests. When requesting multiple cores, we'd really like them to be on the same host! | 26 February 2019, 15:33:47 UTC |
| ae590b1 | Thomas B. Mooney | 20 February 2019, 23:37:27 UTC | More and better error messages. | 20 February 2019, 23:37:27 UTC |
| 32f70e8 | Thomas B. Mooney | 20 February 2019, 23:28:01 UTC | New WorkflowBuilder backend: "simple". This `bsub`s a single job which then runs the workflow inline. For one-step workflows (like CwlPipeline or ReferenceSequence) this is approximately the same as launching with Ptero, but without the overhead! For more complicated workflows (like SomaticValidation) this would almost certainly be a poor choice. This is pretty much equivalent to this combo of options from "Workflow": --server-dispatch lsf --job-dispatch inline | 20 February 2019, 23:32:45 UTC |
| 3986888 | Thomas B. Mooney | 20 February 2019, 23:25:35 UTC | Make `lsf_job_id` more like a typical UR property getter/setter. | 20 February 2019, 23:25:35 UTC |
| 27ae736 | Thomas B. Mooney | 20 February 2019, 23:22:54 UTC | Support some resource requests in Genome::Sys::LSF::bsub. | 20 February 2019, 23:22:54 UTC |
| 39ec01a | Thomas B. Mooney | 20 February 2019, 19:40:15 UTC | Merge pull request #1914 from tmooney/cwl_config_subject_mappings_with_multiple_models Add support for multiple matching subject mappings for a sample. | 20 February 2019, 19:40:15 UTC |
| 56d11f5 | Thomas B. Mooney | 20 February 2019, 19:40:00 UTC | Merge pull request #1910 from tmooney/clean_cromwell_environment Get a clean environment for cromwell jobs. | 20 February 2019, 19:40:00 UTC |
| 8808a96 | Thomas B. Mooney | 20 February 2019, 19:39:46 UTC | Merge pull request #1911 from tmooney/add_shortname_option_to_intervallist_dumper Add option to preserve region names to the IntervalList dumper. | 20 February 2019, 19:39:46 UTC |
| ff27d37 | Thomas B. Mooney | 24 January 2019, 17:02:19 UTC | Remove restriction on combinations of merge and preserve names. The test failed due to a limitation of FeatureLists where more than one merged version cannot be generated within the same process. (By "limitation" I probably mean "bug", since it can yield the wrong results instead of simply failing.) | 20 February 2019, 16:30:15 UTC |
| a86075a | Thomas B. Mooney | 14 February 2019, 22:18:27 UTC | Merge pull request #1918 from tmooney/cromwell_config_updates_37 Updates to the Cromwell Config to hopefully support version 37. | 14 February 2019, 22:18:27 UTC |
| a8bd5c6 | Thomas B. Mooney | 14 February 2019, 21:33:35 UTC | Only get straight values from Cromwell. New versions are picky about types and don't like to concatenate numbers to strings. | 14 February 2019, 21:33:35 UTC |
| fe78267 | Thomas B. Mooney | 14 February 2019, 19:10:58 UTC | Merge pull request #1913 from tmooney/prefix_for_output_gatherer Allow for an optional prefix when staging cromwell outputs. | 14 February 2019, 19:10:58 UTC |
| 863cc5e | Thomas B. Mooney | 13 February 2019, 18:25:40 UTC | Merge pull request #1917 from tmooney/when_is_experimental We know that `when` is experimental, and we're okay with that. | 13 February 2019, 18:25:40 UTC |
| ff5d68a | Thomas B. Mooney | 11 February 2019, 23:26:03 UTC | We know that `when` is experimental, and we're okay with that. | 11 February 2019, 23:26:03 UTC |
| 5a03213 | johnegarza | 08 February 2019, 19:05:17 UTC | Using updated picard version to fix a bug in the old version | 08 February 2019, 19:05:17 UTC |
| f974b88 | Thomas B. Mooney | 04 February 2019, 17:21:58 UTC | Merge pull request #1915 from tmooney/cwl_process_outputs Allow for an optional process_outputs.pl script. | 04 February 2019, 17:21:58 UTC |
| 13e5edf | Thomas B. Mooney | 31 January 2019, 22:33:01 UTC | Allow for an optional process_outputs.pl script. This will be our mechanism for submitting files to third-party locations and the like, since those types of operations must be run on completed workflows. | 31 January 2019, 22:33:01 UTC |
| e98ce78 | Thomas B. Mooney | 31 January 2019, 15:12:15 UTC | Add support for multiple matching subject mappings for a sample. | 31 January 2019, 15:12:15 UTC |
| c98883e | Thomas B. Mooney | 31 January 2019, 15:12:04 UTC | Remove breakpoint. | 31 January 2019, 15:12:04 UTC |
| 168cc3b | Thomas B. Mooney | 24 January 2019, 16:32:21 UTC | Merge pull request #1912 from tmooney/use_array_form_of_shellcmd_in_import_fastqtosam Use array form of `shellcmd` in the Importer's FastqsToBam step. | 24 January 2019, 16:32:21 UTC |
| 7ac9a17 | Thomas B. Mooney | 24 January 2019, 15:29:14 UTC | Allow for an optional prefix when staging cromwell outputs. This is one way to address genome/analysis-workflows#546. | 24 January 2019, 15:30:23 UTC |
| 2af4efc | Thomas B. Mooney | 22 January 2019, 23:10:53 UTC | Don't bother checking if the cmd exists. This looks like a holdover from when we were creating a command object. If Perl fails to create an array, we have bigger issues! | 22 January 2019, 23:10:53 UTC |
| ac163b6 | Thomas B. Mooney | 22 January 2019, 23:10:35 UTC | Use the array form of shellcmd. | 22 January 2019, 23:10:35 UTC |
| d6b258e | Thomas B. Mooney | 16 January 2019, 23:15:26 UTC | Add option to presere region names in dumped interval lists. This is framed the opposite of dump-merged-list's "--short-name" option so that the default can be false (which matches existing results). | 16 January 2019, 23:27:18 UTC |
| aec540a | Thomas B. Mooney | 16 January 2019, 23:12:56 UTC | Documentation cleanup. * of course they're command line options! * correct copy-pasta from merge option. | 16 January 2019, 23:12:56 UTC |
| 4ea33b4 | Thomas B. Mooney | 16 January 2019, 16:54:36 UTC | Get a clean environment for cromwell jobs. | 16 January 2019, 23:02:44 UTC |
| 4315629 | Thomas B. Mooney | 15 January 2019, 20:20:32 UTC | Merge pull request #1909 from tmooney/fix_input_show_colorization Pass color to _format_is_many instead of as an extra sprintf argument. | 15 January 2019, 20:20:32 UTC |
| 665770e | Thomas B. Mooney | 15 January 2019, 19:48:31 UTC | Pass color to _format_is_many instead of as an extra sprintf argument. | 15 January 2019, 19:48:31 UTC |
| 9f5098a | Thomas B. Mooney | 14 January 2019, 18:27:55 UTC | Merge pull request #1907 from tmooney/yet_even_more_stager_updates More Tolerant Staging and Handling of Strange Outputs | 14 January 2019, 18:27:55 UTC |
| e6dfd5a | Thomas B. Mooney | 14 January 2019, 17:14:50 UTC | Merge pull request #1908 from tmooney/dumb_default_value_for_volume_hostname Set a default for disk volume hostname. | 14 January 2019, 17:14:50 UTC |
| 1959431 | Thomas B. Mooney | 14 January 2019, 16:25:53 UTC | Set a default for disk volume hostname. New disk volumes are coming through with this value undefined. A better solution would probably be to make a sqitch migration to drop this column and then not worry about it. This is an expedient solution instead. | 14 January 2019, 16:25:53 UTC |
| 8b026ba | Thomas B. Mooney | 10 January 2019, 22:19:09 UTC | Merge pull request #1906 from tmooney/lookup_sambamba_path Eliminate hard-coded sambamba path by looking up allocation. | 10 January 2019, 22:19:09 UTC |
| a9bbefa | Thomas B. Mooney | 10 January 2019, 21:05:02 UTC | Can't change the permissions of a dangling symlink. | 10 January 2019, 21:05:02 UTC |
| c284851 | Thomas B. Mooney | 10 January 2019, 21:04:28 UTC | Ignore simple values. They're definitely not files! | 10 January 2019, 21:04:28 UTC |
| 63640da | Thomas B. Mooney | 10 January 2019, 21:04:04 UTC | Tolerate duplicate file names as long as the files are identical. | 10 January 2019, 21:04:04 UTC |
| 7b635a6 | Thomas B. Mooney | 08 January 2019, 21:23:27 UTC | Eliminate hard-coded sambamba path by looking up allocation. | 08 January 2019, 21:23:27 UTC |
| 73dfdc2 | Thomas B. Mooney | 07 January 2019, 22:11:16 UTC | Merge pull request #1905 from tmooney/more_staging_fixes Better handling of arbitrary array structures when extracting files. | 07 January 2019, 22:11:16 UTC |
| 90b753d | Thomas B. Mooney | 07 January 2019, 22:11:05 UTC | Merge pull request #1904 from tmooney/control_cwlpipeline_runner_memory_usage Make LSF Resource for CWL Runner job configurable. | 07 January 2019, 22:11:05 UTC |
| 15553e4 | Thomas B. Mooney | 07 January 2019, 21:52:12 UTC | Add a very minimal test for walking the output data structure. | 07 January 2019, 21:52:12 UTC |
| c040344 | Thomas B. Mooney | 07 January 2019, 19:10:17 UTC | Better handling of arbitrary array structures when extracting files. We've encountered some workflows that like to have arrays of arrays of files as outputs. Rather than coerce everything into a single level of arrays, let's just recurse until we find the individual entries we're after. | 07 January 2019, 19:10:17 UTC |
| 4fdee1c | Thomas B. Mooney | 06 January 2019, 05:17:53 UTC | Make LSF Resource for CWL Runner job configurable. We've encountered some workflows where Cromwell needs more than the default amount to survive pre-processing, but it's probably better to only request more memory if we really need it. | 06 January 2019, 05:17:53 UTC |
| 7f5f173 | Thomas B. Mooney | 03 January 2019, 20:54:59 UTC | Merge pull request #1863 from tmooney/interval_list_dumping_has_strip_chr_option Add new option whether or not to remove "chr"s during conversion. | 03 January 2019, 20:54:59 UTC |
| e6f709c | Thomas B. Mooney | 03 January 2019, 20:52:37 UTC | Merge pull request #1903 from tmooney/fix_batch_mode This will work better if we specify the necessary key. | 03 January 2019, 20:52:37 UTC |
| 393180a | Thomas B. Mooney | 03 January 2019, 20:52:22 UTC | Merge pull request #1900 from tmooney/import_fastqs_too Support importing LIMS data that may or may not be BAMs. | 03 January 2019, 20:52:22 UTC |
| db26766 | Thomas B. Mooney | 03 January 2019, 19:10:06 UTC | This will work better if we specify the necessary key. Otherwise the value is ignored and `batch` mode is identical to `queued` mode. | 03 January 2019, 19:10:06 UTC |
| 2dff766 | Thomas B. Mooney | 03 January 2019, 16:52:58 UTC | Merge pull request #1901 from tmooney/stage_dangling_symlinks_too Use lower-level `move` for staging results. | 03 January 2019, 16:52:58 UTC |
| 583a718 | Thomas B. Mooney | 02 January 2019, 15:26:00 UTC | Use lower-level move for staging results. When moving symlinks, their targets may or may not exist depending if what they point to already got moved. We need to faithfully relocate them in either circumstance, so verifying `-e` is bad. | 02 January 2019, 15:26:00 UTC |
| 47a7806 | Thomas B. Mooney | 26 December 2018, 14:25:00 UTC | Merge pull request #1899 from tmooney/lsf_bsub_test_doesnt_hardcode_queues Don't use hardcoded queue names in LSF bsub test. | 26 December 2018, 14:25:00 UTC |
| 86eede3 | Thomas B. Mooney | 20 December 2018, 20:55:17 UTC | Support importing LIMS data that may or may not be BAMs. | 20 December 2018, 20:55:17 UTC |
| 30a5a3e | Thomas B. Mooney | 19 December 2018, 21:49:33 UTC | Don't use hardcoded queue names in LSF bsub test. Use the config! Also... don't inject a method into our module that only gets used in this test... just use an array directly. | 19 December 2018, 21:49:33 UTC |
| eaf2cd8 | Thomas B. Mooney | 19 December 2018, 18:42:03 UTC | Merge pull request #1898 from tmooney/more_tolerant_subject_mappings Allow individual configs for a model type to mismatch. | 19 December 2018, 18:42:03 UTC |
| 9991256 | Thomas B. Mooney | 19 December 2018, 15:44:29 UTC | Merge pull request #1892 from tmooney/add_batch_mode_to_search_index Add batch mode to search indexer. | 19 December 2018, 15:44:29 UTC |
| 44bc711 | Thomas B. Mooney | 12 December 2018, 22:08:03 UTC | Really skip when the tags don't match instead of processing as unmapped! Otherwise this generates bogus models in addition to the correct ones. | 12 December 2018, 22:08:03 UTC |
| 6a2d564 | Thomas B. Mooney | 12 December 2018, 21:13:04 UTC | Allow individual configs for a model type to mismatch. When using tags to distinguish between otherwise matching configurations to limit to certain subject mappings, this will occasionally return no result for a given config. For now, let's optimsitically assume that we should always get at least one match for a model type, even though it's possible even this assumption is not valid either. | 12 December 2018, 21:13:04 UTC |
| 01ac9bb | Thomas B. Mooney | 11 December 2018, 17:15:27 UTC | Fix BED file chromosome name for test. The seqdict says `Y` but the BED file said `chrY`. Now both say `Y`. | 11 December 2018, 17:15:27 UTC |
| 4ed3944 | Thomas B. Mooney | 11 December 2018, 16:44:39 UTC | Merge pull request #1896 from tmooney/use_local_disk_tables Use Local `disk.` Tables | 11 December 2018, 16:44:39 UTC |
| cba0458 | Thomas B. Mooney | 11 December 2018, 16:43:58 UTC | Merge pull request #1897 from tmooney/set_cromwell_db_threads Limit threads instead of connections. | 11 December 2018, 16:43:58 UTC |
| 4685f1b | Thomas B. Mooney | 22 August 2018, 13:31:11 UTC | Add new option whether or not to remove "chr"s during conversion. The original behavior was a holdover from when we assumed human build36 or 37. | 11 December 2018, 16:00:05 UTC |
| ab70dee | Thomas B. Mooney | 04 December 2018, 21:36:52 UTC | New synchronize classes for getting disk information from LIMS. | 10 December 2018, 23:06:14 UTC |
| 3a7818d | Thomas B. Mooney | 04 December 2018, 19:41:34 UTC | Use our own bridge table for groups and volumes. | 10 December 2018, 23:06:14 UTC |
| 9137eda | Thomas B. Mooney | 04 December 2018, 19:37:49 UTC | Use our own disk.group table instead of reading out. | 10 December 2018, 23:06:13 UTC |
| d282293 | Thomas B. Mooney | 03 December 2018, 23:31:09 UTC | Use our own disk.volume table instead of reading out. | 10 December 2018, 23:06:13 UTC |
| b87b445 | Thomas B. Mooney | 10 December 2018, 22:54:55 UTC | Limit threads instead of connections. Slick will automatically limit connections based on the thread count but not vice versa. Therefore to get sane numbers let's set the thread count instead. | 10 December 2018, 22:54:55 UTC |
| f80cbae | Thomas B. Mooney | 04 December 2018, 22:14:37 UTC | Merge pull request #1895 from tmooney/better_cromwell_output_staging Handle array outputs and non-file outputs when staging results. | 04 December 2018, 22:14:37 UTC |
| a3fc168 | Thomas B. Mooney | 27 November 2018, 20:08:16 UTC | Handle array outputs and non-file outputs when staging results. | 27 November 2018, 20:10:09 UTC |
| 84a787f | Thomas B. Mooney | 27 November 2018, 19:29:07 UTC | Merge pull request #1894 from tmooney/deal_with_lims_permissions Handle `000`-mode files for BAM imports. | 27 November 2018, 19:29:07 UTC |
| 9ce9cf1 | Thomas B. Mooney | 26 November 2018, 17:56:25 UTC | Merge pull request #1893 from tmooney/set_max_connections_in_cromwell Restrict the number of DB connections for a single Cromwell run. | 26 November 2018, 17:56:25 UTC |
| 91e775b | Thomas B. Mooney | 20 November 2018, 22:13:32 UTC | Set explicit permissions and group on imported BAMs. | 20 November 2018, 22:13:32 UTC |
| a40fb27 | Thomas B. Mooney | 20 November 2018, 22:06:26 UTC | rsync_directory support for chown. | 20 November 2018, 22:06:26 UTC |
| 59503e9 | Thomas B. Mooney | 20 November 2018, 21:48:10 UTC | New option for rsync_directory to overwrite permissions. | 20 November 2018, 22:05:47 UTC |
| c7c5e89 | Thomas B. Mooney | 20 November 2018, 20:49:04 UTC | Restrict the number of DB connections for a single Cromwell run. This was the advice provided in: https://gatkforums.broadinstitute.org/gatk/discussion/12330/call-catching-cromwell-exhausts-all-connection-to-mysqld | 20 November 2018, 20:49:04 UTC |
| b6f577e | Thomas B. Mooney | 19 November 2018, 23:19:07 UTC | Merge pull request #1891 from tmooney/import_bam_from_lims_updates Make the `import-bam-from-lims` command apply to more situations | 19 November 2018, 23:19:07 UTC |
| 8457ee9 | Thomas B. Mooney | 13 November 2018, 17:55:55 UTC | Add batch mode to search indexer. Adds items from the queue up to the specified maximum. (Contrast with the "queued" option that processes everything.) | 13 November 2018, 17:55:55 UTC |
| f5daa75 | Thomas B. Mooney | 12 November 2018, 19:29:01 UTC | Merge pull request #1890 from tmooney/cromwell_output_collection_fixes Some fixes for staging the final outputs from Cromwell. | 12 November 2018, 19:29:01 UTC |
| a84a1c6 | Thomas B. Mooney | 12 November 2018, 19:28:40 UTC | Merge pull request #1889 from tmooney/dont_add_empty_yaml_entries_for_missing_values Don't add empty YAML entries for missing values. | 12 November 2018, 19:28:40 UTC |
