| 656f299 | Kevin Jacobs | 24 August 2017, 14:01:55 UTC | Fix bcf_translate to operate on INFO and FORMAT values that must be freed. Also submitted upstream to htslib. | 24 August 2017, 14:01:55 UTC |
| bd4d8f9 | Andreas Heger | 21 August 2017, 20:07:44 UTC | Merge pull request #530 from pysam-developers/AH-addGetIndexStatistics Ah add get index statistics | 21 August 2017, 20:07:44 UTC |
| 7eef15c | Andreas Heger | 21 August 2017, 15:53:11 UTC | {AH} add total field | 21 August 2017, 15:53:11 UTC |
| bcd812e | Andreas Heger | 21 August 2017, 13:17:17 UTC | {AH} make conda the recommended installation method | 21 August 2017, 13:23:11 UTC |
| b16167d | Andreas Heger | 21 August 2017, 13:06:48 UTC | {AH} update 0.12.0 release notes | 21 August 2017, 13:06:48 UTC |
| da3e8b4 | Kevin Jacobs | 08 August 2017, 19:32:04 UTC | Merge pull request #526 from msto/MS_new_record_bugfix Handle unset ref when creating new records | 08 August 2017, 19:32:04 UTC |
| fce8bf1 | Matt Stone | 08 August 2017, 17:57:50 UTC | Handle unset ref when creating new records | 08 August 2017, 17:57:50 UTC |
| 27d8452 | Kevin Jacobs | 01 August 2017, 14:15:17 UTC | Simplify file detection logic and allow remote index files (#521) ## Scope Simplify data and index loading logic in `AlignmentFile._open`: * Remove attempts to guess data and index file names; this is magic that I recommend we leave to htslib. * Remove file existence check prior to opening files with htslib; do check for errors more carefully afterward and raise the appropriate error (`IOError` for when `errno` is set, `ValueError` otherwise for backward compatibility) * Catch IO errors when explicitly loading an index. * Allow remote indices (tested using S3 signed URLs, not easily added to the test suite). * Document `filepath_index` and make it an alias for `index_filename`. * Added a `require_index` parameter to `AlignmentFile` | 01 August 2017, 14:15:17 UTC |
| 68fc5df | Nils Homer | 01 August 2017, 12:57:12 UTC | Fix link to tabix (#522) @nh13: Thanks for the fix! | 01 August 2017, 12:57:12 UTC |
| 233386e | Andreas Heger | 31 July 2017, 14:42:05 UTC | Merge pull request #520 from vorlonofportland/no-usage-test-without-tty skip usage() tests on autobuilders | 31 July 2017, 14:42:05 UTC |
| 6da12b1 | Steve Langasek | 28 July 2017, 20:48:18 UTC | skip usage() tests on autobuilders The usage() functions check whether stdin is a tty before displaying usage information. Therefore, we should not test this code when running on an autobuilder without a tty. Clever allocation of a pty from within the test suite itself for purposes of testing this path is left as an exercise for the reader. | 28 July 2017, 20:48:18 UTC |
| 41bbc42 | Kevin Jacobs | 26 July 2017, 14:29:42 UTC | Add linking method used prior to PR #489 | 26 July 2017, 14:29:42 UTC |
| cb799af | Kevin Jacobs | 26 July 2017, 14:19:38 UTC | Fix bcf_translate to skip deleted FORMAT fields to avoid segfaults (#513) | 26 July 2017, 14:19:38 UTC |
| 28c6388 | Andreas Heger | 24 July 2017, 15:48:46 UTC | {AH} add get_index_statistics method, closes #512 | 24 July 2017, 15:48:46 UTC |
| 78e95ef | Kevin Jacobs | 24 July 2017, 13:12:33 UTC | Expose IO errors via IOError exceptions (#516) ## Scope Restructure error handling in `FastaFile.fetch`, `FastaFile._fetch`, and `VariantFile.fetch` to expose IO errors by raising Python`IOError` exceptions when `errno` is set. The intent is to address hidden data loss (Issue #504), clarify when IO errors are occurring, and incremental to being able to add retry logic. Testing is difficult, since simulating IO errors is not trivial. Also added some missing detection of BCF allocation failures and raise `MemoryException`. ## Concerns `FastaFile.fetch` still raises `ValueError` for non-IO errors. However raising `IOError` is a non-backward compatible change and may cause problems in practice. To me, is justifiable in order to unobfuscate when IO errors occur. | 24 July 2017, 13:12:33 UTC |
| 70ea6a8 | Kevin Jacobs | 23 July 2017, 15:39:48 UTC | Use htslib and utils functions to ensure linking is working | 23 July 2017, 15:39:48 UTC |
| 30ff0c8 | Kevin Jacobs | 22 July 2017, 22:13:20 UTC | Merge pull request #515 from pysam-developers/refactor-parse-region ## Scope * Update glossary entry for CIGAR * Correct error return in pysam_bam_update * Remove incorrect use of inline declarations in external Cython definitions * Refactor region_parsing and related code to HTSFile base class. Doing so exposed a bifurcation in terminology. `AlignmentFile` uses `reference` and `end`, where as `VariantFile` is careful to always use `contig` and `stop`. Although there is more code in the wild using `AlignmentFile`, I *strongly* prefer the latter nomenclature and updated all of the code to use that. However, extra keyword arguments were added to enable backward compatible use of `reference` and `end`. | 22 July 2017, 22:13:20 UTC |
| 055de7f | Kevin Jacobs | 22 July 2017, 19:22:51 UTC | Refactor region_parsing and related code to HTSFile base class. Doing so exposed a bifurcation in terminology. AlignmentFile uses `reference` and `end`, where as VariantFile is careful to always use `contig` and `stop`. Although there is more code in the wild using AlignmentFile, I *strongly* prefer the latter nomenclature and updated all of the code to use that. However, extra keyword arguments were added to enable backward compatible use of `reference` and `end`. I understand this change may be a bit controversial and I am happy to discuss before merging this PR. | 22 July 2017, 19:30:40 UTC |
| 261da66 | Kevin Jacobs | 22 July 2017, 19:21:35 UTC | Remove incorrect use of inline declarations in external Cython definitions | 22 July 2017, 19:30:40 UTC |
| be1d943 | Kevin Jacobs | 22 July 2017, 19:20:57 UTC | Correct error return in pysam_bam_update | 22 July 2017, 19:30:40 UTC |
| 09af3e8 | Kevin Jacobs | 22 July 2017, 19:20:32 UTC | Update glossary entry for CIGAR | 22 July 2017, 19:30:40 UTC |
| 1e50107 | Kevin Jacobs | 22 July 2017, 19:26:49 UTC | Fix temp file paths used in tests (#514) ## Scope Running tests with nose would use the `tests` directory as CWD, so test file paths were all relative to the `tests` directory. In contrast, pytest runs from the `pysam` base directory as CWD, so all test file file paths must be updated. Prior changes were sufficient to ensure that existing test files were found in the correct locations, but many temporary files were created in CWD, rather than the `tests` directory. ## Limitations - There are still a few temp files not created in `tests`, including the 1kGenomes index file. - In future, all temporary files should probably be written to a dedicated directory that can be removed after the test suite is run. | 22 July 2017, 19:26:49 UTC |
| 3b98db3 | Kevin Jacobs | 22 July 2017, 19:01:01 UTC | Fix bcf_translate to skip deleted INFO fields to avoid segfaults (#513) ## Background `bcf_translate` works correctly on newly created or loaded BCF records, but it does not properly account for records which have been modified by having one or more INFO fields deleted. ## Nature of fix Prior to attempt to update the info value (either in place or after resizing), update the entry's key id (to keep all ids valid) and then check whether `line->d.info[i].vptr` is `NULL`. This indicates that the key and value have been deleted. Otherwise, `vptr` is incorrectly assumed to point to a valid value and a segmentation fault results. ## Other issues not addressed in this PR `bcf_translate` uses a data structure stored within the source BCF header data structure. This is a poor design decision, because it presumes that any header will be used to translate to no more than one destination header. Worse, there is no check to verify that the same destination header is used or that the source and destination have not changed since the translation table was initialized. Both requirements are unchecked and can cause crashes or write corrupted data. There is no good reason that the translation data structure is part of the header, except that it would require breaking ABI compatibility. This suggests adding a `bcf_translate2` function that takes an explicit translation table as a parameter with clear documentation on the assumptions and lifetime of the translation struct. ## Notes This fix was also submitted upstream to htslib. See [PR #568](https://github.com/samtools/htslib/pull/568) | 22 July 2017, 19:01:01 UTC |
| 649194d | Andreas Heger | 21 July 2017, 12:20:13 UTC | Merge pull request #511 from pysam-developers/{AH}-tabix_add_line_skip {AH} add tabix line_skip, fixes #487. Remove pileup preset. | 21 July 2017, 12:20:13 UTC |
| f5da479 | Andreas Heger | 21 July 2017, 12:20:02 UTC | Merge branch 'master' into {AH}-tabix_add_line_skip | 21 July 2017, 12:20:02 UTC |
| c892ae9 | Andreas Heger | 21 July 2017, 12:12:17 UTC | Merge pull request #510 from pysam-developers/AH-add-linking-tests Ah add linking tests | 21 July 2017, 12:12:17 UTC |
| 7b1cc91 | Andreas Heger | 21 July 2017, 10:54:25 UTC | {AH} fix Memory error detection | 21 July 2017, 10:54:25 UTC |
| 368dd1b | Andreas Heger | 21 July 2017, 09:18:14 UTC | Merge pull request #502 from pysam-developers/AH-faidx-instantiate-records Ah faidx instantiate records | 21 July 2017, 09:18:14 UTC |
| 3269beb | Andreas Heger | 10 July 2017, 15:37:37 UTC | {AH} fix return code if read_name is identical, issue with memory faults? | 10 July 2017, 15:37:37 UTC |
| 1c766c0 | Andreas Heger | 10 July 2017, 15:30:38 UTC | {AH} update docs of infer_query_length, see #494 | 10 July 2017, 15:30:38 UTC |
| fdd852f | Andreas Heger | 10 July 2017, 14:17:28 UTC | Merge branch 'master' into AH-faidx-instantiate-records | 10 July 2017, 14:17:28 UTC |
| f617dc8 | Andreas Heger | 10 July 2017, 14:16:07 UTC | {AH} add without rpath tests | 10 July 2017, 14:16:07 UTC |
| 200bf2c | Andreas Heger | 10 July 2017, 13:23:31 UTC | {AH} fix BAM_DATADIR | 10 July 2017, 13:23:31 UTC |
| 2381a57 | Andreas Heger | 10 July 2017, 13:22:21 UTC | {AH} add rpath linking test | 10 July 2017, 13:22:21 UTC |
| 19fd140 | Andreas Heger | 10 July 2017, 10:14:11 UTC | {AH} FastxRecord raises ValueError if no name or sequence | 10 July 2017, 10:14:11 UTC |
| cdef6c8 | Andreas Heger | 10 July 2017, 10:13:54 UTC | {AH} FastxRecord raises ValueError if no name or sequence | 10 July 2017, 10:13:54 UTC |
| 1c28fcb | Andreas Heger | 10 July 2017, 09:58:46 UTC | {AH} add FastxRecord, replaces PersistentFastqProxy | 10 July 2017, 09:58:46 UTC |
| af861c2 | Andreas Heger | 06 July 2017, 20:22:15 UTC | Merge pull request #498 from pysam-developers/pytest-testing {AH} import paths via TestUtils | 06 July 2017, 20:22:15 UTC |
| d0716d0 | Andreas Heger | 06 July 2017, 20:22:03 UTC | Merge branch 'master' into pytest-testing | 06 July 2017, 20:22:03 UTC |
| cd0bed0 | Andreas Heger | 05 July 2017, 20:55:49 UTC | {AH} more cleaning up | 05 July 2017, 20:55:49 UTC |
| 788ade2 | Andreas Heger | 05 July 2017, 20:45:27 UTC | {AH} remove superfluous cd | 05 July 2017, 20:45:27 UTC |
| d4cd1b8 | Andreas Heger | 04 July 2017, 20:23:13 UTC | {AH] cleanup after testing | 04 July 2017, 20:23:13 UTC |
| 3b49b59 | Andreas Heger | 04 July 2017, 20:22:13 UTC | {AH] disable vcf test 12 | 04 July 2017, 20:22:13 UTC |
| 9944973 | Andreas Heger | 03 July 2017, 20:11:57 UTC | {AH} remove time, needs to be installed | 03 July 2017, 20:11:57 UTC |
| e085071 | Andreas Heger | 03 July 2017, 14:50:57 UTC | {AH} move htslib etc versions to 1.5 on travis | 03 July 2017, 14:50:57 UTC |
| e6deaac | Andreas Heger | 03 July 2017, 14:38:31 UTC | {AH} use command time instead of /usr/bin/time | 03 July 2017, 14:38:31 UTC |
| 4dcd0ce | Andreas Heger | 03 July 2017, 14:36:33 UTC | {AH} infer_query_length returns None, not 0 if cigar not present, closes #494 | 03 July 2017, 14:36:33 UTC |
| 30c46b0 | Andreas Heger | 03 July 2017, 13:37:32 UTC | {AH} import paths via TestUtils | 03 July 2017, 13:37:32 UTC |
| 6bc6e2a | Andreas Heger | 03 July 2017, 11:03:42 UTC | Merge pull request #496 from pysam-developers/{AH}-htslib-1.5 {ah} htslib 1.5 | 03 July 2017, 11:03:42 UTC |
| ce56d6d | Andreas Heger | 01 July 2017, 22:48:38 UTC | Merge pull request #489 from jvkersch/build/mac-os-linker-paths Use @loader_path for mac OS library paths | 01 July 2017, 22:48:38 UTC |
| 649a178 | Andreas Heger | 01 July 2017, 22:25:47 UTC | {AH} disable tests failng in OSX, related to stdout redirection | 01 July 2017, 22:25:47 UTC |
| c2e5ec1 | Andreas Heger | 28 June 2017, 20:33:35 UTC | {AH} remove superfluous ; | 28 June 2017, 20:34:28 UTC |
| 2bf0869 | Joris Vankerschaver | 28 June 2017, 12:54:29 UTC | Temporarily pin samtools version numbers. | 28 June 2017, 12:54:29 UTC |
| 4ff2415 | Joris Vankerschaver | 28 June 2017, 12:53:23 UTC | Use @rpath for library name. | 28 June 2017, 12:53:23 UTC |
| 54bc34e | Andreas Heger | 27 June 2017, 19:43:29 UTC | {AH} add missing files | 27 June 2017, 19:43:29 UTC |
| 21791fa | Andreas Heger | 26 June 2017, 13:01:21 UTC | {AH} first import of htslib 1.5, some samtools/bcftools tests disabled | 26 June 2017, 13:01:42 UTC |
| 93dc11c | Andreas Heger | 26 June 2017, 08:47:17 UTC | {AH} add tabix line_skip, fixes #487. Remove pileup preset. | 26 June 2017, 09:49:20 UTC |
| ebedce5 | Andreas Heger | 26 June 2017, 08:47:17 UTC | {AH} fix wrong call in GTF.asDict() | 26 June 2017, 08:57:52 UTC |
| 5874050 | Andreas Heger | 19 June 2017, 12:26:48 UTC | {AH} more explicit memory checks | 26 June 2017, 08:20:48 UTC |
| 19d47ac | Joris Vankerschaver | 20 June 2017, 13:20:44 UTC | Use @loader_path for mac OS library paths | 20 June 2017, 13:20:44 UTC |
| d41a7c8 | Andreas Heger | 08 June 2017, 19:07:51 UTC | {AH} increase version to 0.11.2.2 | 08 June 2017, 19:07:51 UTC |
| 0eafc2f | Kevin Jacobs | 08 June 2017, 16:05:23 UTC | Correct VariantRecord edge cases described in Issue #479 (#480) | 08 June 2017, 16:05:23 UTC |
| 31f750e | Andreas Heger | 05 June 2017, 12:14:01 UTC | {AH} add missing B tag to doc strings, fixes #467 | 05 June 2017, 12:14:01 UTC |
| ac9a787 | Andreas Heger | 05 June 2017, 12:05:09 UTC | Merge pull request #476 from pysam-developers/AH-GFF3-Fixes Ah gff3 fixes | 05 June 2017, 12:05:09 UTC |
| 0727a27 | Andreas Heger | 05 June 2017, 10:37:51 UTC | Merge pull request #474 from kyleabeauchamp/write_fastq Add example of fastq output | 05 June 2017, 10:37:51 UTC |
| 1edb33d | Andreas Heger | 05 June 2017, 10:37:12 UTC | Merge pull request #470 from nieder/implicit_dec remove implicit declarations on memset and strlen | 05 June 2017, 10:37:12 UTC |
| c6a132a | Andreas Heger | 05 June 2017, 08:37:04 UTC | {AH} fix load_and_convert import | 05 June 2017, 08:37:04 UTC |
| 28fad67 | Andreas Heger | 04 June 2017, 20:19:15 UTC | {AH} fix GTF/GFF3 parsing | 04 June 2017, 20:19:15 UTC |
| 4aac086 | Kyle Beauchamp | 02 June 2017, 20:22:43 UTC | Add example of fastq output | 02 June 2017, 20:22:43 UTC |
| 67a89b6 | Hanspeter Niederstrasser | 30 May 2017, 08:54:46 UTC | remove implicit declarations on memset and strlen | 30 May 2017, 08:54:46 UTC |
| 268100c | Kevin Jacobs | 23 May 2017, 18:01:27 UTC | [KBJ] Add missing autoconf input file -- prevents libcurl from being detected at build time | 23 May 2017, 18:01:27 UTC |
| 943e881 | Andreas Heger | 16 May 2017, 22:17:22 UTC | {AH} fix typo | 16 May 2017, 22:17:22 UTC |
| adc0382 | Andreas Heger | 16 May 2017, 21:43:14 UTC | {AH} add config option for ReadTheDocs | 16 May 2017, 21:43:14 UTC |
| 0d76fc6 | Kevin Jacobs | 15 May 2017, 19:14:28 UTC | [KBJ] Correct check for deleting INFO/END when rlen == len(ref) | 15 May 2017, 19:14:28 UTC |
| f66f28a | Andreas Heger | 23 April 2017, 21:04:49 UTC | {AH} updates to buildwheels.sh | 13 May 2017, 21:08:27 UTC |
| 0dc59d3 | Andreas Heger | 13 May 2017, 20:40:41 UTC | {AH} update docs for v0.11.2.1 | 13 May 2017, 20:47:09 UTC |
| 887b9f8 | Andreas Heger | 12 May 2017, 19:02:04 UTC | {AH} update release notes | 12 May 2017, 19:02:04 UTC |
| ac4eca1 | Kevin Jacobs | 12 May 2017, 17:46:49 UTC | Improve sanity checking of VariantRecord.rlen | 12 May 2017, 17:46:49 UTC |
| d4535f5 | Kevin Jacobs | 12 May 2017, 17:44:30 UTC | Merge branch 'master' of github.com:pysam-developers/pysam | 12 May 2017, 17:44:30 UTC |
| 43aaed6 | Kevin Jacobs | 12 May 2017, 17:43:46 UTC | [KBJ] Ensure VariantRecord.rlen can never be negative | 12 May 2017, 17:43:46 UTC |
| afa9ce1 | Andreas Heger | 11 May 2017, 18:49:49 UTC | {AH} fix superfluous \r in reference string | 11 May 2017, 18:49:59 UTC |
| 80e2e82 | Kevin Jacobs | 11 May 2017, 13:04:19 UTC | Experimental HFile object should raise AttributeError if fileno is unavailable | 11 May 2017, 13:04:19 UTC |
| 0846c40 | Kevin Jacobs | 11 May 2017, 05:55:25 UTC | Multiple fixes related to VariantRecordInfo and handling of INFO/END (#461) A pysam user has privately reported a bug with the VariantRecord class. The situation is somewhat complicated, but the root cause is that INFO/END is able to get out of sync with the VariantRecord stop coordinate. After trying several simpler workarounds, it seems that the best solution is to not expose INFO/END via VariantRecord.info. This is because it is an artifact of the VCF text format that isn't used in BCF or the htslib in-memory representation. Instead, all access is via VariantRecord.stop and VariantRecord.rlen, while INFO/END is kept in sync behind the scenes. In fixing researching and this bug, I've also found and fixed several other bugs in VariantRecordInfo: * scalar INFO entries could not be set again after being deleted * multiple cases where access to values was attempted before ensuring that lazy parsing had occurred * __bool__ and __len__ counted deleted entries * __delitem__ now allows unset flags to be deleted without raising a KeyError | 11 May 2017, 05:55:25 UTC |
| b13ecbd | Kevin Jacobs | 11 May 2017, 00:25:54 UTC | Fix even more bugs that have resulted from supporting non-diploid data (#460) * Aggressively sync RLEN and INFO/END * Fix pop * Optimize header metadata check for bcf_sync_end * Must restore info.vptr to restore a deleted value * Disable fast path for setting INFO values due to problems with shared pointers | 11 May 2017, 00:25:54 UTC |
| 2026e90 | Andreas Heger | 10 May 2017, 20:50:41 UTC | {AH} limit qname to 251 characters, see #447 | 10 May 2017, 20:50:41 UTC |
| 4ad0f30 | Andreas Heger | 10 May 2017, 20:36:57 UTC | Merge pull request #458 from pysam-developers/AH-GFF3-support {AH} add GFF3 support to tabix, closes #452 | 10 May 2017, 20:36:57 UTC |
| 5dbb8eb | Andreas Heger | 09 May 2017, 19:01:13 UTC | {AH} add GFF3 support to tabix, closes #452 | 10 May 2017, 20:35:32 UTC |
| 4d0c727 | Kevin Jacobs | 10 May 2017, 02:27:10 UTC | Fix error in calling `bcf_get_genotypes` and a few other more minor glitches (#457) Bug fixes: * BCF fixes for non-ordered dicts * Correct pop return value and fix initialization in bcf_genotype_count * Update record header after translating | 10 May 2017, 02:27:10 UTC |
| 01b3b1f | Andreas Heger | 09 May 2017, 19:13:01 UTC | {AH} increase samtools/bcftools version, add release notes | 09 May 2017, 19:13:01 UTC |
| 8666aa4 | Andreas Heger | 09 May 2017, 19:07:50 UTC | Merge pull request #456 from pysam-developers/kbj-dev KBJ's eclectic development branch | 09 May 2017, 19:07:50 UTC |
| 0092e4d | Kevin Jacobs | 09 May 2017, 13:51:47 UTC | Update configure -- it appeared corrupted with random make output | 09 May 2017, 13:51:47 UTC |
| 05eeb0b | Kevin Jacobs | 09 May 2017, 13:40:13 UTC | Merge remote-tracking branch 'origin/AH-htslib-1.4.1' into kbj-dev | 09 May 2017, 13:40:13 UTC |
| 5aa9a09 | Kevin Jacobs | 09 May 2017, 06:52:05 UTC | Update test cases | 09 May 2017, 06:52:05 UTC |
| 0eb08ad | Kevin Jacobs | 09 May 2017, 06:07:44 UTC | Implement file truncation check in AlignmentFile; add argument to ignore truncation to both AlignmentFile and VariantFile. | 09 May 2017, 06:07:44 UTC |
| 4ef68b2 | Kevin Jacobs | 09 May 2017, 03:37:07 UTC | Many improvements to VariantFile and related objects: * Restore __dealloc__ behavior * Correct handling of bcf_str_missing * Added update() and pop() methods to some dict-like proxy objects * Fix handling of INFO/END Experimental features: * Initialization arguments to VariantRecord.new_record * Initial implementation of object comparison * Iintial support for non-ploidy 2 data | 09 May 2017, 03:37:07 UTC |
| 3033234 | Kevin Jacobs | 09 May 2017, 03:36:31 UTC | __dealloc__ really can't call methods, since doing so will revive the object. | 09 May 2017, 03:36:31 UTC |
| 220b0e9 | Kevin Jacobs | 09 May 2017, 03:20:49 UTC | Restore initialization of the return value | 09 May 2017, 03:20:49 UTC |
| f39c12c | Andreas Heger | 08 May 2017, 21:34:00 UTC | Merge branch 'master' into AH-htslib-1.4.1 | 08 May 2017, 21:34:00 UTC |
| 7061955 | Andreas Heger | 08 May 2017, 21:33:18 UTC | Merge branch 'master' into AH-htslib-1.4.1 | 08 May 2017, 21:33:18 UTC |
| d2a3230 | Andreas Heger | 08 May 2017, 16:03:07 UTC | {AH} add missing version file | 08 May 2017, 16:03:07 UTC |
