VCF Improvements and DMD Support

This release involves breaking changes to the D API.

New features since 0.13.3:

• VCF Genotype functionality along with other fixes (#116, #117)
• VCF multi-threading
• DMD support is back! #122
• Overhaul of mechanism to update htslib bindings #129, #130
• Laid groundwork for future iterators and io overhaul #128
• Logo added!

Breaking changes

• getAlignedPairs and Cigar API

What's Changed

• Vcf improvements and fixes by @charlesgregory in #125
• HtslibFile and HtslibIterator by @charlesgregory in #128
• Linker postblit fix by @charlesgregory in #123
• Dstep Bindings V2 by @charlesgregory in #129
• Update the Dstep folder after #129 by @charlesgregory in #130
• Release v0.14.0+htslib-1.13 by @charlesgregory in #131

Full Changelog: v0.13.3...v0.14.0+htslib-1.13

Typesafe Coordinates and Reference Counting

This release involves breaking changes to the D API: Typesafe Coordinates (manuscript forthcoming; see https://github.com/blachlylab/typesafe-coordinates) are used throughout the entire interface to eliminate off-by-one errors.

New features since 0.12.0:

• Support htslib 1.13, 1.12, and 1.11 (including all new functions available through 1.13)
• Moved all wrapper structs (SAM/VCFFile; SAM/VCFReader/Writer, CIGAR, TagValue, etc.) to reference counted with automatic memory cleanup when out of scope
  • This uses new HtslibMemory wrapper struct with transparent pointer aliasing if you are feeling dangerous
• Typesafe Coordinates (https://github.com/blachlylab/typesafe-coordates/)
• Typesafe Coordinates is a subpackage, so you can use dhtslib:coordinates without needing htslib!
• The entire D side of the package (i.e. all high level interfaces, but not not the C bindings) take and/or receive typesafe coordinates.
• Integration of the GFF3D (https://github.com/blachlylab/gff3d) GFF3 reader package; add writer
• BED reader/writer (again, you guessed it, using typesafe coordinates)
• Remove GC allocation from coloration of log messages
• Markedly improved test coverage
• VCF: INFO and FORMAT field getters and setters, along with structs to represent and convert those data
• Support -dip1000 compilation
• Marked many D side functions as one or more of @safe @nogc nothrow pure
• Marked the most of C htslib bindings as @nogc nothrow ; some also @system
• Made some ranges (like BAM RecordRange) a forward range for better Phobos range API compat
• FASTA faidx opDollar (i.e., fai["chrom1", 1 .. $]) indexing

Bugfixes

• Many and sundry
• Tabix bugfixes
• Random buffer overflows
• Rare early-deallocation errors with reference counted wrapper structs

What's Changed

• Initial support for SAMReader["chrWHATEVA", x .. $] by @jblachly in #50
• changes to qscore functions by @charlesgregory in #53
• Cigar and md changes by @charlesgregory in #58
• Bgzf reader by @charlesgregory in #28
• htslib 1.12 compatibility by @charlesgregory in #79
• CoordinateSystem Implementation and Adoption by @charlesgregory in #71
• Other file formats by @charlesgregory in #80
• Fix typo in readme by @thewilsonator in #83
• apt-get update github actions by @jblachly in #87
• Remove GC allocation from coloration of log messages by @thewilsonator in #84
• htslib 1.13 compatibility by @charlesgregory in #90
• Coordinate system improvements by @charlesgregory in #88
• VCF Overhaul by @charlesgregory in #105
• Htslib package changes by @charlesgregory in #109
• Coordinate subpackage by @charlesgregory in #111
• Reference Counting fixes and consolidation by @charlesgregory in #108
• Release v0.13.3+htslib-1.13 by @charlesgregory in #113

New Contributors

• @thewilsonator made their first contribution in #83

Full Changelog: v0.10.3...v0.13.3+htslib-1.13

v0.12.0: Reorganization, Structs, and reference counting

We have moved to using structs and reference counting for SAMRecord and VCFRecord. Reference counting has been added to Cigar and TagValue, solving issues with reference counting and segfaults related to TagValue and Cigar. These structs should be considerably safer now. dhtslib.sam package has been reorganized to be more manageable. SAMRecord moves to dhtslib.sam.record. TagValue moves to dhtslib.sam.tagvalue. SAMReader moves to dhtslib.sam.reader. SAMWriter moves to dhtslib.sam.writer. SAMReader and SAMWriter now primarily deal with SAMHeader rather than bam_hdr_t. The SAMReader.allRecords() should now be able to be called more than once without issue. There were also changes made to BGZFile along with the addition of a fastq reader FastqFile.

This should build against htslib-1.10 or htslib-1.11, but the C bindings are only up to date with htslib-1.10 at the moment. This only means a handful of newest functions are not yet included. You can easily tell whether you have htslib-1.9 (or earlier) versus 1.10+ installed by looking at your shared libraries: if you have htslib.so.3, you are up to date. If htslib.so.2, that reflects htslib-1.9 or earlier and is incompatible with this release.

htslib-1.10, 1.11 support

htslib-1.10 (and 1.11) compatibility

We have been compatible in the develop branch for a long time, but this finally comes to master for building client programs.

Besides adding tons of new APIs and functions in htslib-1.10 (and 1.11, with which this is binary compabile, see below) There are some major breaking changes to be aware of:

1. 64 bit coordinates (previously were 32 bit in 1.9 and earlier)
   • go make all your int coordinates long
2. Revamped namespace to place all convenience wrappers and idiomatic D and functions not included in htslib (like cigar and MD processing) under dhtslib.*, while direct C bindings moved under htslib.* (previously dhtslib.htslib.*)

This should build against htslib-1.10 or htslib-1.11, but the C bindings are only up to date with htslib-1.10 at the moment. This only means a handful of newest functions are not yet included. You can easily tell whether you have htslib-1.9 (or earlier) versus 1.10+ installed by looking at your shared libraries: if you have htslib.so.3, you are up to date. If htslib.so.2, that reflects htslib-1.9 or earlier and is incompatible with this release.

Windows support

Better support for building on Windows platforms. Please let us know of any trouble.

Huge update, new fatures, some API-breaking

Overall

• new features
• bugfixes
• speedups/reduce allocs
• codestyle
• fix warnings
• documentation

VCF -- massive changes: bugfixes, new features, speed
SAM -- massive changes: SamWriter; SAMFile -> SAMReader
SAM -- CRAM support; UBAM writer support; reader autodetect support
Cigar module
Tabix
hts_log function signatures to const(char)[]
hFile module complete
faidx API breaking: fetchSequence supports alternative coordinate systems
faidx expose BGZF; added setCacheSize, setThreads

Huge update, mostly non-API breaking

New ways to slice SAMFile:
bam["chr1:1-2"]
bam["chr1",1]
bam[0,1]
bam["chr1",1..2]

Cigar ops and Cigar module / Cigar helpers in SAMFile

Parse SAM from text record/string

VCFReader/VCFWriter

SAMRecord bam1_core_t getters/setters