Easy install

.gitignore

@@ -0,0 +1,138 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+pip-wheel-metadata/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# Apptainer build containers
+*.sif
+
+#
+build/
+containers/deprecated/
+
+.snakemake/

.readthedocs.yaml

@@ -0,0 +1,21 @@
+version: 2
+
+build:
+    os: ubuntu-22.04
+    tools:
+        python: "3.10"
+
+sphinx:
+    builder: html
+    configuration: docs/conf.py
+    fail_on_warning: false
+
+python:
+    install:
+        - method: pip
+          path: .
+          extra_requirements: [docs]
+
+submodules:
+    include: [docs/notebooks]
+    recursive: true

LICENSE

@@ -0,0 +1,28 @@
+BSD 3-Clause License
+
+Copyright (c) 2023, Princeton University
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its
+   contributors may be used to endorse or promote products derived from
+   this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

NOTES.md

@@ -0,0 +1,26 @@
+### Questions
+- Why is it necessary to download the SNP panel and phasing panel for an example that provides the reduced matrices?
+- wisdom of naming random realizations calicost/clone* etc.?
+- in config, phasing_panel -> phasing_panel_dir
+- wildcards: outputdir=calicost? NB prevents target rules.
+- GRCh38_resources: what are these and were are they found for other references?
+- genetic_map_GRCh38_merged.tab: tabix index?
+- deprecate bin/* bash scripts.  These aren't used.
+- what resolves outputdir variable?
+- startle snakemake rule?
+- sandbox deprecation
+
+###  Warnings
+- convert_params defined multiple times [hmm_NB_sharedstates.py, utils_distribution_fitting.py]
+- compute_posterior_transition_sitewise defined multiple times [hmm_gaussian.py, utils_hmm.py]
+- compute_posterior_obs [hmm_gaussian.py, utils_hmm.py]
+- update_startprob_sitewise [hmm_gaussian.py, utils_hmm.py]
+- np_sum_ax_squeeze [hmm_gaussian.py, utils_hmm.py]
+- mylogsumexp [hmm_gaussian.py, utils_hmm.py]
+- update_transition_sitewise [hmm_gaussian.py, utils_hmm.py]
+
+
+### NOTES
+- How to pin conda: http://damianavila.github.io/blog/posts/how-to-pin-conda.html
+- Apple silicon installs can be facilitated with Rosetta emulation of the x86 instruction set, see e.g. [here](https://taylorreiter.github.io/2022-04-05-Managing-multiple-architecture-specific-installations-of-conda-on-apple-M1/) - note, brew install iterm2 as duplication of the terminal app. is no longer supported.
+-  poetry config virtualenvs.prefer-active-python false

README.md

@@ -1,2 +1,159 @@
-# Locality clustering for CNV inference
-A repo for the locality clustering (HMM) module in CNV calling across various data modalities.
+# CalicoST
+
+<p align="center">
+<img src="https://github.com/raphael-group/CalicoST/blob/main/docs/_static/img/overview4_combine.png?raw=true" width="100%" height="auto"/>
+</p>
+
+[CalicoST](https://www.biorxiv.org/content/10.1101/2024.03.09.584244v1) is a probabilistic model that infers allele-specific copy number aberrations and tumor phylogeography from spatially resolved transcriptomics.  CalicoST has the following key features:
+1. Identifies allele-specific integer copy numbers for each transcribed region, revealing events such as Copy Neutral Loss of Heterozygosity (CNLOH) and mirrored subclonal CNAs that are invisible to total copy number analysis.
+2. Assigns each spot a clone label indicating whether the spot is primarily normal cells or a cancer clone with an aberration copy number profile.
+3. Infers a phylogeny relating the identified cancer clones as well as a phylogeography that combines genetic evolution and spatial dissemination of clones.
+4. Handles normal cell admixture in SRT technologies that are not single-cell resolution (e.g. 10x Genomics Visium) to ensure more accurate allele-specific copy numbers and cancer clones.
+5. Simultaneously analyzes multiple regions or aligned SRT slices from the same tumor.
+
+# System requirements
+The package has been tested on the following Linux operating systems: SpringdaleOpenEnterprise 9.2 (Parma) and CentOS Linux 7 (Core).
+
+# Installation
+First, setup a conda environment from the `environment.yml` file:
+```
+cd CalicoST
+
+conda env create -f environment.yml --name calicost_env
+
+conda activate calicost
+```
+Next download [Eagle2](https://alkesgroup.broadinstitute.org/Eagle/) by
+```
+wget https://storage.googleapis.com/broad-alkesgroup-public/Eagle/downloads/Eagle_v2.4.1.tar.gz
+tar -xzf Eagle_v2.4.1.tar.gz
+```
+Next, we need to install [Startle](https://github.com/raphael-group/startle).  Its dependencies
+include [LEMON](https://lemon.cs.elte.hu/trac/lemon/wiki/InstallLinux), [CPLEX](https://www.ibm.com/products/ilog-cplex-optimization-studio/cplex-optimizer),
+perl and python3.  We do so with conda
+```
+conda install -c schmidt73 startle
+```
+or by building from source,
+```
+git clone --recurse-submodules https://github.com/raphael-group/startle.git
+
+cd startle
+
+mkdir build; cd build
+
+cmake -DLIBLEMON_ROOT=<lemon path>\
+        -DCPLEX_INC_DIR=<cplex include path>\
+        -DCPLEX_LIB_DIR=<cplex lib path>\
+        -DCONCERT_INC_DIR=<concert include path>\
+        -DCONCERT_LIB_DIR=<concert lib path>\
+        ..
+make
+```
+Note this will install a copy of cellsnp-lite to the environment directory, which must be updated
+in the config.yaml, i.e. with the new path to the installed cellsnp-lite. 
+
+Finally, install CalicoST using pip in the root directory with
+```
+pip install --no-deps -e .
+```
+Setting up the conda environments takes around 10 minutes on an HPC head node.  Make sure to use the
+[mamba](https://mamba.readthedocs.io/en/latest/installation/mamba-installation.html) backend to ensure
+fast builds.
+
+# Getting started
+CalicoST requires annotations for genes and SNPs - these are available for the GRCh38 reference are available in the [example tarballs](https://github.com/raphael-group/CalicoST/tree/main/examples). Specify the information file paths, your input SRT data paths, and running configurations in `config.yaml`, and then you can run CalicoST by
+```
+snakemake --cores <number cores> --configfile config.yaml --snakefile calicost.smk all (--use-singularity) (--use-conda)
+```
+
+Check out our [readthedocs](https://calicost.readthedocs.io/en/latest/) for tutorials on the simulated data and prostate cancer data.
+
+# Run on a simulated example data
+### Download data
+The simulated count matrices are available from [`examples/CalicoST_example.tar.gz`](https://github.com/raphael-group/CalicoST/blob/main/examples/CalicoST_example.tar.gz).
+CalicoST requires a reference SNP panel and phasing panel, which can be downloaded directly with wget and the links below
+* [SNP panel](https://downloads.sourceforge.net/project/cellsnp/SNPlist/genome1K.phase3.SNP_AF5e4.chr1toX.hg38.vcf.gz?ts=gAAAAABmDbjZ1jaoDHw8fbmTQcP1y_WA9KfnTJH3aLrm0O7S4voV89YyU55O3jJdtO163_SpSBquChmB7dIl4dZ7pB-64L-W8A%3D%3D) - 0.5GB in size.
+* [Phasing panel](http://pklab.med.harvard.edu/teng/data/1000G_hg38.zip) - 9.0GB in size.
+
+Other SNP panels are available at [cellsnp-lite webpage](https://cellsnp-lite.readthedocs.io/en/latest/main/data.html).
+
+### Run CalicoST
+Gunzip the downloaded example data and replace the following paths in the provide `example_config.yaml` with those on your machine,
+* region_vcf: the path to the downloaded SNP panel vcf.
+* phasing_panel: the path to the downloaded and unzipped phasing panel directory.
+* spaceranger_dir: the path to Visium Space Ranger output directory.
+
+To avoid falling into local maxima in CalicoST's optimization objective, we recommend running CalicoST with multiple random initializations that are specified by the `random_state` variable in each config. Those provided use five random initializations, but this may be made smaller in testing.
+
+Finally, run CalicoST with
+```
+snakemake --cores <number cores> --configfile example_config.yaml --snakefile <calicost_dir>/calicost.smk all
+```
+CalicoST takes just ove an hour to complete this example with 5 cores.
+
+### Understanding the results
+The above snakemake run will create a folder `calicost` in the directory of downloaded example data. Within this folder, each random initialization of CalicoST generates a subdirectory of `calicost/clone*`. 
+
+CalicoST generates the following key files for each random initialization:
+* clone_labels.tsv: The inferred clone labels for each spot.
+* cnv_seglevel.tsv: Allele-specific copy numbers for each clone for each genome segment.
+* cnv_genelevel.tsv: The projected allele-specific copy numbers from genome segments to the covered genes.
+* cnv_*_seglevel.tsv and cnv_*_genelevel.tsv: Allele-specific copy numbers when enforcing a ploidy of {diploid, triploid, tetraploid} for each genome segment & each gene.
+
+See the following examples of these key files.
+```
+head -n 10 calicost/clone3_rectangle0_w1.0/clone_labels.tsv
+BARCODES        clone_label
+spot_0  2
+spot_1  2
+spot_2  2
+spot_3  2
+spot_4  2
+spot_5  2
+spot_6  2
+spot_7  2
+spot_8  0
+```
+
+```
+head -n 10 calicost/clone3_rectangle0_w1.0/cnv_seglevel.tsv
+CHR     START   END     clone0 A        clone0 B        clone1 A        clone1 B        clone2 A        clone2 B
+1       1001138 1616548 1       1       1       1       1       1
+1       1635227 2384877 1       1       1       1       1       1
+1       2391775 6101016 1       1       1       1       1       1
+1       6185020 6653223 1       1       1       1       1       1
+1       6785454 7780639 1       1       1       1       1       1
+1       7784320 8020748 1       1       1       1       1       1
+1       8026738 9271273 1       1       1       1       1       1
+1       9292894 10375267        1       1       1       1       1       1
+1       10398592        11922488        1       1       1       1       1       1
+```
+
+```
+head -n 10 calicost/clone3_rectangle0_w1.0/cnv_genelevel.tsv
+gene    clone0 A        clone0 B        clone1 A        clone1 B        clone2 A        clone2 B
+A1BG    1       1       1       1       1       1
+A1CF    1       1       1       1       1       1
+A2M     1       1       1       1       1       1
+A2ML1-AS1       1       1       1       1       1       1
+AACS    1       1       1       1       1       1
+AADAC   1       1       1       1       1       1
+AADACL2-AS1     1       1       1       1       1       1
+AAK1    1       1       1       1       1       1
+AAMP    1       1       1       1       1       1
+```
+
+CalicoST creates the following plots to visualize the spatial distribution of the inferred cancer clones and allele-specific copy number profiles for each realization.
+* ```plots/clone_spatial.pdf```: The spatial distribution of inferred cancer clones and normal regions - grey color, clone 0 by default.
+* ```plots/rdr_baf_defaultcolor.pdf```: The Read Depth Ratio (RDR) and B-Allele Frequency (BAF) along the genome for each clone.  Higher RDR indicates higher total copy numbers and BAF deviations from 0.5 indicates allelic imbalance due to CNAs.
+* ```plots/acn_genome.pdf```: The default allele-specific copy numbers along the genome.
+* ```plots/acn_genome_*.pdf```: Allele-specific copy numbers when enforcing a ploidy of {diploid, triploid, tetraploid}.
+
+The allele-specific copy number plots have the following color legend.
+<p align="left">
+<img src="https://github.com/raphael-group/CalicoST/blob/main/docs/_static/img/acn_color_palette.png?raw=true" width="20%" height="auto"/>
+</p>
+
+# Software dependencies
+TODO