actually, lets trust the processed data

Author: tristan-f-r

datasets/egfr/README.md

@@ -30,9 +30,8 @@ We want to generate the [egfr-prizes.txt](https://github.com/gitter-lab/tps/blob
 The script depends on three files, all inside the TPS [data/timeseries](https://github.com/koksal/tps/tree/bb58d6d89e24dbc39e976a02f1e31387dbe17dfb/data/timeseries)
 folder, which also depend on their own raw files.
 
-`firstfile` and `prevfile`, mapped to `p-values-first.tsv` and `p-values-prev.tsv` are processed from the raw data provided by first supplementary data in the TPS paper.
-
-We first get the raw data with `fetch-tps-data.py`, then TODO.
+`firstfile` and `prevfile`, mapped to `p-values-first.tsv` and `p-values-prev.tsv` are processed from the raw data provided by first supplementary data in the TPS paper. However, since the raw data inside the paper is unlikely
+to be updated with the same output format, we trust that the second supplementary data (or the processed data) is correct.
 
 `mapfile`, or `peptide-mapping.tsv`, is, as quoted by the TPS paper:
 > Obtained by mapping the UniProt accession number (e.g. P00533) to the UniProt ID (e.g. EGFR_HUMAN, also known as the UniProt entry name).

datasets/egfr/Snakefile

@@ -13,15 +13,22 @@ rule prizes:
 
 rule prizes_unsprased:
     input:
-        "raw/p-values-first.tsv",
-        "raw/p-values-prev.tsv",
+        "download/tps_input_pvals_first.tsv",
+        "download/tps_input_pvals_prev.tsv",
         "raw/peptide-mapping.tsv"
     output:
         # We haven't made the final SPRAS adjustments yet.
         "processed/egfr-prizes-unSPRASed.txt"
     shell:
         "uv run scripts/generate_prizes.py " \
-            "--firstfile=raw/p-values-first.tsv " \
-            "--prevfile=raw/p-values-prev.tsv " \
+            "--firstfile=download/tps_input_pvals_first.tsv " \
+            "--prevfile=download/tps_input_pvals_prev.tsv " \
             "--mapfile=raw/peptide-mapping.tsv " \
             "--outfile={output}"
+
+rule get_prize_files:
+    output:
+        "download/tps_input_pvals_first.tsv",
+        "download/tps_input_pvals_prev.tsv"
+    shell:
+        "uv run scripts/fetch-tps-data.py"

datasets/egfr/scripts/fetch-tps-data.py

@@ -1,9 +1,10 @@
 """
-Fetches the first supplementary data from the TPS paper:
-https://doi.org/10.1016/j.celrep.2018.08.085.
+Fetches the second supplementary data from the TPS paper:
+https://doi.org/10.1016/j.celrep.2018.08.085. We trust the processed data from the paper:
+see the README for motivation.
 
-This contains the raw data necessary to get `p-values-first.tsv` and `p-values-prev.tsv`,
-which are fed in to generate the prizes.
+The ZIP contains `p-values-first.tsv` and `p-values-prev.tsv`,
+which are fed in to generate the prizes file.
 """
 
 import requests
@@ -14,7 +15,7 @@
 
 current_directory = Path(os.path.dirname(os.path.realpath(__file__)))
 
-DOWNLOAD_URL = "https://ars.els-cdn.com/content/image/1-s2.0-S2211124718313895-mmc2.zip"
+DOWNLOAD_URL = "https://ars.els-cdn.com/content/image/1-s2.0-S2211124718313895-mmc3.zip"
 
 def main():
     # https://stackoverflow.com/a/14260592/7589775
@@ -25,8 +26,5 @@ def main():
     download_folder.mkdir(exist_ok=True)
     zipf.extractall(current_directory / '..' / 'download')
 
-    initial_xlsx = download_folder / 'initial.xlsx'
-    assert initial_xlsx.exists(), "initial.xlsx should be present from the DOWNLOAD_URL!"
-
 if __name__ == '__main__':
     main()