TCIA_Segmentations.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "cpb6dbKi6s4v",
   "metadata": {
    "id": "cpb6dbKi6s4v"
   },
   "source": [
    "You can download and run this notebook locally, or you can run it for free in a cloud environment using Colab or Sagemaker Studio Lab:\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/kirbyju/TCIA_Notebooks/blob/main/TCIA_Segmentations.ipynb)\n",
    "\n",
    "[![Open In SageMaker Studio Lab](https://studiolab.sagemaker.aws/studiolab.svg)](https://studiolab.sagemaker.aws/import/github.com/kirbyju/TCIA_Notebooks/blob/main/TCIA_Segmentations.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "KmXfYFZtja2F",
   "metadata": {
    "id": "KmXfYFZtja2F"
   },
   "source": [
    "# Summary\n",
    "\n",
    "Access to large, high-quality datasets is essential for researchers to understand disease and precision medicine pathways, especially in cancer. However, HIPAA constraints make sharing medical images outside an individual institution complex. [The Cancer Imaging Archive (TCIA)](https://www.cancerimagingarchive.net/) is a public service funded by the National Cancer Institute that addresses this challenge by providing hosting and de-identification services that take major burdens of data sharing off researchers.\n",
    "\n",
    "**This notebook is focused on steps to identify an example segmentation file, find the corresponding reference series and visualize them together.**  If you're interested in additional TCIA notebooks and coding examples, check out the tutorials at https://github.com/kirbyju/TCIA_Notebooks."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "AruUGe3lmjkh",
   "metadata": {
    "id": "AruUGe3lmjkh"
   },
   "source": [
    "# 1 Learn about Available Collections on the TCIA Website\n",
    "\n",
    "[Browsing Collections](https://www.cancerimagingarchive.net/collections) and viewing [Analysis Results](https://www.cancerimagingarchive.net/tcia-analysis-results/) of TCIA datasets are the easiest ways to become familiar with what is available. These pages will help you quickly identify datasets of interest, find valuable supporting data that are not available via our APIs (e.g. clinical spreadsheets and non-DICOM segmentation data), and answer the most common questions you might have about the datasets.  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bqulqBEDMSS6",
   "metadata": {
    "id": "bqulqBEDMSS6"
   },
   "source": [
    "# 2 Setup\n",
    "\n",
    "The following installs and imports **[tcia_utils](https://pypi.org/project/tcia-utils/)**, which contains a variety of useful functions for accessing TCIA via Python and Juptyter Notebooks.  It also ensures that the necessary imports are performed and logging settings are adjusted for Google Colab."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "YPrZ5pc-yYQl",
   "metadata": {
    "id": "YPrZ5pc-yYQl"
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "\n",
    "# install tcia utils\n",
    "!{sys.executable} -m pip install --upgrade -q tcia_utils\n",
    "\n",
    "# install simpleDicomViewer and forked pydicom-seg dependency\n",
    "!{sys.executable} -m pip install --upgrade -q git+https://github.com/kirbyju/pydicom-seg.git@master\n",
    "!{sys.executable} -m pip install --upgrade -q simpleDicomViewer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "LaKRUf-GJ_4C",
   "metadata": {
    "id": "LaKRUf-GJ_4C",
    "tags": []
   },
   "outputs": [],
   "source": [
    "import requests\n",
    "import pandas as pd\n",
    "from tcia_utils import nbia\n",
    "from simpleDicomViewer import dicomViewer\n",
    "\n",
    "# set logging level to INFO in Google Colab (not necessary in Jupyter)\n",
    "if 'google.colab' in sys.modules:\n",
    "  import logging\n",
    "\n",
    "  for handler in logging.root.handlers[:]:\n",
    "      logging.root.removeHandler(handler)\n",
    "\n",
    "  # Set handler with level = info\n",
    "  logging.basicConfig(format='%(asctime)s:%(levelname)s:%(message)s',\n",
    "                      level=logging.INFO)\n",
    "\n",
    "  print(\"Google Colab Logging set to INFO\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "UmZ1gP5XsvD9",
   "metadata": {
    "id": "UmZ1gP5XsvD9"
   },
   "source": [
    "# 3 Download and visualize a sample DICOM SEG\n",
    "Here we'll walk through some steps to identify an example segmentation file, find the corresponding reference series and visualize them together in the notebook.\n",
    "\n",
    "First, let's pull a list of segmentation series UIDs of interest.  We'll use the [C4KC-KiTS](https://doi.org/10.7937/TCIA.2019.IX49E8NX) collection as an example, which contains CT scans and segmentations from subjects from the training set of the [2019 Kidney and Kidney Tumor Segmentation Challenge (KiTS19)](https://kits19.grand-challenge.org/) in DICOM SEG format.  \n",
    "\n",
    "We can get an inventory of all scans in the collection using **nbia.getSeries()**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "NNfqtMZUE4xE",
   "metadata": {
    "id": "NNfqtMZUE4xE"
   },
   "outputs": [],
   "source": [
    "df = nbia.getSeries(collection = \"C4KC-KiTS\", format = \"df\")\n",
    "sorted = df.sort_values([\"PatientID\", \"SeriesDescription\"])\n",
    "sorted.head(4)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "JDtkd13_Hr7i",
   "metadata": {
    "id": "JDtkd13_Hr7i"
   },
   "source": [
    "Here we can see that patient KiTS-00000 has 3 CT series and one SEG series.  How do we know which one of the CTs goes with the SEG?  In many cases you can figure this out by looking at the Reference Series UID tag in the segmentation series.  Let's try it by saving the SEG series UID to a variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "WFEOLIYRs33G",
   "metadata": {
    "id": "WFEOLIYRs33G"
   },
   "outputs": [],
   "source": [
    "segSeries = sorted.loc[df['Modality'] == 'SEG', 'SeriesInstanceUID'].iloc[0]\n",
    "\n",
    "print(segSeries)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8NVZAb3qu_X4",
   "metadata": {
    "id": "8NVZAb3qu_X4"
   },
   "source": [
    "Next, let's determine the Reference Series Instance UID of the CT scan that goes with the segmentation using **nbia.getSegRefSeries()**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "jogkmh9kvMoW",
   "metadata": {
    "id": "jogkmh9kvMoW"
   },
   "outputs": [],
   "source": [
    "refSeries = nbia.getSegRefSeries(segSeries)\n",
    "\n",
    "print(refSeries)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "KkMkyHoYwC2F",
   "metadata": {
    "id": "KkMkyHoYwC2F"
   },
   "source": [
    "Now let's download these two series."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1P-s8sZBwIIz",
   "metadata": {
    "id": "1P-s8sZBwIIz"
   },
   "outputs": [],
   "source": [
    "nbia.downloadSeries([refSeries, segSeries], input_type= \"list\", format = \"df\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6HU0fGOwvxxO",
   "metadata": {
    "id": "6HU0fGOwvxxO"
   },
   "source": [
    "Finally, we can look at the images and segmentation together.  You can move the slider to flip through the images and toggle the segmentation layer on/off.  \n",
    "\n",
    "**Tip:** Once the slider is selected, sometimes it's easier to move between images using the left/right arrow keys on your keyboard than to use your mouse."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "gZb33aCNv24K",
   "metadata": {
    "id": "gZb33aCNv24K"
   },
   "outputs": [],
   "source": [
    "# Assuming you didn't change the default download options for downloadSeries\n",
    "imgPath = \"tciaDownload/\" + refSeries\n",
    "\n",
    "# The annotation path has to be a file name (not directory name).  Since there is generally\n",
    "# only one file in a segmentation series we can assume it will always be called 1-1.dcm\n",
    "segPath = \"tciaDownload/\" + segSeries + \"/1-1.dcm\"\n",
    "\n",
    "# Display the viewer\n",
    "dicomViewer.viewSeriesAnnotation(imgPath, segPath)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ntPetyDFNaew",
   "metadata": {
    "id": "ntPetyDFNaew"
   },
   "source": [
    "# 4 Download and visualize a sample DICOM RTSTRUCT\n",
    "RTSTRUCT is another common format used to save segmentations.  Let's take a look at the [Annotations for The Clinical Proteomic Tumor Analysis Consortium Pancreatic Ductal Adenocarcinoma Collection (CPTAC-PDA-Tumor-Annotations) dataset](https://doi.org/10.7937/BW9V-BX61) as an example.  This [Analysis Result](https://www.cancerimagingarchive.net/tcia-analysis-results/) dataset analyzed images from the [CPTAC-PDA](https://doi.org/10.7937/K9/TCIA.2018.SC20FO18) collection.\n",
    "\n",
    "This time around, let's use the **modality** parameter in getSeries() to only return the RTSTRUCT series."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7TUxwnmYNYsK",
   "metadata": {
    "id": "7TUxwnmYNYsK"
   },
   "outputs": [],
   "source": [
    "df = nbia.getSeries(collection = \"CPTAC-PDA\", modality = \"RTSTRUCT\", format = \"df\")\n",
    "display(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "HnLV3G7mNYsK",
   "metadata": {
    "id": "HnLV3G7mNYsK"
   },
   "source": [
    "If you look at the Series Description column you'll note that in some cases these RTSTRUCT series are listed as \"seed point\" or \"no finding\".  These ones would not be particularly useful to visualize so let's make sure to avoid those. You can update the code below to use any of the other series UIDs you prefer, but let's start with **1.2.826.0.1.534147.667.2747872357.2023429821032.4** which has a description of **\"Pre-dose, PANCREAS - 1\"**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "vettaObhNYsL",
   "metadata": {
    "id": "vettaObhNYsL"
   },
   "outputs": [],
   "source": [
    "segSeries = \"1.2.826.0.1.534147.667.2747872357.2023429821032.4\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1v-GBguFNYsL",
   "metadata": {
    "id": "1v-GBguFNYsL"
   },
   "source": [
    "Next, let's determine the Reference Series Instance UID of the CT scan that goes with the segmentation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "sbDQNm4bNYsL",
   "metadata": {
    "id": "sbDQNm4bNYsL"
   },
   "outputs": [],
   "source": [
    "refSeries = nbia.getSegRefSeries(segSeries)\n",
    "\n",
    "print(refSeries)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "QeftUEEFNYsL",
   "metadata": {
    "id": "QeftUEEFNYsL"
   },
   "source": [
    "Now let's download these two series.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "iT7n0ahDNYsL",
   "metadata": {
    "id": "iT7n0ahDNYsL"
   },
   "outputs": [],
   "source": [
    "nbia.downloadSeries([refSeries, segSeries], input_type= \"list\", format = \"df\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "SqRhk5xCNYsL",
   "metadata": {
    "id": "SqRhk5xCNYsL"
   },
   "source": [
    "Finally, we can look at the images and segmentation together.  You can move the slider to flip through the images and toggle the segmentation layer on/off."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "qZmqZmcgNYsL",
   "metadata": {
    "id": "qZmqZmcgNYsL"
   },
   "outputs": [],
   "source": [
    "# Assuming you didn't change the default download options for downloadSeries\n",
    "imgPath = \"tciaDownload/\" + refSeries\n",
    "\n",
    "# The annotation path has to be a file name (not directory name).  Since there is generally\n",
    "# only one file in a segmentation series we can assume it will always be called 1-1.dcm\n",
    "segPath = \"tciaDownload/\" + segSeries + \"/1-1.dcm\"\n",
    "\n",
    "# Display the viewer\n",
    "dicomViewer.viewSeriesAnnotation(imgPath, segPath)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b19fe9a1-1f0b-4125-bb77-2aed09020de1",
   "metadata": {},
   "source": [
    "# Acknowledgements\n",
    "[The Cancer Imaging Archive (TCIA)](https://www.cancerimagingarchive.net/) is a service which de-identifies and hosts a large publicly available archive of medical images of cancer.  TCIA is funded by the [Cancer Imaging Program (CIP)](https://imaging.cancer.gov/), a part of the United States [National Cancer Institute (NCI)](https://www.cancer.gov/), and is managed by the [Frederick National Laboratory for Cancer Research (FNLCR)](https://frederick.cancer.gov/).\n",
    "\n",
    "This notebook was created by [Justin Kirby](https://www.linkedin.com/in/justinkirby82/).  If you leverage TCIA datasets in your work please be sure to comply with the [TCIA Data Usage Policy](https://wiki.cancerimagingarchive.net/x/c4hF) and include all relevant citations.\n",
    "\n",
    "## TCIA Citation\n",
    "\n",
    "Clark, K., Vendt, B., Smith, K., Freymann, J., Kirby, J., Koppel, P., Moore, S., Phillips, S., Maffitt, D., Pringle, M., Tarbox, L., & Prior, F. (2013). The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository. Journal of Digital Imaging, 26(6), 1045–1057. https://doi.org/10.1007/s10278-013-9622-7\n",
    "\n",
    "## Dataset Citations\n",
    "\n",
    "Please cite the datasets as shown below if you use them in your work:\n",
    "\n",
    "1. Heller, N., Sathianathen, N., Kalapara, A., Walczak, E., Moore, K., Kaluzniak, H., Rosenberg, J., Blake, P., Rengel, Z., Oestreich, M., Dean, J., Tradewell, M., Shah, A., Tejpaul, R., Edgerton, Z., Peterson, M., Raza, S., Regmi, S., Papanikolopoulos, N., Weight, C.  (2019) Data from C4KC-KiTS  [Data set]. The Cancer Imaging Archive. DOI: 10.7937/TCIA.2019.IX49E8NX\n",
    "2. National Cancer Institute Clinical Proteomic Tumor Analysis Consortium (CPTAC). (2018). The Clinical Proteomic Tumor Analysis Consortium Pancreatic Ductal Adenocarcinoma Collection (CPTAC-PDA) (Version 14) [Data set]. The Cancer Imaging Archive. https://doi.org/10.7937/K9/TCIA.2018.SC20FO18\n",
    "3. Rozenfeld, M., & Jordan, P. (2023). Annotations for The Clinical Proteomic Tumor Analysis Consortium Pancreatic Ductal Adenocarcinoma Collection (CPTAC-PDA-Tumor-Annotations) (Version 1) [Data set]. The Cancer Imaging Archive. https://doi.org/10.7937/BW9V-BX61"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}