SMIT CT Lung GTV segmentation model #108

locastre · 2025-03-17T15:10:41Z

Submitting the SMIT CT Lung GTV segmentation model for mHub

LennyN95

Thank you for your contribution 🚀

Can we avoid using conda and instead install all requirements with uv? Our base image comes with uv installed and a virtual environment set-up. However, it is suggested that you create your own virtual environment with uv, e.g., uv venv -p 3.10 .venv310
The contents of the meta.json are used to populate the model card on our website under mhub.ai/models. The more information, the better.
To test-build the model and to move forward with our test routine, an mhub.toml file needs to be created.

LennyN95 · 2025-03-18T16:30:55Z

models/msk_smit_lung_gtv/dockerfiles/Dockerfile

+
+WORKDIR ${WORK_DIR}/msk_smit_lung_gtv/src
+ENV WEIGHTS_URL=https://mskcc.box.com/shared/static/sf7jic4m2dk67413cipbbq6hddvhpj61.gz
+ENV CONDA_URL=https://mskcc.box.com/shared/static/d580gfjzzmt26v8klwp8pivb6wafomag.gz


Is this using a conda environment for dependencies?

We should avoid conda in open source projects due to it's licensing scheme.

I suggest we install all dependencies with uv (which comes with our base image), e.g., you could provide a pyproject.toml to make src installable or just provide a list with dependencies (like a requirements.txt) and install with uv pip install.

LennyN95 · 2025-03-18T16:31:36Z

models/msk_smit_lung_gtv/meta.json

+{
+    "id": "", 
+    "name": "msk_smit_lung_gtv", 
+    "title": "CT Lung GTV SMIT Segmentation",


What does SMIT stand for :)?

LennyN95 · 2025-03-18T16:31:52Z

models/msk_smit_lung_gtv/meta.json

+    "name": "msk_smit_lung_gtv", 
+    "title": "CT Lung GTV SMIT Segmentation",
+    "summary": {
+    "description": "GTV segmentation from CT scan",


Could be slightly longer, this is what the user sees on the website.

LennyN95 · 2025-03-18T16:32:37Z

models/msk_smit_lung_gtv/meta.json

+            "description": "Segmentation of the lung GTV from NIfTI CT images.",
+            "type": "Segmentation",                       
+            "classes": [  
+                "GTV"


See comment in SMITrunner.py.

LennyN95 · 2025-03-18T16:33:41Z

models/msk_smit_lung_gtv/meta.json

+        "version": "1.0.0", 
+        "devteam": "",  
+        "authors": ["Jue Jiang, Harini Veeraraghavan"],      
+        "type": "it is a 3D Swin transformer based segmentation net",   


Just Segmentation?

LennyN95 · 2025-03-18T16:40:28Z

models/msk_smit_lung_gtv/utils/SMITrunner.py

+
+    @IO.Instance()
+    @IO.Input('scan', 'nifti:mod=ct',  the='input ct scan')
+    @IO.Output('gtv_mask', 'gtv_mask.nii.gz', 'nifti:mod=seg:model=SMIT:roi=GTV',data='scan', the='predicted lung gtv')


The segmentation ID GTV does not exist in SegDB. You can find an overview of available segmentations here.

Note, that a tumor is usually described in the context of a parent structure, e.g., segmentation ID LUNG, LEFT_LUNG, etc. A potential fit could be LUNG+NEOPLASM_MALIGNANT_PRIMARY.

LennyN95 · 2025-03-18T16:43:28Z

models/msk_smit_lung_gtv/utils/SMITrunner.py

+    @IO.Instance()
+    @IO.Input('scan', 'nifti:mod=ct',  the='input ct scan')
+    @IO.Output('gtv_mask', 'gtv_mask.nii.gz', 'nifti:mod=seg:model=SMIT:roi=GTV',data='scan', the='predicted lung gtv')
+    def task(self, instance: Instance, scan: InstanceData, gtv_mask: InstanceData) -> None:


The input ct image is available under scan.abspath, the segmentation mask generated by the model should be written or copied into gtv_mask.abspath.

LennyN95 · 2025-03-18T16:44:14Z

models/msk_smit_lung_gtv/utils/SMITrunner.py

+    @IO.Output('gtv_mask', 'gtv_mask.nii.gz', 'nifti:mod=seg:model=SMIT:roi=GTV',data='scan', the='predicted lung gtv')
+    def task(self, instance: Instance, scan: InstanceData, gtv_mask: InstanceData) -> None:
+
+        workDir = os.path.join(os.environ['WORK_DIR'],'models','msk_smit_lung_gtv','src')   # Needs to be defined in docker file as ENV WORK_DIR=path_to_dir e.g. /app/models/SMIT/workDir


Could we use pathlib for all path concatenations to increase readability?

LennyN95 · 2025-03-18T16:46:25Z

models/msk_smit_lung_gtv/utils/SMITrunner.py

+        #condaEnvDir = os.path.join(wrapperInstallDir,'conda-pack')
+        #condaEnvActivateScript = os.path.join(condaEnvDir, 'bin', 'activate')
+        wrapperPath = os.path.join(workDir,'bash_run_SMIT_Segmentation.sh')
+        load_weight_name = os.path.join(workDir,'trained_weights','model.pt')


Are there multiple weights trained for this model architecture? Otherwise, I'd suggest we make this parameter fixed, to increase the readability.

LennyN95 · 2025-03-18T16:47:20Z

models/msk_smit_lung_gtv/meta.json

+        }
+    ],
+    "model": {
+        "architecture": "Swin Transformer based segmentation, self-supervised pretrained with 10k CT data", 


Just Swin3D?

locastre · 2025-03-18T16:59:36Z

@LennyN95 Thank you for the in-depth feedback! We're working to address the suggestions.

I have an additional question: when we upload our test data to Zenodo (for the mhub.toml), should we refer to the mHub DOI# 13785615 or create a new DOI from an independent/new entry?

fedorov · 2025-03-18T20:04:54Z

I would recommend renaming the PR to "SMIT CT Lung GTV segmentation model" (or something similar).

LennyN95 · 2025-03-19T15:24:40Z

You're more than welcome. Thank you and your team for the great work.

I have an additional question: when we upload our test data to Zenodo (for the mhub.toml), should we refer to the mHub DOI# 13785615 or create a new DOI from an independent/new entry?

@locastre We mainly use Zenodo for it's reproducible storage mechanism, so you can create a new DOI for your sample & reference.

LennyN95 · 2025-03-19T15:28:29Z

@locastre FYI; There are also some errors in the compliance check that need to be resolved:

Errors:

Model folder models/msk_smit_lung_gtv does not contain a mhub.toml
https://github.com/MHubAI/documentation/blob/main/documentation/mhub_models/model_folder_structure.md
https://github.com/MHubAI/documentation/blob/main/documentation/mhub_contribution/testing_phase.md#test-procedure

Model meta json is not compliant with the schema: 'non-contrast' is a required property
https://github.com/MHubAI/documentation/blob/main/documentation/mhub_models/model_json.md

WORKDIR must not be set to any other than /app as defined in our base image. WORKDIR ${WORK_DIR}/msk_smit_lung_gtv/src
https://github.com/MHubAI/documentation/blob/main/documentation/mhub_models/the_mhub_dockerfile.md

locastre added 20 commits March 10, 2025 10:57

Create Dockerfile for MSK SMIT Lung GTV

6ffb8e3

1st draft Dockerfile

90e36a1

Create README.md

4de5a52

draft SMITrunner.py

54f36b7

add src files

84ffd42

add utils

903d37c

Add files via upload

d38cfe4

Add files via upload

1802f70

Update Dockerfile

8aa9989

mkdir for conda-pack

ec99930

add default.yml

483c92a

add config.json

5aa9531

Rename config.json to meta.json

874da7e

Update meta.json

5ccb70c

Update meta.json

20b0960

Update meta.json

d5733f6

Update meta.json

d346ec2

Update meta.json

28a2adf

Update Dockerfile

c20d489

Update SMITrunner.py

523ea46

LennyN95 reviewed Mar 18, 2025

View reviewed changes

LennyN95 changed the title ~~New Model Submission~~ SMIT CT Lung GTV segmentation model Mar 19, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SMIT CT Lung GTV segmentation model #108

SMIT CT Lung GTV segmentation model #108

locastre commented Mar 17, 2025

LennyN95 left a comment

LennyN95 Mar 18, 2025

LennyN95 Mar 18, 2025

LennyN95 Mar 18, 2025

LennyN95 Mar 18, 2025

LennyN95 Mar 18, 2025

LennyN95 Mar 18, 2025

LennyN95 Mar 18, 2025

LennyN95 Mar 18, 2025

LennyN95 Mar 18, 2025

LennyN95 Mar 18, 2025

locastre commented Mar 18, 2025

fedorov commented Mar 18, 2025

LennyN95 commented Mar 19, 2025 •

edited

Loading

LennyN95 commented Mar 19, 2025

SMIT CT Lung GTV segmentation model #108

Are you sure you want to change the base?

SMIT CT Lung GTV segmentation model #108

Conversation

locastre commented Mar 17, 2025

LennyN95 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

locastre commented Mar 18, 2025

fedorov commented Mar 18, 2025

LennyN95 commented Mar 19, 2025 • edited Loading

LennyN95 commented Mar 19, 2025

LennyN95 commented Mar 19, 2025 •

edited

Loading