Demorun 2 Running small training and prediction with mito testsample dataset

Training with test dataset on Cloud

The following instructions use the mito_testsample dataset stored with this repository as an example on how to perform the entire process from training through prediction. Although this training dataset is too small to offer good segmentation, it is quick to process so one can try all the steps without long periods of waiting.

To just try prediction with a pre-trained mitochrondria model click here

Step 1) Connect via ssh to instance created by these instructions

ssh ubuntu@PublicDNS_VALUEFROM_WEBPAGE_ABOVE

Quick and Dirty run

Here are the commands to run once logged into the instance. Below in Steps 2+ are more detailed instructions on what is happening.

cd ~
PreprocessTrainingData.m ~/cdeep3m/mito_testsample/training/images/ ~/cdeep3m/mito_testsample/training/labels/ ~/mito_testaugtrain
runtraining.sh --numiterations 100 ~/mito_testaugtrain ~/train_out
PreprocessImageData.m ~/cdeep3m/mito_testsample/testset/ ~/mito_testaugimages
runprediction.sh ~/train_out/ ~/mito_testaugimages/ ~/predictout
EnsemblePredictions.m ~/predictout/1fm ~/predictout/3fm ~/predictout/5fm ~/predictout/ensembled
# results are in ~/predictout/ensembled
ls ~/predictout/ensembled

Step 2) Run PreprocessTrainingData.m

More information about PreprocessTrainingData.m can be found here

cd ~
PreprocessTrainingData.m ~/cdeep3m/mito_testsample/training/images/ ~/cdeep3m/mito_testsample/training/labels/ ~/mito_testaugtrain

Output:

octave: X11 DISPLAY environment variable not set
octave: disabling GUI features
Starting Training data Preprocessing
Training Image Path:
/home/ubuntu/cdeep3m/mito_testsample/training/images/
Training Label Path:
/home/ubuntu/cdeep3m/mito_testsample/training/labels/
Output Path:
/home/ubuntu/mito_testaugtrain
Loading:
/home/ubuntu/cdeep3m/mito_testsample/training/images/
Image importer loading ...
/home/ubuntu/cdeep3m/mito_testsample/training/images/
Reading file: /home/ubuntu/cdeep3m/mito_testsample/training/images/images.010.png
.
.
/home/ubuntu/cdeep3m/mito_testsample/training/labels/
Image importer loading ...
/home/ubuntu/cdeep3m/mito_testsample/training/labels/
Reading file: /home/ubuntu/cdeep3m/mito_testsample/training/labels/mitos_3D.010.png
.
.
Create variation 8 and 16
Saving: /home/ubuntu/mito_testaugtrain/training_full_stacks_v8.h5
Elapsed time is 15.1134 seconds.
-> Training data augmentation completed
Training data stored in /home/ubuntu/mito_testaugtrain
For training your model please run CreateTrainJob.m /home/ubuntu/mito_testaugtrain <desired output directory>

Step 3) Run runtraining.sh

Run runtraining.sh to train CDeep3M and created trained models.

runtraining.sh --numiterations 100 ~/mito_testaugtrain ~/train_out

Output:

octave: X11 DISPLAY environment variable not set
octave: disabling GUI features
Verifying input training data is valid ... success
Copying over model files and creating run scripts ... success

A new directory has been created: /home/ubuntu/train_out
In this directory are 3 directories 1fm,3fm,5fm which
correspond to 3 caffe models that need to be trainedas well as two scripts:

caffe_train.sh -- Runs caffe for a single model
run_all_train.sh -- Runs caffe_train.sh serially for all 3 models

Running 1fm train, this could take a while
real 540.15
user 460.86
sys 79.90
Running 3fm train, this could take a while
real 181.50
user 166.45
sys 30.35
Running 5fm train, this could take a while
real 112.99
user 102.26
sys 20.05
Training has completed. Have a nice day!
Training has completed. Results are stored in /home/ubuntu/train_out
Have a nice day!

Step 4) Run PreprocessImageData.m

More information about PreprocessImageData.m can be found here

PreprocessImageData.m ~/cdeep3m/mito_testsample/testset/ ~/mito_testaugimages

Output:

octave: X11 DISPLAY environment variable not set
octave: disabling GUI features
Starting Image Augmentation
Check image size of:
/home/ubuntu/cdeep3m/mito_testsample/testset/
Reading file: /home/ubuntu/cdeep3m/mito_testsample/testset/images.040.png
warning: your version of GraphicsMagick limits images to 16 bits per pixel
Reading file: /home/ubuntu/cdeep3m/mito_testsample/testset/images.044.png
Padding images
.
.
.
Saving: /home/ubuntu/mito_testaugimages/Pkg001_Z01/test_data_full_stacks_v16.h5
Elapsed time is 1.75296 seconds.
Image Augmentation completed
Created 1 packages in x/y with 1 z-stacks
Data stored in:
 /home/ubuntu/mito_testaugimages

Step 5) Run runprediction.sh

runprediction.sh ~/train_out/ ~/mito_testaugimages/ ~/predictout

Output:

octave: X11 DISPLAY environment variable not set
octave: disabling GUI features
Verifying input training data is valid ... skipping check, TODO need to fix this.
Verifying image data and getting Pkg folders ... skipping check, TODO need to fix this.
Creating output directories and creating run scripts ... success

A new directory has been created: /home/ubuntu/predictout
In this directory are 3 directories 1fm,3fm,5fm which
will contain the results from running prediction with caffeThere are also two scripts:

caffe_predict.sh -- Runs caffe prediction single model
run_all_predict.sh -- Runs caffe_predict.sh serially for all 3 models

To run prediction for all 3 models run this: /home/ubuntu/predictout/run_all_predict.sh


Running Prediction

Trained Model Dir: /home/ubuntu/train_out/
Image Dir: /home/ubuntu/mito_testaugimages/

Running 1fm predict 1 package(s) to process

Step 6) Run EnsemblePredictions.m

Merges segmentations from 1fm, 3fm, and 5fm models. Final segmentations will be put in directory specified by last argument to EnsemblePredictions.m in case below its ~/predictout/ensembled which will contain a set of png files.

EnsemblePredictions.m ~/predictout/1fm ~/predictout/3fm ~/predictout/5fm ~/predictout/ensembled