Tutorial 4 Post processing of Automatically Generated Segmentations

This page provides instructions on how to create and quantify 3D models from CDeep3M results created in:

Tutorial 3: Run CDeep3M

Goals

1. Convert output of CDeep3M results to 3D models
2. Apply filters to the automatic output to improve its accuracy
3. Obtain quantitative metrics (volume, surface area, centroid) of segmented objects

Requirements

• Running CDeep3M instance from Tutorial 2: Launch and train CDeep3M
• Command line terminal (on Windows the Cygwin Terminal that is installed with IMOD will work)
• ssh client (on Windows the Cygwin Terminal that is installed with IMOD will work)
• IMOD If not done earlier install instructions can be found here

Step 1 Download dataset four

A. Download the ZIP file entitled datasetfour.zip from:

https://github.com/CRBS/cdeep3m/wiki/data/datasetfour.zip

B. Unzip the contents to an easily accessible location:

• For Windows 7/8/10: Unzip the file into a new folder in your home directory. If you used the IMOD cygwin install, the home directory will be C:\Users\<username> (for windows 10 it may be C:\cygwin\home\<username>)

From the cygwin terminal this can be done with this command ($HOMEPATH is an environment variable in Windows set to the user's home directory):

unzip $HOMEPATH/Downloads/datasetfour.zip

• For Mac: Unzip the file into a new folder (e.g. /Users/<username>/datasetfour).

• For Linux: Unzip the file into a new folder (e.g. /home/<username>/datasetfour).

C. If not already open already, open your terminal program

D. Navigate to the directory to which you unzipped the dataset, using the cd command in the terminal:

cd ~/datasetfour

E. List the directory's contents using the ls command. you should see one file, sbem_volume.mrc and one directory, mitochondria

NOTE: If you want to use the mitochondria directory generated from tutorial 3 simply rename the existing mitochondria directory under datasetfour and copy or move the mitochondria directory from datasetthree directory.

Step 2 Convert Mitochondrial Probability Maps to MRC

A. Breakdown of contents of datasetfour directory:

• sbem_volume.mrc, an MRC stack that has 80 slices
• mitochondria, a directory which contains the CDeep3M probability maps for mitochondria

B. The probability maps have already been converted from PNG to TIF. As of version 4.7 most IMOD programs can process TIF images in the same manner as MRC files. Therefore, we can directly append the TIF files to a single MRC stack. Use the following commands:

cd mitochondria
newstack *.tif pm_mito.mrc

This will append all of the newly made TIF files, in alphanumerical order, to a new MRC stack named in the above case pm_mito.mrc

Step 3 Alter the Header Info of Converted MRC Stack

A. View the header information of the raw image stack:

header ../sbem_volume.mrc

Note that the Pixel Spacing entry is "59 59 400" and the Origin entry is "0 0 0". While the origin is the same as that of the new created pm_mito.mrc, the pixel spacing is not(it is set to "1 1 1" by default). We will correct this using the IMOD program alterheader

B. At the command line, enter the following:

alterheader -del "59 59 400" pm_mito.mrc

Step 4 Generate Contours from Probability Map Stack

A. For this tutorial, we will apply a single-level threshold to the probability maps and generate contours around the output. The IMOD program imodauto is used to do this. The following imodauto arguments are useful to use and understand for this purpose:

• -h Pixel intensity value for the high threshold. Contours will be drawn about all the pixels above this value. Since the data are 8-bit, values ranging from 0 to 255 are valid.
• -u Uses unscaled pixel intensity values for the threshold cutoff. Highly recommended.
• -R Proportion of points to remove from the final contours. This can have the effects of producing smoother contours and reducing file size. Useful values typically range from 0.25 - 0.5.
• -k Smooths the probability map stack with a kernel filter whose Gaussian sigma is given by the entered value.

For these data, a good segmentation can be achieved using a 50% cutoff for pixel intensity (= 128).

Run the following command which will create an MRC model file named seg_mito.mod:

imodauto -h 128 -u pm_mito.mrc seg_mito.mod

B. Though the above command will yield good output, it is worthwhile to take some time experimenting with different combinations of arguments and visualizing how they affect the output. For example, increasing the high threshold value will reduce false positives, but may also increase false negatives by being too stringent. Similarly, check how the -R and -k arguments change the output.

Step 5 Create 3D Meshes from Contours

A. imodauto outputs all contours as part of one, massive object. This is not desirable, ideally each spatially separated mitochondrion to be its own object. To achieve this, a mesh for the model file needs to be generated. This is done using the program imodmesh There are man optional arguments for imodmesh, and it is a good idea to review them on the program's man page. FOr now, let's mesh the contours using the following command:

imodmesh -CTs -P 3 seg_mito.mod seg_mito.mod

The most important options to note here are the -s and -P arguments. The combination of both will cause meshes to be made across slices of the object that have missing data. An object containing a slice with missing data is common, and may occur due to artifacts of the imaging process. The integer value entered after the -P flag tells imodmesh to skip up to three sections when forming meshes. If this value is too large, there is a risk of erroneously connecting spatially separated objects that shouldn't be joined together (false merge error).

Step 6 Split the Model into Spatially Separated Objects

A. Once the single object has mesh information, it can be split into separate objects based on 3D connectivity by the imodsortsurf command. Let's run imodsortsurf on the data, using the -s argument to create new objects:

imodsortsurf -s seg_mito.mod seg_mito_sort.mod

B. Once the process has completed, the split objects can be visualized using 3dmod. But first, the objects need to be re-meshed to update the mesh info to reflect the colors of the new, sorted objects:

imodmesh -CTs -P 3 seg_mito_sort.mod seg_mito_sort.mod

C. Open the model file in 3dmod:

3dmod seg_mito_sort.mod

D. Make sure the header information for the model file is set properly. From the 3dmod window, navigate to Edit -> Model -> Header. Set the 'Pixel Size' and 'Total Z-scale' to correspond with the values of the MRC stack entered when alterheader command was run. These values should be (yes type nm) :

Pixel Size = 5.9 nm
Total Z-scale = 6.78    (which is equal to 400/59, the ratio of Z:XY pixel sizes)

Step 7 Filter the Objects Based on Morphology

Using the suggested parameters, the above step slit the model into X putative mitochondria. Some of these objects are false positives, and can be removed by applying some simple morphological filters. As a starting point, we'll remove all objects that have less then a certain number of contours.

Doing this through the IMOD GUI is very tedious, and IMOD doesn't have any command line programs to automate this process. We will use a set of Python classes to automate this filtering on the CDeep3M instance spun up earlier. First we need to copy our model file to the CDeep3M instance:

A. Copy the file to the CDeep3M instance:

scp -i ~/.ssh/id_rsa seg_mito_sort.mod ubuntu@<PUBLIC DNS>:/home/ubuntu/.

Replace "<PUBLIC DNS>" with....

B. Connect to CDeep3M instance

ssh -i ~/.ssh/id_rsa ubuntu@<PUBLIC DNS>

C. Clone the [PyIMOD][pyimod] repository from Github:

git clone https://github.com/CRBS/PyIMOD.git

D. Start Python, import PyIMOD, and perform some basic filtering of the model file:

python
import PyIMOD
mod = PyIMOD.ImodModel('seg_mito_sort.mod')
mod.filterByNContours('>',3)
PyIMOD.ImodWrite(mod, 'seg_mito_sort_filter.mod')
exit()

Starting with the 'import PyIMOD' line, this will import PyIMOD module you cloned. Next, the model file you copied to the CDeep3M instance is read into an ImodModel Python object. The next line performs the filtering. This particular commadn will keep all objects taht have greater then 3 contours, and remove all objects that have less then or equal to 3 contours. This is a very quick and dirty way to filter out potential false positives. Finally, the filtered model file is written to a new file under a different name, and then Python is exited.

E. Exit from Cdeep3M instance and copy the filtered model file back to your computer. Be sure to include a space and the period character at the end of the scp command below:

scp -i ~/.ssh/id_rsa ubuntu@<PUBLIC DNS>:/home/ubuntu/seg_mito_sort_filter.mod .

F. Re-mesh the model file:

imodmesh -CTs -P 3 seg_mito_sort_filter.mod seg_mito_sort_filter.mod

G. Open the model file using the 3D model viewer. Note the differences between this model and the unfiltered version.

NOTE: If you have some extra time at the end of the demo, you can try experimenting with some of the different filters in PyIMOD. Instance of filtering by number of contours, you can, for example, filter by object volume using PyIMOD.filterByVolume

To remove all objects with volumes less then 0.01 um^3, for example, you would use:

PyIMOD.filterByVolume('>=', 0.01)

Step 8 Quantifying the Results Using IMOD

Step 9 Generate Contours using thresholding

Step 10 Manual Proofreading/Correction of the Results

End

Congratulations you have finished the tutorials!!!