validation -> extract info

README.md

@@ -11,8 +11,9 @@ Validation of weak-lensing catalogues (galaxy and star shapes and other paramete
 | [![coc](https://img.shields.io/badge/conduct-read-lightgrey)](https://github.com/martin.kilbinger/sp_validation/blob/master/CODE_OF_CONDUCT.md) | [![Updates](https://pyup.io/repos/github/martin.kilbinger/sp_validation/shield.svg)](https://pyup.io/repos/github/martin.kilbinger/sp_validation/) | |
 
 ---
-> Authors: <a href="www.cosmostat.org" target="_blank" style="text-decoration:none; color: #F08080">CosmoStat</a> lab at CEA Paris-Saclay, including:
-  Axel Guinot, Martin Kilbinger, Lucie Baumont, Sacha Guerrini, Fabian Hervas Peters, Samuel Farrens, Emma Ayçoberry.</a>
+> Authors: <a href="www.cosmostat.org" target="_blank" style="text-decoration:none; color: #F08080">CosmoStat</a> lab at CEA Paris-Saclay;
+  Martin Kilbinger, Axel Guinot.
+> Contributors: Emma Ayçoberry, Lucie Baumont, Clara Bonini, Cale Daley, Samuel Farrens, Sacha Guerrini, Fabian Hervas Peters.</a>
 > Email: <a href="mailto:[email protected]" style="text-decoration:none; color: #F08080">[email protected]</a>  
 ---
 
@@ -33,9 +34,17 @@ tasks that can be performed by `sp_validation` are:
   correlation function.
 
 
+## Flow chart
+
+The following flow chart illustrates the steps required to go from ShapePipe output products
+to calibrated and well-selected galaxy catalogues.
+
+![Flow chart](docs/images/flow_chart.png)
+
+
 ## Run shear validation
 
-See the [documentation](docs/source/run_validation.md) for instructions how to set up and run `sp_validation`.
+See the [documentation](docs/source/run_validation.md) for instructions on how to set up and run `sp_validation`.
 
 
 ## Post processing

docs/source/run_validation.md

@@ -1,6 +1,11 @@
-## Shear validation
+## Extract shear information
 
-### Set up a shear validation run
+(<= SP v1.4.1: Formerly known as shapepipe validation.)
+
+This step extracts the relevant information from the merged final_cat hdf5 catalogue (<= v1.4.1: fits)
+including the sheared values for metacalibration.
+
+### Set up
 
 All inputs and settings are contained in the python configuration script
 `notebooks/params.py`, that needs to be edited accordingly.
@@ -24,46 +29,6 @@ See the script `prepare_patch_for_spval.sh` for an example of copying
 the required input files to where the validation is to be run.
 
 
-### Run shear validation
-
-There are two possibilities to carry out the shear validation, by running the jupyter notebooks
-in `jupyter-lab`, or by running a python script in the command line via `ipython`.  
-
-#### Running the jupyter notebooks                                              
-
-1. In the run directory start JupyterLab:                                       
-  ```bash                                                                       
-  jupyer-lab                                                                    
-  ```                                                                           
-
-2. Load and run first notebook `main_set_up.ipynb`:                             
-   1. Double-click in file manager tab on the left.                             
-   2. Run notebook.                                                             
-
-3. Run all further notebooks:                                                   
-   1. Double-click as above.                                                    
-   2. Change kernel to `main_set_up.ipynb`:                                     
-      Either via the menu `Kernel -> Change Kernel` or                          
-      by clicking on `Python 3` on the top left of the notebook.                
-   3. Run notebook.                                                             
-
-Run the notebooks in the following order:                                       
-   1. `main_set_up.ipynb` (main notebook)                                       
-   2. `metacal_global.ipynb`                                                    
-   3. [`metacal_local.ipynb`] optional                                          
-   4. `psf_leakage.ipynb`                                                       
-   5. `write_cat.ipynb`                                                         
-   6. `maps.ipynb`                                                              
-   7. `cosmology.ipynb`                                                         
-
-#### Running the python script                                                  
-
-1. Create the python script from the jupyter notbooks. In `notebooks`:          
-  ```bash                                                                       
-  python config_convert.py                                                      
-  ```                                                                           
-
-2. Run python script. In run directory:                                         
-  ```bash                                                                       
-   ipython /path/to/sp_validation/notebooks/validation.py                       
-   ```
+### Run
+
+Run the jupyter notebook `extract_information.ipynb` or python script `extract_information.py`.