SciCatProject · rkweehinzmann · Jun 27, 2025 · Jul 2, 2025 · Jul 2, 2025 · Jul 2, 2025
diff --git a/.github/mkdocs/mkdocs.yaml b/.github/mkdocs/mkdocs.yaml
@@ -5,7 +5,7 @@ docs_dir: ../../docs
 nav: 
   - Home: index.md
   - SciCat User Guide:  
-    - user-manual/index.md
+    - user-guide/index.md
     - Login:
       - login/index.md
       - Anonymous: login/Anonymous.md
@@ -14,15 +14,16 @@ nav:
       - datasets/index.md
       - Publishing data: datasets/Publishing.md
       - Publishing data Advanced: datasets/PublishingAdvanced.md
+      - Data Retrieval: datasets/jobs.md 
     - Proposals: 
       - proposals/index.md
     - Samples: samples/index.md
-    - Instruments: instruments.md
+    - Instruments: instruments/index.md
     - Troubleshooting:
       - troubleshoot/index.md
 
   - SciCat Operator Guide: 
-    - operator-manual/index.md
+    - operator-guide/index.md
     - swagger/index.md
     - backendconfig/index.md
     - backendconfig/authorization/index.md

diff --git a/docs/backendconfig/authorization/authorization_datasets.md b/docs/backendconfig/authorization/authorization_datasets.md
@@ -1,4 +1,4 @@
-# Datasets Authoorization
+# Datasets Authorization
 ## CASL ability actions
 This is the list of the permissions methods available for datasets and all their endpoints and more fine-grained instance authorization. 
 

diff --git a/docs/backendconfig/authorization/index.md b/docs/backendconfig/authorization/index.md
@@ -2,43 +2,30 @@
 
 ## General 
 For how authorization is handled in SciCat, see [general](./authorization.md) description.
-
 Developers information, see [github](https://github.com/SciCatProject/documentation/blob/master/Development/v4.x/backend/authorization/authorization.md).
 
-
 ## Authorization Datasets
-Summary of how authorization for Datasets are handeld [better visible](./authorization_datasets.md) displayed.
 
 Developers information, see [github](https://github.com/SciCatProject/documentation/blob/master/Development/v4.x/backend/authorization/authorization_datasets.md).
 
 
 ## Authorization OrigDatablocks
 
-Summary of how authorization for OrigDatablocks are handeld [better visible](./authorization_origdatablocks.md) displayed.
-
 Developers information, see [github](https://github.com/SciCatProject/documentation/blob/master/Development/v4.x/backend/authorization/authorization_origdatablocks.md).
 
 ## Authorization Jobs
 
-Summary of how authorization for OrigDatablocks are handeld [better visible](./authorization_jobs.md) displayed.
-
 Developers information, see [github](https://github.com/SciCatProject/documentation/blob/master/Development/v4.x/backend/authorization/authorization_jobs.md).
 
 ## Authorization Users
 
-Summary of how authorization for Users are handeld [better visible](./authorization_users.md) displayed.
-
 Developers information, see [github](https://github.com/SciCatProject/documentation/blob/master/Development/v4.x/backend/authorization/authorization_users.md).
 
 ## Authorization Proposals
 
-Summary of how authorization for proposals are handeld [better visible](./authorization_proposals.md) displayed.
-
 Developers information, see [github](https://github.com/SciCatProject/documentation/blob/master/Development/v4.x/backend/authorization/authorization_proposals.md).
 
 ## Authorization Intruments
 
-Summary of how authorization for instruments are handeld [better visible](./authorization_instruments.md) displayed.
-
 Developers information, see [github](https://github.com/SciCatProject/documentation/blob/master/Development/v4.x/backend/authorization/authorization_instruments.md).
 
diff --git a/docs/datasets/PublishingAdvanced.md b/docs/datasets/PublishingAdvanced.md
@@ -4,13 +4,16 @@ The previously described options to [publish datasets](Publishing.md) in SciCat
 
 ## Implementation workflow target
 
-This diagram shows the essential steps in the workflow to be implemented. Please note, that SciCat datasets are *always* only *meta datasets*, SciCat has no direct to the storage system there is no default coupling to such systems.
+Please note only metadata is stored in SciCat, there is no direct coupling between the software and the storage system. If you wish to publish both the metadata and data, please speak to your operator or consult the developer documentation for examples.
 
 ### 1. Create a list of selected datasets
-User can select datasets to create a **dataset list**; more datasets can be added and removed in several sessions. He can cancel the process at any time. New will be that while examining single datasets he can directly add or remove them to or from the selection in the cart.
+You can select datasets to create a **dataset list**; more datasets can be added and removed in several sessions. You can cancel the process at any time. New will be that while examining single datasets he can directly add or remove them to or from the selection in the cart. Before proceeding, you will be asked to verify the selection of datasets. You as user have finalized the dataset selection for which you want to register the dataset with a DOI.
+
+##### Internal review (to be implemented)
+Some institutions may introduce an internal review step at this point: other authenticated user (as part of a dedicated reviewer-group) review the selected datasets. If OK, proceed to next form and the initial you can continue the DOI minting process.
 
 ### 2. Fill the form for this dataset selection 
-The user will be forwarded to a form where one can **provide metadata specific to this selection** which can e.g. match site specific information about e.g. grants, associated projects, etc. All selected datasets will be made public. You will be asked to verify the selection of datasets. Owners and Admins are allowed to update this form. Again this shall be possible within several sessions.
+You will be forwarded to a form where you provide **metadata specific to this selection** already conform to DataCite metadata fields to match site specific information about e.g. grants, associated projects, etc. All selected datasets will be made public. Owners (and Admins) are allowed to update this form. Again, this shall be possible within several sessions.
 
 ### 3. Publish the selection
 After hitting button all selected datasets become publicly visible: not only the owner can view all the metadata of the data, date of creation, associated files names, location, PI, etc. This is **prerequisite** for DOI registration.  

diff --git a/docs/datasets/img/dataset_details.png b/docs/datasets/img/dataset_details.png
diff --git a/docs/datasets/img/datasets_SearchBar.png b/docs/datasets/img/datasets_SearchBar.png
diff --git a/docs/datasets/img/datasets_detailedView.png b/docs/datasets/img/datasets_detailedView.png
diff --git a/docs/datasets/img/datasets_filterNConditions_1.png b/docs/datasets/img/datasets_filterNConditions_1.png
diff --git a/docs/datasets/img/datasets_filterNConditions_2.png b/docs/datasets/img/datasets_filterNConditions_2.png
diff --git a/docs/datasets/index.md b/docs/datasets/index.md
@@ -1,21 +1,44 @@
 # Datasets
-SciCat datasets are sets of meta data and can include several files which e.g. comprise a self-contained measurement - which is fully customizeable during ingestion of meta data. Users can search, view, list the meta data of a dataset. 
+SciCat datasets are sets of metadata and can include several files which e.g. comprise a self-contained measurement - which is fully customizeable during ingestion of metadata. Users can search and view different formats (e.g. in tree, tables or as JSON) of the dataset and list them. 
 
-To group and tag datasets is depicted [here](grouping_tagging_ds.md). Datasets can also be issued to be **published**: either removing the restricted view or triggering the process of obtaining a DOI for the selected datasets, see description of [publication of SciCat datasets](Publishing.md).
+## Features
+A very handy feature is to **group and tag** datasets. Find [here](grouping_tagging_ds.md) more details how to group them using tags. 
 
-### How to query datasets
+Datasets can also be selected for **several actions**:
+One such action is the *publication* of that selection. For more details see [publication of SciCat datasets](Publishing.md).
+Generally, actions depend on what is implemented at your site and can cover a wide range from
+comprising them into a *new datacollection of a custom type* [(see advanced documentation)](../datasets/datasetTypes.md) to
+using that selection of datasets to *run an analysis* on them.
 
+## How to search for datasets
+Datasets can be queried in several places: 
+
+* The search bar at the top of the page provides a quick free text search.
+* The filter & conditions column on the left allow you to customize your filters and conditions, adjust the filters to those that you find interesting and define your own conditions making use of your specific scientific metadata.
+
+The bar looks like this:
+![search bar](img/datasets_SearchBar.png)
+
+Filtering by conditions can be applied through the option box on the left.
+
+![filterColumn](img/datasets_filterNConditions_1.png)
+
+If you chose "More Filters" a pop-up window appears where you can chose which of the filters you want to display. You can also add your own conditions as well (visible in the background under conditions):
+![apply filters and conditions](../datasets/img/datasets_filterNConditions_2.png)
+
+## Dataset details
+The main tab shows the details of a dataset.
+
+![example of dataset details page](img/dataset_details_PSI.png)
 
 ## Dataset file listing
-Here is the view of files belonging to a dataset: Below the PID on the top, one finds the tab **Datafiles**:
+A dataset can have several associated files to it. They can be listed by clicking on the tab **Datafiles** just right to the Details tab:
 ![list](img/dataset_details_filelist.png)
 
 
-
 ## Dataset attachments
-What kind of attachement can be saved? Will they be searchable? Can also other formats be attached than pngs?
+Another tab is for the attachements of a dataset, e.g. PNG or TIFF images.
 
-On the dataset details page, you can click on the Attachments tab
 ![Choose an image file, must be under 16 MB limit](img/dataset_attachments_PSI.png)
 
 Simply follow the instructions to upload an image. The size is restricted to be below 16 MB.
@@ -30,13 +53,6 @@ Scientific meta data is shown in JSON under its section and looks like this:
 One can also get the JSON file via the swagger API. If set up, one can directly access the API endpoints of SciCat backend. Usually the address is in the form: ```my-scicat-instance.country/explorer```, swagger is accessible via the explorer. One needs to authenticate by copying the token from the GUI into the field **authorize**, then find the dataset of interest, by trying it out it will display you dataset and you can download it in JSON format.
 
 ## Edit Scientific meta data
-// WARNING: this is the old text! 
-If enabled, fields in the scientific metadata can be modified and edited by the owner of the data by hitting the "Edit" Icon. The user can add,remove or change metadata fields, every change will create a new record in the databse with it's history.
+If enabled, fields in the scientific metadata can be modified and edited by any member of the [Owner Group](../backendconfig/authorization/authorization_datasets.md) of the data by hitting the "Edit" Icon. The user can add, remove or change metadata fields, every change will create a new record in the databse with it's history [feature is soon available again from 2025-07-02].
 
 ![Image edit metadata](img/editMetadata.png)
-
-
-## New developments on dataset types
-Generalize datatypes to remove restrictions of ```raw``` and ```derived``` types (difference was a set of dataset properties).
-
-[datasetTypes](datasetTypes.md)
diff --git a/docs/datasets/jobs.md b/docs/datasets/jobs.md
@@ -0,0 +1,6 @@
+# Archival and retrieval of experimental data
+
+SciCat can be setup to interconnect to your local storage system which allows you to do the following:
+* Send datasets to your site's archive via a click of a button
+* Retrieve your datasets asynchronously via a button click.
+For more information please contact your site administrators. 
diff --git a/docs/index.md b/docs/index.md
@@ -1,4 +1,4 @@
 # Welcome to SciCat Documentation 
 
-Find [**SciCat USERS Guide**](user-manual/index.md) or [**SciCat Operator's Guide**](operator-manual/index.md).
-Developers can read long the ```READMEs``` in github of the [projects page](https://www.scicatproject.org) and in both guides as well.
+Find [**SciCat USERS Guide**](user-guide/index.md) or [**SciCat Operator's Guide**](operator-guide/index.md).
+Developers can read along the ```READMEs``` in github of the [projects page](https://www.scicatproject.org) and in both guides as well.
diff --git a/docs/instruments/index.md b/docs/instruments/index.md
@@ -0,0 +1,3 @@
+## Instruments
+
+Instruments contains the metadata of the different instruments available at the facility.
diff --git a/docs/login/Dashboard.md b/docs/login/Dashboard.md
@@ -1,21 +1,39 @@
 # Dashboard
 
-The dashboard is the first page that you see when you are logged in. When set to datasets as main access point, it will show an overview of all datasets that you have access to. <!--Soon it will be possible to obtain this overview page for the other objects such as proposals, samples, published data and instruments. This can be configured by your site-admin.-->
+SciCat's Dashboard or also sometimes called Landing Page is the first page that you see independent of whether you are logged in or not. If you want to login click *Sign in* on the top right button, see [here](./index.md) for more information. When set to datasets as main access point, it will show an overview of all datasets that you have access to. If you do not login you see those that are public. <!--Soon it will be possible to obtain this overview page for the other objects such as proposals, samples, published data and instruments. This can be configured by your site-admin.-->
 
 ![dashboard](img/dashboard.png)
 
+SciCat offers now new features for viewing metadata as you like with adjustable columns. 
+
+![dashboard_adjustableColumns](img/dashboard_adjustableColumns_rk.png)
+
+You can change the columns to be shown by chosing from the three right dots "Column setting" and select those you would like. You can also drag columns by hovering over dots that appear just next to the label, click and pull it where you want to place it, then release. 
+
+![dashboard_newFeatures](img/dashboard_optionsPerColumn.png)
+
+You can
+
+1. sort columns (click on the name and pull)
+2. adjust width of columns (left block of dots)
+3. remove or add columns (selection from Column settings)
+4. invert order of display (click arrow next to the name)
+5. apply a filter directly on that dataset with various options ("contains", "equals", "startsWith", "endsWith", "empty", "notEmpty") and either add (+), or (||) and exclude (x) another filter.
+
+![Overall Menu](img/dashboard_filterOnColumns.png)
+
 ## Menu access to different information pages
 
 You can always navigate to other parts of the application, simply by clicking on the user icon on the top right corner 
 
 ![Overall Menu](img/menu_dropdown.png)
 
 ## Finding Datasets
-SciCat provides several possibilities for finding the right datasets. One can use the tob search bar, one can narrow down your selection by applying filters and/or conditions and the user can search on his scientific metadata as well.
+SciCat provides several possibilities for finding the right datasets. You can use the top search bar, you can narrow down your selection by applying filters and/or conditions and the user can search on scientific metadata as well.
 
 ### Using Filters and Conditions
 
-On the left one can apply most common filters. Currently there are
+On the left you can apply most common filters. Currently there are
 
 1. Location: location of creation of the dataset.
 2. PID: Identifier of the dataset.
@@ -29,20 +47,11 @@ The text fields provide an auto completion, which becomes visible as you type.
 
 You can click on the date calendar to select the start date and a second to select end date. Make sure you select 2 dates.
 
-One can configure the selection of filters and add specific _conditions_. An example shows two additional conditions added:
+You can configure the selection of filters and add specific _conditions_. An example shows two additional conditions added:
 ![filters](./img/dashboard_filters.png)
 
-## Searching
-
-The text field at the top of the navigation bar allows you to search the metadata for any word contained in the metadata (but not arbitrary substrings). The search starts automatically when to start to type in this textfield, so better type fast ;-) 
-
-## Configure table columns
-
-The cog wheel symbol on the top right allows to define the columns, that you want to see in the table
-
 ## View Details
-
-To view a dataset simply click on it in the table and a more detailed view will load (this is covered in the next section)
+To view a dataset simply click on it in the table and a more detailed view will load (this is covered in the datasets section)
 
 
 

diff --git a/docs/login/img/dashboard.png b/docs/login/img/dashboard.png
diff --git a/docs/login/img/dashboard_adjustableColumns_rk.png b/docs/login/img/dashboard_adjustableColumns_rk.png
diff --git a/docs/login/img/dashboard_filterOnColumns.png b/docs/login/img/dashboard_filterOnColumns.png
diff --git a/docs/login/img/dashboard_optionsPerColumn.png b/docs/login/img/dashboard_optionsPerColumn.png
diff --git a/docs/login/index.md b/docs/login/index.md
@@ -2,17 +2,14 @@
 
 To get access to **all** the data, for which you have read access, you first have to login. Otherwise you can browse only public datasets, see [anonymous browsing](Anonymous.md).
 
-To login hit the "Sign in" Icon at the top right corner.
+To login hit the "Sign in" icon at the top right corner.
 
 ![Login to ESS](img/login.png)
 
-There are two types of account associated with the DataCatalog: *Functional* and *User*. A *functional* account will primarily be used by software and system administrators to deal with backups and other tasks.
-
-*User* accounts are tied into the login system that is used by your institution, for example: Active Directory. You are able to log in to the system using the same credentials you use on that account. This process is called *authenitication* in IT tech terminology
+User accounts are tied into the login system that is used by your institution, for example: Active Directory. You are able to log in to the system using the same credentials you use on that account. This process is called *authenitication* in IT tech terminology.
 
 When you login as a user your user management system will assign groups to the logged in user. Each dataset is also assigned to one such group (via the so called ownerGroup field), and you can view the datasets only, if you are member of the corresponding group. The logic that defines, what parts of the data you can see, is called "authorization" in IT terminology. The first page you'll see after login is the ["Dashboard"](Dashboard.md).
 
 Another  example login page from PSI is here
-
 ![Login to PSI](img/login-psi.png)
 
diff --git a/docs/operator-manual/img/DacatDataflowV3.png → docs/operator-guide/img/DacatDataflowV3.png b/docs/operator-manual/img/DacatDataflowV3.png → docs/operator-guide/img/DacatDataflowV3.png
diff --git a/docs/operator-manual/img/job-assembler.png → docs/operator-guide/img/job-assembler.png b/docs/operator-manual/img/job-assembler.png → docs/operator-guide/img/job-assembler.png
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,3 @@
		## Instruments

		Instruments contains the metadata of the different instruments available at the facility.