Skip to content

Updating the ESS-DIVE submission #270

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Updating the ESS-DIVE submission #270

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
fc9781d
Create ess-dive-submissions.md
stephpenn1 Mar 6, 2025
d3140ce
Remove header info
stephpenn1 Mar 6, 2025
ec7de12
Update check-workflow.yaml
bpbond Mar 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/check-workflow.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ jobs:
any::ggplot2
- name: Install other needed packages
run: |
pak::pkg_install("COMPASS-DOE/compasstools")
pak::pkg_install("COMPASS-DOE/compasstools@v0.2")
shell: Rscript {0}
- name: Run data workflow
run: |
Expand Down
38 changes: 38 additions & 0 deletions pipeline/docs/ess-dive-submissions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@

# Steps to update ESS-DIVE submissions

0. Run processing pipeline to produce L1 data for synoptic and TEMPEST sites

1. Move L1 data to folders labeled `vX-X` \> `vX-X_SYN` and `vX-X` \> `vX-X_TMP` . Each folder gets a copy of the `READMD.txt` and `README_vX-X.txt` files.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

README.md ?


2. Prep package metadata files

- Produce flmd using `flmd-generator.R`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it obvious how to do this? Do you need to point the script to where the L1 data are?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question, I think currently flmd-generator requires the files to be in the L1 folder (at least i havent tried otherwise) which would impact the steps here

- Copy over `dd.csv` from previous version (this file does not change unless we change or add a new L1 column)
- Run `availability_graph.R` to produce availability graphs.
- Add saved graphs to quick start documents located in the FME Data Management folder
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you create a new copy of the documents for each release?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

- Export Google document as a PDF and move to submission folders

3. Prep data

- Compress all folders into \_.zip\_ format

4. ESS-DIVE submission

- If a minor version change:

- Remove all data files

- Add new files

- Update data date range in submission metadata

- Change version name in title and abstract

- Finally, email ESS-DIVE support to start the re-review

- If a major version change:

- Create new submission

- *More steps to come*...