Skip to content

Conversation

@bpbond
Copy link
Member

@bpbond bpbond commented Apr 17, 2025

This PR adds time zone conversion capabilities to L1_normalize.qmd. In other words, we now can process data from sites with catalogers set to different local times, and everything is converted to a single output timezone (by default, "Etc/GMT+5" or EST) in the output.

A small data file from a fake site "TZTEST" in a different time zone is included in the ./data_TEST/Raw/ to exercise the timestamp-adjustment code in L1_normalize.

Raw file:
Screenshot 2025-04-17 at 20 58 01

After L1_normalize processing:
Screenshot 2025-04-17 at 20 58 48

You can see that the TZTEST site's data (in central standard time, per the L1_metadata_timezones.csv file) is now in Eastern time.

The output time zone information is now written to both the overall README

COMPASS-FME Level 1 data
Version: ???
Date: ???
Observations: 40,890
Data time zone: Etc/GMT+5
Git commit: e55d476

and site-specific metadata files:

File columns:
—----------------------------------
Site            Site code; see site info above (character)
Plot            Plot code; see site info above (character)
TIMESTAMP       Datalogger timestamp in Etc/GMT+5 (POSIXct)
Instrument      Name of measurement instrument (character)

See #298

@bpbond bpbond marked this pull request as ready for review April 18, 2025 00:55
@bpbond bpbond requested a review from stephpenn1 April 18, 2025 00:55
@bpbond bpbond changed the title First pass at flexible time zone handling Flexible time zone handling Apr 18, 2025
@bpbond bpbond added this to the L1 v2-0 milestone Apr 18, 2025
@bpbond bpbond linked an issue Apr 18, 2025 that may be closed by this pull request
@bpbond
Copy link
Member Author

bpbond commented Apr 23, 2025

Feedback welcome @stephpenn1

Copy link
Member

@stephpenn1 stephpenn1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great, and I like the metadata additions. Thank you! This is important metadata to capture explicitly

@bpbond bpbond merged commit 5e74e57 into dev-v2 Apr 25, 2025
1 check passed
@bpbond bpbond deleted the tz branch April 25, 2025 18:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Arbitrary time zone data

3 participants