[1pt] PR: Update get_sample_data.py for lidar bridge elevation data

[mluck]

Updates data/get_sample_data.py for changes in input data resulting from the addition of lidar bridge elevation. Resolves #1443 and supersedes #1452 and closes #1575.

This also includes the latest new input types such as roads and fema.

Other changes including:

• Updating a partial AWS communication system, but major upgrades to it to allow for near tasks where our python scripts wanting to talk to S3.
• Small adjustments to our git PR template.
• Fix a hardcoded pathing to the wbd.
• Fix bug for the duration calc tool for hucs that fail to process.
• Adjustment to the src/srct_adjust_ras2fim_rating.py which fails when running get_samples when it hits a huc that has no ras2fim processing data.
• Add a new standardized shared logger basic function which handles screen and file at the same time.

Added

• data\aws\s3_shared_functions.py: A collection of functions to talk to AWS S3 buckets.

Changes

• .github/PULL_REQUEST_TEMPLATE.md

• data

  • get_sample_data.py: Updated input files and creates a VRT for bridge elevation diff.
  • aws
    • aws_base.py: Marked as deprecated.
    • s3.py: Marked as deprecated.

• fim_post_processing.sh: Used WBD environment variable instead of hardcoding.

• src

  • duration_system.py: Fix bug for the duration calc tool for hucs that fail to process.
  • src/src_adjust_ras2fim_rating.py: Fix bug related to sample_data when a huc does not have any ras2fim data plus remove tests for maximum cpu.s
  • fim_logger.py: Marked as deprecated.
  • shared_functions: Added a new function to setup a logger for a file for both screen and file.

Testing

Ran fim_pipeline.sh on HUCs 06010108, 10280103, and 12090301. Tested against efs, FIM S3, and ESIP.

Deployment Plan (For developer use)

How does the changes affect the product?

• Code only?
• If applicable, has a deployment plan be created with the deployment person/team?
• Require new or adjusted data inputs? Does it have start, end and duration code (in UTC)?
• If new or updated data sets, has the FIM code been updated and tested with the new/adjusted data (subset is fine, but must be a subset of the new data)?
• Require new pre-clip set?
• Has new or updated python packages?

Issuer Checklist (For developer use)

You may update this checklist before and/or after creating the PR. If you're unsure about any of them, please ask, we're here to help! These items are what we are going to look for before merging your code.

• Informative and human-readable title, using the format: [_pt] PR: <description>
• Links are provided if this PR resolves an issue, or depends on another other PR
• If submitting a PR to the dev branch (the default branch), you have a descriptive Feature Branch name using the format: dev-<description-of-change> (e.g. dev-revise-levee-masking)
• Changes are limited to a single goal (no scope creep)
• The feature branch you're submitting as a PR is up to date (merged) with the latest dev branch
• pre-commit hooks were run locally
• Any change in functionality is tested
• New functions are documented (with a description, list of inputs, and expected output)
• Placeholder code is flagged / future todos are captured in comments
• CHANGELOG updated with template version number, e.g. 4.x.x.x
• Add yourself as an assignee in the PR as well as the FIM Technical Lead

Merge Checklist (For Technical Lead use only)

• Update CHANGELOG with latest version number and merge date
• Update the Citation.cff file to reflect the latest version number in the CHANGELOG
• If applicable, update README with major alterations

[RobHanna-NOAA]

Matt started this, I upgraded it adding some of the latest inputs plus aws / s3 hooks. Matt reviewed it but does not have perms to approve. I approved it once he re-tested it.