You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 7df527a
Browse filesBrowse the repository at this point in the historyBrowse files
Update tutorials to include memory benchmarks (#163)
* Updated mem usage + copy edits to CryoCloud_demo
* updated GrIMP tutorial memory requirements (not measured any Dask tasks). Copy+edits. Flagging dependency issue with Numba library req older numpy.
* updated GrIMP tutorial memory requirements (not measured any Dask tasks). Copy+edits. Flagging dependency issue with Numba library req older numpy.
* added memory benchmark. Tutorial runs again after re-release of ATL15 data that caused earthdata to return null
* added memory benchmark and patch dependency issue so tutorial runs
* added memory benchark and learning objectives
* added memory benchmark and fixed icepyx version issue
* added memory benchmark and fixed icepyx .vars -> .variables issue
* updated sliderule_applications to include memory benchmark. Runs fine on cryo-geo environment
* Renamed and split SWOT tutorial files: (0) download/access (1) plot coverage (2) plot comparison with IS2
* fix:typo, date
* More tutorial updates
Minor file structure org, memory benchmarks, etc.
Note that some tutorials are still broken due to mismatched dependencies. This is a task for later.
* cleanup old CC files
---------
Co-authored-by: Tasha Snow <tsnow@mines.edu>
Copy file name to clipboardExpand all lines: book/external/ICESat2_ATL10-h5coro_large_scale_time_series.ipynb
+57-24Lines changed: 57 additions & 24 deletions
Original file line number
Diff line number
Diff line change
@@ -38,7 +38,51 @@
38
38
"tags": [],
39
39
"user_expressions": []
40
40
},
41
-
"source": "## **1. Tutorial Overview**\n\nThis tutorial is designed for the \"DAAC data access in the cloud hands-on experience\" session at the 2023 NSIDC DAAC [User Working Group (UWG)](https://nsidc.org/data/data-programs/nsidc-daac/about-daac#anchor-2) Meeting. \n\nThe NSIDC DAAC archives and distributes Daily and Monthly Gridded [Sea Ice Freeboard (ATL20)](https://nsidc.org/data/atl20) and [Polar Sea Surface Height Anomaly (ATL21)](https://nsidc.org/data/atl21) data sets from the ICESat-2 Mission, derived from the lower level [ATL10](https://nsidc.org/data/atl10) data set. However, we may want these lower level point data to be gridded and averaged at a weekly cadence, or using a different projection or other gridding parameters. \n\nThis tutorial session is in two parts: \n* We will first guide you through this Jupyter Notebook running in the AWS `us-west-2` region, where data are hosted in the NASA Earthdata Cloud. The notebook utilizes several libraries to performantly search, access, read, and grid the data including `earthaccess`, `h5coro`, and `geopandas`.\n\n* This notebook will focus on the Ross Sea, Antarctica. But let's say we want to scale this analysis to the entire continent. In the second portion, we will present how to scale and run this same workflow from a script (see [workflow.py](./h5cloud/workflow.py) in the `h5cloud` folder within this notebook's directory) that can be run from your laptop, using [Coiled](https://www.coiled.io/). \n\n### **Credits**\n\nThe notebook was created by Andy Barrett and Luis Lopez of NSIDC.\n\nFor questions regarding the notebook, or to report problems, please create a new issue in the [NSIDC-Data-Tutorials repo](https://github.com/nsidc/NSIDC-Data-Tutorials/issues).\n\n### **Learning Objectives**\n\nBy the end of this demonstration you will be able to: \n1. Use `earthaccess` to authenticate with Earthdata Login, search for ICESat-2 data using spatial and temporal filters, and directly access files in the cloud.\n2. Open data granules using `h5coro` to efficiently read HDF5 data from the NSIDC DAAC S3 bucket.\n3. Load data into a geopandas.DataFrame containing geodetic coordinates, ancillary variables, and date/time converted from ATLAS Epoch.\n4. Grid track data to EASE-Grid v2 6.25 km projected grid using drop-in-the-bucket resampling. \n5. Calculate mean statistics and assign aggregated data to grid cells. \n5. Visualize aggregated sea ice height data on a map.\n\n### **Prerequisites**\n\n1. We are running this notebook in the [CryoCloud](https://book.cryointhecloud.com/) JupyterHub. For more information, see the CryoCloud [Getting Started](https://book.cryointhecloud.com/getting-started) documentation.\n**It is advised that you use at least a 16GB instance for this notebook.** \n2. An Earthdata Login is required for data access. If you don't have one, you can register for one [here](https://urs.earthdata.nasa.gov/).\n3. It is recommended that you create a .netrc file that contains your Earthdata Login credentials, stored in your home directory. If you do not have a .netrc file, `earthaccess` will prompt you to enter your Earthdata Login username and password.\n\n### **Example of end product** \nAt the end of this tutorial, the following figure will be generated, demonstrating a year's worth of ATL10 Sea Ice Freeboard height data gridded over the Ross Sea, Antarctica:\n<center>\n<img src='./img/icesat2.atl10.gridded.count_segments.ross_sea.png'/>\n</center>\n\n### **Time requirement**\n\nAllow approximately 40 minutes to complete this tutorial."
41
+
"source": [
42
+
"## **1. Tutorial Overview**\n",
43
+
"\n",
44
+
"This tutorial is designed for the \"DAAC data access in the cloud hands-on experience\" session at the 2023 NSIDC DAAC [User Working Group (UWG)](https://nsidc.org/data/data-programs/nsidc-daac/about-daac#anchor-2) Meeting. \n",
45
+
"\n",
46
+
"The NSIDC DAAC archives and distributes Daily and Monthly Gridded [Sea Ice Freeboard (ATL20)](https://nsidc.org/data/atl20) and [Polar Sea Surface Height Anomaly (ATL21)](https://nsidc.org/data/atl21) data sets from the ICESat-2 Mission, derived from the lower level [ATL10](https://nsidc.org/data/atl10) data set. However, we may want these lower level point data to be gridded and averaged at a weekly cadence, or using a different projection or other gridding parameters. \n",
47
+
"\n",
48
+
"This tutorial session is in two parts: \n",
49
+
"* We will first guide you through this Jupyter Notebook running in the AWS `us-west-2` region, where data are hosted in the NASA Earthdata Cloud. The notebook utilizes several libraries to performantly search, access, read, and grid the data including `earthaccess`, `h5coro`, and `geopandas`.\n",
50
+
"\n",
51
+
"* This notebook will focus on the Ross Sea, Antarctica. But let's say we want to scale this analysis to the entire continent. In the second portion, we will present how to scale and run this same workflow from a script (see [workflow.py](./h5cloud/workflow.py) in the `h5cloud` folder within this notebook's directory) that can be run from your laptop, using [Coiled](https://www.coiled.io/). \n",
52
+
"\n",
53
+
"### **Credits**\n",
54
+
"\n",
55
+
"The notebook was created by Andy Barrett and Luis Lopez of NSIDC.\n",
56
+
"\n",
57
+
"For questions regarding the notebook, or to report problems, please create a new issue in the [NSIDC-Data-Tutorials repo](https://github.com/nsidc/NSIDC-Data-Tutorials/issues).\n",
58
+
"\n",
59
+
"### **Learning Objectives**\n",
60
+
"\n",
61
+
"By the end of this demonstration you will be able to: \n",
62
+
"1. Use `earthaccess` to authenticate with Earthdata Login, search for ICESat-2 data using spatial and temporal filters, and directly access files in the cloud.\n",
63
+
"2. Open data granules using `h5coro` to efficiently read HDF5 data from the NSIDC DAAC S3 bucket.\n",
64
+
"3. Load data into a geopandas.DataFrame containing geodetic coordinates, ancillary variables, and date/time converted from ATLAS Epoch.\n",
65
+
"4. Grid track data to EASE-Grid v2 6.25 km projected grid using drop-in-the-bucket resampling. \n",
66
+
"5. Calculate mean statistics and assign aggregated data to grid cells. \n",
67
+
"5. Visualize aggregated sea ice height data on a map.\n",
68
+
"\n",
69
+
"### **Prerequisites**\n",
70
+
"\n",
71
+
"1. We are running this notebook in the [CryoCloud](https://book.cryointhecloud.com/) JupyterHub. For more information, see the CryoCloud [Getting Started](https://book.cryointhecloud.com/getting-started) documentation.\n",
72
+
"**It is advised that you use at least a 16GB instance for this notebook.** \n",
73
+
"2. An Earthdata Login is required for data access. If you don't have one, you can register for one [here](https://urs.earthdata.nasa.gov/).\n",
74
+
"3. It is recommended that you create a .netrc file that contains your Earthdata Login credentials, stored in your home directory. If you do not have a .netrc file, `earthaccess` will prompt you to enter your Earthdata Login username and password.\n",
75
+
"\n",
76
+
"### **Example of end product** \n",
77
+
"At the end of this tutorial, the following figure will be generated, demonstrating a year's worth of ATL10 Sea Ice Freeboard height data gridded over the Ross Sea, Antarctica:\n",
0 commit comments