Part 2b: Using the icepyx python library to access ICESat-2 data (ATL06)

Tutorial Overview

This notebook demonstrates how to search for, access, and plot a cloud-hosted ICESat-2 dataset using the icepyx package.

icepyx is a community and software library for searching, downloading, and reading ICESat-2 data. While opening data should be straightforward, there are some oddities in navigating the highly nested organization and hundreds of variables of the ICESat-2 data. icepyx provides tools to help with those oddities.

Thanks to contributions from countless community members, icepyx can (for ICESat-2 data):

• search for available data granules (data files)

• order and download data or access it directly in the cloud

• order a subset of data: clipped in space, time, containing fewer variables, or a few other options provided by NSIDC

• navigate the available ICESat-2 data variables

• read ICESat-2 data into Xarray DataArrays, including merging data from multiple files

• access coincident Argo data via the QUEST (Query Unify Explore SpatioTemporal) module

• add new datasets to QUEST via a template

Under the hood, icepyx relies on earthaccess to help handle authentication, especially for obtaining S3 tokens to access ICESat-2 data in the cloud. All this happens without the user needing to take any action other than supplying their Earthdata Login credentials using one of the methods described in the earthaccess tutorial.

In this tutorial we will look at the ATL06 Land Ice Height product.

Learning Objectives

In this tutorial you will learn:

1. how to use icepyx to search for ICESat-2 data using spatial and temporal filters;

2. how to open and combine data multiple HDF5 groups into an xarray.Dataset using icepyx.Read;

3. how to begin your analysis, including plotting.

Prerequisites

The workflow described in this tutorial forms the initial steps of an Analysis in Place workflow that would be run on a AWS cloud compute resource. You will need:

1. a JupyterHub, such as CryoCloud, or AWS EC2 instance in the us-west-2 region.

2. a NASA Earthdata Login. If you need to register for an Earthdata Login see the Getting an Earthdata Login section of the ICESat-2 Hackweek 2023 Jupyter Book.

3. A .netrc file, that contains your Earthdata Login credentials, in your home directory. See Configure Programmatic Access to NASA Servers to create a .netrc file.

Credits

This notebook is based on an icepyx Tutorial originally created by Rachel Wegener, Univ. Maryland and updated by Amy Steiker, NSIDC, and Jessica Scheick, Univ. of New Hampshire for the “Cloud Computing and Open-Source Scientific Software for Cryosphere Communities” Learning Workshop at the 2023 AGU Fall Meeting (using a different data product). A version of it was presented at the 2024 NEGM meeting.

It was updated in May 2024 to utilize (at a minimum) v1.0.0 of icepyx.

Using icepyx to search and access ICESat-2 data

We won’t dive into using icepyx to search for and download data in this tutorial, since we already discussed how to do that with earthaccess. The code to search and download is still provided below for the curious reader. The icepyx documentation shows more detail about different search parameters and how to inspect the results of a query.

import icepyx as ipx
ipx.__version__

'1.0.0'

import datashader
import geoviews as gv
import hvplot.xarray

%matplotlib inline

# Use our search parameters to setup a search Query
short_name = 'ATL06'
spatial_extent = [-39, 66.2, -37.7, 66.6]
date_range = ['2019-05-04','2019-08-04']
region = ipx.Query(short_name, spatial_extent, date_range)

# Visualize our spatial extent
region.visualize_spatial_extent()

# Display if any data files, or granules, matched our search
region.avail_granules(ids=True)

[['ATL06_20190509202526_06350305_006_02.h5',
  'ATL06_20190513201705_06960305_006_02.h5',
  'ATL06_20190525070709_08710303_006_02.h5',
  'ATL06_20190611185304_11380305_006_02.h5',
  'ATL06_20190615184444_11990305_006_02.h5',
  'ATL06_20190623054310_13130303_006_02.h5',
  'ATL06_20190710172853_01930405_006_02.h5',
  'ATL06_20190714172035_02540405_006_02.h5',
  'ATL06_20190726041049_04290403_006_02.h5',
  'ATL06_20190730040232_04900403_006_02.h5']]

# We can also get the S3 urls
region.avail_granules(ids=True, cloud=True)

[['ATL06_20190509202526_06350305_006_02.h5',
  'ATL06_20190513201705_06960305_006_02.h5',
  'ATL06_20190525070709_08710303_006_02.h5',
  'ATL06_20190611185304_11380305_006_02.h5',
  'ATL06_20190615184444_11990305_006_02.h5',
  'ATL06_20190623054310_13130303_006_02.h5',
  'ATL06_20190710172853_01930405_006_02.h5',
  'ATL06_20190714172035_02540405_006_02.h5',
  'ATL06_20190726041049_04290403_006_02.h5',
  'ATL06_20190730040232_04900403_006_02.h5'],
 ['s3://nsidc-cumulus-prod-protected/ATLAS/ATL06/006/2019/05/09/ATL06_20190509202526_06350305_006_02.h5',
  's3://nsidc-cumulus-prod-protected/ATLAS/ATL06/006/2019/05/13/ATL06_20190513201705_06960305_006_02.h5',
  's3://nsidc-cumulus-prod-protected/ATLAS/ATL06/006/2019/05/25/ATL06_20190525070709_08710303_006_02.h5',
  's3://nsidc-cumulus-prod-protected/ATLAS/ATL06/006/2019/06/11/ATL06_20190611185304_11380305_006_02.h5',
  's3://nsidc-cumulus-prod-protected/ATLAS/ATL06/006/2019/06/15/ATL06_20190615184444_11990305_006_02.h5',
  's3://nsidc-cumulus-prod-protected/ATLAS/ATL06/006/2019/06/23/ATL06_20190623054310_13130303_006_02.h5',
  's3://nsidc-cumulus-prod-protected/ATLAS/ATL06/006/2019/07/10/ATL06_20190710172853_01930405_006_02.h5',
  's3://nsidc-cumulus-prod-protected/ATLAS/ATL06/006/2019/07/14/ATL06_20190714172035_02540405_006_02.h5',
  's3://nsidc-cumulus-prod-protected/ATLAS/ATL06/006/2019/07/26/ATL06_20190726041049_04290403_006_02.h5',
  's3://nsidc-cumulus-prod-protected/ATLAS/ATL06/006/2019/07/30/ATL06_20190730040232_04900403_006_02.h5']]

s3urls = region.avail_granules(ids=True, cloud=True)[1]

Reading a file with icepyx

To read a file with icepyx there are several steps:

1. Create a Read object. This sets up an initial connection to your file(s) and validates the metadata.

2. Tell the Read object what variables you would like to read

3. Load your data!

Create a Read object

Here we are creating a read object to set up an initial connection to your file(s). It will ask you if you’d like to proceed - enter “y”.

reader = ipx.Read(s3urls)

Enter your Earthdata Login username:  icepyx_devteam
Enter your Earthdata password:  ········
Do you wish to proceed (not recommended) y/[n]? y

Tip: If you don't want to type your Earthdata Login information every time they are required you can setup more automatic methods of authentication. Two common methods are 1) Add your earthdata password and username to as environment variables as EARTHDATA_USERNAME and EARTHDATA_PASSWORD. 2) setup a .netrc file in your home directory. See the Openscapes tutorial

reader

<icepyx.core.read.Read at 0x7f9ed8d565d0>

Select your variables

To view the variables contained in your dataset you can call .vars on your data reader.

reader.vars.avail()

['ancillary_data/atlas_sdp_gps_epoch',
 'ancillary_data/control',
 'ancillary_data/data_end_utc',
 'ancillary_data/data_start_utc',
 'ancillary_data/end_cycle',
 'ancillary_data/end_delta_time',
 'ancillary_data/end_geoseg',
 'ancillary_data/end_gpssow',
 'ancillary_data/end_gpsweek',
 'ancillary_data/end_orbit',
 'ancillary_data/end_region',
 'ancillary_data/end_rgt',
 'ancillary_data/granule_end_utc',
 'ancillary_data/granule_start_utc',
 'ancillary_data/qa_at_interval',
 'ancillary_data/release',
 'ancillary_data/start_cycle',
 'ancillary_data/start_delta_time',
 'ancillary_data/start_geoseg',
 'ancillary_data/start_gpssow',
 'ancillary_data/start_gpsweek',
 'ancillary_data/start_orbit',
 'ancillary_data/start_region',
 'ancillary_data/start_rgt',
 'ancillary_data/version',
 'ancillary_data/land_ice/dt_hist',
 'ancillary_data/land_ice/fit_maxiter',
 'ancillary_data/land_ice/fpb_maxiter',
 'ancillary_data/land_ice/max_res_ids',
 'ancillary_data/land_ice/min_dist',
 'ancillary_data/land_ice/min_gain_th',
 'ancillary_data/land_ice/min_n_pe',
 'ancillary_data/land_ice/min_n_sel',
 'ancillary_data/land_ice/min_signal_conf',
 'ancillary_data/land_ice/n_hist',
 'ancillary_data/land_ice/nhist_bins',
 'ancillary_data/land_ice/n_sigmas',
 'ancillary_data/land_ice/proc_interval',
 'ancillary_data/land_ice/qs_lim_bsc',
 'ancillary_data/land_ice/qs_lim_hrs',
 'ancillary_data/land_ice/qs_lim_hsigma',
 'ancillary_data/land_ice/qs_lim_msw',
 'ancillary_data/land_ice/qs_lim_snr',
 'ancillary_data/land_ice/qs_lim_sss',
 'ancillary_data/land_ice/rbin_width',
 'ancillary_data/land_ice/sigma_beam',
 'ancillary_data/land_ice/sigma_tx',
 'ancillary_data/land_ice/t_dead',
 'ancillary_data/land_ice/txp_maxiter',
 'gt1l/land_ice_segments/atl06_quality_summary',
 'gt1l/land_ice_segments/delta_time',
 'gt1l/land_ice_segments/h_li',
 'gt1l/land_ice_segments/h_li_sigma',
 'gt1l/land_ice_segments/latitude',
 'gt1l/land_ice_segments/longitude',
 'gt1l/land_ice_segments/segment_id',
 'gt1l/land_ice_segments/sigma_geo_h',
 'gt1l/land_ice_segments/bias_correction/fpb_mean_corr',
 'gt1l/land_ice_segments/bias_correction/fpb_mean_corr_sigma',
 'gt1l/land_ice_segments/bias_correction/fpb_med_corr',
 'gt1l/land_ice_segments/bias_correction/fpb_med_corr_sigma',
 'gt1l/land_ice_segments/bias_correction/fpb_n_corr',
 'gt1l/land_ice_segments/bias_correction/med_r_fit',
 'gt1l/land_ice_segments/bias_correction/tx_mean_corr',
 'gt1l/land_ice_segments/bias_correction/tx_med_corr',
 'gt1l/land_ice_segments/dem/dem_flag',
 'gt1l/land_ice_segments/dem/dem_h',
 'gt1l/land_ice_segments/dem/geoid_free2mean',
 'gt1l/land_ice_segments/dem/geoid_h',
 'gt1l/land_ice_segments/fit_statistics/dh_fit_dx',
 'gt1l/land_ice_segments/fit_statistics/dh_fit_dx_sigma',
 'gt1l/land_ice_segments/fit_statistics/dh_fit_dy',
 'gt1l/land_ice_segments/fit_statistics/h_expected_rms',
 'gt1l/land_ice_segments/fit_statistics/h_mean',
 'gt1l/land_ice_segments/fit_statistics/h_rms_misfit',
 'gt1l/land_ice_segments/fit_statistics/h_robust_sprd',
 'gt1l/land_ice_segments/fit_statistics/n_fit_photons',
 'gt1l/land_ice_segments/fit_statistics/n_seg_pulses',
 'gt1l/land_ice_segments/fit_statistics/sigma_h_mean',
 'gt1l/land_ice_segments/fit_statistics/signal_selection_source',
 'gt1l/land_ice_segments/fit_statistics/signal_selection_source_status',
 'gt1l/land_ice_segments/fit_statistics/snr',
 'gt1l/land_ice_segments/fit_statistics/snr_significance',
 'gt1l/land_ice_segments/fit_statistics/w_surface_window_final',
 'gt1l/land_ice_segments/geophysical/bckgrd',
 'gt1l/land_ice_segments/geophysical/bsnow_conf',
 'gt1l/land_ice_segments/geophysical/bsnow_h',
 'gt1l/land_ice_segments/geophysical/bsnow_od',
 'gt1l/land_ice_segments/geophysical/cloud_flg_asr',
 'gt1l/land_ice_segments/geophysical/cloud_flg_atm',
 'gt1l/land_ice_segments/geophysical/dac',
 'gt1l/land_ice_segments/geophysical/e_bckgrd',
 'gt1l/land_ice_segments/geophysical/layer_flag',
 'gt1l/land_ice_segments/geophysical/msw_flag',
 'gt1l/land_ice_segments/geophysical/neutat_delay_total',
 'gt1l/land_ice_segments/geophysical/r_eff',
 'gt1l/land_ice_segments/geophysical/solar_azimuth',
 'gt1l/land_ice_segments/geophysical/solar_elevation',
 'gt1l/land_ice_segments/geophysical/tide_earth',
 'gt1l/land_ice_segments/geophysical/tide_earth_free2mean',
 'gt1l/land_ice_segments/geophysical/tide_equilibrium',
 'gt1l/land_ice_segments/geophysical/tide_load',
 'gt1l/land_ice_segments/geophysical/tide_ocean',
 'gt1l/land_ice_segments/geophysical/tide_pole',
 'gt1l/land_ice_segments/ground_track/ref_azimuth',
 'gt1l/land_ice_segments/ground_track/ref_coelv',
 'gt1l/land_ice_segments/ground_track/seg_azimuth',
 'gt1l/land_ice_segments/ground_track/sigma_geo_at',
 'gt1l/land_ice_segments/ground_track/sigma_geo_r',
 'gt1l/land_ice_segments/ground_track/sigma_geo_xt',
 'gt1l/land_ice_segments/ground_track/x_atc',
 'gt1l/land_ice_segments/ground_track/y_atc',
 'gt1l/residual_histogram/bckgrd_per_m',
 'gt1l/residual_histogram/bin_top_h',
 'gt1l/residual_histogram/count',
 'gt1l/residual_histogram/delta_time',
 'gt1l/residual_histogram/ds_segment_id',
 'gt1l/residual_histogram/lat_mean',
 'gt1l/residual_histogram/lon_mean',
 'gt1l/residual_histogram/pulse_count',
 'gt1l/residual_histogram/segment_id_list',
 'gt1l/residual_histogram/x_atc_mean',
 'gt1l/segment_quality/delta_time',
 'gt1l/segment_quality/record_number',
 'gt1l/segment_quality/reference_pt_lat',
 'gt1l/segment_quality/reference_pt_lon',
 'gt1l/segment_quality/segment_id',
 'gt1l/segment_quality/signal_selection_source',
 'gt1l/segment_quality/signal_selection_status/signal_selection_status_all',
 'gt1l/segment_quality/signal_selection_status/signal_selection_status_backup',
 'gt1l/segment_quality/signal_selection_status/signal_selection_status_confident',
 'gt1r/land_ice_segments/atl06_quality_summary',
 'gt1r/land_ice_segments/delta_time',
 'gt1r/land_ice_segments/h_li',
 'gt1r/land_ice_segments/h_li_sigma',
 'gt1r/land_ice_segments/latitude',
 'gt1r/land_ice_segments/longitude',
 'gt1r/land_ice_segments/segment_id',
 'gt1r/land_ice_segments/sigma_geo_h',
 'gt1r/land_ice_segments/bias_correction/fpb_mean_corr',
 'gt1r/land_ice_segments/bias_correction/fpb_mean_corr_sigma',
 'gt1r/land_ice_segments/bias_correction/fpb_med_corr',
 'gt1r/land_ice_segments/bias_correction/fpb_med_corr_sigma',
 'gt1r/land_ice_segments/bias_correction/fpb_n_corr',
 'gt1r/land_ice_segments/bias_correction/med_r_fit',
 'gt1r/land_ice_segments/bias_correction/tx_mean_corr',
 'gt1r/land_ice_segments/bias_correction/tx_med_corr',
 'gt1r/land_ice_segments/dem/dem_flag',
 'gt1r/land_ice_segments/dem/dem_h',
 'gt1r/land_ice_segments/dem/geoid_free2mean',
 'gt1r/land_ice_segments/dem/geoid_h',
 'gt1r/land_ice_segments/fit_statistics/dh_fit_dx',
 'gt1r/land_ice_segments/fit_statistics/dh_fit_dx_sigma',
 'gt1r/land_ice_segments/fit_statistics/dh_fit_dy',
 'gt1r/land_ice_segments/fit_statistics/h_expected_rms',
 'gt1r/land_ice_segments/fit_statistics/h_mean',
 'gt1r/land_ice_segments/fit_statistics/h_rms_misfit',
 'gt1r/land_ice_segments/fit_statistics/h_robust_sprd',
 'gt1r/land_ice_segments/fit_statistics/n_fit_photons',
 'gt1r/land_ice_segments/fit_statistics/n_seg_pulses',
 'gt1r/land_ice_segments/fit_statistics/sigma_h_mean',
 'gt1r/land_ice_segments/fit_statistics/signal_selection_source',
 'gt1r/land_ice_segments/fit_statistics/signal_selection_source_status',
 'gt1r/land_ice_segments/fit_statistics/snr',
 'gt1r/land_ice_segments/fit_statistics/snr_significance',
 'gt1r/land_ice_segments/fit_statistics/w_surface_window_final',
 'gt1r/land_ice_segments/geophysical/bckgrd',
 'gt1r/land_ice_segments/geophysical/bsnow_conf',
 'gt1r/land_ice_segments/geophysical/bsnow_h',
 'gt1r/land_ice_segments/geophysical/bsnow_od',
 'gt1r/land_ice_segments/geophysical/cloud_flg_asr',
 'gt1r/land_ice_segments/geophysical/cloud_flg_atm',
 'gt1r/land_ice_segments/geophysical/dac',
 'gt1r/land_ice_segments/geophysical/e_bckgrd',
 'gt1r/land_ice_segments/geophysical/layer_flag',
 'gt1r/land_ice_segments/geophysical/msw_flag',
 'gt1r/land_ice_segments/geophysical/neutat_delay_total',
 'gt1r/land_ice_segments/geophysical/r_eff',
 'gt1r/land_ice_segments/geophysical/solar_azimuth',
 'gt1r/land_ice_segments/geophysical/solar_elevation',
 'gt1r/land_ice_segments/geophysical/tide_earth',
 'gt1r/land_ice_segments/geophysical/tide_earth_free2mean',
 'gt1r/land_ice_segments/geophysical/tide_equilibrium',
 'gt1r/land_ice_segments/geophysical/tide_load',
 'gt1r/land_ice_segments/geophysical/tide_ocean',
 'gt1r/land_ice_segments/geophysical/tide_pole',
 'gt1r/land_ice_segments/ground_track/ref_azimuth',
 'gt1r/land_ice_segments/ground_track/ref_coelv',
 'gt1r/land_ice_segments/ground_track/seg_azimuth',
 'gt1r/land_ice_segments/ground_track/sigma_geo_at',
 'gt1r/land_ice_segments/ground_track/sigma_geo_r',
 'gt1r/land_ice_segments/ground_track/sigma_geo_xt',
 'gt1r/land_ice_segments/ground_track/x_atc',
 'gt1r/land_ice_segments/ground_track/y_atc',
 'gt1r/residual_histogram/bckgrd_per_m',
 'gt1r/residual_histogram/bin_top_h',
 'gt1r/residual_histogram/count',
 'gt1r/residual_histogram/delta_time',
 'gt1r/residual_histogram/ds_segment_id',
 'gt1r/residual_histogram/lat_mean',
 'gt1r/residual_histogram/lon_mean',
 'gt1r/residual_histogram/pulse_count',
 'gt1r/residual_histogram/segment_id_list',
 'gt1r/residual_histogram/x_atc_mean',
 'gt1r/segment_quality/delta_time',
 'gt1r/segment_quality/record_number',
 'gt1r/segment_quality/reference_pt_lat',
 'gt1r/segment_quality/reference_pt_lon',
 'gt1r/segment_quality/segment_id',
 'gt1r/segment_quality/signal_selection_source',
 'gt1r/segment_quality/signal_selection_status/signal_selection_status_all',
 'gt1r/segment_quality/signal_selection_status/signal_selection_status_backup',
 'gt1r/segment_quality/signal_selection_status/signal_selection_status_confident',
 'gt2l/land_ice_segments/atl06_quality_summary',
 'gt2l/land_ice_segments/delta_time',
 'gt2l/land_ice_segments/h_li',
 'gt2l/land_ice_segments/h_li_sigma',
 'gt2l/land_ice_segments/latitude',
 'gt2l/land_ice_segments/longitude',
 'gt2l/land_ice_segments/segment_id',
 'gt2l/land_ice_segments/sigma_geo_h',
 'gt2l/land_ice_segments/bias_correction/fpb_mean_corr',
 'gt2l/land_ice_segments/bias_correction/fpb_mean_corr_sigma',
 'gt2l/land_ice_segments/bias_correction/fpb_med_corr',
 'gt2l/land_ice_segments/bias_correction/fpb_med_corr_sigma',
 'gt2l/land_ice_segments/bias_correction/fpb_n_corr',
 'gt2l/land_ice_segments/bias_correction/med_r_fit',
 'gt2l/land_ice_segments/bias_correction/tx_mean_corr',
 'gt2l/land_ice_segments/bias_correction/tx_med_corr',
 'gt2l/land_ice_segments/dem/dem_flag',
 'gt2l/land_ice_segments/dem/dem_h',
 'gt2l/land_ice_segments/dem/geoid_free2mean',
 'gt2l/land_ice_segments/dem/geoid_h',
 'gt2l/land_ice_segments/fit_statistics/dh_fit_dx',
 'gt2l/land_ice_segments/fit_statistics/dh_fit_dx_sigma',
 'gt2l/land_ice_segments/fit_statistics/dh_fit_dy',
 'gt2l/land_ice_segments/fit_statistics/h_expected_rms',
 'gt2l/land_ice_segments/fit_statistics/h_mean',
 'gt2l/land_ice_segments/fit_statistics/h_rms_misfit',
 'gt2l/land_ice_segments/fit_statistics/h_robust_sprd',
 'gt2l/land_ice_segments/fit_statistics/n_fit_photons',
 'gt2l/land_ice_segments/fit_statistics/n_seg_pulses',
 'gt2l/land_ice_segments/fit_statistics/sigma_h_mean',
 'gt2l/land_ice_segments/fit_statistics/signal_selection_source',
 'gt2l/land_ice_segments/fit_statistics/signal_selection_source_status',
 'gt2l/land_ice_segments/fit_statistics/snr',
 'gt2l/land_ice_segments/fit_statistics/snr_significance',
 'gt2l/land_ice_segments/fit_statistics/w_surface_window_final',
 'gt2l/land_ice_segments/geophysical/bckgrd',
 'gt2l/land_ice_segments/geophysical/bsnow_conf',
 'gt2l/land_ice_segments/geophysical/bsnow_h',
 'gt2l/land_ice_segments/geophysical/bsnow_od',
 'gt2l/land_ice_segments/geophysical/cloud_flg_asr',
 'gt2l/land_ice_segments/geophysical/cloud_flg_atm',
 'gt2l/land_ice_segments/geophysical/dac',
 'gt2l/land_ice_segments/geophysical/e_bckgrd',
 'gt2l/land_ice_segments/geophysical/layer_flag',
 'gt2l/land_ice_segments/geophysical/msw_flag',
 'gt2l/land_ice_segments/geophysical/neutat_delay_total',
 'gt2l/land_ice_segments/geophysical/r_eff',
 'gt2l/land_ice_segments/geophysical/solar_azimuth',
 'gt2l/land_ice_segments/geophysical/solar_elevation',
 'gt2l/land_ice_segments/geophysical/tide_earth',
 'gt2l/land_ice_segments/geophysical/tide_earth_free2mean',
 'gt2l/land_ice_segments/geophysical/tide_equilibrium',
 'gt2l/land_ice_segments/geophysical/tide_load',
 'gt2l/land_ice_segments/geophysical/tide_ocean',
 'gt2l/land_ice_segments/geophysical/tide_pole',
 'gt2l/land_ice_segments/ground_track/ref_azimuth',
 'gt2l/land_ice_segments/ground_track/ref_coelv',
 'gt2l/land_ice_segments/ground_track/seg_azimuth',
 'gt2l/land_ice_segments/ground_track/sigma_geo_at',
 'gt2l/land_ice_segments/ground_track/sigma_geo_r',
 'gt2l/land_ice_segments/ground_track/sigma_geo_xt',
 'gt2l/land_ice_segments/ground_track/x_atc',
 'gt2l/land_ice_segments/ground_track/y_atc',
 'gt2l/residual_histogram/bckgrd_per_m',
 'gt2l/residual_histogram/bin_top_h',
 'gt2l/residual_histogram/count',
 'gt2l/residual_histogram/delta_time',
 'gt2l/residual_histogram/ds_segment_id',
 'gt2l/residual_histogram/lat_mean',
 'gt2l/residual_histogram/lon_mean',
 'gt2l/residual_histogram/pulse_count',
 'gt2l/residual_histogram/segment_id_list',
 'gt2l/residual_histogram/x_atc_mean',
 'gt2l/segment_quality/delta_time',
 'gt2l/segment_quality/record_number',
 'gt2l/segment_quality/reference_pt_lat',
 'gt2l/segment_quality/reference_pt_lon',
 'gt2l/segment_quality/segment_id',
 'gt2l/segment_quality/signal_selection_source',
 'gt2l/segment_quality/signal_selection_status/signal_selection_status_all',
 'gt2l/segment_quality/signal_selection_status/signal_selection_status_backup',
 'gt2l/segment_quality/signal_selection_status/signal_selection_status_confident',
 'gt2r/land_ice_segments/atl06_quality_summary',
 'gt2r/land_ice_segments/delta_time',
 'gt2r/land_ice_segments/h_li',
 'gt2r/land_ice_segments/h_li_sigma',
 'gt2r/land_ice_segments/latitude',
 'gt2r/land_ice_segments/longitude',
 'gt2r/land_ice_segments/segment_id',
 'gt2r/land_ice_segments/sigma_geo_h',
 'gt2r/land_ice_segments/bias_correction/fpb_mean_corr',
 'gt2r/land_ice_segments/bias_correction/fpb_mean_corr_sigma',
 'gt2r/land_ice_segments/bias_correction/fpb_med_corr',
 'gt2r/land_ice_segments/bias_correction/fpb_med_corr_sigma',
 'gt2r/land_ice_segments/bias_correction/fpb_n_corr',
 'gt2r/land_ice_segments/bias_correction/med_r_fit',
 'gt2r/land_ice_segments/bias_correction/tx_mean_corr',
 'gt2r/land_ice_segments/bias_correction/tx_med_corr',
 'gt2r/land_ice_segments/dem/dem_flag',
 'gt2r/land_ice_segments/dem/dem_h',
 'gt2r/land_ice_segments/dem/geoid_free2mean',
 'gt2r/land_ice_segments/dem/geoid_h',
 'gt2r/land_ice_segments/fit_statistics/dh_fit_dx',
 'gt2r/land_ice_segments/fit_statistics/dh_fit_dx_sigma',
 'gt2r/land_ice_segments/fit_statistics/dh_fit_dy',
 'gt2r/land_ice_segments/fit_statistics/h_expected_rms',
 'gt2r/land_ice_segments/fit_statistics/h_mean',
 'gt2r/land_ice_segments/fit_statistics/h_rms_misfit',
 'gt2r/land_ice_segments/fit_statistics/h_robust_sprd',
 'gt2r/land_ice_segments/fit_statistics/n_fit_photons',
 'gt2r/land_ice_segments/fit_statistics/n_seg_pulses',
 'gt2r/land_ice_segments/fit_statistics/sigma_h_mean',
 'gt2r/land_ice_segments/fit_statistics/signal_selection_source',
 'gt2r/land_ice_segments/fit_statistics/signal_selection_source_status',
 'gt2r/land_ice_segments/fit_statistics/snr',
 'gt2r/land_ice_segments/fit_statistics/snr_significance',
 'gt2r/land_ice_segments/fit_statistics/w_surface_window_final',
 'gt2r/land_ice_segments/geophysical/bckgrd',
 'gt2r/land_ice_segments/geophysical/bsnow_conf',
 'gt2r/land_ice_segments/geophysical/bsnow_h',
 'gt2r/land_ice_segments/geophysical/bsnow_od',
 'gt2r/land_ice_segments/geophysical/cloud_flg_asr',
 'gt2r/land_ice_segments/geophysical/cloud_flg_atm',
 'gt2r/land_ice_segments/geophysical/dac',
 'gt2r/land_ice_segments/geophysical/e_bckgrd',
 'gt2r/land_ice_segments/geophysical/layer_flag',
 'gt2r/land_ice_segments/geophysical/msw_flag',
 'gt2r/land_ice_segments/geophysical/neutat_delay_total',
 'gt2r/land_ice_segments/geophysical/r_eff',
 'gt2r/land_ice_segments/geophysical/solar_azimuth',
 'gt2r/land_ice_segments/geophysical/solar_elevation',
 'gt2r/land_ice_segments/geophysical/tide_earth',
 'gt2r/land_ice_segments/geophysical/tide_earth_free2mean',
 'gt2r/land_ice_segments/geophysical/tide_equilibrium',
 'gt2r/land_ice_segments/geophysical/tide_load',
 'gt2r/land_ice_segments/geophysical/tide_ocean',
 'gt2r/land_ice_segments/geophysical/tide_pole',
 'gt2r/land_ice_segments/ground_track/ref_azimuth',
 'gt2r/land_ice_segments/ground_track/ref_coelv',
 'gt2r/land_ice_segments/ground_track/seg_azimuth',
 'gt2r/land_ice_segments/ground_track/sigma_geo_at',
 'gt2r/land_ice_segments/ground_track/sigma_geo_r',
 'gt2r/land_ice_segments/ground_track/sigma_geo_xt',
 'gt2r/land_ice_segments/ground_track/x_atc',
 'gt2r/land_ice_segments/ground_track/y_atc',
 'gt2r/residual_histogram/bckgrd_per_m',
 'gt2r/residual_histogram/bin_top_h',
 'gt2r/residual_histogram/count',
 'gt2r/residual_histogram/delta_time',
 'gt2r/residual_histogram/ds_segment_id',
 'gt2r/residual_histogram/lat_mean',
 'gt2r/residual_histogram/lon_mean',
 'gt2r/residual_histogram/pulse_count',
 'gt2r/residual_histogram/segment_id_list',
 'gt2r/residual_histogram/x_atc_mean',
 'gt2r/segment_quality/delta_time',
 'gt2r/segment_quality/record_number',
 'gt2r/segment_quality/reference_pt_lat',
 'gt2r/segment_quality/reference_pt_lon',
 'gt2r/segment_quality/segment_id',
 'gt2r/segment_quality/signal_selection_source',
 'gt2r/segment_quality/signal_selection_status/signal_selection_status_all',
 'gt2r/segment_quality/signal_selection_status/signal_selection_status_backup',
 'gt2r/segment_quality/signal_selection_status/signal_selection_status_confident',
 'gt3l/land_ice_segments/atl06_quality_summary',
 'gt3l/land_ice_segments/delta_time',
 'gt3l/land_ice_segments/h_li',
 'gt3l/land_ice_segments/h_li_sigma',
 'gt3l/land_ice_segments/latitude',
 'gt3l/land_ice_segments/longitude',
 'gt3l/land_ice_segments/segment_id',
 'gt3l/land_ice_segments/sigma_geo_h',
 'gt3l/land_ice_segments/bias_correction/fpb_mean_corr',
 'gt3l/land_ice_segments/bias_correction/fpb_mean_corr_sigma',
 'gt3l/land_ice_segments/bias_correction/fpb_med_corr',
 'gt3l/land_ice_segments/bias_correction/fpb_med_corr_sigma',
 'gt3l/land_ice_segments/bias_correction/fpb_n_corr',
 'gt3l/land_ice_segments/bias_correction/med_r_fit',
 'gt3l/land_ice_segments/bias_correction/tx_mean_corr',
 'gt3l/land_ice_segments/bias_correction/tx_med_corr',
 'gt3l/land_ice_segments/dem/dem_flag',
 'gt3l/land_ice_segments/dem/dem_h',
 'gt3l/land_ice_segments/dem/geoid_free2mean',
 'gt3l/land_ice_segments/dem/geoid_h',
 'gt3l/land_ice_segments/fit_statistics/dh_fit_dx',
 'gt3l/land_ice_segments/fit_statistics/dh_fit_dx_sigma',
 'gt3l/land_ice_segments/fit_statistics/dh_fit_dy',
 'gt3l/land_ice_segments/fit_statistics/h_expected_rms',
 'gt3l/land_ice_segments/fit_statistics/h_mean',
 'gt3l/land_ice_segments/fit_statistics/h_rms_misfit',
 'gt3l/land_ice_segments/fit_statistics/h_robust_sprd',
 'gt3l/land_ice_segments/fit_statistics/n_fit_photons',
 'gt3l/land_ice_segments/fit_statistics/n_seg_pulses',
 'gt3l/land_ice_segments/fit_statistics/sigma_h_mean',
 'gt3l/land_ice_segments/fit_statistics/signal_selection_source',
 'gt3l/land_ice_segments/fit_statistics/signal_selection_source_status',
 'gt3l/land_ice_segments/fit_statistics/snr',
 'gt3l/land_ice_segments/fit_statistics/snr_significance',
 'gt3l/land_ice_segments/fit_statistics/w_surface_window_final',
 'gt3l/land_ice_segments/geophysical/bckgrd',
 'gt3l/land_ice_segments/geophysical/bsnow_conf',
 'gt3l/land_ice_segments/geophysical/bsnow_h',
 'gt3l/land_ice_segments/geophysical/bsnow_od',
 'gt3l/land_ice_segments/geophysical/cloud_flg_asr',
 'gt3l/land_ice_segments/geophysical/cloud_flg_atm',
 'gt3l/land_ice_segments/geophysical/dac',
 'gt3l/land_ice_segments/geophysical/e_bckgrd',
 'gt3l/land_ice_segments/geophysical/layer_flag',
 'gt3l/land_ice_segments/geophysical/msw_flag',
 'gt3l/land_ice_segments/geophysical/neutat_delay_total',
 'gt3l/land_ice_segments/geophysical/r_eff',
 'gt3l/land_ice_segments/geophysical/solar_azimuth',
 'gt3l/land_ice_segments/geophysical/solar_elevation',
 'gt3l/land_ice_segments/geophysical/tide_earth',
 'gt3l/land_ice_segments/geophysical/tide_earth_free2mean',
 'gt3l/land_ice_segments/geophysical/tide_equilibrium',
 'gt3l/land_ice_segments/geophysical/tide_load',
 'gt3l/land_ice_segments/geophysical/tide_ocean',
 'gt3l/land_ice_segments/geophysical/tide_pole',
 'gt3l/land_ice_segments/ground_track/ref_azimuth',
 'gt3l/land_ice_segments/ground_track/ref_coelv',
 'gt3l/land_ice_segments/ground_track/seg_azimuth',
 'gt3l/land_ice_segments/ground_track/sigma_geo_at',
 'gt3l/land_ice_segments/ground_track/sigma_geo_r',
 'gt3l/land_ice_segments/ground_track/sigma_geo_xt',
 'gt3l/land_ice_segments/ground_track/x_atc',
 'gt3l/land_ice_segments/ground_track/y_atc',
 'gt3l/residual_histogram/bckgrd_per_m',
 'gt3l/residual_histogram/bin_top_h',
 'gt3l/residual_histogram/count',
 'gt3l/residual_histogram/delta_time',
 'gt3l/residual_histogram/ds_segment_id',
 'gt3l/residual_histogram/lat_mean',
 'gt3l/residual_histogram/lon_mean',
 'gt3l/residual_histogram/pulse_count',
 'gt3l/residual_histogram/segment_id_list',
 'gt3l/residual_histogram/x_atc_mean',
 'gt3l/segment_quality/delta_time',
 'gt3l/segment_quality/record_number',
 'gt3l/segment_quality/reference_pt_lat',
 'gt3l/segment_quality/reference_pt_lon',
 'gt3l/segment_quality/segment_id',
 'gt3l/segment_quality/signal_selection_source',
 'gt3l/segment_quality/signal_selection_status/signal_selection_status_all',
 'gt3l/segment_quality/signal_selection_status/signal_selection_status_backup',
 'gt3l/segment_quality/signal_selection_status/signal_selection_status_confident',
 'gt3r/land_ice_segments/atl06_quality_summary',
 'gt3r/land_ice_segments/delta_time',
 'gt3r/land_ice_segments/h_li',
 'gt3r/land_ice_segments/h_li_sigma',
 'gt3r/land_ice_segments/latitude',
 'gt3r/land_ice_segments/longitude',
 'gt3r/land_ice_segments/segment_id',
 'gt3r/land_ice_segments/sigma_geo_h',
 'gt3r/land_ice_segments/bias_correction/fpb_mean_corr',
 'gt3r/land_ice_segments/bias_correction/fpb_mean_corr_sigma',
 'gt3r/land_ice_segments/bias_correction/fpb_med_corr',
 'gt3r/land_ice_segments/bias_correction/fpb_med_corr_sigma',
 'gt3r/land_ice_segments/bias_correction/fpb_n_corr',
 'gt3r/land_ice_segments/bias_correction/med_r_fit',
 'gt3r/land_ice_segments/bias_correction/tx_mean_corr',
 'gt3r/land_ice_segments/bias_correction/tx_med_corr',
 'gt3r/land_ice_segments/dem/dem_flag',
 'gt3r/land_ice_segments/dem/dem_h',
 'gt3r/land_ice_segments/dem/geoid_free2mean',
 'gt3r/land_ice_segments/dem/geoid_h',
 'gt3r/land_ice_segments/fit_statistics/dh_fit_dx',
 'gt3r/land_ice_segments/fit_statistics/dh_fit_dx_sigma',
 'gt3r/land_ice_segments/fit_statistics/dh_fit_dy',
 'gt3r/land_ice_segments/fit_statistics/h_expected_rms',
 'gt3r/land_ice_segments/fit_statistics/h_mean',
 'gt3r/land_ice_segments/fit_statistics/h_rms_misfit',
 'gt3r/land_ice_segments/fit_statistics/h_robust_sprd',
 'gt3r/land_ice_segments/fit_statistics/n_fit_photons',
 'gt3r/land_ice_segments/fit_statistics/n_seg_pulses',
 'gt3r/land_ice_segments/fit_statistics/sigma_h_mean',
 'gt3r/land_ice_segments/fit_statistics/signal_selection_source',
 'gt3r/land_ice_segments/fit_statistics/signal_selection_source_status',
 'gt3r/land_ice_segments/fit_statistics/snr',
 'gt3r/land_ice_segments/fit_statistics/snr_significance',
 'gt3r/land_ice_segments/fit_statistics/w_surface_window_final',
 'gt3r/land_ice_segments/geophysical/bckgrd',
 'gt3r/land_ice_segments/geophysical/bsnow_conf',
 'gt3r/land_ice_segments/geophysical/bsnow_h',
 'gt3r/land_ice_segments/geophysical/bsnow_od',
 'gt3r/land_ice_segments/geophysical/cloud_flg_asr',
 'gt3r/land_ice_segments/geophysical/cloud_flg_atm',
 'gt3r/land_ice_segments/geophysical/dac',
 'gt3r/land_ice_segments/geophysical/e_bckgrd',
 'gt3r/land_ice_segments/geophysical/layer_flag',
 'gt3r/land_ice_segments/geophysical/msw_flag',
 'gt3r/land_ice_segments/geophysical/neutat_delay_total',
 'gt3r/land_ice_segments/geophysical/r_eff',
 'gt3r/land_ice_segments/geophysical/solar_azimuth',
 'gt3r/land_ice_segments/geophysical/solar_elevation',
 'gt3r/land_ice_segments/geophysical/tide_earth',
 'gt3r/land_ice_segments/geophysical/tide_earth_free2mean',
 'gt3r/land_ice_segments/geophysical/tide_equilibrium',
 'gt3r/land_ice_segments/geophysical/tide_load',
 'gt3r/land_ice_segments/geophysical/tide_ocean',
 'gt3r/land_ice_segments/geophysical/tide_pole',
 'gt3r/land_ice_segments/ground_track/ref_azimuth',
 'gt3r/land_ice_segments/ground_track/ref_coelv',
 'gt3r/land_ice_segments/ground_track/seg_azimuth',
 'gt3r/land_ice_segments/ground_track/sigma_geo_at',
 'gt3r/land_ice_segments/ground_track/sigma_geo_r',
 'gt3r/land_ice_segments/ground_track/sigma_geo_xt',
 'gt3r/land_ice_segments/ground_track/x_atc',
 'gt3r/land_ice_segments/ground_track/y_atc',
 'gt3r/residual_histogram/bckgrd_per_m',
 'gt3r/residual_histogram/bin_top_h',
 'gt3r/residual_histogram/count',
 'gt3r/residual_histogram/delta_time',
 'gt3r/residual_histogram/ds_segment_id',
 'gt3r/residual_histogram/lat_mean',
 'gt3r/residual_histogram/lon_mean',
 'gt3r/residual_histogram/pulse_count',
 'gt3r/residual_histogram/segment_id_list',
 'gt3r/residual_histogram/x_atc_mean',
 'gt3r/segment_quality/delta_time',
 'gt3r/segment_quality/record_number',
 'gt3r/segment_quality/reference_pt_lat',
 'gt3r/segment_quality/reference_pt_lon',
 'gt3r/segment_quality/segment_id',
 'gt3r/segment_quality/signal_selection_source',
 'gt3r/segment_quality/signal_selection_status/signal_selection_status_all',
 'gt3r/segment_quality/signal_selection_status/signal_selection_status_backup',
 'gt3r/segment_quality/signal_selection_status/signal_selection_status_confident',
 'orbit_info/bounding_polygon_lat1',
 'orbit_info/bounding_polygon_lon1',
 'orbit_info/crossing_time',
 'orbit_info/cycle_number',
 'orbit_info/lan',
 'orbit_info/orbit_number',
 'orbit_info/rgt',
 'orbit_info/sc_orient',
 'orbit_info/sc_orient_time',
 'quality_assessment/qa_granule_fail_reason',
 'quality_assessment/qa_granule_pass_fail',
 'quality_assessment/gt1l/delta_time',
 'quality_assessment/gt1l/lat_mean',
 'quality_assessment/gt1l/lon_mean',
 'quality_assessment/gt1l/signal_selection_source_fraction_0',
 'quality_assessment/gt1l/signal_selection_source_fraction_1',
 'quality_assessment/gt1l/signal_selection_source_fraction_2',
 'quality_assessment/gt1l/signal_selection_source_fraction_3',
 'quality_assessment/gt1r/delta_time',
 'quality_assessment/gt1r/lat_mean',
 'quality_assessment/gt1r/lon_mean',
 'quality_assessment/gt1r/signal_selection_source_fraction_0',
 'quality_assessment/gt1r/signal_selection_source_fraction_1',
 'quality_assessment/gt1r/signal_selection_source_fraction_2',
 'quality_assessment/gt1r/signal_selection_source_fraction_3',
 'quality_assessment/gt2l/delta_time',
 'quality_assessment/gt2l/lat_mean',
 'quality_assessment/gt2l/lon_mean',
 'quality_assessment/gt2l/signal_selection_source_fraction_0',
 'quality_assessment/gt2l/signal_selection_source_fraction_1',
 'quality_assessment/gt2l/signal_selection_source_fraction_2',
 'quality_assessment/gt2l/signal_selection_source_fraction_3',
 'quality_assessment/gt2r/delta_time',
 'quality_assessment/gt2r/lat_mean',
 'quality_assessment/gt2r/lon_mean',
 'quality_assessment/gt2r/signal_selection_source_fraction_0',
 'quality_assessment/gt2r/signal_selection_source_fraction_1',
 'quality_assessment/gt2r/signal_selection_source_fraction_2',
 'quality_assessment/gt2r/signal_selection_source_fraction_3',
 'quality_assessment/gt3l/delta_time',
 'quality_assessment/gt3l/lat_mean',
 'quality_assessment/gt3l/lon_mean',
 'quality_assessment/gt3l/signal_selection_source_fraction_0',
 'quality_assessment/gt3l/signal_selection_source_fraction_1',
 'quality_assessment/gt3l/signal_selection_source_fraction_2',
 'quality_assessment/gt3l/signal_selection_source_fraction_3',
 'quality_assessment/gt3r/delta_time',
 'quality_assessment/gt3r/lat_mean',
 'quality_assessment/gt3r/lon_mean',
 'quality_assessment/gt3r/signal_selection_source_fraction_0',
 'quality_assessment/gt3r/signal_selection_source_fraction_1',
 'quality_assessment/gt3r/signal_selection_source_fraction_2',
 'quality_assessment/gt3r/signal_selection_source_fraction_3']

Thats a lot of variables!

One key feature of icepyx is the ability to browse the variables available in the dataset. There are typically hundreds of variables in a single dataset, so that is a lot to sort through! Let’s take a moment to get oriented to the organization of ATL06 variables, starting with a few important pieces of the algorithm.

To create higher level variables like land ice height, the ATL06 algorithms goes through a series of steps:

1. Identify signal photons from noise photons

2. Filter out photons not over (or near) land

3. Group the signal photons into 40m segments. If there are a sufficient number of photons in that group, calculate statistics (ex. mean height, max height, standard deviation, etc.)

Providing all the potentially useful information from all these processing steps results in a data file that looks analogous to this diagram for an ATL08 file:

Another way to visualize these structure is to download one file and open it using https://myhdf5.hdfgroup.org/.

Further information about each one of the variables is available in the Algorithm Theoretical Basis Document (ATBD) for ATL06.

There is lots to explore in these variables, but we will move forward using a common ATL06 variable: h_li, land ice height (ATBD definition).

reader.vars.append(beam_list=['gt1r'], var_list=['h_li', 'latitude', 'longitude'])

Note that adding variables is a required step before you can load the data.

reader.vars.wanted

{'h_li': ['gt1r/land_ice_segments/h_li'],
 'latitude': ['gt1r/land_ice_segments/latitude'],
 'longitude': ['gt1r/land_ice_segments/longitude']}

Load the data!

%%time
ds = reader.load()

CPU times: user 24.8 s, sys: 4.63 s, total: 29.5 s
Wall time: 1min 29s

ds

<xarray.Dataset> Size: 23MB
Dimensions:              (photon_idx: 80672, spot: 1, gran_idx: 10)
Coordinates:
  * photon_idx           (photon_idx) int64 645kB 0 1 2 3 ... 80669 80670 80671
  * spot                 (spot) uint8 1B 2
  * gran_idx             (gran_idx) float64 80B 1.93e+04 2.54e+04 ... 1.313e+05
    source_file          (gran_idx) object 80B <File-like object S3FileSystem...
    delta_time           (gran_idx, photon_idx) datetime64[ns] 6MB 2019-07-10...
Data variables:
    sc_orient            (gran_idx) int8 10B 0 0 0 0 0 0 0 0 0 0
    cycle_number         (gran_idx) int8 10B 4 4 4 4 3 3 3 3 3 3
    rgt                  (gran_idx) int16 20B 193 254 429 490 ... 1138 1199 1313
    atlas_sdp_gps_epoch  (gran_idx) datetime64[ns] 80B 2018-01-01T00:00:18 .....
    data_start_utc       (gran_idx) datetime64[ns] 80B 2019-07-10T17:28:51.71...
    data_end_utc         (gran_idx) datetime64[ns] 80B 2019-07-10T17:32:50.53...
    h_li                 (spot, gran_idx, photon_idx) float32 3MB nan ... nan
    latitude             (spot, gran_idx, photon_idx) float64 6MB 79.99 ... nan
    longitude            (spot, gran_idx, photon_idx) float64 6MB -29.97 ... nan
    gt                   (gran_idx, spot) <U4 160B 'gt1r' 'gt1r' ... 'gt1r'
Attributes:
    data_product:  ATL06
    Description:   The land_ice_height group contains the primary set of deri...
    data_rate:     Data within this group are sparse.  Data values are provid...

xarray.Dataset

• Dimensions:
  • photon_idx: 80672
  • spot: 1
  • gran_idx: 10
• Coordinates: (5)
  • photon_idx
    (photon_idx)
    int64
    0 1 2 3 ... 80668 80669 80670 80671

    array([    0,     1,     2, ..., 80669, 80670, 80671])

  • spot
    (spot)
    uint8
    2

    array([2], dtype=uint8)

  • gran_idx
    (gran_idx)
    float64
    1.93e+04 2.54e+04 ... 1.313e+05

    array([ 19303.,  25403.,  42903.,  49003.,  63502.,  69602.,  87102., 113802.,
           119902., 131302.])

  • source_file
    (gran_idx)
    object
    <File-like object S3FileSystem, ...

    array([<File-like object S3FileSystem, nsidc-cumulus-prod-protected/ATLAS/ATL06/006/2019/07/10/ATL06_20190710172853_01930405_006_02.h5>,
           <File-like object S3FileSystem, nsidc-cumulus-prod-protected/ATLAS/ATL06/006/2019/07/14/ATL06_20190714172035_02540405_006_02.h5>,
           <File-like object S3FileSystem, nsidc-cumulus-prod-protected/ATLAS/ATL06/006/2019/07/26/ATL06_20190726041049_04290403_006_02.h5>,
           <File-like object S3FileSystem, nsidc-cumulus-prod-protected/ATLAS/ATL06/006/2019/07/30/ATL06_20190730040232_04900403_006_02.h5>,
           <File-like object S3FileSystem, nsidc-cumulus-prod-protected/ATLAS/ATL06/006/2019/05/09/ATL06_20190509202526_06350305_006_02.h5>,
           <File-like object S3FileSystem, nsidc-cumulus-prod-protected/ATLAS/ATL06/006/2019/05/13/ATL06_20190513201705_06960305_006_02.h5>,
           <File-like object S3FileSystem, nsidc-cumulus-prod-protected/ATLAS/ATL06/006/2019/05/25/ATL06_20190525070709_08710303_006_02.h5>,
           <File-like object S3FileSystem, nsidc-cumulus-prod-protected/ATLAS/ATL06/006/2019/06/11/ATL06_20190611185304_11380305_006_02.h5>,
           <File-like object S3FileSystem, nsidc-cumulus-prod-protected/ATLAS/ATL06/006/2019/06/15/ATL06_20190615184444_11990305_006_02.h5>,
           <File-like object S3FileSystem, nsidc-cumulus-prod-protected/ATLAS/ATL06/006/2019/06/23/ATL06_20190623054310_13130303_006_02.h5>],
          dtype=object)

  • delta_time
    (gran_idx, photon_idx)
    datetime64[ns]
    2019-07-10T17:28:52.064189720 .....

    contentType :

    referenceInformation

    description :

    Number of GPS seconds since the ATLAS SDP epoch. The ATLAS Standard Data Products (SDP) epoch offset is defined within /ancillary_data/atlas_sdp_gps_epoch as the number of GPS seconds between the GPS epoch (1980-01-06T00:00:00.000000Z UTC) and the ATLAS SDP epoch. By adding the offset contained within atlas_sdp_gps_epoch to delta time parameters, the time in gps_seconds relative to the GPS epoch can be computed.

    long_name :

    Elapsed GPS seconds

    source :

    section 4.4

    standard_name :

    time

    array([['2019-07-10T17:28:52.064189720', '2019-07-10T17:28:52.067194592',
            '2019-07-10T17:28:52.070098464', ...,
                                      'NaT',                           'NaT',
                                      'NaT'],
           ['2019-07-14T17:20:38.442619152', '2019-07-14T17:20:38.445450712',
            '2019-07-14T17:20:38.448280272', ...,
                                      'NaT',                           'NaT',
                                      'NaT'],
           ['2019-07-26T04:12:15.835573120', '2019-07-26T04:12:15.838086928',
            '2019-07-26T04:12:15.840752032', ...,
                                      'NaT',                           'NaT',
                                      'NaT'],
           ...,
           ['2019-06-11T18:53:04.298630480', '2019-06-11T18:53:04.301459904',
            '2019-06-11T18:53:04.304285392', ...,
                                      'NaT',                           'NaT',
                                      'NaT'],
           ['2019-06-15T18:44:44.575919656', '2019-06-15T18:44:44.694307904',
            '2019-06-15T18:44:44.702760376', ...,
                                      'NaT',                           'NaT',
                                      'NaT'],
           ['2019-06-23T05:44:45.075140048', '2019-06-23T05:44:45.077695296',
            '2019-06-23T05:44:45.080217568', ...,
                                      'NaT',                           'NaT',
                                      'NaT']], dtype='datetime64[ns]')

• Data variables: (10)
  • sc_orient
    (gran_idx)
    int8
    0 0 0 0 0 0 0 0 0 0

    array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int8)

  • cycle_number
    (gran_idx)
    int8
    4 4 4 4 3 3 3 3 3 3

    array([4, 4, 4, 4, 3, 3, 3, 3, 3, 3], dtype=int8)

  • rgt
    (gran_idx)
    int16
    193 254 429 490 ... 1138 1199 1313

    array([ 193,  254,  429,  490,  635,  696,  871, 1138, 1199, 1313],
          dtype=int16)

  • atlas_sdp_gps_epoch
    (gran_idx)
    datetime64[ns]
    2018-01-01T00:00:18 ... 2018-01-...

    array(['2018-01-01T00:00:18.000000000', '2018-01-01T00:00:18.000000000',
           '2018-01-01T00:00:18.000000000', '2018-01-01T00:00:18.000000000',
           '2018-01-01T00:00:18.000000000', '2018-01-01T00:00:18.000000000',
           '2018-01-01T00:00:18.000000000', '2018-01-01T00:00:18.000000000',
           '2018-01-01T00:00:18.000000000', '2018-01-01T00:00:18.000000000'],
          dtype='datetime64[ns]')

  • data_start_utc
    (gran_idx)
    datetime64[ns]
    2019-07-10T17:28:51.719024 ... 2...

    array(['2019-07-10T17:28:51.719024000', '2019-07-14T17:20:38.103241000',
           '2019-07-26T04:12:14.952859000', '2019-07-30T04:04:02.589239000',
           '2019-05-09T20:25:25.949206000', '2019-05-13T20:17:05.409521000',
           '2019-05-25T07:08:35.164495000', '2019-06-11T18:53:03.920437000',
           '2019-06-15T18:44:44.048325000', '2019-06-23T05:44:42.577083000'],
          dtype='datetime64[ns]')

  • data_end_utc
    (gran_idx)
    datetime64[ns]
    2019-07-10T17:32:50.538123 ... 2...

    array(['2019-07-10T17:32:50.538123000', '2019-07-14T17:24:37.394595000',
           '2019-07-26T04:16:14.491682000', '2019-07-30T04:07:52.078075000',
           '2019-05-09T20:29:25.010041000', '2019-05-13T20:21:09.109380000',
           '2019-05-25T07:12:34.982977000', '2019-06-11T18:57:04.060639000',
           '2019-06-15T18:49:01.320586000', '2019-06-23T05:48:35.816682000'],
          dtype='datetime64[ns]')

  • h_li
    (spot, gran_idx, photon_idx)
    float32
    nan nan nan nan ... nan nan nan nan

    contentType :

    physicalMeasurement

    description :

    Standard land-ice segment height determined by land ice algorithm, corrected for first-photon bias, representing the median- based height of the selected PEs

    long_name :

    Land Ice height

    source :

    section 4.4

    units :

    meters

    array([[[      nan,       nan,       nan, ...,       nan,       nan,
                   nan],
            [1687.3898, 1687.3912, 1687.3843, ...,       nan,       nan,
                   nan],
            [      nan,       nan,       nan, ...,       nan,       nan,
                   nan],
            ...,
            [1645.5754, 1645.5027, 1645.3977, ...,       nan,       nan,
                   nan],
            [1720.8497, 1499.9307, 1502.5663, ...,       nan,       nan,
                   nan],
            [      nan,       nan,       nan, ...,       nan,       nan,
                   nan]]], dtype=float32)

  • latitude
    (spot, gran_idx, photon_idx)
    float64
    79.99 79.99 79.99 ... nan nan nan

    contentType :

    physicalMeasurement

    description :

    Latitude of segment center, WGS84, North=+,

    long_name :

    Latitude

    source :

    section 3.10

    standard_name :

    latitude

    units :

    degrees_north

    valid_max :

    90.0

    valid_min :

    -90.0

    array([[[79.9873394 , 79.98716445, 79.98698951, ...,         nan,
                     nan,         nan],
            [79.71909517, 79.71891986, 79.71874457, ...,         nan,
                     nan,         nan],
            [64.97326552, 64.97344122, 64.97361973, ...,         nan,
                     nan,         nan],
            ...,
            [79.99380029, 79.9936253 , 79.9934503 , ...,         nan,
                     nan,         nan],
            [79.98454515, 79.97718805, 79.97666285, ...,         nan,
                     nan,         nan],
            [65.47531434, 65.47550084, 65.47567935, ...,         nan,
                     nan,         nan]]])

  • longitude
    (spot, gran_idx, photon_idx)
    float64
    -29.97 -29.97 -29.97 ... nan nan

    contentType :

    physicalMeasurement

    description :

    Longitude of segment center, , WGS84, East=+

    long_name :

    Longitude

    source :

    section 3.10

    standard_name :

    longitude

    units :

    degrees_east

    valid_max :

    180.0

    valid_min :

    -180.0

    array([[[-29.96751367, -29.9677367 , -29.9679601 , ...,          nan,
                      nan,          nan],
            [-31.06723432, -31.06744263, -31.06765123, ...,          nan,
                      nan,          nan],
            [-37.79214255, -37.79224754, -37.79229347, ...,          nan,
                      nan,          nan],
            ...,
            [-30.29610761, -30.29632976, -30.29655164, ...,          nan,
                      nan,          nan],
            [-31.08650992, -31.09562877, -31.09628802, ...,          nan,
                      nan,          nan],
            [-37.40760598, -37.40765703, -37.4077035 , ...,          nan,
                      nan,          nan]]])

  • gt
    (gran_idx, spot)
    <U4
    'gt1r' 'gt1r' ... 'gt1r' 'gt1r'

    array([['gt1r'],
           ['gt1r'],
           ['gt1r'],
           ['gt1r'],
           ['gt1r'],
           ['gt1r'],
           ['gt1r'],
           ['gt1r'],
           ['gt1r'],
           ['gt1r']], dtype='<U4')

• Indexes: (3)
  • photon_idx
    PandasIndex

    PandasIndex(Index([    0,     1,     2,     3,     4,     5,     6,     7,     8,     9,
           ...
           80662, 80663, 80664, 80665, 80666, 80667, 80668, 80669, 80670, 80671],
          dtype='int64', name='photon_idx', length=80672))

  • spot
    PandasIndex

    PandasIndex(Index([2], dtype='uint8', name='spot'))

  • gran_idx
    PandasIndex

    PandasIndex(Index([ 19303.0,  25403.0,  42903.0,  49003.0,  63502.0,  69602.0,  87102.0,
           113802.0, 119902.0, 131302.0],
          dtype='float64', name='gran_idx'))

• Attributes: (3)

  data_product :

  ATL06

  Description :

  The land_ice_height group contains the primary set of derived ATL06 products. This includes geolocation, height, and standard error and quality measures for each segment. This group is sparse, meaning that parameters are provided only for pairs of segments for which at least one beam has a valid surface-height measurement.

  data_rate :

  Data within this group are sparse. Data values are provided only for those ICESat-2 20m segments where at least one beam has a valid land ice height measurement.

Here we have an xarray Dataset, a common Python data structure for analysis. To visualize the data we can plot it using:

# single mabplotlib axis
ax = ds.isel(gran_idx=0).plot.scatter(x="longitude", y="latitude", hue="h_li")

It would be great if we could see the data on a background map. To do this, we’ll use the GeoViews library.

Notice: The interactive plots are not rendered within this Jupyterbook because the are too big to include in a JupyterNotebook on GitHub.

tile = gv.tile_sources.EsriImagery.opts(width=500, height=500)
tile

First, we must convert our data to geodetic coordinates and add them to the DataSet.

x, y = datashader.utils.lnglat_to_meters(ds.longitude, ds.latitude)

ds = ds.assign(x=x, y=y)

# create our plot
is2 = ds.hvplot.scatter(x="x", y="y", groupby=["data_start_utc"], rasterize=True)

is2

# background via geoviews
is2 * tile

When to Cloud

The astute user has by now noticed that in this tutorial we read in a minimally sized dataset from the S3 bucket. Due to the way ICESat-2 data is stored on disk (because of the file format - it doesn’t matter if it’s a local disk or cloud disk), accessing the data within the file is really slow via a virtual file system. Several efforts are under way to help address this issue, and icepyx will implement them as soon as they are available. Current efforts include:

• storing ICESat-2 data in a cloud-optimized format

• reading data using the h5coro library

• utilizing optimized read paramaters via the underlying C libraries (versus the defaults, which are optimized for a local file system).

Please let Jessica know if you’re interested in joining any of these conversations (or telling us what issues you’ve encountered). We’d love to have your input and use case!

Summary

In this notebook we explored the opening and rendering ATL06 data with icepyx. We saw that icepyx will read in the desired variables directly in the cloud. The ATL06 data has a folder-like structure with many variables to choose from. We focused on h_li.

More information about ATL06 or icepyx can be found in:

• The icepyx documentation

• The Algorithm Theoretical Basis Document (ATBD)