gedih3.gedidriver#

GEDI data file operations and HDF5 I/O.

This module provides classes and functions for working with GEDI HDF5 files, including parsing filenames, extracting data, and building Dask DataFrames.

Attributes#

logger

Classes#

`GEDIFile`	Parser for GEDI HDF5 filenames.
`GEDIShot`	Decoder for GEDI shot numbers.

Functions#

`soc_prod_from_file`(→ str)	Extract GEDI product code from a file path.
`soc_from_file`(→ str)	Get the SOC root directory from a GEDI file path.
`write_soc_manifest`(→ int)	(Re)generate the SOC manifest sentinel by walking soc_dir once.
`soc_file_tree`(→ Union[Dict[str, Dict[str, str]], ...)	Build a structured tree of GEDI SOC files grouped by orbit/track.
`gedi_subset`(source_file, dest_file, variables[, ...])
`gedi_file_glob`([orbit, orbit_granule, track, product, ...])
`gedi_vars_expand`(product_vars[, version])
`expand_var_wildcards`(var_specs, available_vars)	Expand fnmatch-style wildcard patterns in variable specifications.
`gedi_vars_from_h5`(gedi_file)
`gedi_vars_static`(product[, version])	Return the canonical variable list for a GEDI `(product, version)`
`check_soc_file_vars`(soc_file, available_products)
`validate_soc_files`(product_vars[, soc_dir, version, ...])	Validate `product_vars` against the shipped static manifests.
`wfm_pad`(wave[, n_bins, cval])
`process_waveforms`(starts, ends, noises, wfs[, n_bins])
`wfm_extract`(h5_file, beam, idx[, tx])
`load_h5`(→ pandas.DataFrame)	Load data from a GEDI HDF5 file into a pandas DataFrame.
`load_h5_merged`(→ Optional[pandas.DataFrame])	Load and merge data from multiple GEDI product files.
`dask_h5_merged`(→ dask.dataframe.DataFrame)	Create a Dask DataFrame from multiple GEDI product file sets.
`add_special_columns`(df[, lon_col, lat_col, dat_col])

Module Contents#

gedih3.gedidriver.logger#

gedih3.gedidriver.soc_prod_from_file(file_path: str | pathlib.Path) → str[source]#

Extract GEDI product code from a file path.

Parameters:

file_pathstr or Path: Path to a GEDI HDF5 file

Returns:

str: Product code (e.g., ‘L2A’, ‘L4A’)

gedih3.gedidriver.soc_from_file(file_path: str | pathlib.Path) → str[source]#

Get the SOC root directory from a GEDI file path.

Parameters:

file_pathstr or Path: Path to a GEDI HDF5 file

Returns:

str: Path to the SOC root directory

gedih3.gedidriver.write_soc_manifest(soc_dir: str, files=None) → int[source]#

(Re)generate the SOC manifest sentinel by walking soc_dir once.

Producer-driven refresh (R2): every code path that mutates a SOC tree (gh3_download, gh3_build –download, s3_etl_subset, gh3_build -i exit, gh3_doctor –fix soc_health) calls this on exit. Cheaper than letting every consumer re-walk the tree.

Walks the SOC tree in parallel via gedih3.parallel.walk_soc_parallel() (year/doy fan-out on the registered dask Client). Returns the number of files written to the manifest (0 when soc_dir is missing / empty).

Parameters:

soc_dir: SOC root directory.
fileslist[str], optional: Pre-computed absolute file list to use instead of walking. When the caller has already enumerated the tree (e.g. cli/gh3_build.py after its existing-h5 listing), passing the list here avoids the redundant parallel walk.

gedih3.gedidriver.soc_file_tree(file_struct: str | List[str] | List[earthaccess.store.EarthAccessFile], to_list: bool = False, glob_kwargs: Dict | None = None, exclude: List[str] | None = None) → Dict[str, Dict[str, str]] | List[Dict[str, str]][source]#

Build a structured tree of GEDI SOC files grouped by orbit/track.

Organizes GEDI HDF5 files into a nested structure for easy access to related files across different products (L2A, L2B, L4A, etc.).

Parameters:

file_structstr, list of str, or list of EarthAccessFile: Either a directory path containing GEDI files, or a list of file paths or EarthAccessFile objects.
to_listbool, default False: If True, return a list of dicts instead of a nested dict.
glob_kwargsdict, optional: Keyword arguments passed to gedi_file_glob() for filtering files (e.g., version, orbit).
excludelist of str, optional: fnmatch-style patterns matched against each file’s basename. Any file whose basename matches any pattern is dropped from the result. Useful to exclude internal/SGS variants (e.g. ['*_SGS.h5']) that share the SOC tree with public release files.

Returns:

dict or list: If to_list=False: dict keyed by orbit_track identifier, with values being dicts mapping product codes to file paths. If to_list=True: list of product dicts.

Examples

>>> tree = soc_file_tree('/path/to/soc')
>>> tree['O12345_01_T00001']
{'L2A': '/path/to/GEDI02_A_...h5', 'L4A': '/path/to/GEDI04_A_...h5'}
>>> # Exclude NASA SGS internal files from discovery
>>> tree = soc_file_tree('/path/to/soc', exclude=['*_SGS.h5'])

gedih3.gedidriver.gedi_subset(source_file, dest_file, variables, subset_beams=None)[source]#

gedih3.gedidriver.gedi_file_glob(orbit=None, orbit_granule=None, track=None, product: int = None, level: str = None, ppds: int = None, pge: int = None, generation: int = None, version: int = None)[source]#

gedih3.gedidriver.gedi_vars_expand(product_vars, version=None)[source]#

gedih3.gedidriver.expand_var_wildcards(var_specs, available_vars)[source]#

Expand fnmatch-style wildcard patterns in variable specifications.

Supports *, ?, [seq], [!seq]. Non-pattern specs pass through unchanged. Raises GediValidationError if a pattern matches nothing.

gedih3.gedidriver.gedi_vars_from_h5(gedi_file)[source]#

gedih3.gedidriver.gedi_vars_static(product, version=None)[source]#

Return the canonical variable list for a GEDI (product, version) from the static manifest shipped in gedih3.data — no I/O on any target file.

NASA release files of the same (product, version) always carry an identical dataset schema, so this is equivalent to gedi_vars_from_h5() for canonical release files but free.

Returns None when no static manifest exists for the requested combination; callers should fall back to gedi_vars_from_h5().

Note: the result reflects a NASA release file’s schema. Files that were previously subset (e.g. via gedi_subset() or S3 ETL mode) contain a subset of these variables — for those, use gedi_vars_from_h5() to introspect what is actually present.

gedih3.gedidriver.check_soc_file_vars(soc_file, available_products)[source]#

gedih3.gedidriver.validate_soc_files(product_vars: Dict, soc_dir: str = GH3_DEFAULT_SOC_DIR, version: int | None = None, exclude: List[str] | None = None)[source]#

Validate product_vars against the shipped static manifests.

Only authoritative for canonical NASA release files validated against the per-product manifest in src/gedih3/data/GEDI*_DATASETS_*.txt — that is, for the fresh-build default request (Regime A) or for a resume where the user explicitly re-requests default for a product (Regime C). On all other resume paths the build log is the contract, and gh3_build.py bypasses this check entirely. Callers are responsible for filtering product_vars to only the products they actually want to validate against the static manifest — passing the full union of existing + new vars on a resume will produce false negatives whenever the shipped manifest has drifted relative to the list saved in the build log.

class gedih3.gedidriver.GEDIFile(file_path: str | pathlib.Path | earthaccess.store.EarthAccessFile)[source]#

Parser for GEDI HDF5 filenames.

Extracts metadata from GEDI filename convention including orbit, granule, track, version, and date information.

Attributes:

file_pathstr: Full path to the GEDI file
full_namestr: Basename of the file
productstr: Product identifier (e.g., ‘GEDI02’)
levelstr: Processing level (e.g., ‘A’, ‘B’)
datedatetime: Acquisition date and time
product_codestr: Combined product code (e.g., ‘L2A’, ‘L4A’)
orbitint: Orbit number
orbit_granuleint: Granule number within orbit
trackint: Ground track number
versionint: Data version number
yearstr: Year string (e.g., ‘2019’)
doystr: Day-of-year string (e.g., ‘338’)
mission_weekint: Mission week number since GEDI launch (2018-12-13)
suffixstr or None: Extra filename parts after version field, if any

Examples

>>> gf = GEDIFile('/path/to/GEDI02_A_2020123120000_O12345_01_T00001_02_003_01_V002.h5')
>>> gf.orbit
12345
>>> gf.version
2

__str__() → str[source]#

parse_file(f: str) → None[source]#

get_glob_pattern(product: int = 1, level: str = 'B', ppds: int = 2)[source]#

classmethod from_filename(file_path)[source]#

search_file(soc_dir: str = None, product: int = 1, level: str = 'B', other_files=None)[source]#

class gedih3.gedidriver.GEDIShot(shot: int | numpy.ndarray, build_glob: bool = False)[source]#

Bases: GEDIFile

Decoder for GEDI shot numbers.

Extracts orbit, beam, track, and other metadata encoded in GEDI shot numbers. The shot number is a 64-bit integer encoding multiple fields.

Attributes:

shotint or array: The shot number(s)
orbitint or array: Orbit number
orbit_granuleint or array: Granule number within orbit
trackint or array: Ground track number
beamint or array: Beam number (0-7)
beam_strstr or list: Beam identifier string (e.g., ‘BEAM0101’)
powerbool or array: True if power beam (beam > 3)
shot_indexint or array: Shot index within the granule

Examples

>>> shot = GEDIShot(12345678901234567890)
>>> shot.beam_str
'BEAM0101'

parse_shot(shot: int | numpy.ndarray) → None[source]#

get_glob_pattern(product: int = 1, level: str = 'B', ppds: int = 2, pge: int = None, generation: int = None, version: int = None)[source]#

search_file(soc_dir: str)[source]#

gedih3.gedidriver.wfm_pad(wave, n_bins=1420, cval=0)[source]#

gedih3.gedidriver.process_waveforms(starts, ends, noises, wfs, n_bins=1420)[source]#

gedih3.gedidriver.wfm_extract(h5_file, beam, idx, tx=False)[source]#

gedih3.gedidriver.load_h5(fpath: str | pathlib.Path | earthaccess.store.EarthAccessFile, columns: List[str], which_beams: List[str] | None = None, shots: numpy.ndarray | None = None, include_source: bool = True, dropna: bool = True) → pandas.DataFrame[source]#

Load data from a GEDI HDF5 file into a pandas DataFrame.

Parameters:

fpathstr, Path, or EarthAccessFile: Path to the GEDI HDF5 file
columnslist of str: Variable names to extract from the file
which_beamslist of str, optional: Specific beams to load (e.g., [‘BEAM0101’, ‘BEAM1000’]). If None, loads all beams.
shotsarray, optional: Specific shot numbers to extract. If None, loads all shots.
include_sourcebool, default True: Add root_file and root_beam columns to output
dropnabool, default True: Drop rows with NaN values

Returns:

pd.DataFrame: DataFrame indexed by shot_number containing requested variables

gedih3.gedidriver.load_h5_merged(prod_files: Dict[str, str], product_vars: Dict[str, List[str]], which_beams: List[str] | None = None, shots: numpy.ndarray | None = None, dropna: bool = True, suffix_all: bool = False) → pandas.DataFrame | None[source]#

Load and merge data from multiple GEDI product files.

Combines data from different GEDI products (L2A, L2B, L4A, etc.) into a single DataFrame, joining on shot_number.

Parameters:

prod_filesdict: Mapping of product codes to file paths (e.g., {‘L2A’: ‘/path/to/l2a.h5’})
product_varsdict: Mapping of product codes to lists of variable names to extract
which_beamslist of str, optional: Specific beams to load
shotsarray, optional: Specific shot numbers to extract
dropnabool, default True: Drop rows with NaN values
suffix_allbool, default False: If True, suffix all columns with product code (e.g., ‘rh_l2a’). Required when variables have the same name across products.

Returns:

pd.DataFrame or None: Merged DataFrame indexed by shot_number, or None if no data

gedih3.gedidriver.dask_h5_merged(prod_files_list: List[Dict[str, str]], product_vars: Dict[str, List[str]], which_beams: List[str] | None = None, shots: numpy.ndarray | None = None, dropna: bool = True, suffix_all: bool = False, by_beam: bool = False) → dask.dataframe.DataFrame[source]#

Create a Dask DataFrame from multiple GEDI product file sets.

Lazily loads and merges GEDI data for distributed processing.

Parameters:

prod_files_listlist of dict: List of product file dictionaries from soc_file_tree()
product_varsdict: Mapping of product codes to variable lists
which_beamslist of str, optional: Specific beams to load
shotsarray, optional: Specific shot numbers to extract
dropnabool, default True: Drop rows with NaN values
suffix_allbool, default False: Suffix all columns with product code
by_beambool, default False: If True, create separate partitions for each beam (more parallelism)

Returns:

dask.dataframe.DataFrame: Lazy Dask DataFrame for distributed processing

gedih3.gedidriver.add_special_columns(df, lon_col: str = None, lat_col: str = None, dat_col: str = None)[source]#