A tool for downloading and processing wildfire-related geospatial data from multiple sources for machine learning and analysis.

Software DOI: 10.5281/zenodo.20743742.

Requires Python 3.12+ (tested on 3.14) and uv. FireDataForge is run from source (it has no installed console script), so get the code and create the environment with:

git clone https://github.com/xiazeyu/FireDataForge.git
cd FireDataForge
uv sync

uv sync creates a local .venv with every dependency. The python main.py … and python plot.py … commands throughout this README assume that environment is active — either activate it once per shell:

source .venv/bin/activate          # Windows: .venv\Scripts\activate

…or prefix each command with uv run (e.g. uv run python main.py <event_id>) to run it in the project environment without activating.

The first time you run FireDataForge (or anytime via python main.py --setup), a short interactive wizard configures the optional credentials and saves them to a local .env file. Everything is optional — skip any step and the wizard tells you exactly which output features become unavailable.

python main.py --setup

# Authenticate (one-time, opens browser)
earthengine authenticate

# Set your project ID
earthengine set_project <YOUR-PROJECT-ID>

# Request a key (instant, free): https://firms.modaps.eosdis.nasa.gov/api/map_key/
export FIRMS_MAP_KEY=your-map-key-here
# ...or in .env:  FIRMS_MAP_KEY=your-map-key-here

# Download the example fires from Zenodo
python main.py --fetch-examples

# Process a single fire event given its MTBS Event ID
python main.py CA3432611848120191010

The argument is an MTBS Event ID — FireDataForge's canonical event key. Any valid ID works (unrecognized IDs are resolved live from mtbs.gov, no archive needed); see Looking up an MTBS Event ID and Event ID Format for the character scheme. CA3432611848120191010 is the 2019 Saddleridge fire, one of the bundled example events.

python main.py <event_id> [options]

# From comma-separated list  
python main.py --batch CA123,CA456,CA789 [options]

# From a file (one event ID per line)
python main.py --batch events.txt [options]

Target CRS must be projected/metric. --resolution and --buffer are in meters, and the grid snaps the projected bounds to whole multiples of the resolution. A geographic CRS in degrees such as EPSG:4326 is therefore not supported and is rejected up front with a clear error, since meter-valued settings applied in degrees would produce a nonsensical grid. Use a projected, meter-based CRS: the default EPSG:5070 (CONUS Albers), a UTM zone, or similar.

# High resolution processing
python main.py CA3432611848120191010 -r 10 -v

# With temporal interpolation (3 intermediate frames)
python main.py CA3432611848120191010 -t 3

# Custom output directory
python main.py CA3432611848120191010 -o ./my_output

# Batch process from file with 4 workers
python main.py --batch events.txt -w 4 -o results/

# Batch process specific events
python main.py --batch CA123,CA456,CA789 --workers 3

# Process only a single feature (for quick debugging)
python main.py CA3432611848120191010 --only frp_daytime

# Process multiple specific features
python main.py CA3432611848120191010 --only frp_daytime,frp_nighttime,elevation

# Regenerate only weather data
python main.py CA3432611848120191010 --only hrrr

Data is saved as .npy files in output/<event_id>/:

output/CA3432611848120191010/
├── task_summary.json     # Per-layer outcome (ok / skipped / failed) + reasons + metadata
├── task_info.npy         # Processing configuration
├── coordinates.npy       # Pixel-center x/y coordinates + CRS for the grid
├── burn_perimeter.npy    # Fire perimeter time series
├── fireline.npy          # Active fireline time series (perimeter differences)
├── fireline_max_frp.npy  # Max FRP painted onto each fireline segment
├── frp_daytime.npy       # Daytime Fire Radiative Power (MW)
├── frp_nighttime.npy     # Nighttime Fire Radiative Power (MW)
├── elevation.npy         # Terrain elevation
├── canopy_bulk_density.npy  # Canopy Bulk Density
├── canopy_cover.npy      # Canopy Cover
├── recent_burn.npy       # Most-recent burn year per pixel (NIFC IFPH, NaN = unburned)
├── r2.npy                # Relative humidity
├── u10.npy               # Wind U component
├── v10.npy               # Wind V component
├── building_height.npy   # Building heights
├── landcover.npy         # Land cover classes
├── lai.npy               # Leaf Area Index
├── sentinel2_rgb.npy     # RGB Sentinel-2 cloudless mosaic
├── terrain_rgb.npy       # Colored shaded-relief terrain RGB (H, W, 3)
└── wui.npy               # Wildland-Urban Interface classification

Only the layers that were successfully produced are written; any that were skipped or failed are omitted from the directory but always recorded in task_summary.json with the reason.

{
  "event_id": "CA3432611848120191010",
  "name": "SADDLERIDGE", "year": 2019, "status": "partial",
  "crs": "EPSG:5070", "resolution_m": 30, "shape": [275, 377],
  "t_start": "2019-10-01T12:00:00", "t_end": "2019-10-16T12:00:00",
  "t_end_estimated": true,
  "has_feds_archive": false, "earth_engine": true, "firms_key": true,
  "notes": ["t_end is an estimate (t_start + 15 days): no FEDS perimeter ..."],
  "layers": {
    "elevation":      {"status": "ok",      "files": ["elevation.npy"]},
    "frp_daytime":    {"status": "ok",      "files": ["frp_daytime.npy"], "n_frames": 7},
    "burn_perimeter": {"status": "skipped", "reason": "no local FEDS archive"},
    "wui":            {"status": "failed",  "reason": "..."}
  },
  "counts": {"ok": 2, "skipped": 1, "failed": 1}
}

from main import load_numpy

coords = load_numpy('output/CA3432611848120191010/coordinates.npy')
x, y = coords.data                  # 1-D arrays, shape (width,) and (height,)
geo = coords.georeference           # typed GeoReference (see schemas.py)
crs = geo.crs                       # e.g. 'EPSG:5070'
crs_wkt = geo.crs_wkt               # full WKT2 (works without an EPSG db)
crs_proj4 = geo.crs_proj4           # legacy PROJ string
bounds = geo.bounds                 # (minx, miny, maxx, maxy)
height, width = geo.shape
a, b, c, d, e, f = geo.transform    # affine: (col, row) -> (x, y)

# Plot everything (default): the combined overview grid, one PNG per channel,
# and one time-series figure per multi-frame layer
python plot.py CA3432611848120191010

# Only the combined overview grid
python plot.py CA3432611848120191010 --mode overview

# Only one PNG per channel
python plot.py CA3432611848120191010 --mode channels

# Only the time-series figures (one per multi-frame layer)
python plot.py CA3432611848120191010 --mode timeseries

# Overview grid + per-channel PNGs, but skip time series
python plot.py CA3432611848120191010 --mode both

# Also write the overview as PDF (PNG only by default)
python plot.py CA3432611848120191010 --pdf

# Plot and display interactively
python plot.py CA3432611848120191010 --show

# Plot the time series for ONLY one layer (e.g., burn perimeters)
python plot.py CA3432611848120191010 -t burn_perimeter

# Batch plot multiple events
python plot.py --batch events.txt

# Batch plot from comma-separated list
python plot.py --batch CA123,CA456,CA789

from main import load_numpy

# Load a data file
data = load_numpy('output/CA3432611848120191010/elevation.npy')
print(data.name)        # 'elevation'
print(data.data[0].shape)  # (height, width)
print(data.unit)        # 'm'

The output format is a small, stable API defined in schemas.py — the typed dataclasses every layer deserializes into. load_numpy returns a DataLayer, the universal envelope wrapping a layer's frames, timestamps, unit, categories, and (for the grid) a GeoReference; FireEvent, ProcessingTask, and ProcessingArgs describe the event and run configuration, and SCHEMA_VERSION tags the envelope version.

schemas.py is standard-library only (no rasterio / earthengine / other geo dependencies), so a third-party application can import — or simply vendor — this single file to read, type-check, and validate FireDataForge outputs without installing the rest of the pipeline.

from firedataforge import forge_event, get_fire_info, get_task_info, ProcessingArgs

# One call: resolve, retrieve, harmonize, and write every available layer.
summary = forge_event("CA3432611848120191010", ProcessingArgs(resolution=30))
print(summary["counts"])          # {'ok': ..., 'skipped': ..., 'failed': ...}

# Or step through the resolution stages.
fire = get_fire_info("CA3432611848120191010")
print(f"{fire.name}: {fire.acres_burned} acres")
task = get_task_info(fire, resolution=30)
print(f"Grid: {task.shape}, CRS: {task.crs}")

• examples/ml_dataloader.py — a PyTorch Dataset that stacks the static layers of each event into a (C, H, W) tensor (python examples/ml_dataloader.py [output]).

• examples/fire_spread_demo.py — a self-contained NumPy fire-spread automaton that consumes the harmonized terrain/fuel/wind layers and shows a 3-frame progression with matplotlib (python examples/fire_spread_demo.py [output/<event_id>]).

  Both are written as notebook-style # %% cell scripts — run them top-to-bottom, or open them in a Jupyter/VS Code interactive window.

• validation/ — quantitative checks (reprojection round-trip error, FRP conservation, categorical overall accuracy, continuous RMSE, sub-pixel registration); see validation/README.md.

main.py                  CLI entry point + backward-compatible import surface
schemas.py               public data contract — stdlib-only output dataclasses
plot.py                  visualization of saved layers
firedataforge/
├── constants.py         paths + source-API constants
├── config.py            credentials, first-run wizard, dataset discovery
├── events.py            fire-list resolution + ProcessingTask (grid + time window)
├── io.py                .npy + coordinate persistence
├── pipeline.py          fail-soft per-event / batch orchestration + task summaries
├── cli.py               argument parsing
├── examples.py          --fetch-examples: download the example bundle from Zenodo
├── remote_archive.py    on-the-fly range-fetch of FEDS fires/firepix from Zenodo
├── progress.py          download + zip-extraction progress helpers
└── sources/             one module per source family
    ├── mtbs.py          MTBS Event-ID → metadata
    ├── feds.py          perimeter / fireline / fireline_max_frp
    ├── frp.py           VIIRS FRP (FIRMS / firepix)
    ├── gee.py           Earth Engine layers (3DEP, LANDFIRE, GBA, WorldCover, LAI, S2, terrain)
    ├── weather.py       HRRR
    ├── wui.py           Global WUI
    └── nifc.py          recent burns

Every source is reprojected to the target CRS and resampled to the target grid with a single, direction-independent method chosen by the data type — the same method is used whether the native source is finer or coarser than the target grid (there is no conditional on the scale ratio). Categorical layers keep nearest-neighbour deliberately: it introduces no mixed/invented classes and preserves the expected class proportions (majority-vote aggregation would erode thin minority features such as narrow WUI strips).

Choosing the target resolution. A fine target grid does not create information that a coarse source lacks — upsampling 3 km HRRR weather to 30 m yields smooth but not genuinely fine fields, so treat such layers as their native resolution despite the grid spacing. Conversely a coarse target grid discards real local variation in fine sources (terrain, fuels, buildings, WUI). Each layer records its true native_resolution (and, where it differs, its current_resolution) — both in meters — in the saved metadata so the scale mismatch is never hidden.

Each VIIRS active-fire detection reports a Fire Radiative Power (FRP, MW) at a point. Rather than dropping the whole value into one grid cell, the detection's FRP is spread over a Gaussian footprint the size of the VIIRS sensor pixel, so the rasterized field reflects the sensor's true spatial uncertainty while conserving the total radiative power.

For a detection with value F at grid position (pₓ, p_y) (pixel units):

• Footprint width. σ = (source_resolution / target_resolution) / 2 pixels, with source_resolution = 375 m (VIIRS). At a 30 m grid, σ = 375/30/2 = 6.25 px (= 187.5 m), giving a full-width-at-half-maximum FWHM = 2√(2 ln 2)·σ ≈ 2.355 σ ≈ 441 m.
• Kernel extent. Weights are evaluated over a square window of radius ⌈3σ⌉ pixels (covering ±3σ, ≈ 99.7 % of the Gaussian mass).
• Weights. A pixel whose center is at distance d (pixels) from the detection gets w = exp(−d² / (2σ²)).
• Normalization (conservation). The in-grid weights are normalized to sum to one and the deposit is F · w / Σw. Summing the rasterized footprint therefore recovers F exactly for any detection whose footprint lies inside the grid; the only loss is the fraction of a footprint that falls off the grid edge. Each pixel value is thus a share of the detection's FRP (≪ the observed MW), not the observed value itself.

The validation/ suite re-splats each event's detections and reports the relative conservation error (typically far below 1 %); see validation/README.md.

FireDataForge degrades gracefully at three levels — a whole layer, an individual timestep, and an individual pixel — and records every gap, so a partial cube is never silently mistaken for a complete one.

Layer level — fail-soft. If a source is unauthenticated, unreachable, or under maintenance, only the layers that depend on it are dropped; every other layer is still produced. A dropped layer is omitted from the event directory and recorded in task_summary.json with status: "skipped"/"failed" and a human-readable reason (see the Output section's task-summary schema).

Pixel level — per-layer nodata sentinels. There is no universal nodata value; the sentinel follows the data type, so check this table (and each layer's unit) before masking:

Timestep level — the per-source datetime cursor. Time-varying layers (burn_perimeter, fireline, fireline_max_frp, frp_*, and the HRRR fields) store only the frames actually observed, held in DataLayer.data paired one-to-one with DataLayer.timestamps (schemas.py). Sources observe at different, irregular cadences — FEDS perimeters/firelines every 12 h, VIIRS at ~2 overpasses/day, HRRR hourly — and each keeps its own timestamp vector, so cadences are never resampled onto a shared clock. A consumer reads each layer through a datetime cursor: at simulation time t, take the frame at the most recent timestamps[i] <= t for that layer, advancing each source independently — the same "most-recent observation ≤ t" rule the pipeline itself uses to mask FRP to the live perimeter (firedataforge/sources/frp.py). Consequences:

• A skipped or missing overpass is a no-op: the previous frame stays current until the next real observation. The pipeline never forward-fills synthetic values or fabricates a frame for a cadence it did not observe. (Pass --interpolation N to explicitly synthesize N SDF-interpolated perimeter frames between timesteps.)
• The cursor never exposes a not-yet-valid observation: it cannot return a frame whose timestamp is after t, so temporal alignment is correct by construction — there is nothing to "synchronize" after the fact.
• Out-of-window / empty sources are excluded upstream: every frame is bounded to the event's active-burning window (t_start–t_end); a source with zero valid observations in that window degrades to the layer-level fail-soft case above rather than emitting empty frames.

Beyond gaps, every .npy envelope is self-describing: it carries source (provenance/attribution), unit, native_resolution (the source's resolution in meters), timestamps, and — for categorical layers — a categories map, while the grid CRS/transform lives in coordinates.npy. See Data contract (schemas.py).

Any MTBS Event ID (Monitoring Trends in Burn Severity) is a valid input, and you do not need the FEDS-MTBS archive to choose or resolve one: unrecognized IDs are looked up live from mtbs.gov at run time, and python main.py --build-firelist caches the full MTBS list offline for browsing and network-free resolution.

If you have the full FEDS-MTBS archive, the summary fireslist_FEDS25MTBS_2012-2024.geojson (the Event_ID column) is a ready-made list of all 7,739 FEDS-covered events (2012–2024).

See Event ID Format for the {STATE}{LAT}{LON}{DATE} scheme.

python main.py --fetch-examples stages the eight fires below and writes their IDs to events.txt at the repo root (ready for --batch events.txt); they are the events used for the benchmark and the quantitative validation. The bundle — these eight fires plus the benchmark and validation reference outputs — is archived on Zenodo as a citable reproducibility artifact (doi:10.5281/zenodo.20743743). They span 2013–2025 and ~9.7k–189k acres across 5 California (chaparral / coastal, wind-driven) and 3 Colorado (montane conifer / grassland, terrain-driven) fires, chosen to exercise the pipeline across diverse fuels, scales, and spread regimes.

Burned area is the MTBS BurnBndAc. The two 2025 fires (Palisades, Eaton) are not yet in the MTBS final record, so the value marked † is the MTBS Provisional Initial Assessment acreage instead (the pre-final figure FireDataForge resolves these events to; see PROVISIONAL_IA in firedataforge/sources/mtbs.py). It is not directly comparable to the FEDS VIIRS perimeter area in each fire's GeoPackage, which runs larger (~31.8k ac Palisades, ~18.6k ac Eaton) because the 375 m active-fire perimeter over-bounds the refined MTBS burn boundary. The active-fire window is each event's perimeter-growth period taken from the FEDS progression (observation-derived, not the estimated fallback); the exact t_start/t_end (including the time of day) are in each event's task_summary.json. Grid is the output raster size on the default EPSG:5070 30 m grid (100 m buffer); it scales with -r/-b.

If you know a fire by name, year, or location rather than by ID:

1. Open the MTBS Data Explorer (interactive map) or the Direct Download / search page.
2. Filter by fire name, year, and state/region (or click the fire on the map).
3. Read the Fire ID (a.k.a. Event ID) field of the matching record — e.g. CA3432611848120191010 — and pass it to python main.py <Event_ID>.

Querying by the exact MTBS Event ID (rather than a name or bounding box) is what makes event matching unambiguous and reproducible: one ID always resolves to the same fire. To browse offline, run python main.py --build-firelist once and search the resulting cache/mtbs_firelist.csv.

FEDS-MTBS derives fire perimeters and firelines every 12 hours from VIIRS active-fire hotspots via object-based tracking, constrained to MTBS burn records.

Note: The FEDS-MTBS archive is optional. FireDataForge streams each requested fire on demand, a few hundred KB per fire with no full download — run python main.py --fetch-examples to stage the eight example fires, or grab the full archive for heavy offline use. Without it the FEDS layers — burn_perimeter, fireline, fireline_max_frp — are skipped and FRP comes from NASA FIRMS unmasked; every other layer is unaffected.

datasets/FEDS25MTBS/
├── fireslist_FEDS25MTBS_2012-2024.geojson  # Optional: MTBS perimeters + metadata (7,739 fires)
├── fireslist_examples.csv   # Example fire metadata (Event_ID, Year, tst, ted, bbox)
├── firepix/                 # Per-fire VIIRS active-fire CSVs (pre-2025 FRP source)
├── 2012/
│   └── CA3245811923420120801.gpkg
├── ...
└── 2024/
    └── ...

Fire Radiative Power comes from NASA FIRMS VIIRS (Collection 2) active-fire detections. When the FEDS archive is available, pre-2025 fires use the bundled FEDS firepix CSVs and FRP is masked to the perimeter at each timestep; otherwise FRP is streamed from FIRMS for any year and left unmasked (perimeter_masked: false in the layer metadata). The FIRMS Area API archive (S-NPP) covers the full FEDS period.

The ESA WorldCover dataset provides global land cover classification at 10m resolution based on Sentinel-1 and Sentinel-2 data.

The Global Wildland-Urban Interface (WUI) dataset maps where buildings and wildland vegetation meet or intermingle at 10m resolution globally.

cache/GlobalWUI/
├── X0065_Y0040/
│   └── WUI.tif      # streamed + cached on first use
├── X0066_Y0040/
│   └── WUI.tif
└── ...

Every output layer and the underlying source dataset, with primary citation and the exact product version / Earth Engine asset ID or service endpoint used. All Earth Engine layers are accessed through Google Earth Engine (Gorelick et al. 2017).

This work is supported by the University of Virginia's Environmental Institute. PNNL is operated by DOE by the Battelle Memorial Institute under contract DE-AC05-76RL01830.

This product contains modified Copernicus Sentinel data (2017–2025), used to derive the sentinel2_rgb surface-reflectance imagery layer.

See LICENSE.