# Weather GRIB2 Source: https://mostlyright.md/docs/recipes/weather-grib/ Select one field out of a weather model file and read its values at the places a recipe names. ## The question What two-metre air temperature did the 12 September 2026 00Z GFS run forecast for Denver and New York, six hours ahead? ## The source | | | | --- | --- | | Publisher | NOAA NCEP, through the NOMADS GRIB filter | | Address | `https://nomads.ncep.noaa.gov/cgi-bin/filter_gfs_0p25.pl?file=gfs.t00z.pgrb2.0p25.f006&lev_2_m_above_ground=on&var_TMP=on&dir=%2Fgfs.20260912%2F00%2Fatmos` | | Format | GRIB2, served as `application/octet-stream` | | Cadence | four model cycles a day, and NOMADS keeps roughly the last ten days | | Rights | output of a United States federal model, published without copyright, recorded as `public_domain_asserted` | The filter endpoint makes this buildable. A whole GFS 0.25 degree file runs to hundreds of megabytes and holds hundreds of messages. `weather.grib2` takes a container bounded at 16 MiB and returns a table bounded at 16 MiB. Ask whether a source's file and its decoded table fit, not which family opens it. Asking NOMADS for one level and one variable returns a small multi-message file that fits. The dated address in this example stops resolving. NOMADS keeps about ten days. Substitute a current cycle. A forecast cycle's file never changes once published, which is what `closed: true` says. ## The recipe ```json title="recipe.json" { "dataset": { "name": "Surface temperature at named places, GFS forecast" }, "question": { "text": "What two-metre temperature did the 12 September 2026 00Z GFS run forecast for Denver and New York six hours ahead?" }, "table": { "name": "gfs_temperature_2m", "description": "One row per named place, model run and valid time.", "grain": [ "point_id", "run_time", "valid_time" ], "columns": [ { "name": "point_id", "type": "string", "nullable": false, "description": "The name this recipe gave the place.", "presentation": { "chart": "top_values" } }, { "name": "run_time", "type": "timestamp", "nullable": false, "description": "The model cycle the forecast came from.", "presentation": { "chart": "timeline", "bucket": "day" } }, { "name": "valid_time", "type": "timestamp", "nullable": false, "description": "The moment the forecast value is for.", "presentation": { "chart": "timeline", "bucket": "day" } }, { "name": "temperature_k", "type": "decimal", "nullable": true, "description": "Two-metre air temperature at the nearest grid point, as the model published it.", "presentation": { "chart": "histogram", "bins": 8 } }, { "name": "matched_grid_lat", "type": "decimal", "nullable": false, "description": "Latitude of the grid point that answered, so the attribution can be checked." }, { "name": "matched_grid_lon", "type": "decimal", "nullable": false, "description": "Longitude of the grid point that answered." }, { "name": "message_sha256", "type": "string", "nullable": false, "description": "Digest of the exact GRIB2 message the value was decoded from.", "presentation": { "chart": "none" } } ] }, "sources": [ { "name": "gfs_cycle_2026091200", "description": "The two-metre temperature field of one GFS 0.25 degree model cycle, as the NOMADS filter serves it.", "source_class": "user_url", "data_classification": "public", "locator": { "kind": "https_url", "display_locator": "https://nomads.ncep.noaa.gov/cgi-bin/filter_gfs_0p25.pl?file=gfs.t00z.pgrb2.0p25.f006&lev_2_m_above_ground=on&var_TMP=on&dir=%2Fgfs.20260912%2F00%2Fatmos" }, "rights_claim": { "claimed_basis": "public_domain_asserted", "claim_evidence_digest": "sha256:1111111111111111111111111111111111111111111111111111111111111111", "claim_note": "Output of a United States federal weather model, published by NOAA NCEP without copyright." }, "connector": { "adapter_id": "public.https", "credential_mode": "none", "origin": "https://nomads.ncep.noaa.gov", "parameters": [ { "name": "reader.family_id", "value": "weather.grib2" }, { "name": "reader.family_version", "value": "3.0.0" }, { "name": "reader.decode_options", "value": "{\"admission_allowlist_version\":1,\"collection_profile_version\":1,\"points\":[{\"id\":\"denver\",\"latitude\":\"39.7392\",\"longitude\":\"-104.9903\"},{\"id\":\"new_york\",\"latitude\":\"40.7128\",\"longitude\":\"-74.0060\"}],\"selector\":{\"discipline\":0,\"parameter_category\":0,\"parameter_number\":0,\"run_time\":\"2026-09-12T00:00:00Z\",\"surface_type\":103,\"surface_value\":{\"denominator\":1,\"numerator\":2},\"valid_time\":\"2026-09-12T06:00:00Z\"},\"variable_name\":\"temperature_2m\"}" } ] }, "limits": { "max_source_bytes": 3145728, "max_rows": 1000, "max_requests": 2 }, "closed": true } ], "transform": { "engine": "duckdb_sql", "steps": [ { "step_id": "gfs_temperature_2m", "sql": "select point_id, cast(run_time as timestamptz) as run_time, cast(valid_time as timestamptz) as valid_time, try_cast(value as decimal(12, 4)) as temperature_k, try_cast(matched_grid_lat as decimal(12, 6)) as matched_grid_lat, try_cast(matched_grid_lon as decimal(12, 6)) as matched_grid_lon, message_sha256 from gfs_cycle_2026091200", "description": "Rename the Reader's value column and cast the all-text relation. The Reader emits one row per named place." } ] }, "checks": [ { "check_id": "one_row_per_place_and_time", "kind": "key_uniqueness", "enforcement": "required", "columns": [ "point_id", "run_time", "valid_time" ] }, { "check_id": "temperature_plausible", "kind": "value_range", "enforcement": "required", "columns": [ "temperature_k" ], "min_value": "150", "max_value": "350" }, { "check_id": "every_place_answered", "kind": "row_expectation", "enforcement": "required", "min_rows": 2, "max_rows": 2, "description": "Two places were named and the Reader emits one row per place." } ], "units": [ { "column": "temperature_k", "unit": "K" } ], "timezone": "UTC" } ``` ## Part by part ### The Reader pin `weather.grib2@3.0.0` reads a bounded multi-message GRIB2 collection and selects one message. Versions `1.0.0` and `2.0.0` require one already-extracted message and refuse a multi-message GFS or HRRR file. `3.0.0` indexes the collection and selects, then reuses the version-2 decoder. Zero matches and multiple matches both refuse. An unselected record never crosses the native decode boundary. The selector names the message by its own coordinates: | Setting | Value | What it means | | --- | --- | --- | | `discipline` | `0` | meteorological products | | `parameter_category` | `0` | temperature | | `parameter_number` | `0` | temperature (`TMP`) | | `surface_type` | `103` | a specified height above ground | | `surface_value` | `{"numerator": 2, "denominator": 1}` | two metres | | `run_time` | `2026-09-12T00:00:00Z` | the model cycle | | `valid_time` | `2026-09-12T06:00:00Z` | the moment the value is for | `surface_value` is a numerator and a denominator because the document carries no fractional JSON number. The `3.0.0` selector must resolve to a whole level, because the version-2 decoder it reuses takes one. `points` name the places. Their coordinates are decimal text rather than numbers, for the same reason: `"latitude": "39.7392"`. The record carries the digits written there, identically on every host. Each point needs an `id`, and two points may not share one. `admission_allowlist_version` and `collection_profile_version` pin the approved lists this recipe was written against. A recipe frozen before a list grew never starts opening files it used to refuse. ### What the Reader returns One row per named place, in the order the document names them. Version 2 and 3 emit nine columns: `point_id`, `variable`, `level`, `run_time`, `valid_time`, `value`, `matched_grid_lat`, `matched_grid_lon`, `message_sha256`. `matched_grid_lat` and `matched_grid_lon` are the coordinates of the cell that answered. A reader can measure how far the answering point sits from the place asked for, without re-running anything. ### Where a value comes from A model holds values on a mesh a few kilometres apart. The value returned is the one at the mesh point nearest the named place. A place exactly between two points takes the more southerly one, and the more westerly. The rule uses compass directions because one model's files sometimes start at the north edge and sometimes at the south. The answer then depends on where the place is, not on which way round the publisher wrote the file. The Reader refuses a place outside the area a file covers, naming the place and the four corners of the area. It does not fall back to the closest point on the edge. A reading from four hundred kilometres away is still a number, and it would arrive with nothing to say it came from somewhere else. Every read recomputes the file's own stated corners and compares them with the publisher's. A disagreement beyond the file's own precision halts the refresh. ### Transform One step: rename `value` to the column this table declares, and cast the all-text relation. ```sql select point_id, cast(run_time as timestamptz) as run_time, cast(valid_time as timestamptz) as valid_time, try_cast(value as decimal(12, 4)) as temperature_k, … from gfs_cycle_2026091200 ``` The Reader writes times as `YYYY-MM-DDTHH:MM:SSZ`, which casts to `TIMESTAMP WITH TIME ZONE` directly. ### Checks `row_expectation` with `min_rows: 2` and `max_rows: 2` matches the two named places, because the Reader emits one row per place. Any other count means the selection or the decode did not do what the document says. `value_range` between `"150"` and `"350"` bounds a kelvin temperature and catches a field that turned out to be something other than temperature. ### Cadence `closed: true` says the bytes behind this address never change. It authorizes predecessor reuse in a refresh plan, and the per-source report then shows zero bytes fetched and names the rule that closed it. It does not by itself make this table refresh. Every source here is closed, so the plan has nothing to acquire and Studio returns `RESYNC_REQUIRED` before acquisition. One finished cycle is a finished table, and this recipe is complete once its full run has sealed a version. A table that follows the model forward is many sources, one per cycle, unioned. The other shape is one source whose address carries the cycle date in a path template and declares a `window` with `location: "path"`. That shape refreshes. `closed` and `window` are exclusive, and a source states at most one. ## Register, run, inspect Use the real table for one full build. For a bounded test first, follow the [scratch-table workflow](/docs/build/run-and-inspect/#test-before-the-full-build). A successful sample on the real table can publish truncated data. Stop if the forecast refuses; size the recipe before running. ```sh mr-data dataset create --name "Surface temperature at named places, GFS forecast" --json mr-data recipe recipe.json --json mr-data recipe explain RECIPE_ID --json mr-data recipe forecast RECIPE_ID --digest RECIPE_DIGEST --json mr-data run --recipe RECIPE_ID --digest RECIPE_DIGEST --full --json mr-data watch RUN_ID --json mr-data peek RUN_ID --json mr-data checks RUN_ID --json mr-data receipt RUN_ID --json ``` ## What the run reports The source receipt records the Reader name and version, the validated options digest, the raw source digest and the row digest. `message_sha256` on every row names the GRIB2 message the value came out of, so a reader can trace a row back to its bytes. A point the model marks as having no value is still a row, under the document's name for the place, with the value left blank. The Reader never writes a stand-in number like 9999, which would read as a temperature. The build records that it met one, so a blank cell means the publisher marked the value absent. ## Traps The gate is the grid and the packing together. Placing values correctly and unpacking them are separate abilities, and one alone is not enough. A model that changes its grid halts the next refresh before a single value is read. Models do change grids, usually at a version upgrade. The refusal names the edit and the sample file that would accept the new combination. The check costs a few hundred bytes. The Reader refuses a lossy codec, and approving one more method does not change that. A stored dataset says its numbers are the publisher's numbers, and a codec free to invent them cannot support the claim. The file has to be republished losslessly. The Reader refuses a file that claims to be enormous before it allocates anything. A message states how many grid points it holds before it holds any of them, and a two-hundred-byte file can claim two million points. The limit is on the claim. The document names the field as well as the places. A file that holds a different field stops the build, and the message says what it found and what the document asked for. Two fields from one model look identical from the outside, both real numbers on the same mesh at the same moment. A table labelled with one field name and carrying the other is wrong in a way nothing downstream can see. A Reader-pinned source runs on the batch shape, not the warm one. A Reader needs a memory ceiling of its own, and the warm container gives a child none. The worker refuses a Reader-pinned source there before fetching anything, and names that reason. The weather family is a lazy optional dependency. First use without a compatible binding answers `READER_DEPENDENCY_UNAVAILABLE`, a typed refusal rather than a crash.