Use case

Build a training table from per-observation rows

You want an hourly (or finer) feature table for a model. dataset() is deliberately daily — one row per local-standard-time day. The raw per-observation path is obs(granularity=“observation”): every METAR/SPECI row as reported, source-identified. This recipe turns those rows into a clean training table.

Daily vs. per-observation

Call	Granularity	Use it for
`dataset(station, from, to)`	one row per LST day	daily targets, daily features
`obs(station, from, to, granularity="observation")`	one row per observation	hourly features, intraday signals

obs() lives in mostlyright.weather. With granularity="observation" it returns the merged per-report METAR/SPECI rows — the same ones dataset() aggregates — window-trimmed to the queried range with no bucketing or resampling. Pin source= when you need every row to carry a single named provenance so train and infer see identical data. See Source identity.

1. Pull raw per-observation rows

1
from mostlyright.weather import obs
2

3
# Historical archive coverage — pin the IEM feed, per-report grain.
4
rows = obs(
5
    "KNYC", "2024-01-15", "2024-01-20",
6
    source="iem", granularity="observation",
7
)
8
print(len(rows), "rows")          # thousands — one per observation, not per day
9
print(rows[["observed_at", "observation_type", "temp_f", "dewpoint_f"]].head())

Three things to know about the rows you get back:

Use source="iem" (or "iem.archive") for history. The live aviation feed (AWC) only serves roughly the last 168 hours; IEM is the full hourly archive.
Rows are window-trimmed. granularity="observation" trims to the queried UTC range — no whole-calendar-month over-fetch. Callers needing the last LST day’s pre-midnight UTC tail should extend to_date by one day.
METAR + SPECI are mixed. Routine hourly observations are observation_type == "METAR" (near HH:51); off-cycle specials are "SPECI". raw_metar is preserved on every row so you can re-parse with MetPy if you need a field the SDK does not surface.

2. Filter to a clean hourly grid

1
import pandas as pd
2

3
rows["observed_at"] = pd.to_datetime(rows["observed_at"], utc=True)
4
rows = rows.set_index("observed_at").sort_index()
5

6
# Strictly-hourly grid: keep routine METARs, drop off-cycle SPECIs
7
hourly = rows[rows["observation_type"] == "METAR"].copy()

If you want a fixed cadence regardless of reporting gaps, resample and forward-fill within a short limit:

1
hourly = hourly.resample("1h").last().ffill(limit=2)

3. Engineer sub-daily features

1
from mostlyright.transforms import relative_humidity
2

3
hourly["rh"] = relative_humidity(hourly["temp_f"], hourly["dewpoint_f"])
4

5
# Lags + rolling windows over the hourly index
6
hourly["temp_f_lag3h"] = hourly["temp_f"].shift(3)
7
hourly["temp_f_roll6h_max"] = hourly["temp_f"].rolling("6h").max()
8
hourly["dewpoint_f_roll6h"] = hourly["dewpoint_f"].rolling("6h").mean()
9

10
# Time-of-day + calendar
11
hourly["hour"] = hourly.index.hour
12
hourly["doy"] = hourly.index.dayofyear

4. Attach a daily target

A common setup: predict the day’s realized high from features known by mid-morning. Pull the daily target with dataset() and join on the local date:

1
import mostlyright
2

3
daily = mostlyright.dataset("KNYC", "2024-01-15", "2024-01-20")  # one row per LST day
4
daily = daily[["date", "obs_high_f"]].rename(columns={"obs_high_f": "target_high_f"})
5

6
hourly["date"] = hourly.index.tz_convert("America/New_York").strftime("%Y-%m-%d")
7
table = hourly.merge(daily, on="date", how="left")

The target is the realized high — it lies in the future relative to your morning features. That’s the textbook leakage trap. Filter your feature rows to what was knowable at decision time with KnowledgeView, and audit the assembled table before you train:

1
from mostlyright.core.temporal import KnowledgeView
2

3
morning_view = KnowledgeView(table, as_of="2024-01-16T14:00:00Z").filter_known_at()

See Walk-forward model evaluation and Temporal safety.

5. Keep one source end-to-end

Train on source="iem" and you should infer on source="iem". The feeds disagree at the half-degree level (IEM may emit non-integer °F; the aviation feed is integer), which silently degrades a model deployed against the wrong source. Pinning the source with obs(source=...) is the whole point — and validate_dataframe raises SourceMismatchError if a row comes back tagged otherwise. An unpinned fused frame carries the merged.live_v1 tag, which a single-source pinned schema rejects, so a fused frame can never masquerade as single-source.

TypeScript

The TS SDK’s dataset() returns the same daily rows. TS obs() defaults to granularity: "observation" — the per-report rows it has always returned — so a TS caller gets per-observation rows out of the box; granularity: "daily" is not yet ported in TS (documented parity ticket) and throws rather than silently returning per-report rows. For the full offline feature build, Python is first today — build the table in Python, export it, and serve features to a TS runtime.

What this recipe doesn’t do

Gap-fill intelligently. ffill(limit=2) is a blunt instrument; real stations have multi-hour outages you may want to mask, not fill.
Cross-source union. Stitching IEM history onto the live AWC tail needs dedupe by observed_at — and reopens the source-identity question.
Resample the target. This builds hourly features against a daily target. A pure intraday target (next-hour temp) skips step 4 entirely.