mostlyright.core.schemas

mostlyright.core.schemas

Canonical schemas shipped with mostlyright v0.1.

The three schemas — observation, forecast, settlement — are the shape contracts every weather-vertical adapter (IEM, AWC, NWS CLI) normalises to. See docs/design.md §A, §X, and §BB.3 for the column-by-column specification.

Each schema is eagerly registered with the Validator at import time so validate_dataframe(df, schema_id) works against the canonical IDs without any explicit register-call boilerplate.

Module Attributes

SCHEMA_REGISTRY	Public alias for the validator’s registry dict, so callers and tests can look up schemas by id without reaching into core.validator’s underscored internal.

class mostlyright.core.schemas.EarningsFactSchema

Bases: Schema

schema.earnings_fact.v1 — one row per counted mention occurrence.

Carries the six RESEARCH-MARKETS §2 taxonomy dimensions (term match rule, counting mode, speaker scope, window scope, time, tie-break/resolution) plus the D-27.11 role_source provenance and the derived kalshi_counted flag, so the SAME fact rows resolve correctly under each venue’s wording.

mention_count is the integer PRIMARY tally; the boolean (“said ≥ 1”) is derived as mention_count >= 1 — storing only a bool cannot settle Polymarket “say X 5+ times” threshold brackets (RESEARCH-MARKETS §2.1(2)).

COLUMNS : ClassVar[list[ColumnSpec]] = [ColumnSpec(name=‘ticker’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=”), ColumnSpec(name=‘call_id’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=‘provider event id for the call’), ColumnSpec(name=‘term_canonical’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=‘canonical market term (§2.1(3) term.canonical)’), ColumnSpec(name=‘term_accepted_forms’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=‘JSON-encoded list of accepted surface forms (exploded slash + plural/possessive) — §2.1(3) term.accepted_forms’), ColumnSpec(name=‘term_match_rule’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘exact’, ‘plural_possessive_ok_no_tense’, ‘lemma’, ‘regex’), notes=‘§2.1(3) term.match_rule (plural/possessive OK, NOT tense)’), ColumnSpec(name=‘compound_type’, dtype=‘enum’, units=None, nullable=True, enum_values=(‘standalone’, ‘open’, ‘hyphenated’, ‘closed’, ‘affix_derivation’), notes=“D-30 §2.1(3) per-occurrence compound axis — SEPARATE from term_match_rule (do NOT overload MATCH_RULE_VALUES). standalone/open/hyphenated auto-count on both venues; closed is Polymarket candidate-only + Kalshi-No; affix_derivation counts for neither. Nullable: pre-fix rows omit it (default ‘standalone’)”), ColumnSpec(name=‘matched_surface_form’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=“the actually-spoken string (§2.1(3)) — kept verbatim so each occurrence is auditable against each venue’s stricter/looser rule”), ColumnSpec(name=‘mention_count’, dtype=‘int64’, units=None, nullable=False, enum_values=None, notes=‘PRIMARY integer tally (§2.1(2)); boolean is derived as >= 1 — storing only a bool cannot settle threshold brackets’), ColumnSpec(name=‘counting_mode’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘boolean_ge1’, ‘threshold_ge_n’, ‘exact_n’, ‘range’), notes=‘§2.1(2) counting_mode; boolean markets = boolean_ge1 (n=1)’), ColumnSpec(name=‘threshold_n’, dtype=‘int64’, units=None, nullable=True, enum_values=None, notes=‘§2.1(2) threshold_n (inclusive >= N); null for non-threshold modes’), ColumnSpec(name=‘speaker_role’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘company_executive’, ‘company_ir’, ‘operator_ir’, ‘operator_moderator’, ‘sell_side_analyst’, ‘other_external’, ‘unknown’), notes=‘§2.1(4) speaker_role — drives Kalshi/Polymarket scope divergence’), ColumnSpec(name=‘speaker_name’, dtype=‘string’, units=None, nullable=True, enum_values=None, notes=‘§2.1(4) speaker_name; nullable when unattributed’), ColumnSpec(name=‘role_source’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘transcript_structural’, ‘transcript_self_id’, ‘roster_match’, ‘diarization_advisory’), notes=‘D-27.11 provenance of the role attribution; only the three transcript-anchored values are Kalshi-countable (diarization is advisory only)’), ColumnSpec(name=‘segment’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘operator_intro’, ‘prepared_remarks’, ‘qa’, ‘closing’), notes=‘§2.1(5) call segment of the occurrence’), ColumnSpec(name=‘offset_seconds’, dtype=‘int64’, units=‘seconds’, nullable=True, enum_values=None, notes=‘§2.1(5) seconds into the call (audit trail); nullable’), ColumnSpec(name=‘confidence’, dtype=‘float64’, units=None, nullable=False, enum_values=None, notes=‘0..1 STT/attribution confidence (§2.1(6)) — flags borderline hits’), ColumnSpec(name=‘window_scope’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘full_call’, ‘prepared_remarks_only’, ‘qa_only’, ‘press_release_only’, ‘full_event_incl_release’), notes=‘§2.1(5) time-window scope a market resolves over’), ColumnSpec(name=‘resolution_status’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘pending’, ‘provisional’, ‘resolved_yes’, ‘resolved_no’, ‘no_qualifying_event’, ‘voided’, ‘disputed’), notes=“§2.1(6) resolution_status; ‘provisional’ = live early-signal delta (27-10, D-27.16) — NOT a settlement/backtest source”), ColumnSpec(name=‘kalshi_counted’, dtype=‘bool’, units=None, nullable=False, enum_values=None, notes=‘derived via validate_kalshi_counted_occurrence (D-27.11 fail-closed) — True iff role_source + speaker_role are anchorable’), ColumnSpec(name=‘is_final’, dtype=‘bool’, units=None, nullable=True, enum_values=None, notes=‘STT-segment finality ONLY (D-27.14/D-27.16); a provisional fact delta is is_final=True. NEVER gates settlement authority. Optional — batch rows omit/default True’), ColumnSpec(name=‘spoken_at’, dtype=‘timestamp_utc’, units=None, nullable=True, enum_values=None, notes=‘aired/event_time wallclock of the words (27-10, D-27.16); optional streaming marker — null on batch rows’), ColumnSpec(name=‘event_time’, dtype=‘timestamp_utc’, units=None, nullable=False, enum_values=None, notes=‘call air time (event-time)’), ColumnSpec(name=‘as_of_time’, dtype=‘timestamp_utc’, units=None, nullable=True, enum_values=None, notes=‘knowledge-time = transcript availability post-call (NOT event_date); for a live row stamped at STT-finalization/publish time and >= spoken_at (D-27.16); nullable’), ColumnSpec(name=‘source’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=“per-row source-identity overlay == ‘earnings_call’ (D-27.2)”), ColumnSpec(name=‘delivery’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘live’, ‘hosted’), notes=‘delivery-channel lineage {live,hosted}; NOT source identity (D-27.2)’)]

schema_id : ClassVar[str] = ‘schema.earnings_fact.v1’

class mostlyright.core.schemas.EarningsTranscriptSchema

Bases: Schema

schema.earnings_transcript.v1 — one row per transcript segment.

COLUMNS : ClassVar[list[ColumnSpec]] = [ColumnSpec(name=‘ticker’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=”), ColumnSpec(name=‘call_id’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=‘provider event id for the call’), ColumnSpec(name=‘segment_index’, dtype=‘int64’, units=None, nullable=False, enum_values=None, notes=‘ordinal index of the segment within the call’), ColumnSpec(name=‘segment’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘operator_intro’, ‘prepared_remarks’, ‘qa’, ‘closing’), notes=‘call segment (reuses earnings_fact.SEGMENT_VALUES)’), ColumnSpec(name=‘speaker_name’, dtype=‘string’, units=None, nullable=True, enum_values=None, notes=‘speaker name; nullable when unattributed’), ColumnSpec(name=‘speaker_role’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘company_executive’, ‘company_ir’, ‘operator_ir’, ‘operator_moderator’, ‘sell_side_analyst’, ‘other_external’, ‘unknown’), notes=‘speaker role (reuses earnings_fact.SPEAKER_ROLE_VALUES)’), ColumnSpec(name=‘text’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=‘verbatim segment text’), ColumnSpec(name=‘offset_seconds’, dtype=‘int64’, units=‘seconds’, nullable=False, enum_values=None, notes=‘seconds into the call where this segment begins’), ColumnSpec(name=‘is_final’, dtype=‘bool’, units=None, nullable=True, enum_values=None, notes=‘STT-segment finality ONLY (partial vs final text, D-27.14); NEVER gates settlement authority. Optional — batch rows omit/default True’), ColumnSpec(name=‘spoken_at’, dtype=‘timestamp_utc’, units=None, nullable=True, enum_values=None, notes=‘aired/event_time wallclock of the segment (27-10, D-27.16); optional streaming marker — null on batch rows’), ColumnSpec(name=‘stream_seq’, dtype=‘int64’, units=None, nullable=True, enum_values=None, notes=‘monotonic streaming sequence number (27-10); optional — absent on batch rows’), ColumnSpec(name=‘event_time’, dtype=‘timestamp_utc’, units=None, nullable=False, enum_values=None, notes=‘call air time (event-time)’), ColumnSpec(name=‘as_of_time’, dtype=‘timestamp_utc’, units=None, nullable=True, enum_values=None, notes=‘knowledge-time = transcript availability post-call (NOT event_date); for a live row stamped at STT-finalization/publish time and >= spoken_at (D-27.16); nullable’), ColumnSpec(name=‘source’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=“per-row source-identity overlay == ‘earnings_call’ (D-27.2)”), ColumnSpec(name=‘delivery’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘live’, ‘hosted’), notes=‘delivery-channel lineage {live,hosted}; NOT source identity (D-27.2)’)]

schema_id : ClassVar[str] = ‘schema.earnings_transcript.v1’

class mostlyright.core.schemas.ForecastSchema

Bases: StationForecastSchema

Back-compat alias for schema.forecast.iem_mos.v1.

Same class semantics as StationForecastSchema. Retained so existing IEM MOS parity fixtures and Phase 17 callers continue to work unchanged. New code should reference StationForecastSchema and the canonical schema.forecast.station.v1 schema_id.

Phase 20 OM-02.

schema_id : ClassVar[str] = ‘schema.forecast.iem_mos.v1’

class mostlyright.core.schemas.MergedObservationSchema

Bases: Schema

schema.observation.merged.v1 — merged per-report observation rows.

Accepts the fused-policy frame identity merged.live_v1 while allowing the per-row source column to carry any of the bare parser tags {awc, iem, ghcnh} (membership, not equality — enforced by the validator via _registered_row_sources).

COLUMNS : ClassVar[list[ColumnSpec]] = [ColumnSpec(name=‘station_code’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=‘ICAO/ASOS station ID (e.g. KNYC)’), ColumnSpec(name=‘observed_at’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=‘ISO-8601 UTC observation valid time (string, as parsers emit)’), ColumnSpec(name=‘observation_type’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘METAR’, ‘SPECI’), notes=‘METAR | SPECI’), ColumnSpec(name=‘source’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=‘bare parser tag (awc/iem/ghcnh); membership-checked, not equality’), ColumnSpec(name=‘temp_c’, dtype=‘float64’, units=‘celsius’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘temp_f’, dtype=‘float64’, units=‘fahrenheit’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘dewpoint_c’, dtype=‘float64’, units=‘celsius’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘dewpoint_f’, dtype=‘float64’, units=‘fahrenheit’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘wind_dir_degrees’, dtype=‘float64’, units=‘degrees’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘wind_speed_kt’, dtype=‘float64’, units=‘kt’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘wind_gust_kt’, dtype=‘float64’, units=‘kt’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘altimeter_inhg’, dtype=‘float64’, units=‘inHg’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘sea_level_pressure_mb’, dtype=‘float64’, units=‘mb’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘visibility_miles’, dtype=‘float64’, units=‘miles’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘precip_1hr_inches’, dtype=‘float64’, units=‘inches’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘sky_cover_1’, dtype=‘enum’, units=None, nullable=True, enum_values=(‘CLR’, ‘FEW’, ‘SCT’, ‘BKN’, ‘OVC’, ‘VV’), notes=”), ColumnSpec(name=‘sky_base_1_ft’, dtype=‘float64’, units=‘ft’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘sky_cover_2’, dtype=‘enum’, units=None, nullable=True, enum_values=(‘CLR’, ‘FEW’, ‘SCT’, ‘BKN’, ‘OVC’, ‘VV’), notes=”), ColumnSpec(name=‘sky_base_2_ft’, dtype=‘float64’, units=‘ft’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘sky_cover_3’, dtype=‘enum’, units=None, nullable=True, enum_values=(‘CLR’, ‘FEW’, ‘SCT’, ‘BKN’, ‘OVC’, ‘VV’), notes=”), ColumnSpec(name=‘sky_base_3_ft’, dtype=‘float64’, units=‘ft’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘sky_cover_4’, dtype=‘enum’, units=None, nullable=True, enum_values=(‘CLR’, ‘FEW’, ‘SCT’, ‘BKN’, ‘OVC’, ‘VV’), notes=”), ColumnSpec(name=‘sky_base_4_ft’, dtype=‘float64’, units=‘ft’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘weather_codes’, dtype=‘string’, units=None, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘peak_wind_gust_kt’, dtype=‘float64’, units=‘kt’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘peak_wind_dir’, dtype=‘float64’, units=‘degrees’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘peak_wind_time’, dtype=‘string’, units=None, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘snow_depth_inches’, dtype=‘float64’, units=‘inches’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘qc_field’, dtype=‘string’, units=None, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘raw_metar’, dtype=‘string’, units=None, nullable=True, enum_values=None, notes=‘raw METAR text if source has it; null for structured-only sources’)]

schema_id : ClassVar[str] = ‘schema.observation.merged.v1’

class mostlyright.core.schemas.NwpForecastSchema

Bases: Schema

schema.forecast_nwp.v1 — gridded NWP forecast rows.

One row per (station, model, cycle, fxx, variable_column) — the variable column itself is encoded as separate float columns rather than long-form rows so quants can do model arithmetic without a pivot.

The mirror column lets a downstream auditor link a row back to the bytes that produced it (NOAA BDP mirror chain).

COLUMNS : ClassVar[list[ColumnSpec]] = [ColumnSpec(name=‘station’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=”), ColumnSpec(name=‘model’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘hrrr’, ‘gfs’, ‘nbm’, ‘ecmwf_ifs_hres’, ‘ecmwf_ifs_ens’, ‘ecmwf_aifs_single’, ‘ecmwf_aifs_ens’, ‘hrrrak’, ‘gefs’, ‘gdas’, ‘rap’, ‘rrfs’, ‘rtma’, ‘urma’, ‘cfs’, ‘hrdps’, ‘rdps’, ‘gdps’, ‘geps’, ‘reps’, ‘hafs’, ‘nam’, ‘href’, ‘hiresw’), notes=”), ColumnSpec(name=‘mirror’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘aws_bdp’, ‘gcp_bdp’, ‘azure_bdp’, ‘nomads’, ‘ecmwf_data_portal’, ‘ecmwf_aws’, ‘ecmwf_azure’, ‘ecmwf_gcp’, ‘msc’), notes=‘NOAA BDP mirror that served the underlying bytes’), ColumnSpec(name=‘grid_kind’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=‘grid-projection label (lambert_conformal_conus, regular_latlon_global_0p25, …)’), ColumnSpec(name=‘issued_at’, dtype=‘timestamp_utc’, units=None, nullable=False, enum_values=None, notes=‘model run / cycle reference time’), ColumnSpec(name=‘valid_at’, dtype=‘timestamp_utc’, units=None, nullable=False, enum_values=None, notes=‘forecast target time = issued_at + forecast_hour’), ColumnSpec(name=‘forecast_hour’, dtype=‘int64’, units=‘hours’, nullable=False, enum_values=None, notes=‘lead time in hours (alias: fxx)’), ColumnSpec(name=‘grid_dist_km’, dtype=‘float64’, units=‘km’, nullable=False, enum_values=None, notes=‘great-circle distance from station to nearest grid cell’), ColumnSpec(name=‘temp_k_2m’, dtype=‘float64’, units=‘K’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘dewpoint_k_2m’, dtype=‘float64’, units=‘K’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘relative_humidity_pct_2m’, dtype=‘float64’, units=‘percent’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘wind_u_ms_10m’, dtype=‘float64’, units=‘m/s’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘wind_v_ms_10m’, dtype=‘float64’, units=‘m/s’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘wind_gust_ms’, dtype=‘float64’, units=‘m/s’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘precip_mm_1h’, dtype=‘float64’, units=‘mm’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘pressure_pa_surface’, dtype=‘float64’, units=‘Pa’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘pressure_pa_mslp’, dtype=‘float64’, units=‘Pa’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘cloud_cover_pct’, dtype=‘float64’, units=‘percent’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘visibility_m’, dtype=‘float64’, units=‘m’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘cloud_ceiling_m’, dtype=‘float64’, units=‘m’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘qc_status’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘clean’, ‘flagged’, ‘suspect’), notes=‘inline physics-bounds verdict; finer-grained QC lands in Phase 3.4’), ColumnSpec(name=‘retrieved_at’, dtype=‘timestamp_utc’, units=None, nullable=False, enum_values=None, notes=‘wall-clock UTC when the bytes were fetched’)]

schema_id : ClassVar[str] = ‘schema.forecast_nwp.v1’

class mostlyright.core.schemas.ObservationLedgerSchema

Bases: Schema

schema.observation_ledger.v1 — silver-tier rows-per-source ledger.

Natural key: (station_code, observed_at, source, parser_name, as_of_time, ingestion_id).

Multiple rows per (station_code, observed_at) are valid — one per contributing source.

COLUMNS : ClassVar[list[ColumnSpec]] = [ColumnSpec(name=‘station_code’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=”), ColumnSpec(name=‘observed_at’, dtype=‘timestamp_utc’, units=None, nullable=False, enum_values=None, notes=”), ColumnSpec(name=‘observation_type’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘METAR’, ‘SPECI’), notes=”), ColumnSpec(name=‘source’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘awc’, ‘iem’, ‘ghcnh’, ‘ncei’), notes=‘ncei reserved per D-2.1-09; never written in v0.1.0.’), ColumnSpec(name=‘temp_c’, dtype=‘float64’, units=‘celsius’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘dewpoint_c’, dtype=‘float64’, units=‘celsius’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘parser_name’, dtype=‘enum’, units=None, nullable=True, enum_values=(‘mostlyright_v1’, ‘iem’, ‘ncei’, ‘ghcnh’), notes=”), ColumnSpec(name=‘parser_version’, dtype=‘string’, units=None, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘ingestion_id’, dtype=‘string’, units=None, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘as_of_time’, dtype=‘timestamp_utc’, units=None, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘source_received_at’, dtype=‘string’, units=None, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘qc_status’, dtype=‘enum’, units=None, nullable=True, enum_values=(‘clean’, ‘flagged’, ‘suspect’), notes=”), ColumnSpec(name=‘observation_kind’, dtype=‘enum’, units=None, nullable=True, enum_values=(‘METAR’, ‘SPECI’), notes=”), ColumnSpec(name=‘provenance’, dtype=‘enum’, units=None, nullable=True, enum_values=(‘legacy’, ‘reingested’), notes=”), ColumnSpec(name=‘observation_quality’, dtype=‘enum’, units=None, nullable=True, enum_values=(‘clean’, ‘flagged’, ‘suspect’), notes=‘Lineage row-quality flag per LINEAGE-01; distinct from qc_status enum slot AND distinct from the obs_qc_status bitmask column per QC-05.’)]

IMPERIAL_RENAMES : ClassVar[dict[str, str]] = {}

schema_id : ClassVar[str] = ‘schema.observation_ledger.v1’

class mostlyright.core.schemas.ObservationQCSchema

Bases: Schema

schema.observation_qc.v1 — QC sidecar (forward-compat).

COLUMNS : ClassVar[list[ColumnSpec]] = [ColumnSpec(name=‘station_code’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=”), ColumnSpec(name=‘observed_at’, dtype=‘timestamp_utc’, units=None, nullable=False, enum_values=None, notes=”), ColumnSpec(name=‘observation_kind’, dtype=‘enum’, units=None, nullable=True, enum_values=(‘METAR’, ‘SPECI’), notes=”), ColumnSpec(name=‘source’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘awc’, ‘iem’, ‘ghcnh’, ‘ncei’), notes=”), ColumnSpec(name=‘parser_name’, dtype=‘string’, units=None, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘as_of_time’, dtype=‘timestamp_utc’, units=None, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘ingestion_id’, dtype=‘string’, units=None, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘qc_system’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=”), ColumnSpec(name=‘qc_version’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=”), ColumnSpec(name=‘rule_id’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=”), ColumnSpec(name=‘field’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=‘Observation column the rule evaluated (e.g. temp_c).’), ColumnSpec(name=‘flag’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘clean’, ‘flagged’, ‘suspect’), notes=”), ColumnSpec(name=‘detector_metadata’, dtype=‘string’, units=None, nullable=True, enum_values=None, notes=‘JSON-serialized detector payload; shape per qc_system.’)]

IMPERIAL_RENAMES : ClassVar[dict[str, str]] = {}

schema_id : ClassVar[str] = ‘schema.observation_qc.v1’

class mostlyright.core.schemas.ObservationSchema

Bases: Schema

schema.observation.v1 — canonical METAR/SPECI rows.

Metric column set per docs/design.md §A; imperial-mode rename map per the same section preserves the existing mostly-light contract (the 9-column anti-duplication projection: station, utc_datetime, temp_F, dew_point_F, slp_hpa, wind_speed_kt, wind_dir_deg, gust_kt, vsby). slp_hpa is canonical aviation units and unchanged across modes; wind_dir_deg is dimensionless degrees.

COLUMNS : ClassVar[list[ColumnSpec]] = [ColumnSpec(name=‘station’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=‘ICAO/ASOS station ID (e.g. KORD)’), ColumnSpec(name=‘event_time’, dtype=‘timestamp_utc’, units=None, nullable=False, enum_values=None, notes=‘observation valid time’), ColumnSpec(name=‘observation_type’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘METAR’, ‘SPECI’), notes=“METAR | SPECI; defaults METAR when source can’t distinguish (e.g. AWC JSON)”), ColumnSpec(name=‘temp_c’, dtype=‘float64’, units=‘celsius’, nullable=True, enum_values=None, notes=‘bounded TEMP_MIN_C..TEMP_MAX_C’), ColumnSpec(name=‘dew_point_c’, dtype=‘float64’, units=‘celsius’, nullable=True, enum_values=None, notes=‘bounded’), ColumnSpec(name=‘wind_speed_ms’, dtype=‘float64’, units=‘m/s’, nullable=True, enum_values=None, notes=‘converted from kt’), ColumnSpec(name=‘wind_dir_deg’, dtype=‘int64’, units=‘degrees’, nullable=True, enum_values=None, notes=‘0-360, bounded’), ColumnSpec(name=‘wind_gust_ms’, dtype=‘float64’, units=‘m/s’, nullable=True, enum_values=None, notes=‘converted from kt’), ColumnSpec(name=‘slp_hpa’, dtype=‘float64’, units=‘hPa’, nullable=True, enum_values=None, notes=‘sea-level pressure (canonical aviation unit, not converted across modes)’), ColumnSpec(name=‘visibility_m’, dtype=‘float64’, units=‘meters’, nullable=True, enum_values=None, notes=‘converted from statute miles’), ColumnSpec(name=‘precip_mm_1h’, dtype=‘float64’, units=‘mm’, nullable=True, enum_values=None, notes=‘hourly precip (METAR p01i, converted from inches)’), ColumnSpec(name=‘sky_cover_1’, dtype=‘enum’, units=None, nullable=True, enum_values=(‘CLR’, ‘FEW’, ‘SCT’, ‘BKN’, ‘OVC’, ‘VV’), notes=‘first cloud layer cover code’), ColumnSpec(name=‘sky_base_1_m’, dtype=‘float64’, units=‘meters’, nullable=True, enum_values=None, notes=‘first cloud layer base height (converted from feet)’), ColumnSpec(name=‘sky_cover_2’, dtype=‘enum’, units=None, nullable=True, enum_values=(‘CLR’, ‘FEW’, ‘SCT’, ‘BKN’, ‘OVC’, ‘VV’), notes=‘second layer; null if not present’), ColumnSpec(name=‘sky_base_2_m’, dtype=‘float64’, units=‘meters’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘sky_cover_3’, dtype=‘enum’, units=None, nullable=True, enum_values=(‘CLR’, ‘FEW’, ‘SCT’, ‘BKN’, ‘OVC’, ‘VV’), notes=‘third layer; null if not present’), ColumnSpec(name=‘sky_base_3_m’, dtype=‘float64’, units=‘meters’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘sky_cover_4’, dtype=‘enum’, units=None, nullable=True, enum_values=(‘CLR’, ‘FEW’, ‘SCT’, ‘BKN’, ‘OVC’, ‘VV’), notes=‘fourth layer; null if not present’), ColumnSpec(name=‘sky_base_4_m’, dtype=‘float64’, units=‘meters’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘metar_raw’, dtype=‘string’, units=None, nullable=True, enum_values=None, notes=‘raw METAR text if source has it; null for AWC JSON (structured-only)’)]

IMPERIAL_RENAMES : ClassVar[dict[str, str]] = {‘dew_point_c’: ‘dew_point_F’, ‘event_time’: ‘utc_datetime’, ‘precip_mm_1h’: ‘precip_in_1h’, ‘sky_base_1_m’: ‘sky_base_1_ft’, ‘sky_base_2_m’: ‘sky_base_2_ft’, ‘sky_base_3_m’: ‘sky_base_3_ft’, ‘sky_base_4_m’: ‘sky_base_4_ft’, ‘temp_c’: ‘temp_F’, ‘visibility_m’: ‘vsby’, ‘wind_gust_ms’: ‘gust_kt’, ‘wind_speed_ms’: ‘wind_speed_kt’}

Metric → imperial column-name map (docs §A imperial-mode table). station, observation_type, wind_dir_deg, slp_hpa, sky_cover_N and metar_raw are unchanged across modes.

schema_id : ClassVar[str] = ‘schema.observation.v1’

mostlyright.core.schemas.SCHEMA_REGISTRY = {‘schema.cwop.v1’: <class ‘mostlyright.weather.cwop._schema.CwopSchema’>, ‘schema.earnings_fact.v1’: <class ‘mostlyright.core.schemas.earnings_fact.EarningsFactSchema’>, ‘schema.earnings_transcript.v1’: <class ‘mostlyright.core.schemas.earnings_transcript.EarningsTranscriptSchema’>, ‘schema.forecast.iem_mos.v1’: <class ‘mostlyright.core.schemas.forecast.ForecastSchema’>, ‘schema.forecast.station.v1’: <class ‘mostlyright.core.schemas.forecast.StationForecastSchema’>, ‘schema.forecast_nwp.v1’: <class ‘mostlyright.core.schemas.forecast_nwp.NwpForecastSchema’>, ‘schema.observation.merged.v1’: <class ‘mostlyright.core.schemas.observation_merged.MergedObservationSchema’>, ‘schema.observation.v1’: <class ‘mostlyright.core.schemas.observation.ObservationSchema’>, ‘schema.observation_ledger.v1’: <class ‘mostlyright.core.schemas.observation_ledger.ObservationLedgerSchema’>, ‘schema.observation_qc.v1’: <class ‘mostlyright.core.schemas.observation_qc.ObservationQCSchema’>, ‘schema.satellite.v1’: <class ‘mostlyright.core.schemas.satellite.SatelliteSchema’>, ‘schema.settlement.cli.v1’: <class ‘mostlyright.core.schemas.settlement.SettlementSchema’>}

Public alias for the validator’s registry dict, so callers and tests can look up schemas by id without reaching into core.validator’s underscored internal. Phase 20 OM-02.

class mostlyright.core.schemas.SatelliteSchema

Bases: Schema

schema.satellite.v1 — single-pixel satellite scan rows (NATIVE ring).

One row per (station, satellite, product, variable, pressure_level_hpa, scan_start_utc) — the dedup key (mirror-invariant, D9). Ports the 18 2i storage fields and adds the SDK overlay columns source / delivery / qc_status / as_of_time.

Phase 26 W1 generalization (H1). Phase 25 was GOES-geostationary-shaped. The native ring (GOES E/W + Himawari + VIIRS + EUMETSAT) is covered by a SET of distinct source identities (_registered_sources) and a broadened satellite enum, and the row contract is generalized so a polar VIIRS SWATH row validates alongside a geostationary GOES/Himawari/SEVIRI row: sat_lon_used is now NULLABLE (geostationary-only) and pixel_row/ pixel_col mean “native grid index OR swath line/sample index”.

COLUMNS : ClassVar[list[ColumnSpec]] = [ColumnSpec(name=‘station’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=”), ColumnSpec(name=‘satellite’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘goes16’, ‘goes17’, ‘goes18’, ‘goes19’, ‘himawari8’, ‘himawari9’, ‘viirs-npp’, ‘viirs-n20’, ‘viirs-n21’, ‘meteosat-0deg’, ‘meteosat-iodc’), notes=”), ColumnSpec(name=‘product’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=”), ColumnSpec(name=‘variable’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=”), ColumnSpec(name=‘pressure_level_hpa’, dtype=‘float64’, units=‘hPa’, nullable=True, enum_values=None, notes=‘null for 2D products; populated (part of the dedup key) for 3D profiles’), ColumnSpec(name=‘scan_start_utc’, dtype=‘timestamp_utc’, units=None, nullable=False, enum_values=None, notes=“UTC scan/granule start time (event-time, D4); from the source’s filename start token (GOES/Himawari _s, VIIRS granule start, SEVIRI repeat-cycle start)”), ColumnSpec(name=‘scan_end_utc’, dtype=‘timestamp_utc’, units=None, nullable=False, enum_values=None, notes=“UTC scan/granule end time; >= scan_start_utc (the source’s filename end token; equal to start for a duration-less filename)”), ColumnSpec(name=‘pixel_value’, dtype=‘float64’, units=None, nullable=True, enum_values=None, notes=“raw-as-reported single grid cell; null ONLY on the file’s _FillValue”), ColumnSpec(name=‘pixel_dqf’, dtype=‘float64’, units=None, nullable=True, enum_values=None, notes=‘NOAA DQF integer at the cell; null for variables with no DQF’), ColumnSpec(name=‘pixel_row’, dtype=‘int64’, units=None, nullable=False, enum_values=None, notes=‘native pixel row index at the station: geostationary fixed-grid row (y) for GOES/Himawari/SEVIRI, OR the SWATH line index for a VIIRS polar granule’), ColumnSpec(name=‘pixel_col’, dtype=‘int64’, units=None, nullable=False, enum_values=None, notes=‘native pixel col index at the station: geostationary fixed-grid col (x) for GOES/Himawari/SEVIRI, OR the SWATH sample index for a VIIRS polar granule’), ColumnSpec(name=‘units’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=‘physical units from the NetCDF variable units attr’), ColumnSpec(name=‘station_lat’, dtype=‘float64’, units=‘degrees_north’, nullable=False, enum_values=None, notes=‘station latitude copied at ingest time (provenance)’), ColumnSpec(name=‘station_lon’, dtype=‘float64’, units=‘degrees_east’, nullable=False, enum_values=None, notes=‘station longitude copied at ingest time (provenance)’), ColumnSpec(name=‘sat_lon_used’, dtype=‘float64’, units=‘degrees_east’, nullable=True, enum_values=None, notes=‘sub-satellite longitude of the geostationary source (GOES goes_imager_projection / Himawari fixed-grid / SEVIRI sub-sat 0deg|45.5E); NULL for a VIIRS polar SWATH row, which has no single sub-satellite point’), ColumnSpec(name=‘source_object_key’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=‘full S3/GCS object key the row was extracted from (provenance)’), ColumnSpec(name=‘ingested_at’, dtype=‘timestamp_utc’, units=None, nullable=True, enum_values=None, notes=‘when the bytes were fetched; null for historical backfill’), ColumnSpec(name=‘source’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=“per-row source-identity overlay == df.attrs[‘source’], one of {noaa_goes, jma_himawari, noaa_viirs, eumetsat_meteosat} (D1)”), ColumnSpec(name=‘delivery’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘live’, ‘hosted’), notes=‘delivery-channel lineage {live,hosted}; NOT source identity (D2)’), ColumnSpec(name=‘qc_status’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘clean’, ‘flagged’, ‘suspect’), notes=‘annotate-never-drop verdict (D5 severity-inverted)’), ColumnSpec(name=‘as_of_time’, dtype=‘timestamp_utc’, units=None, nullable=True, enum_values=None, notes=‘knowledge-time stamp; nullable (D4)’)]

schema_id : ClassVar[str] = ‘schema.satellite.v1’

class mostlyright.core.schemas.SettlementSchema

Bases: Schema

schema.settlement.cli.v1 — NWS CLI daily settlement rows.

No imperial rename map: CLI settlement IS Fahrenheit, and Fahrenheit is the canonical unit for both Kalshi contract labels and the NWS CLI product itself. Adapters must NOT convert.

COLUMNS : ClassVar[list[ColumnSpec]] = [ColumnSpec(name=‘station’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=‘ICAO/ASOS station ID’), ColumnSpec(name=‘station_tz’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=‘IANA timezone for the station (e.g. America/Chicago for KORD). Required for local-climate-day semantics; see §U.’), ColumnSpec(name=‘observation_date’, dtype=‘date’, units=None, nullable=False, enum_values=None, notes=‘local climate day per NWS convention (no timezone applied to the date itself)’), ColumnSpec(name=‘event_time’, dtype=‘timestamp_utc’, units=None, nullable=False, enum_values=None, notes=‘00:00 local time on observation_date converted to UTC; for sort/join only’), ColumnSpec(name=‘product_release_time’, dtype=‘timestamp_utc’, units=None, nullable=False, enum_values=None, notes=‘parsed from CLI product header (_climate.py::_parse_product_timestamp)’), ColumnSpec(name=‘report_type’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘preliminary’, ‘final’, ‘correction’), notes=‘preliminary | final | correction; dedup priority preliminary < final < correction’), ColumnSpec(name=‘temp_max_F’, dtype=‘float64’, units=‘fahrenheit’, nullable=True, enum_values=None, notes=‘daily high (uppercase F for consistency with obs imperial mode)’), ColumnSpec(name=‘temp_min_F’, dtype=‘float64’, units=‘fahrenheit’, nullable=True, enum_values=None, notes=‘daily low’), ColumnSpec(name=‘precipitation_in’, dtype=‘float64’, units=‘inches’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘snowfall_in’, dtype=‘float64’, units=‘inches’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘cli_data_quality’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘clean’, ‘flagged_instrument’, ‘flagged_late’, ‘flagged_other’, ‘missing’), notes=‘NWS CLI data-quality marker (Pitfall 6/16). Allows downstream code to filter or weight settlement rows by issuer quality without re-parsing the product header.’), ColumnSpec(name=‘settlement_finality’, dtype=‘enum’, units=None, nullable=False, enum_values=(‘provisional’, ‘final’, ‘superseded’), notes=“provisional | final | superseded. Kalshi NHIGH/NLOW settlement contractually requires ‘final’; ‘provisional’ values are kept for early-look research only.”)]

IMPERIAL_RENAMES : ClassVar[dict[str, str]] = {}

Settlement values are already in canonical Fahrenheit / inches; no imperial-mode rename map applies. Schema.column_names("imperial") returns the same names as column_names("metric").

schema_id : ClassVar[str] = ‘schema.settlement.cli.v1’

class mostlyright.core.schemas.StationForecastSchema

Bases: Schema

schema.forecast.station.v1 — unified per-station forecast schema.

Covers IEM MOS shared core + Open-Meteo extras. Source identity via the source column. Two schema_id strings register against the same column set: schema.forecast.station.v1 (canonical) and schema.forecast.iem_mos.v1 (back-compat alias via subclass).

Phase 20 OM-02.

COLUMNS : ClassVar[list[ColumnSpec]] = [ColumnSpec(name=‘station’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=”), ColumnSpec(name=‘issued_at’, dtype=‘timestamp_utc’, units=None, nullable=True, enum_values=None, notes=‘model run time (knowledge_time). Nullable to accommodate Phase 20 open_meteo.seamless rows whose cycle is unrecoverable from the response. LeakageDetector + assert_issued_at_populated() are the runtime gates that reject null issued_at in training-data paths.’), ColumnSpec(name=‘valid_at’, dtype=‘timestamp_utc’, units=None, nullable=False, enum_values=None, notes=‘forecast target time (event_time)’), ColumnSpec(name=‘forecast_hour’, dtype=‘int64’, units=‘hours’, nullable=False, enum_values=None, notes=‘(valid_at - issued_at).total_seconds() / 3600’), ColumnSpec(name=‘model’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=‘e.g. NBE, GFS, LAV, MET, gfs_global, ecmwf_ifs025’), ColumnSpec(name=‘source’, dtype=‘string’, units=None, nullable=False, enum_values=None, notes=‘iem.archive | open_meteo.previous_runs | open_meteo.single_run | open_meteo.live’), ColumnSpec(name=‘temp_c’, dtype=‘float64’, units=‘celsius’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘dew_point_c’, dtype=‘float64’, units=‘celsius’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘wind_speed_ms’, dtype=‘float64’, units=‘m/s’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘wind_dir_deg’, dtype=‘int64’, units=‘degrees’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘precip_probability’, dtype=‘float64’, units=‘probability’, nullable=True, enum_values=None, notes=‘bounded [0, 1]’), ColumnSpec(name=‘sky_cover_pct’, dtype=‘int64’, units=‘percent’, nullable=True, enum_values=None, notes=‘bounded [0, 100]’), ColumnSpec(name=‘apparent_temp_c’, dtype=‘float64’, units=‘celsius’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘shortwave_radiation_wm2’, dtype=‘float64’, units=‘W/m^2’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘direct_radiation_wm2’, dtype=‘float64’, units=‘W/m^2’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘cape_jkg’, dtype=‘float64’, units=‘J/kg’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘precipitation_mm’, dtype=‘float64’, units=‘mm’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘cloud_cover_pct’, dtype=‘int64’, units=‘percent’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘surface_pressure_hpa’, dtype=‘float64’, units=‘hPa’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘pressure_msl_hpa’, dtype=‘float64’, units=‘hPa’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘freezing_level_m’, dtype=‘int64’, units=‘meters’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘snow_depth_m’, dtype=‘float64’, units=‘meters’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘visibility_m’, dtype=‘int64’, units=‘meters’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘wind_gusts_ms’, dtype=‘float64’, units=‘m/s’, nullable=True, enum_values=None, notes=”), ColumnSpec(name=‘weather_code’, dtype=‘int64’, units=‘WMO 4677’, nullable=True, enum_values=None, notes=‘WMO weather code (clear, fog, rain, snow, etc.)’), ColumnSpec(name=‘retrieved_at’, dtype=‘timestamp_utc’, units=None, nullable=False, enum_values=None, notes=‘wall-clock time the row was fetched from upstream’)]

IMPERIAL_RENAMES : ClassVar[dict[str, str]] = {‘apparent_temp_c’: ‘apparent_temp_F’, ‘dew_point_c’: ‘dew_point_F’, ‘temp_c’: ‘temp_F’, ‘wind_gusts_ms’: ‘wind_gusts_kt’, ‘wind_speed_ms’: ‘wind_speed_kt’}

Imperial-mode renames apply to temperature, wind speed, and wind gusts. valid_at / issued_at / retrieved_at are model-internal timestamps and keep their canonical names.

schema_id : ClassVar[str] = ‘schema.forecast.station.v1’

Modules

earnings_fact	Earnings-fact schema (schema.earnings_fact.v1) — Phase 27 (27-01).
earnings_transcript	Earnings-transcript schema (schema.earnings_transcript.v1) — Phase 27 (27-01).
forecast	Phase 20: Unified per-station forecast schema.
forecast_nwp	NWP forecast schema (schema.forecast_nwp.v1) — Phase 3.2.
observation	Canonical METAR/SPECI observation schema (schema.observation.v1).
observation_ledger	schema.observation_ledger.v1 — silver-tier observation ledger.
observation_merged	Merged observation-grain schema (schema.observation.merged.v1).
observation_qc	schema.observation_qc.v1 — QC sidecar schema (forward-compat).
satellite	Satellite schema (schema.satellite.v1) — Phase 25 (25-01 surface).
settlement	NWS CLI settlement schema (schema.settlement.cli.v1).