mostlyright.core.merge

mostlyright.core.merge

Phase 2.1 — read-time merge policy for silver-tier observation ledgers.

Silver-tier ledger schema (schema.observation_ledger.v1) carries one row per (station, observed_at, source) — multiple rows per (station, observed_at) are valid (one per contributing source). This module provides query_time_merge(silver_df, policy=LIVE_V1) that materializes the v0.14.1-equivalent single-row-per-key gold shape for Mode-1 parity callers.

Policy LIVE_V1 uses strict > priority on source_priority (AWC=3, IEM=2, GHCNh=1; ncei reserved per D-2.1-09 but never written in v0.1.0). Ties resolved by deterministic secondary key (source_received_at, ingestion_id).

See .planning/phases/02.1-sprint-2o-lineage-refactor-per-source-provenance/02.1-PLAN-02-merge-policy-port.md.

Module Attributes

LIVE_V1	Canonical Mode-1 parity policy — AWC > IEM > GHCNh; ncei reserved.

Functions

query_time_merge(silver_df, *[, policy])	Materialize gold shape from silver-tier ledger.

Classes

ObservationMergePolicy(name, source_priority)	Read-time merge policy for silver-tier observation ledger.

mostlyright.core.merge.LIVE_V1 = ObservationMergePolicy(name=‘LIVE_V1’, source_priority=mappingproxy({‘awc’: 3, ‘iem’: 2, ‘ghcnh’: 1, ‘ncei’: 0}), secondary_key=(‘source_received_at’, ‘ingestion_id’))

Canonical Mode-1 parity policy — AWC > IEM > GHCNh; ncei reserved.

class mostlyright.core.merge.ObservationMergePolicy(name, source_priority, secondary_key=(‘source_received_at’, ‘ingestion_id’))

Bases: object

Read-time merge policy for silver-tier observation ledger.

Frozen at the dataclass level AND at the source_priority value level (codex Phase 2.1 review HIGH: a frozen dataclass with a mutable dict attribute is not actually immutable — callers could mutate LIVE_V1.source_priority at runtime and globally change merge results). Construction wraps any input dict in MappingProxyType for read-only enforcement.

• Parameters:
  • name (str)
  • source_priority (Mapping [str , int ])
  • secondary_key (tuple [str , … ])

STRICT_PRIORITY : ClassVar[bool] = True

Strict “>” priority comparison — first-row-seen wins at equal priority, secondary key breaks ties deterministically.

apply(silver_df)

Materialize gold (single-row-per-(station, observed_at)) from silver.

• Return type: DataFrame
• Parameters: silver_df (DataFrame)

name : str

secondary_key : tuple[str, ...] = (‘source_received_at’, ‘ingestion_id’)

source_priority : Mapping[str, int]

mostlyright.core.merge.query_time_merge(silver_df, , policy=ObservationMergePolicy(name=‘LIVE_V1’, source_priority=mappingproxy({‘awc’: 3, ‘iem’: 2, ‘ghcnh’: 1, ‘ncei’: 0}), secondary_key=(‘source_received_at’, ‘ingestion_id’)))

Materialize gold shape from silver-tier ledger.

The silver-tier schema.observation_ledger.v1 carries one row per (station, observed_at, source). This function applies policy.source_priority with strict > to keep only the highest-priority source per (station_code, observed_at) key, using the policy’s secondary_key for deterministic tiebreaks.

• Parameters:
  • silver_df (DataFrame) – DataFrame conforming to schema.observation_ledger.v1. MUST carry station_code, observed_at, source columns.
  • policy (ObservationMergePolicy) – Merge policy (default LIVE_V1).
• Return type: DataFrame
• Returns: DataFrame with one row per (station_code, observed_at) — the v0.14.1-equivalent gold shape Mode-1 parity callers expect.