mostlyright.preprocessing

mostlyright.preprocessing

Phase 3.5 — preprocessing primitives.

Standalone callable counterparts to the in-pipeline QC engine + transforms. Quants can run these after mostlyright.research() (or any DataFrame they constructed elsewhere) without going through the QC engine.

Surface:

• clip_outliers(df, column, *, bounds=None, std=3.0)() — winsorize to either physics-based bounds (default) or mean ± std * sigma.
• iem_crosscheck(silver_df, *, tolerance="default")() — standalone IEM-vs-GHCNh disagreement detector. Thin re-export of mostlyright.qc.crosscheck_iem_ghcnh() with sensible defaults.

Module Attributes

PHYSICS_BOUNDS	Physics-based clipping defaults for the canonical observation columns.

Functions

clip_outliers(df, column, *[, bounds, std])	Winsorize df[column] to either physics-based or sigma-based bounds.
iem_crosscheck(silver_df, *[, tolerance])	Standalone-callable IEM-vs-GHCNh crosscheck.

mostlyright.preprocessing.PHYSICS_BOUNDS : dict[str, tuple[float, float]] = {‘dew_point_c’: (-89.0, 35.0), ‘dewpoint_c’: (-89.0, 35.0), ‘precip_mm_1h’: (0.0, 305.0), ‘relative_humidity_pct_2m’: (0.0, 100.0), ‘sea_level_pressure_mb’: (870.0, 1085.0), ‘slp_hpa’: (870.0, 1085.0), ‘temp_c’: (-89.0, 57.0), ‘wind_dir_deg’: (0.0, 360.0), ‘wind_dir_degrees’: (0.0, 360.0), ‘wind_speed_kt’: (0.0, 200.0), ‘wind_speed_ms’: (0.0, 100.0)}

Physics-based clipping defaults for the canonical observation columns. Values are (min, max) tuples in canonical units (°C for temp, m/s for wind, hPa for pressure, percent for humidity).

mostlyright.preprocessing.clip_outliers(df, column, , bounds=None, std=3.0)

Winsorize df[column] to either physics-based or sigma-based bounds.

If bounds is supplied, clip to that explicit (min, max). Else if column has a PHYSICS_BOUNDS entry, clip to that. Else fall back to mean ± std * sigma (matches the historical behaviour of mostlyright.transforms.clip_outliers()).

Returns a NEW Series; the input DataFrame is unchanged.

• Parameters:
  • df (DataFrame) – input DataFrame.
  • column (str) – name of the numeric column to clip.
  • bounds (tuple[float, float] | None) – explicit (lower, upper). Overrides physics defaults.
  • std (float) – sigma multiplier for the fallback branch. Ignored when bounds or a physics entry applies.
• Raises: KeyError – column not in df.
• Return type: Series

Phase 6 W2-T3: accepts pandas OR polars input; returns the same backend the caller passed.

mostlyright.preprocessing.iem_crosscheck(silver_df, , tolerance=‘default’)

Standalone-callable IEM-vs-GHCNh crosscheck.

Splits silver_df by source column (rows whose source starts with "iem" vs "ghcnh") and delegates to mostlyright.qc.crosscheck_iem_ghcnh(). Useful when a quant has already called mostlyright.research() (or another fetcher) and wants the disagreement table without going through the full QC engine.

• Parameters:
  • silver_df (DataFrame) – DataFrame with a source column and per-row (station, event_time, temp_c) (or production-shape station_code, observed_at — both are accepted, the former is auto-derived from the latter when missing).
  • tolerance (str | float) – either "default" (uses 2.0 °C from crosscheck_iem_ghcnh()) or an explicit float in °C.
• Return type: DataFrame
• Returns: DataFrame with one row per disagreement (station, event_time, temp_c_iem, temp_c_ghcnh, delta_c).
• Raises: ValueError – silver_df lacks a source column.

Phase 6 W2-T3: accepts pandas OR polars input; returns the same backend the caller passed.