Quality control

The QC engine runs physics-bound checks against every row at fetch time. Failed checks set bits in a sidecar bitmask. Rows are never silently dropped — you choose how to filter.

Philosophy

Three QC postures:

1. clean — passed all physics checks.
2. flagged — failed one or more checks but the row is plausible (e.g. an out-of-range gust reading).
3. suspect — failed checks that strongly suggest sensor error (e.g. a negative absolute humidity).

Mostlyright never drops rows. The full row is preserved with a QC sidecar so you can:

• Filter to clean rows for model training
• Audit flagged/suspect rows for sensor failure investigation
• Re-process with different thresholds without re-fetching

Enabling QC on research()

Python

from mostlyright import research

df = research(
    "KNYC", "2025-01-06", "2025-01-12",
    qc=True,  # attach qc_* columns
)

# Filter to clean rows
clean = df[df["qc_status"] == "clean"]

# Inspect what failed for flagged rows
flagged = df[df["qc_status"] == "flagged"]
print(flagged[["date", "qc_flags"]])

TypeScript

const rows = await research("KNYC", "2025-01-06", "2025-01-12", { qc: true });

const clean = rows.filter((r) => r.qc_status === "clean");
const flagged = rows.filter((r) => r.qc_status === "flagged");

QC alpha rules

The Phase-3.x qc_alpha ruleset covers the highest-signal physics bounds. Each rule has a bit position in the qc_alpha sidecar bitmask:

Bit	Rule	Field	Suspect / Flagged threshold
0	temp_c_out_of_range	temp_c	< -90 °C or > 60 °C → suspect
1	dewpoint_c_above_temp	dewpoint_c	dewpoint > temp → flagged (rare; usually METAR transcription)
2	wind_speed_out_of_range	wind_speed_kt	> 200 kt → suspect
3	wind_gust_below_speed	wind_gust_kt	gust < sustained → flagged
4	pressure_inhg_out_of_range	pressure_inhg	< 25 or > 33 in Hg → suspect
5	visibility_m_out_of_range	visibility_m	< 0 or > 50000 m → suspect
6	precip_mm_1h_negative	precip_mm_1h	< 0 → suspect
…	… (see schemas/qc-alpha-rules.json)

Bits are stable across SDK versions — bit 0 is always temp_c_out_of_range. New rules append; existing positions never shift.

The qc_alpha sidecar

When qc=True, every row gets:

{
  "qc_status": "clean" | "flagged" | "suspect",
  "qc_alpha_mask": 0b00000100,   # bit 2 → wind_speed_out_of_range
  "qc_flags": ["wind_speed_out_of_range"],
  "qc_severity": "suspect",
}

qc_status is the worst-case across all triggered rules:

• Any suspect rule → status suspect
• No suspect but any flagged rule → status flagged
• Nothing triggered → status clean

Per-rule introspection

from mostlyright.qc import QC_ALPHA_RULES_BY_ID

rule = QC_ALPHA_RULES_BY_ID["wind_speed_out_of_range"]
print(rule)
# {
#   "rule_id": "wind_speed_out_of_range",
#   "bit_position": 2,
#   "description": "Sustained wind speed > 200 kt is sensor error",
#   "field": "wind_speed_kt"
# }

The same registry ships in TS:

import { QC_ALPHA_RULES_BY_ID } from "@mostlyrightmd/core";

const rule = QC_ALPHA_RULES_BY_ID.get("wind_speed_out_of_range");

NWP-specific QC rules

Gridded NWP rows get additional physics rules covering grid-cell artifacts:

Rule	Field	Notes
nwp_rh_out_of_range	rh_pct	< -5 or > 110 → suspect
nwp_surface_p_out_of_range	surface_p_pa	< 50000 or > 110000 Pa → suspect
nwp_mslp_out_of_range	mslp_pa	< 87000 or > 108500 Pa → suspect
nwp_hafs_basin_check	latitude	HAFS rows outside 0..60° N → flagged

See mostlyright.weather.qc.rules_nwp for the full list.

Writing a QC sidecar to disk

qc_sidecar writes a separate parquet per (station, year, month) carrying only the QC fields, so you can join post-hoc without re-fetching:

from mostlyright.weather.qc_sidecar import write_qc_sidecar

write_qc_sidecar(rows, station="KNYC", year=2025, month=1)
# Writes ~/.mostlyright/cache/qc/KNYC/2025/01.parquet

This is what research(qc=True) does internally when the QC engine fires. Filtering downstream:

import pandas as pd

qc_df = pd.read_parquet("~/.mostlyright/cache/qc/KNYC/2025/01.parquet")
obs_df = pd.read_parquet("~/.mostlyright/cache/observations/KNYC/2025/01.parquet")

merged = obs_df.merge(qc_df, on=["station", "observed_at"], how="left")
clean = merged[merged["qc_status"] == "clean"]

When NOT to use QC

For settlement use cases, you don’t filter out flagged/suspect rows — Kalshi NHIGH/NLOW settles on the rawest observation regardless of QC status. Filtering would silently mis-route.

For model training, filtering to clean is usually the right move. For sensor-failure debugging, look at flagged/suspect only.

See also

• Raw as reported — the contract QC doesn’t violate
• Observation schema — fields QC operates on
• API reference: mostlyright.qc · mostlyright.weather.qc · @mostlyrightmd/core/qc