mostlyright.core.schema

mostlyright.core.schema

Declarative schema framework for mostlyright.

Defines the Schema base class, ColumnSpec description records, and SchemaRegistration audit-trail container. These are the shape contracts that the Validator, KnowledgeView, and adapter layers consume.

See docs/design.md §A (canonical schemas), §BB.3 (settlement schema), §BB.4 (audit log API), and §J (allow_source_drift reason-string semantics).

TimePoint will eventually wrap timestamp handling; this module accepts raw datetime objects for now and validates that they are timezone-aware.

Classes

ColumnSpec(name, dtype, units, nullable[, …])	Description of a single column in a Schema.
Schema()	Declarative schema base class.
SchemaRegistration(schema, source, …[, _audit])	Audit-trail container produced by Schema.register(...).

class mostlyright.core.schema.ColumnSpec(name, dtype, units, nullable, enum_values=None, notes=”)

Bases: object

Description of a single column in a Schema.

• Parameters:
  • name (str)
  • dtype (str)
  • units (str | None)
  • nullable (bool)
  • enum_values (tuple [str , … ] | None)
  • notes (str)

name

Column name as it appears in the canonical (metric) projection.

dtype

One of the canonical dtype tags. "enum" requires enum_values to be populated.

units

Free-form unit label ("celsius", "meters", "kt") or None for dimensionless columns.

nullable

Whether the column may contain NaN/NULL values.

enum_values

Allowed values when dtype == "enum". Must be None for non-enum columns.

notes

Free-form documentation surfaced in catalog metadata.

dtype : str

enum_values : tuple[str, ...] | None = None

name : str

notes : str = ”

nullable : bool

units : str | None

class mostlyright.core.schema.Schema

Bases: object

Declarative schema base class.

Subclasses declare:

• schema_id — stable identifier ("schema.observation.v1").
• COLUMNS — ordered list of ColumnSpec.
• IMPERIAL_RENAMES (optional) — metric → imperial column-name map; columns not mentioned keep their metric name.

The base class is intentionally light: no I/O, no DataFrame coupling. Validator and adapter layers consume COLUMNS to perform their work; Schema itself only holds the contract.

COLUMNS : ClassVar[list[ColumnSpec]] = []

IMPERIAL_RENAMES : ClassVar[dict[str, str]] = {}

classmethod column(name)

Look up a ColumnSpec by its (metric) name.

• Raises: KeyError – if name does not name a declared column.
• Return type: ColumnSpec
• Parameters: name (str)

classmethod column_names(mode=‘metric’)

Return the column names in declaration order for the given mode.

mode="metric" returns the canonical names declared in COLUMNS. mode="imperial" applies IMPERIAL_RENAMES (columns absent from the rename map keep their metric name).

• Raises: ValueError – if mode is not "metric" or "imperial".
• Return type: list[str]
• Parameters: mode (str)

classmethod from_dataframe(df, source, retrieved_at)

Infer a schema from a DataFrame (BYO data path). Deferred to v0.1.1.

Per docs/design.md Premise 5 (and §”Layer responsibilities”): schemas can be constructed declaratively (subclasses of Schema) OR inferred from a sample DataFrame. The declarative path ships in v0.1.0; the inference path (this method) ships in v0.1.1.

Track at:

• Return type: Schema
• Parameters:
  • df (DataFrame)
  • source (str)
  • retrieved_at (datetime)

classmethod register(source, retrieved_at, rows)

Register this schema against a concrete pull.

Records the source and retrieved_at provenance and produces a fresh SchemaRegistration carrying the audit log. The registered event is appended automatically.

NOTE

v0.1.0 is a per-call factory. Every call returns a brand-new SchemaRegistration with an independent audit log; there is no cross-call registry that deduplicates by (schema_id, source) or merges audit events across pulls.

Design §BB.4 describes an audit log that persists with the schema across calls; that persistence model lands in v0.1.1 alongside the catalog adapter, which owns the (schema, source) keying and on-disk storage decisions (parquet metadata, JSON sidecar). Callers in v0.1.0 that need a long-lived registration must hold onto the returned object themselves.

Track at:

• Parameters:
  • source (str) – Source identity in "<source>.<endpoint>" form ("iem.archive", "awc.live").
  • retrieved_at (datetime) – Wall-clock timestamp of the pull that produced the rows. MUST be timezone-aware.
  • rows (int) – Row count after schema validation.
• Raises:
  • TypeError – if source is not str, retrieved_at is not a datetime, or rows is not int.
  • ValueError – if source is empty, retrieved_at is naive, or rows is negative.
• Return type: SchemaRegistration

schema_id : ClassVar[str] = ”

class mostlyright.core.schema.SchemaRegistration(schema, source, retrieved_at_min, retrieved_at_max, rows, _audit=)

Bases: object

Audit-trail container produced by Schema.register(...).

Records the (source, retrieved_at range, row count) provenance of a training-time pull and accumulates an append-only audit log. The audit log captures source-drift opt-outs and reproducibility-audit outcomes per docs/design.md §BB.4.

Validator wires audit events in via _append_audit(); this module intentionally does not import Validator (it is implemented elsewhere).

Audit-log seam (contract — read this). The single-underscore prefix on _audit and _append_audit() denotes module-private with one cross-module writer: the Validator (a sibling module) calls _append_audit to record source_drift_allowed and temporal_drift_audit events. Any caller OTHER than Schema.register() or the Validator that invokes _append_audit — or any direct mutation of _audit — is a contract violation: the audit log is meant to be append-only and stamped by trusted writers only. The list is intentionally not frozen so the Validator can append without indirection; this is the explicit seam, documented here so code review can catch unauthorised callers.

Deferred to v0.1.1: volatility_warning (per design §B/§P, surfaced by catalog_search and flagged on schemas whose registered rows fall within the 30-day volatile window of iem.archive) will land as an attribute on this dataclass populated by the catalog adapter at registration time. v0.1.0 carries no such attribute.

• Parameters:
  • schema (type [Schema ])
  • source (str)
  • retrieved_at_min (datetime)
  • retrieved_at_max (datetime)
  • rows (int)
  • _audit (list [dict [str , Any ] ])

audit_log()

Return a defensive copy of the chronologically-ordered audit log.

Both the outer list and each inner entry are shallow-copied so callers cannot mutate stored entries (e.g. reg.audit_log()[0]["event"] = "tampered" is a no-op against the underlying log). The list contains dict entries with at minimum event and ts (ISO-8601 UTC) keys. See _append_audit() for the supported event vocabulary.

• Return type: list[dict[str, Any]]

retrieved_at_max : datetime

retrieved_at_min : datetime

rows : int

schema : type[Schema]

source : str