mostlyright.experimental

mostlyright.experimental — public-experimental, semver-exempt surfaces.

Everything re-exported here is EXPERIMENTAL: its API, semantics, and existence may change in ANY minor release until it graduates. Pin your mostlyright version if you depend on it.

The pattern mirrors pandas’ pandas.api.extensions namespace — a documented, importable home for plugin surfaces that are stable enough to build on but not yet frozen under the semver contract.

Current surface:

• spine() — the BYO bridge spine(df, *, entity, decision_time, y) that lifts a user frame into the guarded (leakage-checked) world.

• align() — the composition verb align(spine, *sources) (as-of join + leakage guard) that the domain training_table() verbs are built on.

• DeferredSource — the deferred-source builder for BYO leakage-safe joins (the escape hatch that composes user data through align()).

• contributor() — register a covariate contributor. A contributor is one function contribute(entity, window, grain) -> {key: {prefixed_col: value}} plus declared metadata (prefix namespace, archive_class, injection_point, and the columns it emits). Registered contributors compose into training_table() via the features=[...] selector without any core change. The FIRST third-party registration emits an ExperimentalFeatureWarning. The labels-only firewall is UNCONDITIONAL — a contributor can NEVER emit or override a label / observed_* / daily_summary_* column.

  A third-party contributor MUST declare columns= (its prefixed column set) so that a contribute() which RAISES or covers zero days still yields an all-NaN prefix block instead of silently dropping the columns. The returned mapping may be keyed by a plain local-standard-date string OR by the declared key tuple (entity, local_standard_date); both bucket to the local standard day.

• ContributorSpec — the immutable descriptor a contributor() registration produces (its prefix, declared columns, archive_class, and injection_point).

• list_contributors() — the registered contributor ids, for discovery.

• registered_contributors() — the {id: ContributorSpec} map of what is registered in the current session.

The features=[...] selector is NOT third-party-only. Four contributors are registered as built-ins and selectable by name: "forecasts" and "trades" (pre-aggregation) and "satellite" and "cwop" (post-join). Each has a deprecated include_* keyword alias on training_table() — include_satellite=True is folded into features=["satellite"] before anything composes, so the two forms hit the same wiring and produce byte-identical frames; the alias emits a DeprecationWarning naming its features= replacement and will be removed no earlier than two minor releases after it was deprecated. features=None / features=[] with every flag False composes no contributor and returns today’s label bytes exactly.

Example:

from mostlyright.experimental import contributor

@contributor(
    "era5_ssrd",
    "ssrd_",
    "refetchable",
    injection_point="post_join",
    columns=("ssrd_mean",),
)
def era5_ssrd(entity, window, grain, *, tz_override=None):
    from_date, to_date = window
    # ... fetch + aggregate to one row per LST local standard day ...
    # keyed by the documented (station, local_standard_date) tuple:
    return {(entity, "2025-01-06"): {"ssrd_mean": 12.3}, ...}

table = mostlyright.weather.training_table("KNYC", "2025-01-06", "2025-01-08",
                                           features=["era5_ssrd"])
# table now carries an additive ssrd_mean column; label columns untouched.

class mostlyright.experimental.ContributorSpec(name, prefix, archive_class, injection_point, fn, key=(‘station’, ‘local_standard_date’), native_grain=‘daily’, columns=(), builtin=False, kind=‘source’)

Bases: object

An immutable registry entry describing one contributor.

• Parameters:
  • name (str)
  • prefix (str)
  • archive_class (Literal [ ‘refetchable’ , ‘ledger’ ])
  • injection_point (Literal [ ‘pre_aggregation’ , ‘post_join’ ])
  • fn (Callable [ [ … ] , dict [str , dict [str , Any ] ] ])
  • key (tuple [str , str ])
  • native_grain (Literal [ ‘daily’ , ‘observation’ ])
  • columns (tuple [str , … ])
  • builtin (bool)
  • kind (Literal [ ‘source’ , ‘label’ ])

name

Unique registry key (also the value used in features=[...]).

prefix

Column namespace this contributor owns (e.g. "sat_"). Every column it emits must start with this prefix; the prefix must be non-empty and end with "_".

archive_class

"refetchable" | "ledger" (source-identity §5).

injection_point

"pre_aggregation" | "post_join" — where the columns land in the dataset() pipeline.

fn

The contribute(entity, window, grain, **kw) callable.

key

The key tuple the returned frame is keyed by. Default ("station", "local_standard_date") (the weather domain); the earnings domain registers with e.g. ("ticker", "call_date").

native_grain

Reserved. Contributors are "daily"; a future contributor may declare "observation" to supply per-report rows.

columns

The declared prefixed column set this contributor emits. Required for third-party contributors, so a contribute() that raises or covers zero days still yields an all-NaN prefix block instead of dropping the columns. Built-ins derive it lazily (their columns are the covariate-helper output), so it may be empty for them; when empty the aligner falls back to the columns first seen across days.

builtin

True for the four built-ins (exempt from the ExperimentalFeatureWarning and the third-party-prefix guard).

kind

"source" (feature / X frame) | "label" (settlement target / y frame). Every registration defaults to "source" so the four built-ins stay byte-stable. mr.registry.source() / mr.registry.label() write this field and enforce the .label. path invariant.

archive_class: Literal['refetchable', 'ledger']

builtin: bool

columns: tuple[str, ...]

fn: Callable[..., dict[str, dict[str, Any]]]

injection_point: Literal['pre_aggregation', 'post_join']

key: tuple[str, str]

kind: Literal['source', 'label']

name: str

native_grain: Literal['daily', 'observation']

prefix: str

class mostlyright.experimental.DeferredSource(contract, entity, builder)

Bases: object

A source whose fetch window is deferred until align runs.

An internal source builder (e.g. the private _deferred_observations that training_table() uses) returns this instead of eagerly fetching, so a source can be handed to align before its window is known. align is the single place the fetch window is inferred — from the spine’s copy of the source’s equality-join key (contract.event_time_col, e.g. the day key local_standard_date) when present, else the decision_time min/max — and it calls materialize() to turn the deferred source into a concrete (frame, SourceContract) source.

• Parameters:
  • contract (SourceContract)
  • entity (str | list [str ] | tuple [str , … ])
  • builder (Callable [ [str | list [str ] | tuple [str , … ] , str , str ] , DataFrame ])

contract

The SourceContract the materialized frame conforms to (prefix / point_in_time_fidelity / entity_key).

entity

The entity (station) the deferred builder will fetch.

builder

A callable (entity, from_date, to_date) -> pd.DataFrame that re-enters the eager fetch path with the align-inferred window. The from_date/to_date are inclusive ISO YYYY-MM-DD strings.

builder: Callable[[str | list[str] | tuple[str, ...], str, str], DataFrame]

contract: SourceContract

entity: str | list[str] | tuple[str, ...]

materialize(from_date, to_date)

Run the deferred builder over [from_date, to_date] (inclusive).

• Return type: tuple[DataFrame, SourceContract]
• Parameters:
  • from_date (str)
  • to_date (str)

mostlyright.experimental.align(spine, *sources)

As-of join each source onto spine.

The signature stays at two concepts: (spine, *sources), with no policy keyword arguments. A source is either a pre-materialized (frame, SourceContract) pair (an optional third tuple element overrides the prefix) or a DeferredSource — a window-deferred builder call that align materializes over the spine’s local-standard-day key (the source’s event_time_col) when the spine carries it, else over the decision_time min/max.

Per source: refuse latest_only → collapse by revision_order authority → namespace non-key columns under the source prefix, raising a ContractError on collision rather than emitting pandas _x / _y → join (equality or backward as-of, inferred from the caller’s spine columns, snapshotted before the first join) → per-source leakage audit.

Join keys are never namespaced, so each source’s declared event_time_col/knowledge_time_col is restored onto the output under its declared name only when that name has a single owner; a name TWO sources declare is dropped for both (one source’s timestamps are not the frame’s), and a name the spine owns is never overwritten. Returns a plain pd.DataFrame stamped attrs["source"] = "align", plus per-column attrs["provenance"] (source id) and attrs["coverage"] (source valid_from/coverage edge) so an all-NaN column prefix is explainable via mr.provenance(frame).

• Return type: DataFrame
• Parameters:
  • spine (DataFrame)
  • sources (tuple *[*DataFrame , SourceContract ] | DeferredSource)

mostlyright.experimental.contributor(name, prefix, archive_class, , injection_point, fn=None, key=(‘station’, ‘local_standard_date’), native_grain=‘daily’, columns=None, _builtin=False)

Register a contributor. Usable as a decorator or a direct call.

Experimental / semver-exempt until it graduates. The public entry point is mostlyright.experimental.contributor(). The registry is module-level and shared by dataset/markets/reconcile; the returned spec is also the decorated function (so @contributor(...) leaves the original callable bound in the caller’s namespace).

Decorator form:

@contributor("era5_ssrd", "ssrd_", "refetchable", injection_point="post_join")
def era5_ssrd(entity, window, grain, *, tz_override=None):
    ...
    return {("KNYC", "<local_standard_date>"): {"ssrd_mean": 12.3}, ...}

Direct form (used by the built-in registrations):

contributor("satellite", "sat_", "refetchable",
            injection_point="post_join", fn=_sat_contribute, _builtin=True)

• Parameters:
  • name (str) – Unique registry key + features=[...] token.
  • prefix (str) – Column namespace (non-empty, ends with "_", not a label prefix). Third-party contributors may not claim a built-in prefix.
  • archive_class (Literal['refetchable', 'ledger']) – "refetchable" | "ledger" (source-identity §5).
  • injection_point (Literal['pre_aggregation', 'post_join']) – "pre_aggregation" | "post_join".
  • fn (Callable[..., dict[str, dict[str, Any]]] | None) – The contribute callable. Omitted in decorator form (the decorated function becomes fn).
  • key (tuple[str, str]) – Returned-frame key tuple (default the weather key). The contribute() return mapping may be keyed by a plain local-standard-date string or by this tuple (entity, local_standard_date); both bucket to the local standard day (the tuple’s last element).
  • native_grain (Literal['daily', 'observation']) – Reserved grain declaration (today always "daily").
  • columns (tuple[str, ...] | list[str] | None) – The prefixed column set the contributor emits. Required for third-party contributors (each column must carry prefix) so a failed / empty contribute() still yields an all-NaN prefix block instead of dropping the columns. Built-ins may omit it (they derive their columns from the covariate-helper output).
  • _builtin (bool) – Internal — set by the four built-in registrations to bypass the ExperimentalFeatureWarning and the third-party-prefix guard.
• Raises: ValueError – duplicate name, malformed / label-colliding prefix, a third-party contributor claiming a reserved built-in prefix, or a third-party contributor that omits columns / declares a column outside its prefix namespace.
• Return type: Any
• Returns: The registered ContributorSpec (direct form) or a decorator that registers then returns the decorated function (decorator form).

mostlyright.experimental.list_contributors()

Return the sorted registered contributor names.

• Return type: list[str]

mostlyright.experimental.registered_contributors()

Return a shallow copy of the registry (name → spec).

• Return type: dict[str, ContributorSpec]

mostlyright.experimental.spine(df, , entity, decision_time, y, label_available_time=None)

Map a user-built frame into the canonical spine shape.

• Parameters:
  • df (DataFrame) – The user-built target frame.
  • entity (str) – The column naming the entity (renamed to the canonical station key).
  • decision_time (str) – The column holding the per-row point-in-time cutoff. If it is absent as a column but lives in a datetime / date-named index, the index is lifted into a column first (reusing _coerce_date_column).
  • y (str | list[str] | tuple[str, ...]) – The target column name(s). A spine with no target is meaningless.
  • label_available_time (str | None) – Optional column naming when the label became known — its own publication time; defaults to decision_time when unmapped, which is conservative and non-leaking.
• Return type: DataFrame
• Returns: A SpineContract-valid frame (station, decision_time, label_available_time, *y).
• Raises: ContractError – a mapped column (entity / decision_time / a y) is absent from the frame; the mapping is never guessed.