mostlyright.core.formats.toon

mostlyright.core.formats.toon

TOON format — DataFrame ↔ TOON tabular string.

Wraps the lifted _toon encoder (encode_tabular) with a DataFrame-aware coercion layer on the dumps side and a focused tabular-form parser on the loads side. TOON is encoder-only in mostlyright; this module adds the loader needed for roundtrip.

The wire form is exactly what encode_tabular produces — a TOON v3.0 tabular block of the form:

rows[N]{col1,col2,col3}:
  v1,v2,v3
  ...

No metadata header is emitted. Loss cases are deterministic and documented per design doc §I:

• pandas Categorical dtype → object (string values; category list dropped).
• Timestamp[ns, <tz>] cells are serialized as ISO-8601 strings at microsecond precision (nanoseconds are truncated). The loader does NOT auto-promote ISO-looking string columns back to datetime64; this prevents user-supplied label columns whose values happen to match the ISO pattern from being silently mutated, and makes the roundtrip idempotent under repeat dumps/loads. Callers who want a timestamp column back must call “pd.to_datetime“ explicitly (e.g. df["event_time"] = pd.to_datetime(df["event_time"], utc=True)).
• Mixed-type object columns containing dict/list values: nested values are stringified deterministically — dict cells via json.dumps(value, sort_keys=True, default=str) (canonical, order- independent), other non-primitives via str(value).
• Int64 nullable integer columns with nulls roundtrip as float64 with NaN for the missing slots (precision loss above 2**53).
• decimal.Decimal cells are coerced to float in the encoder to match TOON’s numeric model (precision loss above 2**53).

For lossless transport prefer parquet.

Functions

dumps(df)	Serialize a DataFrame as a TOON v3.0 tabular string.
loads(data)	Parse a TOON v3.0 tabular string back into a DataFrame.

mostlyright.core.formats.toon.dumps(df)

Serialize a DataFrame as a TOON v3.0 tabular string.

Iterates the DataFrame column-by-column rather than row-by-row. df.iterrows() materializes each row as a Series, which upcasts every cell to the row’s common dtype — so an int64 column with a value above 2**53 sharing a row with a float64 cell would silently lose precision (int(2**60) → float(2**60)). Column-wise iteration preserves each column’s native dtype before per-cell coercion. See module docstring for the full loss matrix.

Phase 6 W2-T6: accepts pandas OR polars input; polars frames are converted to pandas at the boundary because the byte-pinned _toon encoder body is parity-locked (CLAUDE.md) and stays pandas-only.

• Return type: str
• Parameters: df (DataFrame)

mostlyright.core.formats.toon.loads(data)

Parse a TOON v3.0 tabular string back into a DataFrame.

Accepts only the tabular form dumps produces. Other TOON constructs (nested objects, expanded lists) are not supported here — the format module’s wire shape is strictly the tabular block.

• Return type: DataFrame
• Parameters: data (str)