Skip to content

mostlyright.core.formats.parquet

Parquet format — Arrow-backed lossless serialization.

Uses pyarrow as the engine and zstd compression. Lossless for the schemas in this project (string, int, Int64 nullable, float, bool, tz-aware Timestamp[ns, tz], object-of-strings).

Documented exceptions:

  • pandas Categorical dtypes roundtrip as Categorical with pyarrow’s dictionary encoding, but the category ordered flag is preserved while the underlying dtype may rebuild on read.
  • pandas extension dtypes that pyarrow does not natively map may degrade to their object representation.
  • Object columns holding mixed Python types are written using pyarrow’s best-effort type inference; columns mixing dicts/lists and primitives will raise at write time rather than silently coerce.

This module does not enforce decompression-bomb protection. The MCP server applies the per-payload uncompressed-size cap before invoking loads.

FunctionDescription
dumps(df)Serialize a DataFrame to a parquet byte payload.
loads(data)Parse a parquet byte payload back into a DataFrame.

mostlyright.core.formats.parquet.dumps(df)

Section titled “mostlyright.core.formats.parquet.dumps(df)”

Serialize a DataFrame to a parquet byte payload.

Engine: pyarrow. Compression: zstd. The full DataFrame schema — column names, dtypes (including nullable Int64, boolean, and tz-aware Timestamp), and the index unless the caller drops it — is preserved on the wire.

  • Return type: bytes
  • Parameters: df (DataFrame)

mostlyright.core.formats.parquet.loads(data)

Section titled “mostlyright.core.formats.parquet.loads(data)”

Parse a parquet byte payload back into a DataFrame.

Engine: pyarrow. This function applies no size cap of its own: the MCP server enforces a per-payload uncompressed-size cap (1 GB default) before invoking it, and an SDK caller is trusted to pass parquet they themselves produced.

  • Return type: DataFrame
  • Parameters: data (bytes)