mostlyright.core.formats.parquet
Parquet format — Arrow-backed lossless serialization.
Uses pyarrow as the engine and zstd compression. Lossless for the schemas in this project (string, int, Int64 nullable, float, bool, tz-aware Timestamp[ns, tz], object-of-strings).
Documented exceptions:
- pandas
Categoricaldtypes roundtrip asCategoricalwithpyarrow’s dictionary encoding, but the categoryorderedflag is preserved while the underlyingdtypemay rebuild on read. - pandas extension dtypes that pyarrow does not natively map may degrade to
their
objectrepresentation. - Object columns holding mixed Python types are written using pyarrow’s best-effort type inference; columns mixing dicts/lists and primitives will raise at write time rather than silently coerce.
This module does not enforce decompression-bomb protection. The MCP server
applies the per-payload uncompressed-size cap before invoking loads.
Functions
Section titled “Functions”| Function | Description |
|---|---|
dumps(df) | Serialize a DataFrame to a parquet byte payload. |
loads(data) | Parse a parquet byte payload back into a DataFrame. |
mostlyright.core.formats.parquet.dumps(df)
Section titled “mostlyright.core.formats.parquet.dumps(df)”Serialize a DataFrame to a parquet byte payload.
Engine: pyarrow. Compression: zstd. The full DataFrame schema —
column names, dtypes (including nullable Int64, boolean,
and tz-aware Timestamp), and the index unless the caller drops
it — is preserved on the wire.
- Return type:
bytes - Parameters: df (DataFrame)
mostlyright.core.formats.parquet.loads(data)
Section titled “mostlyright.core.formats.parquet.loads(data)”Parse a parquet byte payload back into a DataFrame.
Engine: pyarrow. This function applies no size cap of its own: the MCP server enforces a per-payload uncompressed-size cap (1 GB default) before invoking it, and an SDK caller is trusted to pass parquet they themselves produced.
- Return type:
DataFrame - Parameters: data (bytes)