Readers

A Reader turns fetched bytes into rows. Select an exact family_id@family_version in the recipe.

How a recipe pins a Reader

A v4 source names its Reader on the connector, as three reserved parameters. All three, or none.

the Reader pin

"connector": {
  "adapter_id": "public.https",
  "credential_mode": "none",
  "origin": "https://example.org",
  "parameters": [
    { "name": "reader.family_id",      "value": "json.tabular" },
    { "name": "reader.family_version", "value": "1.1.0" },
    { "name": "reader.decode_options", "value": "{\"columns\":[{\"name\":\"icao_id\",\"pointer\":\"/icaoId\",\"required\":true}],\"document_format\":\"json\",\"expand\":[],\"records_pointer\":\"\"}" }
  ]
}

declared_source has no member for a decoder. The pin rides on connector.parameters, which the source authority digest already folds in. Registration refuses a fourth name under reader. rather than ignoring it.

reader.decode_options is the canonical JSON text of the options the family itself accepts, with every family-defaulted key filled in. A stated subset digests differently from the pin the worker reconstructs, so registration refuses it and quotes the spelling that works. The value runs to 262144 bytes (256 KiB), and every other connector parameter value runs to 2048.

Reader implementations run on the backend and are not included in the thin CLI installation. Use the complete settings shown for the pinned family below. Serialize the admitted options with sorted keys and compact separators; this canonicalizes JSON, not family defaults. A canonical-options refusal supplies the complete expected object.

import json

value = json.dumps({"encoding": "utf-8"}, sort_keys=True, separators=(",", ":"))
# Complete options for text.lines@1.0.0.

A column that may be absent from a record must state "required": false. Publishers routinely omit a key rather than sending null, and a missing required column refuses the record.

Studio checks the envelope at registration without loading Reader code. The (family_id, family_version) pair must be one of the coordinates this page lists. The options must be an object, and the bytes must be canonical JSON inside 256 KiB. The worker runs family-specific admission before it fetches anything.

Where a Reader runs

A Reader needs a memory ceiling of its own, and the warm session worker gives a child none. Admission stamps a Reader-pinned source for the batch shape and refuses it on the warm shape before any fetch. A generic API response always names a Reader inside its frozen configuration, so it takes the same shape.

Three source shapes accept a certified Reader pin:

• a hash-pinned user.file artifact
• a credential-free public HTTPS source
• a bounded ordered public HTTPS collection (public.https.collection@2.0.0). It applies one exact pin to every member and combines equal-schema normalized rows after all members succeed

The worker narrows the response media type to the exact pinned family before it makes the request. Fetched source bytes never leave the confined operation: retrieval, decode and parse are one call into the worker sandbox, and only normalized bytes come back.

On macOS the worker sandbox refuses with SANDBOX_MEMORY_BOUNDARY, because there is no cgroup to bound a Reader’s memory with. It refuses every other sandbox operation with SANDBOX_OS_BOUNDARY. A single-address credential-free public HTTPS acquisition runs in the hosted Linux sandbox from any machine. Bounded collections have no other route.

Picking a version

For an unreviewed source, name the family’s highest certified version. A base version rejects a publisher that merely declined to describe its file. A weak-label widening requires a new family version.

Where a major version has moved, the highest version is a different output contract. weather.grib2@2.0.0 emits columns 1.0.0 does not. A version change is a recipe revision.

The registry

A Reader coordinate is the exact (family_id, family_version) pair, and resolution takes nothing else. There is no plugin scan, entry point or environment override.

Coordinate	Accepted media types	Settings	Optional dependency
archive.gzip@1.0.0	application/gzip application/x-gzip	member_format, plus optional header: true. member is invalid.	none
archive.gzip@1.1.0	application/gzip application/octet-stream application/x-gzip	Same, plus a response served as application/octet-stream.	none
archive.tar@1.0.0	application/x-tar	Same as archive.zip@1.0.0.	none
archive.tar@1.1.0	application/octet-stream application/x-tar	Same, plus a response served as application/octet-stream.	none
archive.tar@2.0.0	application/gzip application/x-gzip application/x-tar application/octet-stream	Explicit compression and encoding; exact member or member list; see physical lines below.	none
text.lines@1.0.0	text/csv text/plain text/tab-separated-values application/octet-stream	Explicit encoding; physical line number and original line text.	none
archive.zip@1.0.0	application/x-zip-compressed application/zip	member, member_format (csv or tsv), and optional header: true.	none
archive.zip@1.1.0	application/x-zip-compressed application/zip	Same, plus member_format: semicolon for publishers such as DWD.	none
archive.zip@1.2.0	application/octet-stream application/x-zip-compressed application/zip	Same as archive.zip@1.1.0, plus a response served as application/octet-stream.	none
archive.zip@1.3.0	application/octet-stream application/x-zip-compressed application/zip	Same as archive.zip@1.2.0; explicit encoding: cp1252 or utf-8, with optional zero-cell record skipping.	none
delimited_text@1.0.0	text/csv text/tab-separated-values	delimiter (comma or tab), optional header: true, optional encoding: utf-8.	none
delimited_text@1.1.0	text/csv text/plain text/tab-separated-values	Same, plus a response served as text/plain, common for public agency CSV such as IEM.	none
delimited_text@2.0.0	application/csv application/octet-stream application/vnd.ms-excel application/vnd.sdmx.data+csv application/x-csv text/csv text/plain text/tab-separated-values text/x-csv	Explicit delimiter, encoding, quoting, header presence, skip_rows, skip_blank_rows, and indexed columns with output names and optional expected_header. The setting unselected_columns requires empty cells unless it is set to allow.	none
geojson.vector@1.0.0	application/geo+json application/json	Required property projection. Optional include_id and include_geometry retain Feature identifiers and RFC 7946 geometries.	none
geopackage.vector@1.0.0	application/geopackage+sqlite3 application/x-sqlite3	table, geometry_column and explicit property columns. Only EPSG:4326 GeoPackage features are accepted.	none
geoparquet.vector@1.0.0	application/vnd.apache.parquet	geometry_column and explicit property columns. Only primary WKB GeoParquet geometry is accepted.	none
html.tabular@1.0.0	text/html	Absolute record element path and relative text/attribute field selectors. A field is required unless it states required: false.	none
html.web_extract@1.0.0	text/html	mode: dom with closed CSS-like record/field selectors, or mode: json with an inert JSON script selector and RFC 6901 field pointers.	none
html.web_extract@1.1.0	text/html	Same settings as 1.0.0. Extracted text follows DOM order, with one space at block-element boundaries and <br>, and none between inline runs.	none
image.rendered@1.0.0	image/jpeg image/png	format plus a bounded x/y/width/height rendered-image crop.	Pillow
json.tabular@1.0.0	application/json application/x-ndjson	columns with output names and RFC 6901 pointers, plus optional document_format (json or ndjson), records_pointer and sequential expand pointers. A column is required unless it states required: false.	none
json.tabular@1.1.0	application/json application/x-ndjson text/plain	Same, plus a response served as text/plain, common for public agency JSON and NDJSON.	none
meteorology.bufr@1.0.0	application/bufr application/octet-stream	Exact message_index and subset_index for the certified cloud fields.	eccodes
meteorology.grib@1.0.0	application/octet-stream application/wmo-grib application/wmo-grib2	Exact GRIB1/GRIB2 message selector plus canonical-decimal named nearest points.	eccodes
microdata.sas_xpt@1.0.0	application/x-sas-transport	Required ordered stored-column projection. SAS Transport XPT only, raw values only.	pyreadstat
microdata.spss_sav@1.0.0	application/x-spss-sav	Required ordered stored-column projection. Raw values only, no labels or display formats.	pyreadstat
microdata.stata_dta@1.0.0	application/x-stata	Required ordered stored-column projection. Raw values only, no labels or display formats.	pyreadstat
official.pcaxis_px@1.0.0	application/x-pcaxis text/plain	dimensions pins STUB-then-HEADING order, with value_column and optional per-code unit. UTF-8 numeric PX tables only.	none
official.sdmx_json@1.0.0	application/json	dimensions pins every series/observation dimension in order, with optional selected attributes (including units) and value_column. Compact one-dataset SDMX-JSON only.	none
parquet.tabular@1.0.0	application/vnd.apache.parquet	Optional columns selects a bounded ordered Parquet projection.	none
protobuf.tabular@1.0.0	application/octet-stream application/protobuf application/vnd.google.protobuf	Digest-pinned base64 FileDescriptorSet, fully-qualified message type, single or length_delimited framing, and fields with an explicit repeated explode or refuse choice.	protobuf
raster.geospatial@1.0.0	image/j2c image/jp2 image/tiff image/x-jp2	format, exact native crs, and an x/y/width/height window.	rasterio
satellite.goes_abi_l2@1.0.0	application/netcdf application/x-netcdf	variable: CMI and exact x/y inclusive-exclusive pixel ranges.	netCDF4
satellite.himawari_hsd@1.0.0	application/octet-stream application/x-himawari-hsd	Certified HSD v1.3 single-segment B13 plus an exact native x/y/width/height radiance window.	numpy
satellite.himawari_hsd_complete@1.0.0	application/zip	Complete HSD v1.3 scan ZIP: band_id, exact ordered segment_members, and a native x/y/width/height radiance window.	numpy
satellite.msg_seviri_l15@1.0.0	application/zip	Exact MSG prologue, eight ordered VIS/IR segments and epilogue. Recorded channel, quality, navigation and calibration, plus a bounded radiance window.	pyPublicDecompWT
satellite.sentinel_safe@1.0.0	application/x-zip-compressed application/zip	SAFE ZIP with exact spectral/quality member names, native CRS, quality layer, and an x/y/width/height window.	rasterio
satellite.xrit_image@1.0.0	application/zip	Exact ZIP prologue, eight ordered HRIT/LRIT image segments and epilogue. Recorded channel, grid and navigation, plus a bounded raw-count window.	pyPublicDecompWT
scientific.fits@1.0.0	application/fits application/octet-stream	band_id plus an exact native x/y/width/height image window.	astropy
scientific.hdf4@1.0.0	application/x-hdf4	Linux x86_64/macOS arm64 HDF4 SD only: exact dataset_name, same-rank ranges, recorded shape/dtype/semantic attributes, and explicit raw values.	pyhdf
scientific.hdf5@1.0.0	application/x-hdf5	Exact dataset_path, same-rank ranges, recorded shape/dtype/chunks/compression/semantic attributes, and explicit raw-value representation.	h5py
scientific.hdf_eos2@1.0.0	application/x-hdf4	Linux x86_64/macOS arm64 HDF-EOS2 Grid/Swath raw field: exact object/field metadata, same-rank ranges, recorded SD attributes, and explicit raw values.	pyhdf
scientific.netcdf@1.0.0	application/netcdf application/x-netcdf	NetCDF-3/4 with exact root variable, same-rank ranges, recorded shape/dtype/chunks/compression/semantic attributes, and explicit raw-value representation.	netCDF4
scientific.zarr@1.0.0	application/zip	ZIP-packaged Zarr v2/v3 with exact array_path, same-rank ranges, recorded shape/dtype/chunks/compression/semantic attributes, and explicit raw-value representation.	zarr
spreadsheet.xlsx@1.0.0	application/vnd.openxmlformats-officedocument.spreadsheetml.sheet	sheet, header_row, and range. The heading row must be the first row of the range.	none
spreadsheet.xlsx@1.1.0	application/octet-stream application/vnd.openxmlformats-officedocument.spreadsheetml.sheet	Same, plus a response served as application/octet-stream.	none
weather.grib2@1.0.0	application/octet-stream application/wmo-grib2	variable, points, and admission_allowlist_version.	mostlyright-grib
weather.grib2@2.0.0	application/octet-stream application/wmo-grib2	Same settings as 1.0.0. The output also includes time, level, matched coordinates and message digest.	mostlyright-grib
weather.grib2@3.0.0	application/octet-stream application/wmo-grib2	Exact collection message selector, variable_name, points, and allowlist version.	mostlyright-grib
xbrl.facts@1.0.0	application/xhtml+xml application/xml	document_kind (xbrl or constrained XML ixbrl) and exact namespace/name fact projection. Contexts, units and metadata remain evidence fields, not joins.	none
xml.tabular@1.0.0	application/rss+xml application/xml text/xml	Absolute record element path, declared namespace prefixes, and relative field selectors ending in text() or @attribute. A field is required unless it states required: false.	none

The runtime loads an optional dependency lazily, so a family can be described without importing it. First use without a compatible binding returns READER_DEPENDENCY_UNAVAILABLE, and the rest of the CLI stays usable.

Settings, family by family

Each block is the decode_options object, before canonical form turns it into the reader.decode_options string. One record is one output row unless the family says otherwise.

Delimited text

One record is one data row. delimited_text@1.0.0 and @1.1.0 read a file whose first row is the heading.

{ "delimiter": ",", "header": true, "encoding": "utf-8" }

delimited_text@2.0.0 reads a file with a publisher preamble, no heading, or padded columns. The recipe selects columns by index, and expected_header asserts what the heading says.

{
  "skip_rows": 2,
  "skip_blank_rows": true,
  "columns": [
    { "index": 0, "name": "Date", "expected_header": "Date" },
    { "index": 1, "name": "food_index", "expected_header": "Food Price Index" },
    { "index": 2, "name": "meat_index", "expected_header": "Meat" }
  ]
}

JSON and NDJSON

One record is one element of the array records_pointer names. records_pointer: "" is the RFC 6901 pointer for a root-level array. expand walks a nested array sequentially, so one parent record becomes one row per child.

{
  "document_format": "json",
  "records_pointer": "/stations",
  "expand": ["/observations"],
  "columns": [
    { "name": "station_id", "pointer": "/id", "required": true },
    { "name": "time", "pointer": "/observations/time", "required": true },
    { "name": "rain", "pointer": "/observations/rain", "required": true }
  ]
}

NDJSON is the same family with document_format: "ndjson" and a root records pointer.

{
  "document_format": "ndjson",
  "records_pointer": "",
  "expand": [],
  "columns": [
    { "name": "station", "pointer": "/station", "required": true },
    { "name": "valid", "pointer": "/valid", "required": true },
    { "name": "tmpf", "pointer": "/tmpf", "required": false }
  ]
}

Physical lines and compressed TAR files

These two coordinates require the Studio API and worker release containing them. Updating the CLI alone does not add Reader support to a deployment; a deployment that has not received that release will refuse the new coordinate.

Use text.lines@1.0.0 for publisher text with preambles, inconsistent delimiters or unbalanced quotes. Set encoding explicitly to utf-8 or cp1252. Output is line_number, raw_line; numbering starts at 1. Blank lines, tabs, quotes and a BOM are preserved. Only CR, LF and CRLF terminators are removed. No field parsing or repairs occur.

archive.tar@2.0.0 adds an explicit compression setting (none or gzip) and requires encoding. Earlier TAR coordinates remain unchanged. To read one file, set member to its exact name and member_format to csv, tsv, semicolon or lines. Delimited forms require header: true; for preambles or inconsistent rows, use lines and parse in the recipe.

For a quarterly bundle, member_format: lines alternatively accepts members, an explicit list of 1–1024 unique exact names. member and members are mutually exclusive. The canonical list is sorted; output follows archive order and carries member_name, line_number, raw_line. A missing, duplicate, sparse, linked or nested selected file refuses the whole decode. There is no wildcard selection or directory extraction. An unrelated file is not added automatically.

Gzip inflation is streamed and checked against the total uncompressed bytes and expansion ratio, including unselected files and the trailer. Only the selected file and canonical output are retained, not the whole inflated TAR. Row, cell and output limits apply across all selected files. This avoids downloading one quarterly archive separately for every daily file. It does not increase the final dataset row limit; plan the final panel within the build limits. A sample row cap does not bypass archive integrity or reader limits.

{"compression":"gzip","encoding":"utf-8","member_format":"lines",
 "members":["prezzo_alle_8-20240101.csv","prezzo_alle_8-20240102.csv"]}

Exact names must come from the publisher’s archive inventory. The example illustrates the option shape, not a claim that every publisher uses those names.

Containers

One record is one data row of the named member. The worker opens the container, reads one member, and decodes that member as delimited text. archive.gzip holds one stream, so member is invalid there.

archive.zip

{ "header": true, "member": "produkt_klima_tag_00044_akt.txt", "member_format": "semicolon" }

archive.tar

{ "header": true, "member": "cities.tsv", "member_format": "tsv" }

archive.gzip

{ "header": true, "member_format": "csv" }

Spreadsheets

One record is one row of the named range. The heading row must be the first row of the range.

{ "sheet": "Cities", "header_row": 1, "range": "A1:C3" }

Receipts identify stored calculated values, so a workbook that shipped computed cells says so.

Parquet

One record is one Parquet row. columns is optional and selects a bounded ordered projection of at most 512 names.

{ "columns": ["symbol", "price"] }

XML

One record is one match of record_selector, an absolute element path. Field selectors are relative and end in text() or @attribute. The recipe declares namespace prefixes, and nothing guesses them.

{
  "record_selector": "/sec:edgarSubmission/sec:offeringData",
  "namespaces": { "sec": "https://www.sec.gov/edgar/formd" },
  "columns": [
    { "name": "issuer_name", "selector": "sec:issuer/sec:issuerName/text()" },
    { "name": "state", "selector": "sec:issuer/sec:issuerAddress/@stateOrCountry" }
  ]
}

HTML tables

One record is one match of record_selector over the parsed document.

{
  "record_selector": "/table/tr",
  "columns": [
    { "name": "id", "selector": "@data-id" },
    { "name": "name", "selector": "th/text()" },
    { "name": "count", "selector": "td/text()" }
  ]
}

HTML pages

html.web_extract has two modes. mode: dom selects records with a closed CSS-like selector grammar. A column with scope: "document" reads once per page, and a column with scope: "record" reads per match.

mode: dom

{
  "mode": "dom",
  "record_selector": "article.story",
  "columns": [
    { "name": "headline", "selector": "h1.headline" },
    { "name": "report", "selector": "div.report" },
    { "name": "price", "selector": "p.price" }
  ]
}

mode: json reads an inert JSON script block and pulls fields out of it with RFC 6901 pointers.

mode: json

{
  "mode": "json",
  "document_selector": "script[type=\"application/ld+json\"]",
  "document_policy": "refuse",
  "records_pointer": "/itemListElement",
  "columns": [
    { "name": "title", "pointer": "/name" },
    { "name": "score", "pointer": "/aggregateRating/ratingValue" },
    { "name": "link", "pointer": "/url" }
  ]
}

1.1.0 changes the text rule and nothing else. Extracted text follows DOM order, with one space at block-element boundaries and at <br>, and none between inline runs. Use 1.1.0 for a new source. 1.0.0 exists so recipes sealed under it still replay.

Protocol buffers

One record is one message, or one exploded repeated field. The descriptor set rides inline as digest-pinned base64, and every repeated field states explode or refuse.

{
  "message_type": ".transit_realtime.FeedMessage",
  "framing": "single",
  "descriptor_set": { "base64": "CqJNChNndGZzLXJlYWx0aW1lLnByb3Rv…", "sha256": "…" },
  "columns": [
    { "name": "entity_id", "path": "entity.id", "repeated": "explode" },
    { "name": "trip_id", "path": "entity.trip_update.trip.trip_id", "repeated": "explode" }
  ]
}

Geospatial vector

One record is one Feature. geojson.vector keeps the Feature id and the RFC 7946 geometry only when asked.

geojson.vector

{
  "columns": [
    { "name": "place", "property": "name" },
    { "name": "population", "property": "population", "required": false }
  ],
  "include_geometry": true,
  "include_id": true
}

geoparquet.vector

{ "geometry_column": "geom", "columns": [{ "name": "place", "field": "name" }] }

geopackage.vector

{ "table": "places", "geometry_column": "geom", "columns": [{ "name": "place", "field": "name" }] }

The reader accepts only primary WKB GeoParquet geometry, and only EPSG:4326 GeoPackage features.

Official statistics

One record is one observation. dimensions pins the order, and nothing infers it from the file.

official.sdmx_json

{
  "dimensions": [
    { "id": "REF_AREA", "name": "area" },
    { "id": "TIME_PERIOD", "name": "period" }
  ],
  "attributes": [
    { "scope": "series", "id": "UNIT_MEASURE", "name": "unit" },
    { "scope": "observation", "id": "CONF_STATUS", "name": "status" }
  ],
  "value_column": "value"
}

official.pcaxis_px

{
  "dimensions": [
    { "id": "Region", "name": "region" },
    { "id": "ContentsCode", "name": "measure" },
    { "id": "Year", "name": "year" }
  ],
  "unit": { "dimension": "ContentsCode", "name": "unit" },
  "value_column": "value"
}

Compact one-dataset SDMX-JSON only. UTF-8 numeric PX tables only.

Statistical microdata

One record is one case. The projection is required and ordered. The values are the stored ones, with no labels and no display formats.

microdata.stata_dta, .spss_sav, .sas_xpt

{ "columns": ["score", "id"] }

Financial filings

One record is one fact. Contexts, units and metadata are evidence fields beside the fact, not joins.

{
  "document_kind": "xbrl",
  "facts": [
    {
      "namespace": "urn:example:us-gaap",
      "name": "RevenueFromContractWithCustomerExcludingAssessedTax"
    }
  ]
}

document_kind: "ixbrl" reads a constrained inline-XBRL XML document instead.

Weather

weather.grib2@1.0.0 and @2.0.0 open one extracted GRIB2 record. They refuse a multi-record GFS or HRRR file. Supply the exact record bytes from the publisher’s index and byte range. @2.0.0 emits the same rows plus time, level, matched coordinates and the message digest.

weather.grib2@1.0.0 and @2.0.0

{
  "admission_allowlist_version": 1,
  "variable": {
    "discipline": 0,
    "name": "temperature_2m",
    "parameter_category": 0,
    "parameter_number": 0,
    "surface_type": 103,
    "surface_value": 2
  },
  "points": [
    { "id": "prague", "latitude": "50.0755", "longitude": "14.4378" },
    { "id": "denver", "latitude": "39.7392", "longitude": "-104.9903" }
  ]
}

weather.grib2@3.0.0 accepts a bounded multi-record collection. It selects one message by its recorded field, surface, run and valid-time values, then reuses the version-2 decoder. Zero matches and multiple matches both refuse.

weather.grib2@3.0.0

{
  "variable_name": "temperature_2m",
  "admission_allowlist_version": 1,
  "collection_profile_version": 1,
  "selector": {
    "discipline": 0,
    "parameter_category": 0,
    "parameter_number": 0,
    "surface_type": 103,
    "surface_value": { "numerator": 2, "denominator": 1 },
    "run_time": "2026-08-05T12:00:00Z",
    "valid_time": "2026-08-05T12:00:00Z"
  },
  "points": [{ "id": "north_west", "latitude": "21.1381", "longitude": "237.2805" }]
}

One record is one point. Latitudes and longitudes are canonical decimal strings, never floats, because the pin is digested.

Meteorology through ecCodes

meteorology.grib

{
  "profile_version": 1,
  "selector": {
    "edition": 1,
    "param_id": 167,
    "type_of_level": "surface",
    "level": 0,
    "data_date": 20070323,
    "data_time": 1200,
    "step_range": "0"
  },
  "points": [{ "id": "origin", "latitude": "0", "longitude": "0" }]
}

meteorology.bufr

{ "message_index": 0, "subset_index": 0 }

Scientific arrays

One record is one cell of the selected window. ranges is same-rank with the array. expected_metadata is fixed: the reader refuses a file whose shape, dtype, chunking, compression or semantic attributes differ from what the pin declares.

scientific.hdf5

{
  "dataset_path": "measurements/temperature",
  "ranges": [{ "start": 0, "stop": 2 }, { "start": 1, "stop": 3 }],
  "expected_metadata": {
    "shape": [2, 3],
    "dtype": "int32",
    "attributes": { "units": "K", "long_name": "air_temperature" },
    "chunks": null,
    "compression": "none"
  },
  "value_representation": "raw"
}

scientific.netcdf takes variable in place of dataset_path. scientific.zarr takes array_path. scientific.hdf4 takes dataset_name. scientific.hdf_eos2 takes dataset_name plus eos_object_type (grid or swath), eos_object_name and field_name.

satellite.goes_abi_l2

{ "variable": "CMI", "y": { "start": 0, "stop": 2 }, "x": { "start": 1, "stop": 3 } }

Raster and rendered images

One record is one pixel for raster.geospatial. image.rendered emits one row for the whole crop.

raster.geospatial

{ "format": "geotiff", "crs": "EPSG:4326", "x": 1, "y": 1, "width": 2, "height": 2 }

image.rendered

{ "format": "png", "x": 1, "y": 0, "width": 2, "height": 2 }

scientific.fits

{ "band_id": "195", "x": 0, "y": 1, "width": 2, "height": 1 }

Satellite segments

satellite.sentinel_safe

{
  "band": "B02",
  "raster_member": "GRANULE/TILE/IMG_DATA/TILE_B02_10m.tif",
  "quality_layer": "SCL",
  "quality_member": "GRANULE/TILE/IMG_DATA/TILE_SCL_10m.tif",
  "crs": "EPSG:4326",
  "x": 1, "y": 0, "width": 2, "height": 2
}

satellite.himawari_hsd

{ "band_id": "B13", "x": 0, "y": 0, "width": 2, "height": 1 }

satellite.himawari_hsd_complete

{
  "band_id": "B13",
  "segment_members": ["segment-01.dat", "segment-02.dat", "segment-03.dat"],
  "x": 0, "y": 1, "width": 2, "height": 3
}

satellite.xrit_image and satellite.msg_seviri_l15 take a prologue member, exactly eight ordered segment members and an epilogue member. They also take the channel and grid constants (band_id, bits_per_pixel, cfac, coff, lfac, loff, columns, lines_per_segment) and a bounded window.

satellite.msg_seviri_l15

{
  "band_id": 9,
  "bits_per_pixel": 10,
  "cfac": -13642337, "coff": 1856,
  "lfac": -13642337, "loff": 1856,
  "columns": 3712,
  "lines_per_segment": 464,
  "prologue_member": "prologue.hrit",
  "segment_members": [
    "segment-01.hrit", "segment-02.hrit", "segment-03.hrit", "segment-04.hrit",
    "segment-05.hrit", "segment-06.hrit", "segment-07.hrit", "segment-08.hrit"
  ],
  "epilogue_member": "epilogue.hrit",
  "x": 0, "y": 0, "width": 2, "height": 8
}

Budgets

Every family declares default budgets. A recipe may narrow them per field and never widen them. Registration refuses a cap that names a field the contract does not have, by name.

Budget group	Families	max_input_bytes	max_output_bytes	max_uncompressed_bytes	max_declared_cells	max_rows	max_columns
Bulk	archive.*, delimited_text, json.tabular, parquet.tabular, geojson.vector, geoparquet.vector, geopackage.vector, official.*	256 MiB	4 GiB	4 GiB	400,000,000	10,000,000	1,024
Streaming tabular	xml.tabular, html.tabular, protobuf.tabular	16 MiB	16 MiB	16 MiB	4,000,000	1,000,000	256
DOM	html.web_extract	16 MiB	16 MiB	16 MiB	4,000,000	250,000	256
Spreadsheet	spreadsheet.xlsx	16 MiB	16 MiB	32 MiB	250,000	250,001	256
Microdata	microdata.*	16 MiB	64 MiB	64 MiB	5,000,000	100,000	128
Filings	xbrl.facts	8 MiB	32 MiB	32 MiB	1,000,000	100,000	6
Contract default	weather.grib2 (all versions)	16 MiB	16 MiB	128 MiB	10,000,000	1,000,000	1,024

The scientific, satellite and raster families each declare their own:

Coordinate	max_input_bytes	max_output_bytes	max_uncompressed_bytes	max_declared_cells	max_rows	max_columns
satellite.goes_abi_l2@1.0.0	128 MiB	64 MiB	256 MiB	4,000,000	1,000,000	4
scientific.hdf5@1.0.0	128 MiB	64 MiB	128 MiB	1,000,000	250,000	33
scientific.netcdf@1.0.0	128 MiB	64 MiB	128 MiB	1,000,000	250,000	33
scientific.hdf4@1.0.0	64 MiB	32 MiB	64 MiB	500,000	125,000	33
scientific.hdf_eos2@1.0.0	64 MiB	32 MiB	64 MiB	500,000	125,000	33
scientific.zarr@1.0.0	64 MiB	64 MiB	96 MiB	1,000,000	250,000	33
raster.geospatial@1.0.0	128 MiB	64 MiB	256 MiB	100,000,000	1,000,000	11
satellite.sentinel_safe@1.0.0	128 MiB	64 MiB	192 MiB	100,000,000	250,000	22
image.rendered@1.0.0	64 MiB	64 MiB	none	1,000,000	1	7
scientific.fits@1.0.0	128 MiB	64 MiB	none	5,000,000	1,000,000	1,024
satellite.himawari_hsd@1.0.0	128 MiB	64 MiB	128 MiB	1,750,000	250,000	7
satellite.himawari_hsd_complete@1.0.0	512 MiB	64 MiB	128 MiB	1,750,000	250,000	7
satellite.xrit_image@1.0.0	128 MiB	32 MiB	64 MiB	1,000,000	250,000	4
satellite.msg_seviri_l15@1.0.0	128 MiB	32 MiB	64 MiB	1,000,000	250,000	11
meteorology.bufr@1.0.0	128 MiB	64 MiB	none	1,000,000	1,000,000	1,024
meteorology.grib@1.0.0	128 MiB	64 MiB	256 MiB	1,500,000	100,000	15

A run narrows max_output_bytes further. The decoded output of one source stops at the lowest of the family’s number, 1 GiB, and one third of the decode child’s address space. So a bulk family’s 4 GiB is the family’s own number and 1 GiB is what a run reaches.

A cell reading none means the family states no value of its own, and the contract default applies: max_uncompressed_bytes 128 MiB, max_expansion_ratio 100, max_container_members 1,024. Two families set the member count below that default: satellite.sentinel_safe to 256, and satellite.himawari_hsd_complete to 99. One sets it above, and that is scientific.zarr at 4,096. meteorology.grib states 1,024, which restates the default rather than narrowing it, and so does scientific.zarr’s max_expansion_ratio of 100. Only the microdata and filing families narrow the expansion ratio, to 4. max_nesting_depth is fixed at 1 and cannot be set. It is a security invariant, not a budget.

Routing and resource feasibility are separate facts. A source may route to the right family and still exceed its uncompressed, output or declared-cell ceilings. It then reports READER_BUDGET, the truthful answer, rather than MEDIA_TYPE. A source’s file and its decoded table both have to fit.

Previewing a Reader before registration

mr-data peek against a run reads the preview the run already wrote. Its --reader, --reader-options and --format flags are accepted and reported as having no effect. No command tries a Reader against a URL.

Settle a Reader by pinning it in the recipe, registering, then taking an unwindowed sample under --max-rows. Read the columns back with peek and the decode evidence with receipt. A wrong pin is a typed refusal on that run, naming the source. See Probe a source for the reading path.

Refusals

Reader settings are closed. Unknown settings, unsupported wrappers, missing archive members, invalid sheets or ranges and unsupported delimiters all produce typed refusals. So do invalid UTF-8, missing headings, multi-record weather files, unapproved grids or packing, lossy codecs, uncovered points, truncated files and corrupt values. Change the recipe or the source. Do not widen validation, and do not report a refused build as an empty result.

Code	Meaning	What the author does
READER_FAMILY_UNKNOWN	the pin names a family the registry does not hold	select a registered certified Reader family
READER_CONTRACT	the pin does not name this family, or the pin is malformed	fix reader.family_id / reader.family_version
READER_OPTIONS	one setting is outside the family’s closed option set: an unknown key, the wrong type, an uncertified value	correct the closed Reader options and run the same acquisition again
READER_ADMISSION	the settings are outside what this family accepts	re-derive decode_options through the family’s own validate_options
READER_BUDGET	the input, output, cell, row or column ceiling was crossed	narrow the projection or the window, or ask the publisher for a smaller slice
READER_DECODE	the bytes did not decode under these settings	check the member, sheet, range, delimiter or selector
READER_DEPENDENCY_UNAVAILABLE	the family’s optional binding is not installed on this worker	the coordinate is not usable on this deployment, so pick another family
READER_MEMBER_MISSING	the named archive member is not in the container	inspect the archive members and set the member option to the exact intended file
READER_MEMBER_AMBIGUOUS	the container holds the named member more than once, and a recipe names exactly one	re-pack the archive, or name a member that appears once
READER_MEMBER_TYPE	the named entry is not a regular file: a directory, symbolic link, hard link, fifo, device or socket, or an encrypted entry	name a regular file. A symbolic link stores the path it points at, not data
READER_NESTED_CONTAINER	the member is itself an archive or an executable, and containers open one layer deep	extract the inner container at the publisher, or name a member that is not one
READER_SHEET_MISSING	the workbook does not hold the named sheet	name a sheet the workbook holds. The refusal lists them
READER_RANGE	the declared window is not a readable range: a header cell that is empty or not text, or a repeated column name	move the header row or the window. A blank or duplicate heading refuses rather than getting an auto-name
READER_GRID_UNKNOWN	the weather message is on a grid template this product has never been certified against	use a message on a certified grid. Approving one means adding the pair to the allowlist and re-certifying
READER_PACKING_UNKNOWN	the weather message uses a packing template this product does not unpack, or pairs a certified grid with a certified packing that has never been opened together	use a certified pair
READER_OUTPUT	a decoded value cannot be represented in the output: it scales outside binary64, or a masked point reached the scaling rule	narrow the window away from the affected points, or take the source at a different resolution
READER_WARMUP	the hosted Reader’s pre-refresh warm-up failed or returned an invalid result	retry. A warm-up that keeps failing is a deployment fault rather than a recipe one
SANDBOX_MEMORY_BOUNDARY	no cgroup to bound the Reader child	the run reached a shape that cannot host a Reader
SANDBOX_OS_BOUNDARY	the worker sandbox has no route on this platform	run it hosted
MEDIA_TYPE	the response media type is not one this family accepts	pin the version whose accepted list carries it

Every source receipt records the source, the byte digest, the row digest, the Reader coordinate and the settings digest.

Refusal codes for the rest of a run are on Run states and errors. Limits and ceilings collects the numeric bounds.