Documents and files

Extract a selected table from a captured PDF and use its certified projection as a dataset source.

A PDF, a scan or an image has no explicit row contract. Document extraction therefore captures evidence first, and it accepts one explicitly selected table projection into a dataset. It is separate from the Reader path. A Reader turns a format that has an explicit row contract into canonical rows, and a PDF does not generally have that property.

What is reachable from mr-data today

The CLI ships no document command. Capture, inspection, extraction and projection handoff run inside the backend worker and belong to the hosted operator, not to this client. No local command stands in for them, and no install profile changes that.

The recipe schema does allow the last step. document.projection@1.0.0 is in the adapter list. The worker redeems a source under it from the operator’s capture store rather than fetching it. That source carries its place in the store in reserved document. connector parameters. It names its own adapter because the receipt records that place as the route the source arrived through.

So a recipe can read a certified projection somebody else produced. You cannot produce one from mr-data. Where a request needs that, say the capability is unavailable for hosted execution. A whole page turned into a one-column table is not a projection, and equal bytes do not restore document provenance.

The six product words

These are product claims, not interchangeable progress labels. Only the last one is eligible to become a dataset.

State	What holds	What it does not imply
captured	Exact raw input bytes and a content-addressed receipt exist. Nothing replaces the raw artifact.	That the file was readable, safe to decode, or contains usable text.
text extracted	The operator can reproduce a native-text artifact and a native-spans artifact from the source digest, page selection, engine name and version, configuration digest and bounds.	That the text is a correct interpretation of visual layout or of a table.
OCR extracted	A pinned OCR engine made a machine transcription from retained rendered images, with page-level boxes and confidence.	That the transcription is correct. Confidence is evidence, not proof.
table candidate	The candidate retains bounded cells, geometry, hypotheses, ambiguity and page or region provenance.	That it is a dataset, or an approved interpretation of a table.
value evidence	The evidence retains exact native or OCR spans within one explicitly selected region.	That it is a numeric value, a normalized value, or a dataset row.
certified table projection	One retained table candidate and an exact projection policy produced CSV or Parquet.	That any unselected surrounding content became data.

None of these is a table. Each stays auditable document evidence.

What capture accepts

Capture accepts four media types: application/pdf, image/png, image/jpeg, image/tiff. PDF handling covers native-text, scanned, mixed and image-only PDFs. A pinned decoder normalizes image inputs before OCR.

Office containers are not PDF inputs, and capture refuses them. It opens no Office XML, no macros, no RAR/7z, no embedded executables and no arbitrary plugins.

The closed operations

Seven, and they are the whole set. The agent does not choose a native PDF decoder, renderer, OCR model, language pack, image encoder or table extractor. The registry fixes those, and the resulting artifact records them.

Operation	Required inputs	What it returns
inspect_pdf	No page selection, no options.	PDF admission and structural evidence only.
extract_native	Explicit pages, no options.	Native text and page or spans evidence.
render_pages	Explicit pages and {dpi, colorspace, rotation}. DPI 150, 200 or 300. Colorspace rgb or gray. Rotation 0/90/180/270.	Retained page images.
ocr_pages	Explicit pages and the complete pinned usable-text policy.	OCR spans only for policy-selected fallback pages.
table_candidates	One page, one bounding box, native or ocr, and the exact retained spans content digest.	Cell geometry and hypotheses. Never a dataset.
select_spans	One page, one bounding box, source kind, the exact spans digest, and refuse or preserve ambiguity policy.	Non-dataset evidence for one or more exact spans.
project_table	The retained candidate’s digest, explicit header_rows and non-empty body_rows, output format, and explicit ambiguity and merged-cell policies.	A CSV or Parquet certified table projection.

The ordinary sequence:

1. Capture the bytes once with their declared media type.
2. inspect_pdf before choosing a decoding path.
3. extract_native for the exact pages the claim rests on.
4. ocr_pages for pages that need transcription, under an explicit usable-text policy rather than a model guess.
5. select_spans for one value, or table_candidates for a possible grid.
6. project_table only for reviewed table evidence.

Native and OCR spans stay separate artifacts in a mixed document. Do not merge them into an invented hybrid table.

What it refuses rather than guesses

One number is not a one-column table. select_spans against the retained spans artifact returns the exact printed token. With ambiguity_policy: "refuse" the region must hold exactly one span. With "preserve" the artifact keeps every matching span in source order, so a reviewer sees that the region was ambiguous. The value evidence preserves the source text exactly. It does not parse 1,234, remove a percent sign, infer a unit, or turn prose into a numerical fact.

A table is a hypothesis, not a property the PDF format guarantees. Tables carry floating labels, diagonal layout, split headers, raster words, merged cells and empty cells. They carry footnotes, duplicate text layers, or an order with no unambiguous row-major reading. A table candidate retains all of that, including footnote spans and the exact fragments where a placement is ambiguous. It can honestly return no usable table region, and it must not invent missing cells or flatten uncertainty.

There is no default that takes every row looking tabular. project_table names header_rows and every body_row explicitly, and prose above or below a table never reaches the output on its own.

This version projects no footnote-bearing table evidence, because no policy can guess its selection or placement. It projects no evidence with conflicting merged-cell occupancy either.

Five conditions reject rather than create plausible-looking data. They are duplicate or empty headers, absent selected rows, unapproved merged cells, ambiguous order under refuse, and cell or output budget excess.

Ceilings

The ceilings are fixed: 128 MiB source bytes, 2,000 PDF pages, 256 selected or OCR pages, one million PDF objects, spans or table cells. It also caps 20,000 images, 40 million pixels per rendered page, 500 million rendered pixels in total, 120 OCR seconds and 180 wall-clock seconds.

The 128 MiB source ceiling covers raw-capture retention, not extraction. Certified operations use a 32 MiB JSON envelope and inline the source. A source at or above 24 MiB refuses before the worker allocates that copy. The usable size is lower again when the operation also carries coordinates or retained evidence. An instruction can narrow a ceiling and cannot widen either one.

Certified execution is Linux-only. It runs with no network authority, bounded memory and tasks, and a filesystem allowlist that limits writes to private scratch. It pins the runtime closure and re-hashes it before each launch. It allows no plugin discovery, no PATH lookup, no model download and no decoder substitution.

What every derivative records

The raw document, inspection, native text, native spans, rendered pages, OCR spans, table candidates, value evidence and certified projections are independent immutable artifacts. A derived artifact records the source content digest and the exact engine and ordered decoder chain, binary and model digests included. It also records the configuration and policy digests, the exact page and region selection, and the parent manifest digests. It records its own content digest, media type and byte size.

Use a manifest digest when reading an artifact or following parentage. The content digest addresses its bytes. The manifest digest addresses those bytes plus source, engine, configuration, pages, region, bounds and parents. Two rendered pages can legitimately carry the same image bytes and different manifest digests.

What document extraction does not claim

• It does not make a generic PDF into a canonical table.
• It does not infer meaning, units, numeric types, headers or rows outside a pinned policy.
• It does not use OCR merely because native text is inconvenient, and it does not silently combine OCR and native text.
• It does not open arbitrary Office documents, macros, archives or executables.
• It does not make a document-derived claim without the raw-byte receipt and retained provenance.

Scientific and satellite formats

These go through the ordinary Reader path, not document extraction. A source pins one Reader name and version and closed decode options, and the Reader returns canonical CSV. Scientific inputs add an exact product profile, a bounded native selection, immutable asset evidence, and calibration, geolocation or quality evidence where the profile requires it. A file signature alone never counts as support.

Input	Reader name and version
GRIB2 collections	weather.grib2@3.0.0
Single-record GRIB2	weather.grib2@1.0.0, weather.grib2@2.0.0
Operational GRIB1/GRIB2	meteorology.grib@1.0.0
BUFR	meteorology.bufr@1.0.0
GOES ABI L2 NetCDF-4	satellite.goes_abi_l2@1.0.0
NetCDF-3/4 physical arrays	scientific.netcdf@1.0.0
HDF5 / HDF-EOS5	scientific.hdf5@1.0.0
HDF4 SD	scientific.hdf4@1.0.0
HDF-EOS2 Grid/Swath	scientific.hdf_eos2@1.0.0
Zarr v2/v3 (ZIP-packaged)	scientific.zarr@1.0.0
TIFF, BigTIFF, GeoTIFF, COG, J2K/JP2	raster.geospatial@1.0.0
Sentinel SAFE	satellite.sentinel_safe@1.0.0
Himawari HSD	satellite.himawari_hsd@1.0.0, satellite.himawari_hsd_complete@1.0.0
MSG-shaped HRIT/LRIT	satellite.xrit_image@1.0.0
MSG SEVIRI Level 1.5	satellite.msg_seviri_l15@1.0.0
FITS	scientific.fits@1.0.0
PNG and JPEG	image.rendered@1.0.0

The GRIB2 versions are three contracts, not three revisions of one. weather.grib2@1.0.0 and @2.0.0 each open exactly one GRIB2 record and refuse a multi-record GFS or HRRR file. Supply the exact record bytes taken from the publisher’s index and byte range. @3.0.0 instead accepts a bounded multi-record collection and selects exactly one message by its recorded field, surface, run and valid-time values. Zero matches and multiple matches both refuse. Version 2 adds time, level, matched coordinates and message digest to the output, which version 1 does not emit. Where a family’s major version has moved, the highest version is a different output contract rather than a wider door onto the same one. Pick the version whose output you want.

Reader settings stay closed. Unknown settings, unsupported wrappers, missing archive members, invalid sheets or ranges and unsupported delimiters all produce typed refusals. So do invalid UTF-8, missing headings, multi-record weather files, unapproved grids or packing, lossy codecs, uncovered points, truncated files and corrupt values. Change the recipe or the source. Do not widen validation, and do not report a refused build as an empty result.

A Reader reads a native-bounded window, not the whole array. A scientific Reader takes an exact inclusive-exclusive window: ranges matching the array’s rank, an x/y/width/height raster window, or named nearest points. The profile caps how many cells one selection may return, and a request over that cap refuses.

Every source receipt records the source, the byte digest, the row digest, and the Reader name, version and settings.

Fetching one message out of a large object

No hosted route exists for a slice order, the fetch-by-byte-range-per-message pattern a GRIB2 model run wants. Where the publisher exposes the addresses separately, declare them as separate sources in the recipe. Say plainly that the slice order was not available. Never present N separate acquisitions as one slice order: a slice order shares one DNS epoch across all its requests, and N acquisitions do not.

Next

Readers is the full list, with every family’s required settings and how a recipe pins a Reader through connector.parameters. Write a recipe covers where the pin sits in the document.