Documents and PDFs

Turn one selected table inside a captured PDF into a dataset through a certified projection.

This document registers, and mr-data cannot produce the eight digests it needs. The capture store, the certified decoder and the operator command that performs capture, table selection and handoff_projection all run inside the backend worker. This client exposes no document command and no local stand-in. The eight digests here have to come from an operator who ran that pipeline. Everything else here is exact, and the recipe validates against the v4 schema as printed.

The question

What figures does the published document state, row by row, and which retained projection does each row come from?

Why a PDF is handled differently

A Reader opens a format with an explicit row contract and turns it into exact rows. A PDF carries no such contract. Its apparent rows and tables are visual hypotheses. Documents go through a separate capture pipeline. It records evidence first, then accepts one explicitly selected table projection into ordinary dataset processing.

The pipeline’s state words are claims, not progress labels:

State	What is true	What is not implied
captured	the exact raw bytes and a content-addressed receipt exist	the file was readable or safe to decode
text extracted	native text and spans can be reproduced from the source digest, page selection, engine name and version, and configuration digest	the text is a correct reading of the visual layout
OCR extracted	a pinned OCR engine transcribed retained rendered images, with boxes and confidence	the transcription is correct
table candidate	the pipeline retained bounded cells, geometry, hypotheses, ambiguity and page provenance	it is a dataset or an approved reading of a table
value evidence	the pipeline retained exact spans inside one selected region	it is a number, a normalized value or a row
certified table projection	one retained candidate plus an exact projection policy produced CSV or Parquet	any unselected surrounding content became data

Only a certified table projection can become a dataset. It enters through handoff_projection and the document.projection@1.0.0 adapter.

The operator’s steps, before the recipe

The pipeline accepts application/pdf, image/png, image/jpeg and image/tiff. It refuses office containers. Certified execution runs on Linux only, inside the worker sandbox, with no network authority.

Capture the original bytes once with their declared media type, and keep the returned receipt digest, which addresses the record of what was captured. A capture proves raw-byte retention, not extraction.
Run inspect_pdf before choosing a decoding path, and read the retained admission and text-layer evidence.
Run extract_native for the exact pages the claim needs. The spans carry boxes and reading order, so the selection stays inspectable.
Run ocr_pages for pages without usable native text, under an explicit policy. The policy decides which pages fall back, not a model guess, and native and OCR spans stay separate artifacts.
Run table_candidates against one page, one bounding box and one span kind, never a mix of native and OCR.
Run project_table with the candidate’s exact digest, explicit header_rows and non-empty body_rows, an output format, and explicit ambiguity and merged-cell policies. No default selects the rows that look tabular.
Run handoff_projection to get the closed set of digests the recipe carries. An exported CSV acquired again is ordinary user_file input, and equal bytes do not restore document provenance.

Footnote-bearing table evidence stays readable. This version does not project it, because no policy can guess its selection or placement.

The recipe

{
  "dataset": {
    "name": "Certified document figures, one projection",
    "description": "Each row is one body row of a table an operator selected out of a captured PDF and projected to CSV under an exact projection policy."
  },
  "question": {
    "text": "What figures does the published document state, row by row, and which retained projection does each row come from?"
  },
  "table": {
    "name": "reported_figures",
    "description": "One row per body row of the projected table.",
    "grain": [
      "row_ordinal"
    ],
    "columns": [
      {
        "name": "row_ordinal",
        "type": "integer",
        "nullable": false,
        "description": "The one-based position of this row inside the projected table.",
        "presentation": {
          "chart": "none"
        }
      },
      {
        "name": "line_item",
        "type": "string",
        "nullable": false,
        "description": "The label the document prints in the first column of the table.",
        "presentation": {
          "chart": "top_values"
        }
      },
      {
        "name": "period",
        "type": "string",
        "nullable": true,
        "description": "The period heading the figure sits under, exactly as the document prints it."
      },
      {
        "name": "reported_value",
        "type": "string",
        "nullable": true,
        "description": "The figure as the document prints it, including any thousands separator or sign the publisher used.",
        "presentation": {
          "chart": "none"
        }
      }
    ]
  },
  "sources": [
    {
      "name": "annual_report_table",
      "description": "One table an operator selected out of a captured annual report and projected to CSV.",
      "source_class": "user_file",
      "data_classification": "internal",
      "locator": {
        "kind": "document_capture",
        "display_locator": "document-capture://receipt/sha256:4444444444444444444444444444444444444444444444444444444444444444"
      },
      "rights_claim": {
        "claimed_basis": "permission_asserted",
        "claim_evidence_digest": "sha256:5555555555555555555555555555555555555555555555555555555555555555",
        "claim_note": "The document was supplied to this workspace by its publisher for internal analysis."
      },
      "connector": {
        "adapter_id": "document.projection@1.0.0",
        "credential_mode": "none",
        "parameters": [
          {
            "name": "document.receipt_digest",
            "value": "sha256:4444444444444444444444444444444444444444444444444444444444444444"
          },
          {
            "name": "document.raw_document_digest",
            "value": "sha256:6666666666666666666666666666666666666666666666666666666666666666"
          },
          {
            "name": "document.projection_manifest_digest",
            "value": "sha256:7777777777777777777777777777777777777777777777777777777777777777"
          },
          {
            "name": "document.projection_content_digest",
            "value": "sha256:8888888888888888888888888888888888888888888888888888888888888888"
          },
          {
            "name": "document.candidate_manifest_digest",
            "value": "sha256:9999999999999999999999999999999999999999999999999999999999999999"
          },
          {
            "name": "document.projection_policy_digest",
            "value": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
          },
          {
            "name": "document.media_type",
            "value": "text/csv"
          },
          {
            "name": "document.source_version",
            "value": "mostlyright-document-projection-source.v1"
          }
        ]
      },
      "limits": {
        "max_source_bytes": 3145728,
        "max_rows": 100000,
        "max_requests": 1
      }
    }
  ],
  "transform": {
    "engine": "duckdb_sql",
    "steps": [
      {
        "step_id": "reported_figures",
        "sql": "select cast(row_ordinal as integer) as row_ordinal, line_item, period, reported_value from annual_report_table",
        "description": "Carry the projected columns through under the names the projection policy gave them. The projection is sealed as CSV, so every column reaches the transform as text."
      }
    ]
  },
  "checks": [
    {
      "check_id": "one_row_per_projected_row",
      "kind": "key_uniqueness",
      "enforcement": "required",
      "columns": [
        "row_ordinal"
      ]
    },
    {
      "check_id": "line_item_present",
      "kind": "null_ceiling",
      "enforcement": "required",
      "columns": [
        "line_item"
      ],
      "max_null_ppm": 0
    },
    {
      "check_id": "rows_present",
      "kind": "row_expectation",
      "enforcement": "required",
      "min_rows": 1
    }
  ],
  "units": [],
  "timezone": "UTC"
}

Part by part

The source

locator.kind is document_capture and adapter_id is document.projection@1.0.0. The receipt records that adapter name as the route the source arrived through. The run fetches nothing. It redeems the projection from the operator’s capture store.

credential_mode is none. The source names no origin, because there is no host.

The eight reserved parameters

declared_source is closed and carries no member for a capture address, so the eight digests ride on connector.parameters. That array already folds into the source authority digest, so an edit in transit moves the recipe digest.

Parameter	What it addresses
`document.receipt_digest`	the retained receipt the projection belongs to
`document.raw_document_digest`	the original captured bytes
`document.projection_manifest_digest`	the projection artifact, as bytes plus source, engine, configuration, pages, region, bounds and parents
`document.projection_content_digest`	the projection’s bytes alone
`document.candidate_manifest_digest`	the table candidate the projection was made from
`document.projection_policy_digest`	the exact header, body, ambiguity and merged-cell policy
`document.media_type`	`text/csv` or `application/vnd.apache.parquet`
`document.source_version`	`mostlyright-document-projection-source.v1`

All eight or none. A source that names half the set cannot be replayed from its own record, so a partial set fails with RUN_RECIPE_INVALID naming the missing members. The engine also refuses a parameter inside the reserved document. namespace that matches none of the eight, instead of ignoring it. A typo would otherwise pass as a digest the worker never read.

The adapter reopens the named receipt from its configured capture store and re-verifies all eight digests before parsing the retained CSV inside the worker sandbox. Deployment configuration sets the store’s location. No recipe and no run document can select it.

Transform

The projection is stored as CSV, so every column reaches the SQL as text and each declared type needs an explicit cast. The column names come from the projection policy, chosen at project_table time rather than here.

Columns

reported_value is declared string, not decimal. The pipeline keeps the source text as printed. It does not parse 1,234, remove a percent sign, infer a unit, or turn prose into a number. Cast to a number in the transform, where the cast is visible and can fail, and keep the printed text beside it. The receipt cites what the document said.

Checks

key_uniqueness on row_ordinal and a null_ceiling of zero on line_item catch a projection whose rows did not come out as the policy expected.

What the receipt records

Every derived artifact records the source content digest and the primary engine. It records the ordered decoder chain with binary and OCR model digests, the canonical configuration digest, and the sandbox policy and bounds digest. It records the page and region selection too, with parent manifest digests and its own content digest, media type and size.

Read an artifact, or follow its parentage, by manifest digest. The content digest addresses its bytes. The manifest digest addresses those bytes plus everything that produced them. Two rendered pages can have the same image bytes and different manifest digests.

Traps

Do not turn a whole page into a one-column table to extract one number. Name the page and box that hold the printed token, then run select_spans against the retained spans artifact. Under ambiguity_policy: "refuse" the region must contain one span. Under "preserve" the pipeline retains every matching span in source order, so a reviewer can see the region was ambiguous.

Do not merge native and OCR spans into a hybrid table. Retain the OCR spans and their image geometry first, then build the candidate against those.

The 128 MiB source ceiling is a retention limit, not an extraction guarantee. Certified operations use a 32 MiB JSON transport envelope and inline the source as base64. A source at or above 24 MiB refuses before that copy is allocated. The usable size drops further when the operation also carries digests or retained evidence.

An instruction can narrow a certified ceiling and can never widen one.

Confidence is evidence, not proof. It never upgrades an OCR claim to a native-text claim.