Recipe document

One JSON file names the dataset, the question, the sources, the SQL that builds the table, and the checks the result has to pass.

Register it with one call:

mr-data recipe RECIPE.json --json

The server puts the document into canonical form, takes its SHA-256, and registers it. One transaction covers the dataset, the question, every source, every connector configuration and the table plan.

What comes back

{
  "schema_version": "4.0.0",
  "recipe_id": "0a4f…",
  "recipe_digest": "9c1f…",
  "dataset_id": "4d01d86e-…",
  "table_id": "b7e2…",
  "source_ids": ["…", "…"]
}

Field	Type	Meaning
schema_version	"4.0.0"	the v4 envelope constant
recipe_id	UUID	the recipe resource
recipe_digest	64 lowercase hex, unprefixed	the digest of the canonical document
dataset_id	UUID	the dataset the table belongs to
table_id	UUID	the table this recipe builds, stable across revisions
source_ids	array of UUID, 1 to 256	one id per declared source, in declaration order

mr-data run takes recipe_id and recipe_digest together, so the run states which recipe it means and which bytes of it.

mr-data recipe show RECIPE_ID reads the recipe back and adds workspace_id, created_at and the stored document.

Two rules that cost a round trip

No fractional number appears anywhere in the document. The digest covers harness canonical bytes, which refuse floats, so every quantity is an integer or a decimal string. A value range bound is "min_value": "-50.5", not -50.5. mr-data recipe refuses a fractional number by position before it sends anything, with THIN_RECIPE_DOCUMENT_INVALID.

The client does not compute the digest, and the document cannot state one. The submit body carries document and at most workspace_id and require_category. It has no digest property and refuses unknown ones. Registration is idempotent by the digest the server derived, so identical bytes sent twice register one recipe and return the same ids. A revised document produces a new recipe_digest against the same dataset_id and table_id.

The document

Eight members, all required. Registration refuses anything else.

Member	Type	Required	Meaning
dataset	object	yes	which dataset this table belongs to
question	object	yes	the question the table answers
table	object	yes	the table’s name, grain and columns
sources	array, 1 to 256	yes	where the rows come from
transform	object	yes	how the sources become the table
checks	array, 0 to 128	yes	what has to hold about the result
units	array, 0 to 512	yes	what the numbers are measured in
timezone	IANA name or UTC	yes	must be UTC, see Time zone

An empty checks or units array is valid. The member itself is not optional.

Identifier grammar

A table name, a column name, a source name, a step id and a check id are each an identifier: ^[a-z][a-z0-9_]{0,62}$. Lowercase, starts with a letter, at most 63 characters.

dataset

Path	Type	Required	Constraints	Meaning
dataset.name	string	yes	1 to 200 chars	the dataset’s name
dataset.id	UUID	no	must belong to the registering workspace	bind to a dataset that already exists
dataset.description	string	no	1 to 4000 chars	the page documentation, as Markdown

dataset.id is what mr-data dataset create printed. With it, registration attaches to a page that already exists. Without it, the server resolves the dataset from (workspace_id, name) and creates it when the name is free. A stated name that disagrees with a bound dataset’s name does not rename it. A dataset in another workspace answers 409 DATASET_MISMATCH.

mr-data recipe asks the server to enforce a saved primary category. In practice dataset.id is required and the dataset must already carry a category. Without the id the answer is 422 DATASET_ID_REQUIRED, and without a saved category it is 422 DATASET_CATEGORY_REQUIRED. The categories are in Units and vocabulary.

dataset.id must be a lowercase canonical UUID. mr-data recipe refuses anything else before sending rather than rewriting it.

Leave description out of a document that carries id. Registration writes it, so a later revision overwrites a description somebody edited by hand. Use mr-data dataset set ID --description-file F instead, and state description only where the registration creates the dataset.

question

Path	Type	Required	Constraints	Meaning
question.text	string	yes	1 to 4000 chars	the question the table answers, written out

table

Path	Type	Required	Constraints	Meaning
table.name	identifier	yes	^[a-z][a-z0-9_]{0,62}$	the table’s name
table.grain	array of identifier	yes	1 to 32, unique	the columns whose tuple is one row
table.columns	array of object	yes	1 to 512	the columns, in order
table.description	string	no	1 to 2000 chars	a sentence about the table

Table identity is (workspace_id, dataset_id, name) and holds across revisions. A revised document builds the same table_id, with no _v2 renaming.

Every name in grain must be a column columns declares, or registration answers 422 RECIPE_DOCUMENT_INCOHERENT with the pointer /table/grain/N. The grain is also the natural key a key_uniqueness check falls back to when it names no columns of its own.

table.columns[]

Path	Type	Required	Constraints	Meaning
name	identifier	yes	unique across the table	the column name
type	enum	yes	one of eight names	the declared output type
nullable	boolean	no	absent means true	whether it may be missing
description	string	no	1 to 2000 chars	what the column means
presentation	object	no	none	what a reader should see

type is one of:

string   integer   decimal   float   boolean   date   timestamp   json

decimal is exact and refuses a floating result. float takes the row decimal refuses, so declare it where the measurement is approximate. Transforms says which engine result type satisfies each declared type.

description is optional to the schema and counted by the client. mr-data recipe reports columns_without_description on the receipt it prints, and names up to twelve of them as a warning. Registration goes through either way.

table.columns[].presentation

Path	Type	Required	Constraints	Meaning
chart	enum	yes	histogram, timeline, top_values, share, none	what to draw
story	string	no	1 to 240 chars	the sentence a reader should be able to say
bins	integer	no	4 to 12 on histogram and timeline only	how many marks the chart gets
bucket	enum	no	hour, day, week, month, quarter, year, on timeline only	the grain one bucket covers

A chart describes only some column types, and registration refuses the pairs it cannot draw:

chart	what it draws	allowed type
histogram	a measurement, binned	integer, decimal, float
timeline	an event time, in contiguous buckets	date, timestamp
top_values	the commonest values, counted	anything but json
share	one split bar, the true part against the false part	boolean
none	no marks, and the distinct count in words	any type

Registration refuses bins on a chart with no bins, and bucket on a chart with no grain, with the pointer /table/columns/N/presentation/bins.

A column that states no presentation still gets a chart, picked from the declared type. The dataset page records that the builder made the choice.

sources

One entry per source, at least one and at most 256. A many-page collection is one entry however many pages it covers.

Path	Type	Required	Constraints	Meaning
name	identifier	yes	unique across the document	the relation name the SQL reads
source_class	enum	yes	one of eight names	what kind of source it is
data_classification	enum	yes	public, internal, confidential, restricted	how sensitive the bytes are
locator	object	yes	none	where it is
rights_claim	object	yes	none	the stated basis for using it
connector	object	yes	may arrive as connection instead	how it is fetched
limits	object	yes	none	how much of it to fetch
description	string	no	1 to 240 chars	one sentence saying what this source is
credential	object	no	required exactly when credential_mode is opaque_reference	a workspace secret or a saved connection
binding	enum	no	text (default) or typed	how its columns reach the SQL
closed	boolean	no	absent means false	this source’s range is over
window	object	no	none	which address parameters carry time
collection	object	no	required on public.https.collection@2.0.0, refused elsewhere	how the pages of one corpus are listed

source_class is one of:

mostlyright_sdk   external_adapter   user_file   user_url
user_api          database_extract   webhook      stream

The dataset page draws description beside the endpoint. Write it about the data, like Daily climatological reports, the official highs and lows., not about the run.

binding decides what the transform sees. text is the default, and it binds every relation column VARCHAR. typed keeps the types the bytes carry: Parquet and JSON/NDJSON keep theirs, and the Reader infers CSV columns over the whole file.

sources[].locator

Path	Type	Required	Constraints	Meaning
kind	enum	yes	sdk_connector, https_url, artifact, document_capture, webhook, stream, database	what sort of address this is
display_locator	string	yes	1 to 2048 chars	the address a person would recognise
artifact	object	required when kind is artifact	none	where the stored snapshot is
connector_contract_version	string	no	^[1-9][0-9]*\.[0-9]+\.[0-9]+$	the connector contract version

locator.artifact carries snapshot_id (UUID), content_digest (sha256:…), size_bytes and media_type.

Where a source states both an origin and an https_url locator, the two must name the same host. The worker fetches it from that one host, even when a sibling source declares another.

sources[].rights_claim

Path	Type	Required	Constraints	Meaning
claimed_basis	enum	yes	one of six names	the stated basis for using the source
claim_evidence_digest	string	yes	^sha256:[0-9a-f]{64}$	the digest of the evidence for that claim
claim_note	string	no	1 to 2000 chars	the claim in words

unknown   prohibited   permission_asserted
public_domain_asserted   contractual_license_asserted   terms_of_service_asserted

The document records the claim its author makes, and the run carries it. Studio derives the rights outcome itself. Registration refuses claimed_basis: "prohibited".

sources[].connector

Path	Type	Required	Constraints	Meaning
adapter_id	enum	yes	one of ten names	the adapter the worker acquires through
credential_mode	enum	yes	none or opaque_reference	whether a credential is presented
origin	string	conditional	canonical https://host, no port, path, query or fragment, ≤255 chars	the one host this source reaches
parameters	array	no	≤64 entries	connector parameters
generic_api	object	required on the two generic adapters, refused elsewhere	none	the frozen request, auth, reader and pagination
stream_connector_document_digest	string	no	sha256:…	the connector document a capture ran under

adapter_id is one of:

external.openligadb
public.https
public.https.collection@2.0.0
snapshot.artifact@1.0.0
authenticated.https.api_key@1.0.0
authenticated.https.api_key@2.0.0
authenticated.https.connector@3.0.0
stream.wss@1.0.0
stream.wss@2.0.0
document.projection@1.0.0

What each adapter requires:

adapter_id	credential_mode	also required	forbidden
authenticated.https.api_key@1.0.0	opaque_reference	origin, exactly one credential_scheme parameter valued bearer or x-api-key	nothing
authenticated.https.api_key@2.0.0	opaque_reference	origin, generic_api	a credential_scheme parameter, generic_api.limits, generic_api.job
authenticated.https.connector@3.0.0	opaque_reference	origin, generic_api, generic_api.limits	a credential_scheme parameter
public.https.collection@2.0.0	none	collection, limits.max_requests	window, closed
every other adapter	any	nothing	generic_api, collection

The adapters themselves are in Source kinds and connectors.

sources[].connector.parameters[]

Each entry is a name and a text value.

Path	Type	Required	Constraints
name	string	yes	^[A-Za-z][A-Za-z0-9._-]*$, 1 to 128 chars
value	string	yes	≤2048 chars, except reader.decode_options at ≤262144

The grammar accepts any name. These names are reserved and mean something to Studio:

Parameter	Used by	Meaning
reader.family_id	any adapter that fetches	the Reader family the bytes are decoded with
reader.family_version	any adapter that fetches	that family’s version
reader.decode_options	any adapter that fetches	the family’s settings, as canonical JSON
stream.members	stream.wss@*	the exact members this source reads, comma separated
stream.series	stream.wss@*	the series whose members it follows
document.receipt_digest and seven siblings	document.projection@1.0.0	the capture’s exact address

The three reader.* parameters are all or none, and registration refuses one or two. The settings must be the canonical form of what the family accepts, with every default written out. Registration refuses a subset and quotes the spelling that works. It refuses any other reader.-prefixed name rather than ignoring it. See Readers.

A stream source names stream.members or stream.series or neither, never both. Registration refuses the pair.

document.projection@1.0.0 requires all eight reserved document. parameters: document.receipt_digest, document.raw_document_digest, document.projection_manifest_digest, document.projection_content_digest, document.candidate_manifest_digest, document.projection_policy_digest, document.media_type and document.source_version.

sources[].limits

Path	Type	Required	Constraints	Meaning
max_source_bytes	integer	yes	≥1	where the fetch stops for this source
max_rows	integer	no	≥1	the row ceiling for this source
max_requests	integer	no	≥1	the per-run request ceiling

The schema requires max_source_bytes. Exceeding this clamp fails every mode, including sample.

It says where the fetch stops, not how big the source is, so a large declaration registers. How far a source may fetch depends on how it will be read. A Reader-pinned source and an unpinned csv source reach 256 MiB. An unpinned ndjson source reaches 37,745,664 bytes. An unpinned json or parquet source reaches 16 MiB, and pinning a Reader is the way past that. Limits carries the table. mr-data recipe warns on the receipt when a declaration is larger than a fetch can reach. A lower ceiling turns a full run into CLAMP_EXCEEDED and truncates a sample, so lowering it is not the fix for a source that really is larger. Split the range across several sources and union them in the first transform step.

sources[].credential

Exactly one of two shapes, present exactly when credential_mode is opaque_reference:

{"secret_name": "example_api_key"}

{"connection_id": "…-uuid", "connection_version_digest": "sha256:…"}

Path	Type	Constraints
secret_name	string	^[a-z][a-z0-9]*(?:[-_.][a-z0-9]+)*$, ≤128 chars
connection_id	UUID	none
connection_version_digest	string	^sha256:[a-f0-9]{64}$

The value never appears in the recipe, a receipt, an event or a log. The worker redeems a one-time handle at acquisition. A secret_name that is not enrolled in the workspace answers 422 SECRET_UNKNOWN at registration. See Source credentials.

Registration refuses a credential that is present without opaque_reference, or absent with it, at /sources/N/credential.

sources[].closed

true says the bytes behind this address cannot change any more. A full build or explicit resync fetches it. With a predecessor, closed: true is the one condition that authorizes reuse_predecessor in a strict refresh plan, and a closed source rides beside a request window in a plan the current materializer executes. A plan of closed sources alone has no bounded partition to replace, so Studio returns RESYNC_REQUIRED before acquisition. Omitting it means false.

Write it only where it holds whenever anybody asks. It states something about the source, not about a date. A query naming a range that has ended is no promise, because publishers correct historical data and republish it under the same address. A normal refresh is predecessor-bound; use a full build or explicit resync when a whole-source acquisition is intended.

The schema refuses closed on the collection adapter, and refuses closed: true beside a window.snapshot. A closed source skips the fetch that would stamp today’s partition. Beside an ordinary window it registers and does nothing. Declare one or the other.

sources[].window

Which parameters of this source’s own address carry time, so a refresh asks for a bounded range instead of the whole corpus.

Path	Type	Required	Constraints	Meaning
start_at	UTC timestamp	yes	none	the first moment this corpus begins, a UTC midnight
granularity	const	yes	"day"	the unit every window is aligned to
timezone	const	yes	"UTC"	the zone the rendered parameters are read in
lookback_seconds	integer	yes	0 to 2592000	how far before the committed boundary a refresh re-requests
max_span_seconds	integer	yes	3600 to 315360000	the widest single request
request	object	exactly one of request or snapshot	none	which parameters carry the range
snapshot	object	exactly one of request or snapshot	none	partition by the acquisition day instead
merge	object	yes	none	how the slice becomes the stored relation
physical_scopes	object	no	refused beside snapshot	per-scope completeness bindings
bootstrap	object	no	refused beside snapshot	the seed scopes an empty table’s first refresh requests

request.start and request.end are each one endpoint:

Path	Type	Required	Constraints
encoding	enum	yes	date_parts, iso_date, epoch_seconds
bound	enum	yes	inclusive or exclusive, what the publisher’s parameter means
parameters	object	with date_parts only	year, month, day, each ^[A-Za-z0-9_.~-]{1,64}$
parameter	string	with iso_date or epoch_seconds only	same grammar
pad	enum	date_parts only	zero (default) or none
location	enum	no	query (default) or path

The engine’s window is always half-open [start, end) in UTC, and the renderer converts. Start/inclusive renders the start, and start/exclusive renders the day before it. End/exclusive renders the end, and end/inclusive renders the day before it. iso_date renders YYYY-MM-DD. epoch_seconds renders the unpadded decimal seconds of that UTC midnight.

merge is how the slice becomes the relation:

Path	Type	Required	Constraints	Meaning
materialization	const	yes	"partition_replace"	the only merge there is
partition.column	identifier	yes	a column of the acquired relation	the cell that carries a row’s time
partition.key	enum	yes	iso_date_prefix, iso_date_value, compact_utc_hour	how one day is read out of that cell
row_identity	array of identifier	yes	1 to 8, unique	what the merged relation must be unique on

partition.column is a column of the acquired relation, and one more rule applies where a refresh plan names more than one source. Every window in such a plan has to name a partition.column the table also declares, and all of them have to agree on the materialization and the partition key. The multi-source lane writes one table manifest and maps no renamed field, so a partition column that never reaches the table is refused with RESYNC_REQUIRED before acquisition. A single-source windowed recipe has its own lane and takes a renamed column.

window.snapshot records a publisher’s current state by acquisition day. It takes column (an identifier) and max_partitions (1 to 3660). Every decoded row carries the acquisition day. With it, lookback_seconds is 0 and max_span_seconds is 86400. merge.partition.column must be that column, merge.partition.key must be iso_date_value, and merge.row_identity must carry the column. The source is never closed.

A snapshot address has no normal-refresh action. A normal or scheduled refresh returns RESYNC_REQUIRED before acquisition. Use an explicit resync when you deliberately need another whole snapshot; do not describe a snapshot’s daily partition as a diff from its publisher.

The worker merges the slice into the previous run’s relation before the transform runs. The transform, the checks and a replay all see the whole relation. A window that gained nothing ends the run unchanged.

sources[].collection

Required on public.https.collection@2.0.0 and refused on every other adapter. It has two members, discovery and pages. That adapter also requires limits.max_requests and credential_mode: "none", and refuses window and closed. The full grammar is in Many-page sources.

transform

Path	Type	Required	Constraints	Meaning
transform.engine	enum	yes	duckdb_sql or none	the engine
transform.steps	array	yes	≤64 entries	the steps, in array order
steps[].step_id	identifier	yes	unique, and never equal to a source name	the relation this step defines
steps[].sql	string	yes	1 to 65536 chars	exactly one SELECT
steps[].description	string	no	1 to 2000 chars	what the step does

The schema accepts engine: "none" and an empty steps. A run refuses both with TRANSFORM_PLAN_INVALID. State duckdb_sql and at least one step.

Steps run in array order, and the last step’s result is the table. Transforms has the dialect, the relation names, what the statement must return and what it may not read.

checks

Deterministic checks the worker runs inline over the finished table. At most 128.

Path	Type	Required	Constraints	Meaning
check_id	identifier	yes	unique across the document	the name the result is reported under
kind	enum	yes	key_uniqueness, null_ceiling, value_range, continuity, row_expectation	what kind of check it is
enforcement	enum	yes	required or advisory	whether failing it stops the run
columns	array of identifier	no	≤64, unique, each a declared column	which columns it is about
max_null_ppm	integer	no	0 to 1000000	null ceiling in parts per million
min_value, max_value	decimal string	no	^-?(?:0|[1-9][0-9]{0,38})(?:\.[0-9]{1,18})?$	the inclusive range
min_rows	integer	no	≥0	the row floor
max_rows	integer	no	≥1	the row ceiling
max_gap_seconds	integer	no	≥1	the widest gap allowed
description	string	no	1 to 2000 chars	what the check is for

The schema puts every optional bound on every kind. Checks says which bounds a kind actually reads, and what happens when a document states one it does not.

units

One entry per column that has a unit.

Path	Type	Required	Constraints	Meaning
units[].column	identifier	yes	must be a declared column	the column
units[].unit	string	yes	1 to 64 chars	the code its numbers are in

Declaration order is part of the digested document, and no column appears twice under two spellings of one key. Registration checks that the column exists and nothing else about the code. The accepted vocabulary is in Units and vocabulary.

Time zone

timezone accepts any IANA zone name or UTC at registration, and the worker builds every table in UTC. A document naming another zone registers and then refuses on its first run:

RUN_RECIPE_INVALID: the member recipe.timezone states America/New_York, and every table
this worker builds is built in UTC; state UTC and put the conversion in the steps, where
the result of it is sealed

State UTC and write the conversion into the SQL.

Refresh cadence

The recipe document carries no cadence member. Refresh control is per source: a request window narrows what a refresh fetches, closed can describe predecessor reuse, and window.merge says how the slice becomes the relation. A source with no normal-refresh action, or an action shape with no current bounded materializer, makes Studio return RESYNC_REQUIRED before acquisition rather than re-read a whole source. mr-data promote records the schedule and mr-data reschedule changes it. See Publish and keep it fresh.

Recorded streams

stream.wss@1.0.0 (an open venue) and stream.wss@2.0.0 (a keyed one) name a source built from a capture rather than from an address. The source states a stream locator whose display_locator is the wss:// endpoint, credential_mode: "none", and connector.stream_connector_document_digest, the digest of the connector document the capture ran under. A build never reaches the venue.

The relation is fixed, and the document does not declare it. The connector picks one of two shapes: eight text columns, or nine when it declares a member field, with member fourth, directly after event_time. A connector that says nothing about attribution writes the eight columns it has always written.

Column	Content
epoch	the connection this frame arrived on, as text
event_id	the identity the Recorder assigned the frame
event_time	the declared event time in the connector’s kind, or the RFC 3339 arrival time when that frame was timed by receipt
member	which member of the venue the frame is about, present only when the connector declares a member field. A table can partition by it without parsing the message
message	the provider frame, as canonical JSON text
message_class	snapshot, delta, or whatever else the connector classifies
message_type	the venue’s own message type
received_at	RFC 3339 UTC receipt time
sequence	the frame’s position within its connection, as text

event_time can hold two kinds of value row by row, so a blind cast returns nulls for half the column with no error. See Live streams.

The digest

recipe_digest is sha256(canonical_json_bytes(document)), unprefixed lowercase hex. Canonical form sorts object keys, uses the tightest separators and emits UTF-8 without ASCII escaping. It refuses floats, NaN, duplicate keys, byte-order marks and integers outside the 64-bit signed range.

The server hashes the document after it resolves the connection shorthand and before anything else. It hashes every other member as written. Any edit, a reordered array, a changed description, one more space inside a SQL string, produces a different digest and a different recipe.

The digest is also the idempotency key, so it takes the unprefixed 64-character form. Every other digest on the v4 wire carries the sha256: prefix.

Submitted form versus registered form

One member exists on the way in and not in the stored document. A source may arrive with connection instead of connector and credential:

Path	Type	Required	Meaning
connection.connection_id	UUID	yes	the granted saved connection
connection.version_digest	sha256:-prefixed digest	yes	the connection version being used
connection.parameter_values	object, at most 32 members, each a string, integer or boolean	yes	what the connection’s own parameters are set to for this source
connection.response	object	yes	how the response is read, in the same shape a connector source states it
connection.limits	object	no	the per-source limits, when this source narrows them

The server replaces it with the resolved connector and a credential reference before it takes the digest. The stored document and the digest describe the resolved connector. The dataset must already hold an explicit grant for that connection.

A complete document

This validates against recipe.schema.json. Two sources, one windowed and one closed, joined in two steps, with every check kind and two declared units.

recipe.json

{
  "dataset": {
    "id": "4d01d86e-1f2a-4c3b-9e7d-8f1a2b3c4d5e",
    "name": "New York Central Park weather history, 2026"
  },
  "question": {
    "text": "What was the hourly air temperature and relative humidity at Central Park in 2026?"
  },
  "table": {
    "name": "station_hourly_readings",
    "description": "One row per station and observation time, from 2026-01-01 onward.",
    "grain": ["station", "valid"],
    "columns": [
      {
        "name": "station",
        "type": "string",
        "nullable": false,
        "description": "The reporting ASOS station identifier.",
        "presentation": { "chart": "top_values" }
      },
      {
        "name": "station_name",
        "type": "string",
        "nullable": false,
        "description": "The station's published name.",
        "presentation": { "chart": "none" }
      },
      {
        "name": "valid",
        "type": "timestamp",
        "nullable": false,
        "description": "The instant the observation was taken. The acquired relation names this column valid, and the table declares it under that name so a refresh can partition by it.",
        "presentation": {
          "chart": "timeline",
          "bucket": "day",
          "story": "Readings arrive every hour with no long outages."
        }
      },
      {
        "name": "air_temp_c",
        "type": "decimal",
        "nullable": true,
        "description": "Air temperature at the station, converted from the published Fahrenheit reading.",
        "presentation": { "chart": "histogram", "bins": 8 }
      },
      {
        "name": "relative_humidity_pct",
        "type": "decimal",
        "nullable": true,
        "description": "Relative humidity at the station.",
        "presentation": { "chart": "histogram", "bins": 8 }
      }
    ]
  },
  "sources": [
    {
      "name": "asos_observations",
      "description": "Hourly airport surface observations, the official temperature and humidity readings.",
      "source_class": "user_url",
      "data_classification": "public",
      "locator": {
        "kind": "https_url",
        "display_locator": "https://mesonet.agron.iastate.edu/cgi-bin/request/asos.py?station=NYC&data=tmpf&data=relh&year1=2026&month1=1&day1=1&year2=2026&month2=9&day2=12&format=onlycomma&tz=Etc/UTC&report_type=3&latlon=no&missing=empty&trace=empty"
      },
      "rights_claim": {
        "claimed_basis": "public_domain_asserted",
        "claim_evidence_digest": "sha256:fc6ec632f676ccb16126dc24d86f07ffc4bb1051fc7c299d8b91762628050510",
        "claim_note": "Federal observations redistributed by the Iowa Environmental Mesonet, whose archive page states the data are in the public domain."
      },
      "connector": {
        "adapter_id": "public.https",
        "credential_mode": "none",
        "origin": "https://mesonet.agron.iastate.edu",
        "parameters": [{ "name": "format", "value": "onlycomma" }]
      },
      "limits": {
        "max_source_bytes": 3145728,
        "max_rows": 65536,
        "max_requests": 4
      },
      "window": {
        "start_at": "2026-01-01T00:00:00Z",
        "granularity": "day",
        "timezone": "UTC",
        "lookback_seconds": 172800,
        "max_span_seconds": 2592000,
        "request": {
          "start": {
            "encoding": "date_parts",
            "bound": "inclusive",
            "pad": "none",
            "parameters": { "year": "year1", "month": "month1", "day": "day1" }
          },
          "end": {
            "encoding": "date_parts",
            "bound": "exclusive",
            "pad": "none",
            "parameters": { "year": "year2", "month": "month2", "day": "day2" }
          }
        },
        "merge": {
          "materialization": "partition_replace",
          "partition": { "column": "valid", "key": "iso_date_prefix" },
          "row_identity": ["station", "valid"]
        }
      }
    },
    {
      "name": "station_metadata",
      "description": "The published station roster, one row per ASOS station.",
      "source_class": "user_url",
      "data_classification": "public",
      "closed": true,
      "locator": {
        "kind": "https_url",
        "display_locator": "https://mesonet.agron.iastate.edu/sites/networks.php?network=NY_ASOS&format=csv&nohtml=on"
      },
      "rights_claim": {
        "claimed_basis": "public_domain_asserted",
        "claim_evidence_digest": "sha256:fc6ec632f676ccb16126dc24d86f07ffc4bb1051fc7c299d8b91762628050510"
      },
      "connector": {
        "adapter_id": "public.https",
        "credential_mode": "none",
        "origin": "https://mesonet.agron.iastate.edu"
      },
      "limits": { "max_source_bytes": 1048576 }
    }
  ],
  "transform": {
    "engine": "duckdb_sql",
    "steps": [
      {
        "step_id": "readings",
        "description": "Carry the publisher's own valid column through, cast the readings into the declared types, and drop rows with no timestamp.",
        "sql": "select station, valid, cast((try_cast(tmpf as decimal(6,2)) - 32) * 5 / 9 as decimal(6,2)) as air_temp_c, try_cast(relh as decimal(5,2)) as relative_humidity_pct from asos_observations where valid is not null and valid <> ''"
      },
      {
        "step_id": "joined",
        "description": "Attach the station name, read the publisher's timestamp as a UTC instant, and return the declared columns.",
        "sql": "select r.station, m.station_name, cast(concat(r.valid, ':00+00') as timestamp with time zone) as valid, r.air_temp_c, r.relative_humidity_pct from readings r join (select stid as station, name as station_name from station_metadata) m on m.station = r.station"
      }
    ]
  },
  "checks": [
    {
      "check_id": "grain_unique",
      "kind": "key_uniqueness",
      "enforcement": "required",
      "columns": ["station", "valid"],
      "description": "One row per station and observation time."
    },
    {
      "check_id": "temperature_present",
      "kind": "null_ceiling",
      "enforcement": "required",
      "columns": ["air_temp_c"],
      "max_null_ppm": 20000,
      "description": "At most two per cent of readings may be missing a temperature."
    },
    {
      "check_id": "humidity_in_range",
      "kind": "value_range",
      "enforcement": "required",
      "columns": ["relative_humidity_pct"],
      "min_value": "0",
      "max_value": "100"
    },
    {
      "check_id": "hourly_continuity",
      "kind": "continuity",
      "enforcement": "advisory",
      "columns": ["valid"],
      "max_gap_seconds": 7200,
      "description": "Two consecutive observations more than two hours apart is a reporting outage."
    },
    {
      "check_id": "enough_rows",
      "kind": "row_expectation",
      "enforcement": "required",
      "min_rows": 24,
      "max_rows": 1000000
    }
  ],
  "units": [
    { "column": "air_temp_c", "unit": "Cel" },
    { "column": "relative_humidity_pct", "unit": "%" }
  ],
  "timezone": "UTC"
}

In that document grain is ["station", "valid"], and grain_unique names the same pair. Step joined is the last step, so its result is the table. asos_observations declares a request window whose incremental action re-requests only the last two days plus whatever is new. Its merge.partition.column names valid, a column of the acquired relation, and the table declares a column of that name as well. station_metadata declares closed: true, which describes predecessor reuse. A closed source may ride beside a request window in a strict refresh plan. This plan names two sources, so the window’s partition column has to be a column the table declares. It is one here, so Studio admits the refresh and the table is partitioned by the day it reads off that column.

Step readings carries valid through under the publisher’s own name, and step joined reads it as a UTC instant. Deriving an observed_at from it instead would leave the table unpartitioned and the refresh refused. Both sources bind all-text, so every declared type is an explicit cast. air_temp_c is declared decimal, so the statement ends in a cast(… as decimal(6,2)). A bare division would come back DOUBLE and the run would refuse it. humidity_in_range states min_value and max_value as strings, because the document carries no fractional number.

What registration refuses

Status and code	When
422 CONTRACT_VALIDATION_FAILED	the document does not satisfy the JSON Schema
422 RECIPE_DOCUMENT_INCOHERENT	the document is internally inconsistent: a grain naming an undeclared column, duplicate column, check or step ids, a chart a type cannot carry, a credential that does not match credential_mode, a source declared prohibited
422 RECIPE_DOCUMENT_NOT_CANONICAL	the document cannot be canonicalized, because of a float or an oversized integer
422 DATASET_ID_REQUIRED	category enforcement was requested and dataset.id is absent
422 DATASET_CATEGORY_REQUIRED	the bound dataset has no saved primary category
422 SECRET_UNKNOWN	a credential.secret_name is not enrolled in the workspace
409 DATASET_MISMATCH	dataset.id names a dataset in another workspace

A RECIPE_DOCUMENT_INCOHERENT answer carries one detail per problem, each with a JSON Pointer into the document and a reason.

Two classes of refusal land later, at run time rather than at registration: the transform dialect (TRANSFORM_*, RUN_RECIPE_INVALID) and the check dialect (DECLARED_CHECK*). Both cost a queued run. Transforms and Checks list them.