# Recipes and sources Source: https://mostlyright.md/docs/reference/cli/recipe/ Find a source with `catalog`, question it with `probe`, store its credential with `keys`, then register the whole recipe document with `recipe`. ## recipe ```bash mr-data recipe FILE [--no-activity] [--json] mr-data recipe show RECIPE_ID [--json] mr-data recipe readiness [--json] ``` `mr-data recipe` registers one whole recipe document and prints the identifiers the server derived: the recipe, its digest, the dataset, the table and the sources. Identical bytes converge on one recipe by digest, so registering the same file twice is idempotent. A changed document is a revision against the same dataset and table. Registration is the slowest step a person watches, so the command tells the dataset page it is drafting before it sends. Write the document first ([recipe reference](/docs/reference/recipe/)), register it, then start a run with the identifier and digest this prints. The positional is a path to a JSON file, or the word `show` followed by a recipe identifier. `recipe show` on its own refuses by name rather than registering a file called `show`. Name such a file with a directory in front of it. | Flag | Argument | Required | Meaning | Default | | --- | --- | --- | --- | --- | | `FILE` | path or `show` | yes | The recipe document to register, or the word `show`. | none | | `RECIPE_ID` | id | with `show` | The recipe to read back. | none | | `--no-activity` | none | no | Do not tell the dataset page this command is drafting. From 0.27.1 the line is also held back, without the flag, while the page shows a current line the agent set itself. | off | | `--json` | none | no | One JSON object instead of human lines. | off | This client checks the cheap half before it sends. The file parses as strict JSON, it is one object and it is at most 1 MiB. It carries `dataset`, `question`, `table`, `sources`, `transform`, `checks`, `units` and `timezone`. Studio validates the whole document against `recipe.schema.json`. ```json title="mr-data recipe recipe.json --json" { "schema_version": "mostlyright-thin-client-recipe.v1", "status": "recipe_registered", "lane": "hosted", "recipe_id": "1a2b3c4d-5e6f-4a7b-8c9d-0e1f2a3b4c5d", "recipe_digest": "9f3c2b…", "dataset_id": "d31017e0", "table_id": "3f8c2d10-6a4b-4c8e-9d1f-5b7a0c2e4d68", "source_ids": ["2c7d5b18-9e04-4a63-8b1d-6f3a9c0e5d72", "8b1f0a63-4d29-4e75-9c08-3a6d2b7f1e40"], "columns_without_description": 3, "warnings": { "warning 1": "…", "warning 2": "…" }, "idempotent_by": "recipe_digest" } ``` Values are illustrative. A clean registration carries no `columns_without_description` and no `warnings`. `recipe show` answers `schema_version: "mostlyright-thin-client-recipe-read.v1"`, `status: "recipe_reported"`, the same five identifiers, and `document` carrying the registered document verbatim. Errors: | Code | When | | --- | --- | | `THIN_RECIPE_DOCUMENT_UNREADABLE` | The file could not be read, or runs over 1 MiB. | | `THIN_RECIPE_DOCUMENT_INVALID` | Not strict JSON, not one object, or missing a required member. The refusal names the path, the reason and the member rather than handing back a 422. A recipe document carries integers, strings, booleans, null, objects and lists, and never a fractional number. | | `THIN_REQUEST_INVALID` | `recipe show` with no identifier, or a second positional after a file. | | `THIN_RESPONSE_INVALID` | Studio registered a recipe without one of the five identifiers, or without any source identifiers. | | `THIN_STUDIO_*` | Studio's own schema validation and contract refusals, carried through under their own codes. | ```bash title="register, then run" REG=$(mr-data recipe recipe.json --json) mr-data run --recipe "$(jq -r .recipe_id <<<"$REG")" \ --digest "$(jq -r .recipe_digest <<<"$REG")" \ --full --json ``` ## recipe readiness ```bash mr-data recipe readiness [--json] ``` Read every recipe revision in the active workspace and classify whether its current table can refresh incrementally. This is an inventory read: it starts no run and changes no recipe. Each row names the immutable recipe revision, whether it is current, the table's live state, `refresh_ready`, every source's classification (`closed`, `window`, `collection`, `recorded`, or `resync_only`), and the names blocking incremental refresh. Use it before a broad recipe migration. A `refresh_ready: false` row is not permission to retry a whole source on the schedule; correct its recipe or use the explicit table resync operation when a full reread is intentional. ## probe ```bash mr-data probe SOURCE QUESTION --dataset DATASET [--kind {source_inspect,sample_rows,profile_columns,evaluate_expression}] [--request JSON | --request-file FILE] [--json] ``` `mr-data probe` asks one source one question through a warm worker, before a recipe commits to it. It opens or reuses a session and closes it, so there is no session to name. It closes the session it opened even when the probe is refused. Use it to learn a source's shape, its columns and its first rows, while the table is still undecided. `source_inspect`, the default, reports the shape and no cell values. Two limits. The command reaches the frozen v3 session routes, and a deployment that has switched them off refuses it. It also takes a question identifier, which no v4 registration receipt returns. On a hosted-only workspace, register the recipe, take an unwindowed sample under `--max-rows`, then run `peek`, `query` and `receipt`. [Probe a source](/docs/build/probe-sources/) walks through that. | Flag | Argument | Required | Meaning | Default | | --- | --- | --- | --- | --- | | `SOURCE` | id | yes | The source to ask about, by its identifier. | none | | `QUESTION` | id | yes | The question being explored, by its identifier. A session is bound to one. | none | | `--dataset` | `DATASET` | yes | The dataset the source and the question belong to. | none | | `--kind` | one of four | no | What to ask. `source_inspect` reports shape only. `sample_rows` needs a `limit` in the request body. `profile_columns` and `evaluate_expression` take their own bodies. | `source_inspect` | | `--request` | JSON object | no | The rest of the request body as one JSON object. The source is filled in from `SOURCE`. | none | | `--request-file` | `FILE` | no | The same body read from a JSON file instead of the command line. | none | | `--json` | none | no | One JSON object instead of human lines. | off | ```json title="mr-data probe SRC QST --dataset D --json" { "schema_version": "mostlyright-thin-client-probe.v1", "status": "probe_answered", "lane": "hosted", "source_id": "2c7d5b18-9e04-4a63-8b1d-6f3a9c0e5d72", "question_id": "6a0d3e91-7c58-4b2f-8d16-4e9b0c3a7f52", "dataset_id": "d31017e0", "probe_kind": "source_inspect", "answer": {}, "session": { "reused": true, "closed": true }, "not_evidence": "…" } ``` Values are illustrative. `status` is `probe_answered` when the worker answered, and `probe_settled` when it settled without an answer, having failed, expired or been cancelled. On `probe_settled`, `answer` is `null`, `failure_code` carries the worker's code when there is one, and the command exits `2`. Errors: `THIN_RESEARCH_SESSION_ENDED` and `THIN_RESEARCH_SOURCE_FORBIDDEN`. `THIN_SESSION_BINDING` when Studio answered about a different session, probe or session run than the one asked about, which says nothing about the session's own state. `THIN_REQUEST_INVALID` for a `--request` that is not one JSON object, or a `--kind` that needs a body and got none. A workspace already at its bound of live research sessions refuses with `retry_after_seconds` on the refusal. ```bash mr-data probe "$SOURCE_ID" "$QUESTION_ID" --dataset "$DATASET_ID" --kind sample_rows \ --request '{"limit": 20}' --json ``` ## catalog ```bash mr-data catalog search QUESTION [--format FORMAT] [--limit N] [--json] ``` `mr-data catalog search` asks the public-source catalogue one question and reads back the entries it ranks against it, best first. Every ranked entry carries the disposition its own facts earned, an entry the catalogue could not vouch for included. It fetches nothing and registers no source. The answer is discovery evidence over one provider's partial sweep, and it is never exhaustive. Search elsewhere too. | Flag | Argument | Required | Meaning | Default | | --- | --- | --- | --- | --- | | `search` | none | yes | The one verb. It asks the catalogue and ranks what it holds. | none | | `QUESTION` | text | yes | What the data has to be about, in plain language. The grain, the place and the period, not a filename. | none | | `--format` | `csv`, `json`, … | no | The one data format this question requires. It states the requirement and does not filter. An entry that does not declare the format still comes back, ranked, with `disposition: "refused"` and `filters_match: false`. The catalogue accepts one format per question. | none. No entry is rejected for its format alone, and most entries declare none | | `--limit` | `1` to `25` | no | How many entries to rank. | `10` | | `--json` | none | no | One JSON object instead of human lines. | off | ```json title="mr-data catalog search '…' --json" { "schema_version": "mostlyright-thin-client-v4-catalog-search.v1", "status": "catalog_searched", "lane": "hosted", "question": "daily maximum temperature by US city since 2020", "incomplete_code": null, "requirement": { "data_format": "csv" }, "generation": { "store_digest": "9f3c…", "entry_count": 262144 }, "work": { "ranges_scanned": 12, "ranges_pruned": 40, "candidates_examined": 900, "member_bytes_read": 811000 }, "candidates": { "source 1": { "rank": 1, "title": "…", "publisher": "…", "description": "…", "spatial_scope": "…", "data_formats": ["csv"], "formats": "csv", "access_kind": "…", "authentication_required": false, "rights": "…", "disposition": "admitted", "filters_match": true, "provider_id": "…", "provider_record_id": "…", "provider_record_url": "https://…", "entry_id": "…", "entry_coordinate": "…", "entry_digest": "…" } }, "next_step": "Each entry is discovery evidence, not a source: …" } ``` Values are illustrative. `formats` is the one member this client derives. It is `"unknown"` when the catalogue holds no format for the entry, and `"none declared"` when the entry declares an empty list. `data_formats` beside it is the contract's own value, untouched. `status` is `catalog_searched`, `catalog_no_admitted_source`, `catalog_search_incomplete` or `catalog_unavailable`. On `catalog_search_incomplete`, `incomplete_code` says why the search stopped early. `catalog_unavailable` is not an error. That deployment has no catalogue to search, the payload includes `refusal: {"code": "THIN_CATALOG_UNAVAILABLE", "detail": "…"}` with every other block `null`, and the command exits `0`. Errors: `THIN_RESPONSE_INVALID` when Studio answers without the `question`, `generation` or `work` blocks that make a ranking citable, or with an unreadable entry. `THIN_REQUEST_INVALID` for a `--limit` outside 1 to 25, or a second `--format`. ```bash mr-data catalog search "hourly electricity demand by US balancing authority" --format csv --limit 5 --json ``` ## connections ```bash mr-data connections [--dataset DATASET] [--json] ``` `mr-data connections` lists the saved connections this workspace holds, so a recipe can name one. It returns the connection identifiers only, no credential, and it changes no grant. A recipe references a connection as `source.connection`, using `connection_id` and `version_digest`. The dataset needs an explicit grant, so set `dataset.id` in the recipe. | Flag | Argument | Required | Meaning | Default | | --- | --- | --- | --- | --- | | `--dataset` | `DATASET` | no | Only show connections granted to this dataset ID. | every connection this workspace holds | | `--json` | none | no | One JSON object instead of human lines. | off | ```json title="mr-data connections --json" { "schema_version": "mostlyright-thin-connections.v1", "connections": [ { "connection_id": "0f5b8c27-3a19-4d64-9e73-2c8d1a6f0b45", "name": "kalshi-prod", "display_name": "Kalshi (production)", "status": "active", "version_digest": "9f3c…", "origin": "https://api.elections.kalshi.com", "source_parameters": {} } ], "truncated": false, "next_step": "Use connection_id and version_digest in source.connection. The dataset must have an explicit grant; set dataset.id in the recipe." } ``` Values are illustrative. A row leaves out a member the record does not carry rather than writing `null`. Errors: `THIN_RESPONSE_INVALID` when Studio returns no connection list. `THIN_AUTHORIZATION_DENIED`, `THIN_NOT_FOUND`. ```bash mr-data connections --dataset "$DATASET_ID" --json | jq '.connections[] | {connection_id, version_digest}' ``` ## keys ```bash mr-data keys set NAME (--value-file FILE | --part NAME=VALUE …) [--json] mr-data keys list [--json] mr-data keys rm NAME [--json] ``` `mr-data keys` stores a source credential by name so a recipe can reference it as `{"secret_name": "…"}`. It reads the value from a file or from standard input, never from the command line. Argv shows up in `ps`, in shell history and in CI logs. The recipe carries the name. The worker redeems a one-time handle for the value when it fetches the source. Values never appear in recipes, receipts, events or logs. | Flag | Argument | Required | Meaning | Default | | --- | --- | --- | --- | --- | | `set\|list\|rm` | word | yes | `set` stores a credential under `NAME`, `list` names the ones this workspace holds, `rm` withdraws one. | none | | `NAME` | name | `set`, `rm` | Lowercase letters, digits, dashes and underscores, starting with a letter, at most 63 characters. | none | | `--value-file` | `FILE` | with `set` | The file holding the value. `-` reads standard input. One trailing newline is removed, because a credential stored with the one `echo` added fails at the source and nothing says why. At most 65536 bytes. | none | | `--part` | `NAME=VALUE` | with `set` | One named part of a shaped credential, which is what a stream venue authenticates with. `NAME=@PATH` reads that part off disk, which is how a signing key arrives. Repeatable, up to 4 parts. Cannot be combined with `--value-file`. | none | | `--json` | none | no | One JSON object instead of human lines. | off | The parts named decide the shape, and the four sets are disjoint: | Shape | Parts | | --- | --- | | `token` | `token` | | `key_pair` | `key_id`, `secret` | | `key_pair_passphrase` | `key_id`, `secret`, `passphrase` | | `key_id_pem` | `key_id`, `private_key_pem` | ```json title="mr-data keys set NAME --value-file - --json" { "schema_version": "mostlyright-thin-client-v4-secret.v1", "status": "secret_stored", "lane": "hosted", "secret": { "name": "kalshi-api", "status": "active", "created_at": "2026-09-12T09:00:00Z", "rotated_at": null, "rotation_generation": 1 }, "rotated": false, "trailing_newline_removed": true, "parts": [], "recipe_reference": { "secret_name": "kalshi-api" }, "note": "The value is stored and was not printed. …" } ``` Values are illustrative. `parts` holds the part names on a shaped credential and never a value. `rotated` says this was a rotation rather than a first enrolment. ```json title="mr-data keys list --json" { "schema_version": "mostlyright-thin-client-v4-secrets.v1", "status": "secrets_listed", "lane": "hosted", "count": 2, "secrets": { "credential 1": { "name": "kalshi-api", "status": "active", "created_at": "2026-08-…", "rotated_at": null } } } ``` ```json title="mr-data keys rm NAME --json" { "schema_version": "mostlyright-thin-client-v4-secret-withdrawn.v1", "status": "secret_withdrawn", "lane": "hosted", "secret": { "name": "kalshi-api", "status": "revoked", "created_at": "…", "rotated_at": "…", "rotation_generation": 2 }, "note": "Nothing resolves kalshi-api any more: no future run redeems a handle for it and no recipe may reference it. Runs that already fetched with it are unchanged." } ``` Errors: | Code | When | | --- | --- | | `THIN_REQUEST_INVALID` | A name outside the grammar. `set` with neither `--value-file` nor `--part`, or with both. A `--part` that is not `NAME=VALUE`. More than four parts. A set of part names matching none of the four shapes. `--part NAME=@PATH` naming a file that could not be opened. | | `THIN_SECRET_MATERIAL_INVALID` | The value, or one part, runs over 65536 bytes. The refusal names a size and never the value. | | `THIN_STUDIO_*` | The vault's own refusals, carried through under their own codes. | ```bash title="store a token without it touching argv" printf '%s' "$KALSHI_TOKEN" | mr-data keys set kalshi-api --value-file - --json ``` ```bash title="store a signing key pair for a stream venue" mr-data keys set kalshi-ws \ --part key_id=1d2c3b4a \ --part private_key_pem=@./kalshi.pem \ --json ``` ## See also - [CLI reference](/docs/reference/cli/): conventions, exit codes, environment variables - [Runs](/docs/reference/cli/run/): starting a run against what `recipe` registered - [Streams](/docs/reference/cli/stream/): the venues a shaped credential authenticates against ## recipe explain ```sh mr-data recipe explain recipe.json --json mr-data recipe explain RECIPE_ID --json ``` Explain accepts a local document or a registered recipe ID. Read `studio_capabilities` before using a feature: only literal `true` means it is available. Read each refresh finding and `fast_features` before registration. Fix a missed feature or state why it does not fit this source. Registration can refuse `THIN_RECIPE_SLOW`. `--accept-slow "reason"` records an intentional exception; it is not a general way to ignore findings. `--no-lint` skips local checks, not server validation. `--delegated "words"` records an actual user delegation, not one invented by the agent. ## recipe forecast ```sh mr-data recipe forecast RECIPE_ID --digest RECIPE_DIGEST --json mr-data recipe forecast RECIPE_ID --mode refresh --json ``` Forecast reports the work a full read or refresh would ask of its sources. It starts no run. With no `--digest`, the command reads the registered digest back. Exit 3 means resize before running. `--no-observed` omits observed source-rate and platform-binding constraints; keep them unless you need that specific comparison. Inspect provider-paid calls separately from Mostly Right usage. Changing the recipe digest can require a full resync and repeat upstream calls. Use the forecast to set source budgets before starting the work. ## recipe test ```sh mr-data recipe test RECIPE_ID --digest RECIPE_DIGEST \ --max-pages-per-source 1 --max-upstream-rows 25 --json ``` This asks for a bounded test report, not a sample build on the live table. A listing walk can fetch 1–5 pages (default 1); a per-row source can read 1–1000 upstream rows (default 25). By default, a credentialed per-row source is skipped and its relation reaches the transform empty. Read that limitation before interpreting a failed join or an empty result. `--allow-paid` permits those credentialed requests and can incur provider charges. `--no-wait` returns after submission; repeating the same bounds reads the same test back. A bounded test does not establish complete coverage. For inspecting a run's artifacts, use the separate [scratch-table workflow](/docs/build/run-and-inspect/#test-before-the-full-build). ## recipe propose ```sh mr-data recipe propose revision.json --sources-from RUN_ID --json ``` Register a revision and start one replay using retained raw inputs from the named successful run. Replay never becomes live. `--confirm` settles a projected-size hold. Use the returned new run ID to inspect results; replay is not permission to promote or publish a revision. ## Filter the readiness inventory `recipe readiness --table TABLE_ID`, `--recipe RECIPE_ID`, and `--current-only` narrow the immutable-revision inventory. Read `refresh_ready`, source classifications and blockers. A revision that cannot refresh does not become refresh-ready by waiting: change its continuation plan, declare an appropriate rebuild, or use an explicit resync.