Skip to content

Units and vocabulary

Describe each measured column with a unit from the supported UCUM vocabulary.

In the recipe document’s units array, one entry per column that has one:

"units": [
{ "column": "air_temp_c", "unit": "Cel" },
{ "column": "relative_humidity_pct", "unit": "%" },
{ "column": "wind_speed", "unit": "km/h" },
{ "column": "pm25", "unit": "ug/m3" },
{ "column": "station_count", "unit": "{count}" }
]

An array rather than a map, so declaration order belongs to the digested document and no column appears twice under two spellings of one key. At most 512 entries, and each unit is 1 to 64 characters.

The recipe schema only bounds the length of unit. Registration checks that units[].column names a declared column and returns 422 RECIPE_DOCUMENT_INCOHERENT at /units/N/column when it does not. Neither registration nor a v4 run resolves the code. The code reaches the table, dataset page and public schema exactly as written. A code outside the accepted subset passes the build but creates work for every reader. Use a code from the subset.

Declare a unit only on a column that holds a quantity. Text, dates and identifiers do not hold quantities. Say nothing rather than guess.

The codes come from UCUM, the Unified Code for Units of Measure. Mostly Right pins version 2.2 (release 2024-06-28) and transcribes that subset here. Resolution is deterministic and offline, in exact fractions.

Forty-seven symbols. Seven are the base units and every one of them takes a prefix.

Code Unit Prefix
m metre yes
s second yes
g gram yes
rad radian yes
K kelvin yes
C coulomb yes
cd candela yes

Dimensionless quantities:

Code Unit Prefix
1 unity, a ratio no
% percent no
[ppth] parts per thousand no
[ppm] parts per million no
[ppb] parts per billion no
[pptr] parts per trillion no

Derived SI units:

Code Unit Prefix
mol mole yes
Hz hertz yes
N newton yes
Pa pascal yes
J joule yes
W watt yes
A ampere yes
V volt yes

Metric units in ordinary use:

Code Unit Prefix
L, l litre yes
t tonne yes
bar bar yes
ar are yes
m[Hg] metre of mercury column yes
m[H2O] metre of water column yes

Calendar durations, none of which takes a prefix:

Code Unit
min minute
h hour
d day
wk week
a_j mean Julian year
a year

Customary lengths, masses and volumes, none of which takes a prefix:

Code Unit Code Unit
[in_i] inch [lb_av] pound
[ft_i] foot [oz_av] ounce
[yd_i] yard [gal_us] US gallon
[mi_i] mile [qt_us] US quart
[nmi_i] nautical mile [pt_us] US pint
[foz_us] US fluid ounce

The two temperature scales that are not ratio scales, neither of which takes a prefix:

Code Unit
Cel degree Celsius
[degF] degree Fahrenheit

The twenty classic decimal prefixes, and only on an atom the table marks as taking one.

Code Name Factor Code Name Factor
Y yotta 10²⁴ d deci 10⁻¹
Z zetta 10²¹ c centi 10⁻²
E exa 10¹⁸ m milli 10⁻³
P peta 10¹⁵ u micro 10⁻⁶
T tera 10¹² n nano 10⁻⁹
G giga 10⁹ p pico 10⁻¹²
M mega 10⁶ f femto 10⁻¹⁵
k kilo 10³ a atto 10⁻¹⁸
h hecto 10² z zepto 10⁻²¹
da deka 10¹ y yocto 10⁻²⁴

km is a kilometre. kh is not a kilo-hour, because an hour takes no prefix. That rule is UCUM’s, not a local restriction.

A symbol the table holds outright never reads as something else. cd is the candela, not a centi-day. Pa is the pascal, not a peta-year. Where the whole symbol is unknown, exactly one prefix split may explain it. The parser refuses a symbol with two possible splits instead of choosing one silently.

Written Means
. multiply, as in N.m
/ divide, as in km/h
a trailing integer a power from −9 to 9, never zero, as in m3 and s-1
a leading / a reciprocal, as in /s

Division binds one term at a time, left to right. J/kg.K is joules per kilogram, multiplied by kelvin. Joules per kilogram-kelvin is J/kg/K.

A code is at most 64 characters and 16 terms, and resolves to a magnitude within 10^±120.

A {…} suffix of up to thirty-two letters, digits and underscores labels a quantity without changing its dimension.

{count} is a count and 1 is a dimensionless ratio. They share a dimension and are not the same unit, and the annotation is the only thing that says so.

An annotation with nothing in front of it stands for unity carrying that label. {count} is a term on its own, as a numerator only. The parser refuses m/{a} because a label is not a thing to divide by.

A code carries its annotations as a set rather than per term, so m{a}/s{b} and m{b}/s{a} are the same unit. Write the distinction into the column’s description when it matters.

Written Not accepted because
(m/s) a parenthesised sub-expression is outside the subset
10*3, 10^3, 4.[pi] a numeric factor is outside the subset, and the parser refuses both * and ^ by name
degrees C a space is outside the subset
furlong the symbol is not an atom in the pinned table
ha a year takes no prefix, so hecto-year is not a unit
kh an hour takes no prefix either
m99, m0 a power runs from −9 to 9 and is never zero
m{a-b} an annotation is letters, digits and underscores
m/{a} a bare annotation cannot be a denominator

Currency has no spelling here. It sits outside the unit standard’s scope.

Celsius and Fahrenheit are affine. Zero degrees Celsius is not zero of anything, so no multiplier turns a Celsius reading into a Fahrenheit one. A conversion by multiplier alone would be wrong by thirty-two degrees at every temperature.

Each resolves with an exact scale and an exact offset:

From To Scale Offset
Cel [degF] 9/5 32
[degF] Cel 5/9 −160/9
Cel K 1 5463/20

The parser refuses both inside a larger expression. Cel alone is a unit, Cel/m is not, and mCel is not. An offset unit has no meaning as a factor in a product. Every other unit in the subset is a ratio scale with an offset of zero.

The names the vocabulary began with keep working. Fourteen are aliases for the codes they always meant and resolve to the same unit. A table declared with one and a plan authored with the other agree.

Name Code Name Code
count {count} meter m
ratio 1 kilometer km
percent % mile [mi_i]
celsius Cel microgram_per_cubic_meter ug/m3
fahrenheit [degF] second s
kelvin K minute min
hectopascal hPa hour h

The fifteenth, none, is not a unit and does not become one. It says there is nothing here to convert, which is a different claim from a dimensionless 1.

Prefer the code over the alias in new work. ug/m3 composes and microgram_per_cubic_meter does not.

A code that resolves is a well-formed claim, not a verified one. Unit validation does not read the data, so it cannot catch a column declared in metres that contains feet. Two rules follow for a recipe:

  • Convert in the transform, then declare the unit of what the column ends up holding.
  • Leave a computed column out of units when the transform does not establish its unit. A density from a mass and a volume has no inferred unit, and neither does a rate from a distance and a duration. Unit validation does not combine the input units. Omitting a unit does not break downstream reads.

A dataset carries one primary category, saved before you register the recipe:

Terminal window
mr-data dataset set DATASET_ID --category climate-environment

mr-data recipe asks the server to enforce it, so a dataset with no saved category answers 422 DATASET_CATEGORY_REQUIRED. Read the vocabulary with its definitions using mr-data dataset categories --json.

ID Category Covers
climate-environment Climate & Environment weather, climate, pollution, ecosystems, environmental measurements
crypto Crypto cryptocurrencies, blockchain networks, DeFi, digital-asset markets
finance Finance banking, securities, investment, insurance, financial-market activity
economics-business Economics & Business economic indicators, businesses, commerce, industry, consumer activity
government-politics Government & Politics elections, public policy, government institutions and administration
people-society People & Society population, demographics, communities, social conditions, human behaviour
health-medicine Health & Medicine clinical medicine, healthcare, diseases, public health
science-research Science & Research physical and natural sciences, experiments, astronomy, research metadata
technology Technology software, computing, AI, internet, telecommunications
energy Energy electricity, fuels, energy production, consumption, supply infrastructure
agriculture-food Agriculture & Food farming, livestock, fisheries, food production, food systems
transport-mobility Transport & Mobility movement of people and goods by road, rail, air and sea
places-infrastructure Places & Infrastructure geography, maps, property, buildings, housing, non-energy infrastructure
education Education schools, universities, learning, students, educational outcomes
sports Sports sports competitions, athletes, teams, athletic performance
arts-media-entertainment Arts, Media & Entertainment culture, arts, news media, music, film, gaming, esports, audience activity
law-public-safety Law & Public Safety legal systems, courts, crime, policing, emergency response
other Other a meaningful subject no category covers after reading their definitions

Classify the main subject, not the source, the cadence, the file format or a secondary variable. Bitcoin price markets are Crypto. Airport weather observations are Climate & Environment. Crop yields with weather covariates are Agriculture & Food. Keep the saved category across refreshes and recipe revisions, and reconsider it only when the dataset’s principal subject changes. Never invent, combine or subdivide an ID.

Topics are separate from the category: three to eight lowercase descriptive tags, each at most 40 characters, carried by --topics. They are search keywords, not categories, and no tag decides the primary category.

A v4 recipe document has no prediction-label member, and no operation produces one. The retired local prediction_label operation is gone. Its horizon is a claim about what somebody intends to predict, which is a modelling choice rather than a property of the sources. A target belongs to the step that consumes the dataset, not to the dataset.

Build the observed values as columns and let the consumer shift them.

Term Meaning
Recipe the one JSON document that says how a table is built
Run one execution of a recipe, which writes one immutable table version
Reader versioned code that converts an accepted source format into canonical rows
Worker sandbox the confined process that parses or decodes untrusted bytes
Courier the hosted role that performs allowlisted public network retrieval
Receipt the machine-readable evidence one operation produced
Bookmark the position inside a recording a successful run committed
Unit code what a column declares it measures in, from the accepted subset
Studio the hosted service that owns scheduling, credentials, versions and release