Units and vocabulary
Describe each measured column with a unit from the supported UCUM vocabulary.
Where a unit is declared
Section titled “Where a unit is declared”In the recipe document’s units array, one entry per column that has one:
"units": [ { "column": "air_temp_c", "unit": "Cel" }, { "column": "relative_humidity_pct", "unit": "%" }, { "column": "wind_speed", "unit": "km/h" }, { "column": "pm25", "unit": "ug/m3" }, { "column": "station_count", "unit": "{count}" }]An array rather than a map, so declaration order belongs to the digested document and no column appears twice under two spellings of one key. At most 512 entries, and each unit is 1 to 64 characters.
The recipe schema only bounds the length of unit. Registration checks that units[].column names a declared column and returns 422 RECIPE_DOCUMENT_INCOHERENT at /units/N/column when it does not. Neither registration nor a v4 run resolves the code. The code reaches the table, dataset page and public schema exactly as written. A code outside the accepted subset passes the build but creates work for every reader. Use a code from the subset.
Declare a unit only on a column that holds a quantity. Text, dates and identifiers do not hold quantities. Say nothing rather than guess.
The accepted subset
Section titled “The accepted subset”The codes come from UCUM, the Unified Code for Units of Measure. Mostly Right pins version 2.2 (release 2024-06-28) and transcribes that subset here. Resolution is deterministic and offline, in exact fractions.
Forty-seven symbols. Seven are the base units and every one of them takes a prefix.
| Code | Unit | Prefix |
|---|---|---|
m |
metre | yes |
s |
second | yes |
g |
gram | yes |
rad |
radian | yes |
K |
kelvin | yes |
C |
coulomb | yes |
cd |
candela | yes |
Dimensionless quantities:
| Code | Unit | Prefix |
|---|---|---|
1 |
unity, a ratio | no |
% |
percent | no |
[ppth] |
parts per thousand | no |
[ppm] |
parts per million | no |
[ppb] |
parts per billion | no |
[pptr] |
parts per trillion | no |
Derived SI units:
| Code | Unit | Prefix |
|---|---|---|
mol |
mole | yes |
Hz |
hertz | yes |
N |
newton | yes |
Pa |
pascal | yes |
J |
joule | yes |
W |
watt | yes |
A |
ampere | yes |
V |
volt | yes |
Metric units in ordinary use:
| Code | Unit | Prefix |
|---|---|---|
L, l |
litre | yes |
t |
tonne | yes |
bar |
bar | yes |
ar |
are | yes |
m[Hg] |
metre of mercury column | yes |
m[H2O] |
metre of water column | yes |
Calendar durations, none of which takes a prefix:
| Code | Unit |
|---|---|
min |
minute |
h |
hour |
d |
day |
wk |
week |
a_j |
mean Julian year |
a |
year |
Customary lengths, masses and volumes, none of which takes a prefix:
| Code | Unit | Code | Unit | |
|---|---|---|---|---|
[in_i] |
inch | [lb_av] |
pound | |
[ft_i] |
foot | [oz_av] |
ounce | |
[yd_i] |
yard | [gal_us] |
US gallon | |
[mi_i] |
mile | [qt_us] |
US quart | |
[nmi_i] |
nautical mile | [pt_us] |
US pint | |
[foz_us] |
US fluid ounce |
The two temperature scales that are not ratio scales, neither of which takes a prefix:
| Code | Unit |
|---|---|
Cel |
degree Celsius |
[degF] |
degree Fahrenheit |
Prefixes
Section titled “Prefixes”The twenty classic decimal prefixes, and only on an atom the table marks as taking one.
| Code | Name | Factor | Code | Name | Factor | |
|---|---|---|---|---|---|---|
Y |
yotta | 10²⁴ | d |
deci | 10⁻¹ | |
Z |
zetta | 10²¹ | c |
centi | 10⁻² | |
E |
exa | 10¹⁸ | m |
milli | 10⁻³ | |
P |
peta | 10¹⁵ | u |
micro | 10⁻⁶ | |
T |
tera | 10¹² | n |
nano | 10⁻⁹ | |
G |
giga | 10⁹ | p |
pico | 10⁻¹² | |
M |
mega | 10⁶ | f |
femto | 10⁻¹⁵ | |
k |
kilo | 10³ | a |
atto | 10⁻¹⁸ | |
h |
hecto | 10² | z |
zepto | 10⁻²¹ | |
da |
deka | 10¹ | y |
yocto | 10⁻²⁴ |
km is a kilometre. kh is not a kilo-hour, because an hour takes no prefix. That rule is UCUM’s, not a local restriction.
A symbol the table holds outright never reads as something else. cd is the candela, not a centi-day. Pa is the pascal, not a peta-year. Where the whole symbol is unknown, exactly one prefix split may explain it. The parser refuses a symbol with two possible splits instead of choosing one silently.
Composition
Section titled “Composition”| Written | Means |
|---|---|
. |
multiply, as in N.m |
/ |
divide, as in km/h |
| a trailing integer | a power from −9 to 9, never zero, as in m3 and s-1 |
a leading / |
a reciprocal, as in /s |
Division binds one term at a time, left to right. J/kg.K is joules per kilogram, multiplied by kelvin. Joules per kilogram-kelvin is J/kg/K.
A code is at most 64 characters and 16 terms, and resolves to a magnitude within 10^±120.
Annotations
Section titled “Annotations”A {…} suffix of up to thirty-two letters, digits and underscores labels a quantity without changing its dimension.
{count} is a count and 1 is a dimensionless ratio. They share a dimension and are not the same unit, and the annotation is the only thing that says so.
An annotation with nothing in front of it stands for unity carrying that label. {count} is a term on its own, as a numerator only. The parser refuses m/{a} because a label is not a thing to divide by.
A code carries its annotations as a set rather than per term, so m{a}/s{b} and m{b}/s{a} are the same unit. Write the distinction into the column’s description when it matters.
Outside the subset
Section titled “Outside the subset”| Written | Not accepted because |
|---|---|
(m/s) |
a parenthesised sub-expression is outside the subset |
10*3, 10^3, 4.[pi] |
a numeric factor is outside the subset, and the parser refuses both * and ^ by name |
degrees C |
a space is outside the subset |
furlong |
the symbol is not an atom in the pinned table |
ha |
a year takes no prefix, so hecto-year is not a unit |
kh |
an hour takes no prefix either |
m99, m0 |
a power runs from −9 to 9 and is never zero |
m{a-b} |
an annotation is letters, digits and underscores |
m/{a} |
a bare annotation cannot be a denominator |
Currency has no spelling here. It sits outside the unit standard’s scope.
Temperature is not a ratio
Section titled “Temperature is not a ratio”Celsius and Fahrenheit are affine. Zero degrees Celsius is not zero of anything, so no multiplier turns a Celsius reading into a Fahrenheit one. A conversion by multiplier alone would be wrong by thirty-two degrees at every temperature.
Each resolves with an exact scale and an exact offset:
| From | To | Scale | Offset |
|---|---|---|---|
Cel |
[degF] |
9/5 | 32 |
[degF] |
Cel |
5/9 | −160/9 |
Cel |
K |
1 | 5463/20 |
The parser refuses both inside a larger expression. Cel alone is a unit, Cel/m is not, and mCel is not. An offset unit has no meaning as a factor in a product. Every other unit in the subset is a ratio scale with an offset of zero.
The fifteen names
Section titled “The fifteen names”The names the vocabulary began with keep working. Fourteen are aliases for the codes they always meant and resolve to the same unit. A table declared with one and a plan authored with the other agree.
| Name | Code | Name | Code | |
|---|---|---|---|---|
count |
{count} |
meter |
m |
|
ratio |
1 |
kilometer |
km |
|
percent |
% |
mile |
[mi_i] |
|
celsius |
Cel |
microgram_per_cubic_meter |
ug/m3 |
|
fahrenheit |
[degF] |
second |
s |
|
kelvin |
K |
minute |
min |
|
hectopascal |
hPa |
hour |
h |
The fifteenth, none, is not a unit and does not become one. It says there is nothing here to convert, which is a different claim from a dimensionless 1.
Prefer the code over the alias in new work. ug/m3 composes and microgram_per_cubic_meter does not.
What a declared unit does not prove
Section titled “What a declared unit does not prove”A code that resolves is a well-formed claim, not a verified one. Unit validation does not read the data, so it cannot catch a column declared in metres that contains feet. Two rules follow for a recipe:
- Convert in the transform, then declare the unit of what the column ends up holding.
- Leave a computed column out of
unitswhen the transform does not establish its unit. A density from a mass and a volume has no inferred unit, and neither does a rate from a distance and a duration. Unit validation does not combine the input units. Omitting a unit does not break downstream reads.
Dataset categories
Section titled “Dataset categories”A dataset carries one primary category, saved before you register the recipe:
mr-data dataset set DATASET_ID --category climate-environmentmr-data recipe asks the server to enforce it, so a dataset with no saved category answers 422 DATASET_CATEGORY_REQUIRED. Read the vocabulary with its definitions using mr-data dataset categories --json.
| ID | Category | Covers |
|---|---|---|
climate-environment |
Climate & Environment | weather, climate, pollution, ecosystems, environmental measurements |
crypto |
Crypto | cryptocurrencies, blockchain networks, DeFi, digital-asset markets |
finance |
Finance | banking, securities, investment, insurance, financial-market activity |
economics-business |
Economics & Business | economic indicators, businesses, commerce, industry, consumer activity |
government-politics |
Government & Politics | elections, public policy, government institutions and administration |
people-society |
People & Society | population, demographics, communities, social conditions, human behaviour |
health-medicine |
Health & Medicine | clinical medicine, healthcare, diseases, public health |
science-research |
Science & Research | physical and natural sciences, experiments, astronomy, research metadata |
technology |
Technology | software, computing, AI, internet, telecommunications |
energy |
Energy | electricity, fuels, energy production, consumption, supply infrastructure |
agriculture-food |
Agriculture & Food | farming, livestock, fisheries, food production, food systems |
transport-mobility |
Transport & Mobility | movement of people and goods by road, rail, air and sea |
places-infrastructure |
Places & Infrastructure | geography, maps, property, buildings, housing, non-energy infrastructure |
education |
Education | schools, universities, learning, students, educational outcomes |
sports |
Sports | sports competitions, athletes, teams, athletic performance |
arts-media-entertainment |
Arts, Media & Entertainment | culture, arts, news media, music, film, gaming, esports, audience activity |
law-public-safety |
Law & Public Safety | legal systems, courts, crime, policing, emergency response |
other |
Other | a meaningful subject no category covers after reading their definitions |
Classify the main subject, not the source, the cadence, the file format or a secondary variable. Bitcoin price markets are Crypto. Airport weather observations are Climate & Environment. Crop yields with weather covariates are Agriculture & Food. Keep the saved category across refreshes and recipe revisions, and reconsider it only when the dataset’s principal subject changes. Never invent, combine or subdivide an ID.
Topics are separate from the category: three to eight lowercase descriptive tags, each at most 40 characters, carried by --topics. They are search keywords, not categories, and no tag decides the primary category.
Prediction labels
Section titled “Prediction labels”A v4 recipe document has no prediction-label member, and no operation produces one. The retired local prediction_label operation is gone. Its horizon is a claim about what somebody intends to predict, which is a modelling choice rather than a property of the sources. A target belongs to the step that consumes the dataset, not to the dataset.
Build the observed values as columns and let the consumer shift them.
Words this documentation uses
Section titled “Words this documentation uses”| Term | Meaning |
|---|---|
| Recipe | the one JSON document that says how a table is built |
| Run | one execution of a recipe, which writes one immutable table version |
| Reader | versioned code that converts an accepted source format into canonical rows |
| Worker sandbox | the confined process that parses or decodes untrusted bytes |
| Courier | the hosted role that performs allowlisted public network retrieval |
| Receipt | the machine-readable evidence one operation produced |
| Bookmark | the position inside a recording a successful run committed |
| Unit code | what a column declares it measures in, from the accepted subset |
| Studio | the hosted service that owns scheduling, credentials, versions and release |