Loading dataset

Description

See which web technologies are gaining or losing websites each month, and which ones large US listed companies put on their home pages. Monthly counts for 3,565 technologies from the HTTP Archive crawl sit beside a yearly record of scripts found on the archived home pages of 228 US listed companies. Compare vendors inside a category, follow a product's share over time, or check when a company added or removed a tool.

The company part is small and partial. It reads one archived home page every second year and only sees technologies loaded through a script tag with an address, so tools installed by an inline snippet, such as most Google Analytics and Google Tag Manager setups, are often missed.

Coverage

  • Window: January 2020 to October 2026 for monthly adoption. Rank buckets start in March 2021. Company captures are the first of 2016, 2018, 2020, 2022, 2024 and 2026.
  • Grain: technology_adoption_monthly has one row per technology, month, rank bucket and geography (1,103,966 rows). category_market_share_monthly has one row per category, technology and month (259,204 rows). technologies has one row per technology (3,565 rows). company_website_detections has one row per company home page capture and technology (2,879 rows). company_technology_timeline has one row per company website and technology (1,299 rows).
  • Cadence: the three HTTP Archive tables rebuild weekly and gain one month after each monthly crawl. The two company tables are a fixed snapshot.
  • Geography: all sites worldwide, plus the United States alone. Companies are US listed with a market value of at least 50 billion dollars.

Columns

  • technology: the product name as the HTTP Archive spells it. It joins all five tables.
  • month, rank, geo: crawl month, site popularity bucket (ALL, Top 1k to Top 10M) and geography.
  • origins_mobile, origins_desktop: websites where the technology was detected.
  • share_mobile_pct, share_of_category_mobile_pct: percent of all crawled sites, and percent of detections in the category.
  • first_seen_month, last_seen_month, lifecycle_event: when a technology enters or leaves the crawl.
  • business_group: one of twelve groups, set by rule from the source category. For the 81 technologies filed under Miscellaneous a small classification model (gemini-decision-1) sets the group when its probability is at least 0.5, which settled 37.
  • is_ai_product, ai_product_probability: the same model's label for whether the product is itself an AI product. A label with a probability, not a fact.
  • vendor, vendor_ticker: the company behind the product, from a hand-written list of product name prefixes, with the ticker confirmed against the US stocks tables.
  • symbol, domain, capture_date: the company, its website and the date of the archived page.
  • first_seen_date, last_seen_date, status, adopted_after, dropped_before: when a technology appeared on and left a company home page, bounded by the captures on each side.

Missing values

An empty vendor means the hand-written list has no entry, not that the product has no owner. An empty business_group or primary_category in the company tables marks a technology the HTTP Archive does not track. A technology with no row in a month was not detected that month. About one archived page in nine could not be read and has no rows.

Suitable for

  • Market share and trend charts for CMS, analytics, payment, CDN, chat and advertising products
  • Tracking a listed vendor's footprint on the web by rank bucket
  • Case studies of when large companies adopted consent managers or new frameworks

Source and rights

Adoption counts come from the HTTP Archive Tech Report API, published under CC BY 4.0, with detections by its Wappalyzer based crawler. Company pages come from the Internet Archive Wayback Machine. Detection rules for company pages are hand-written for this dataset and use no third-party fingerprint list.

Tables5

  • Table healthStatus unavailable
    category_market_share_monthly
    Table overviewPublished rows259,204Columns16 rows1 Jan 2020 to 1 Oct 2026 · latest event 43 d before refresh · 50 anomalies
    Status unavailable
  • Table healthStatus unavailable
    company_technology_timeline
    Table overviewPublished rows1,299Columns23 rows1 Jan 2016 to 27 Mar 2026 · latest event 231 d before refresh · 9 anomalies
    Status unavailable
  • Table healthStatus unavailable
    company_website_detections
    Table overviewPublished rows2,879Columns14 rows1 Jan 2016 to 27 Mar 2026 · latest event 231 d before refresh · 26 anomalies
    Status unavailable
  • Table healthStatus unavailable
    technologies
    Table overviewPublished rows3,565Columns19 rows
    Status unavailable
  • Table healthStatus unavailable
    technology_adoption_monthly
    Table overviewPublished rows1,103,966Columns19 rows1 Jan 2020 to 1 Oct 2026 · latest event 43 d before refresh · 50 anomalies
    Status unavailable

Sources

TablePermission asserted.2 endpoints
Usage rights
Permission asserted.
Collection
2 requests across 2 endpoints

Collected endpoints

The company_website_detections table of this dataset.det1 request
The technologies table of this dataset.tech1 request

Details

Contents
5 tables · 1,370,913 rows · 91 columns
Category
Technology
Updated
Published
Version
2026-11-12
License
Not stated
Visibility
Public
Publisher
Mostly Right

Activity

Views245+245 in the last 30 days
Likes0+0 in the last 30 days
Uses6+6 in the last 30 days

Support

Talk to Mostly Right

Loading chat…