Skip to content

perf: let metadata and section-pages start on their dependency's first chunk - #1240

Merged
avivkeller merged 6 commits into
mainfrom
perf/stream-metadata
Nov 18, 2026
Merged

avivkeller merged 6 commits into
mainfrom
perf/stream-metadata

Conversation

@ovflowd

@ovflowd ovflowd commented Nov 3, 2026 •

Copy link
Copy Markdown
Member

Description

This PR lets metadata and section-pages start on their dependency's first chunk instead of waiting for all of it, so with more than one thread parsing, metadata and section pages overlap. It's the fourth part of a stack that splits #1205.

  • section-pages passes each batch of entries on as it arrives and builds that batch's chunk pages, which it adds after the last batch, in module order, as if built from the whole input at once.
  • Each of metadata's chunks holds whole modules, so section-pages reads its input as it comes, with no regrouping. creating-generators.md now states that guarantee.
  • A generator that sets streamsInput now always gets an async iterable. With one thread its input is still collected first, but arrives as its only chunk instead of as the array, so the generator reads it with one for await either way.

Why it's built this way

Why does this barely move the site build? The overlap here is between ast, metadata and section-pages, and stops there. -t section-pages is delivered through html and sitemap, so the site build also runs jsx-ast, html and sitemap. But jsx-ast, which reads section-pages, still takes its input whole, and so do json, orama-db and llms-txt, which read metadata. jsx-ast and html stream in #1241, which is where the site build gets its -17% (4 threads) and -20% (6 threads) against main. So this layer is mostly a precondition. On its own it saves 0.14 to 0.36s. In the site build that's within run-to-run spread. In the legacy build it isn't: at both thread counts, every run on this branch beat every run on #1239. So overlapping ast and metadata does save time.

The in-flight limit doesn't stop this. threads * CHUNKS_PER_THREAD is counted per worker.stream call, so at 4 threads ast and metadata can each have up to 12 chunks in the pool at once. A benchmark like #1253 can't see this PR. With one thread, core hands every generator its input whole and worker.stream runs each chunk on the main thread, so nothing overlaps. It also builds no section pages.

Why no regrouping of metadata's chunks? The first version had groupWholeModules, because I assumed a chunk could end partway through a module. It can't. parseApiDoc sets api once per file, from the file's path, and worker.stream splits a batch by item, so each chunk holds whole files, which means whole modules. Aviv pointed this out. Dropping it removed the helper (61 lines) and its 92 lines of tests, and made no measurable difference to #1241 at 6 threads over 7 runs. The guarantee is documented because jsx-ast relies on it in #1241 too.

Why hold the chunk pages until the end? Only to keep the output identical. Each chunk page's first entry is promoted to depth 1, and sitemap lists depth-1 entries in input order. Yielding each batch's chunk pages right away would mix them in with the module pages and reorder sitemap.xml. It costs nothing next to main, which also built every chunk page after the whole input and returned everything at the end. From #1241 on, html puts its pages in site order itself. So if we want the chunk pages to go out earlier, the cheaper fix is a sort in sitemap. I'd rather do that in its own PR.

Why always an async iterable, and still collected with one thread? Aviv's idea: until now a streaming generator got the collected array with one thread and an iterable with more, so each one went through a toBatches helper to accept both. Core now wraps the collected array as a single chunk, so toBatches and the array-or-iterable types go, here and in jsx-ast, html and html-all later in the stack. With one thread the input is still collected first, as before: streaming there would only interleave the stages and keep each stage's data alive at the same time, and -p 1 is the build that has to fit in 512MB. streamsInput isn't released yet (it's in #1238's pending changeset), so nobody outside depends on the array. On the full Node.js docs at -p 1, the whole stack with and without this change builds the site in 21.99s and 21.98s (medians of 5), with peaks of 1945MB and 1927MB, within the spread of the runs.

Benchmarks

Full Node.js docs on Node 26. Each number is the median of 5 interleaved runs (4 for #1239 at 6 threads, and for main's legacy build at 6 threads). Peak is RSS summed over the process tree. Site is -t orama-db -t json -t llms-txt -t section-pages, and legacy is -t legacy-html-all -t legacy-json-all.

Build Threads #1239 This PR vs #1239 vs main
site 4 9.37s, 1686MB 9.02s, 1707MB -4% time, +1% mem -3%, -2%
site 6 8.27s, 2306MB 8.08s, 2240MB -2%, -3% -2%, -5%
legacy 4 4.37s, 1531MB 4.23s, 1533MB -3%, +0% -4%, +0%
legacy 6 4.14s, 1980MB 3.96s, 1928MB -4%, -3% -6%, -1%

Memory is flat. The site build's -3% at 6 threads is within #1239's spread there (2189 to 2399MB over 4 runs).

Validation

Added tests for once, for runGenerators giving a streaming generator its whole input as one chunk with one thread, and for section-pages building the same output from several batches as from its whole input, as it gets it with one thread, and passing each batch on before the next one arrives. Built on 12 of Node.js's API docs at 1 and 4 threads, the output is byte-identical to main's.

Related Issues

Refs: #1205

Check List

  • I have read the Contributing Guidelines and made commit messages that follow the guideline.
  • I have run node --run test and all tests passed.
  • I have check code formatting with node --run format:check & node --run lint.
  • I've covered new added functionality with unit tests if necessary.

@vercel

vercel Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
api-docs-tooling Ready Ready Preview Nov 17, 2026 2:44pm UTC

Request Review

@codecov

codecov Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.79%. Comparing base (5db2f94) to head (7649f2d).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1240      +/-   ##
==========================================
+ Coverage   93.76%   93.79%   +0.02%     
==========================================
  Files         294      294              
  Lines       30330    30476     +146     
  Branches     3089     3100      +11     
==========================================
+ Hits        28438    28584     +146     
  Misses       1870     1870              
  Partials       22       22              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

🚀 Deploying Preview to Cloudflare 🚀

Preview Deployments by commit

Status Deployment URL Commit Updated (UTC) See this deployment's details
  • Build: Success ✅

View logs ↗
No Preview URL. Enable ↗ 7649f2d 2026-11-17T14:46:10.254Z View logs ↗
  • Build: Success ✅

View logs ↗
No Preview URL. Enable ↗ 54a4f5a 2026-11-13T16:49:38.537Z View logs ↗
  • Build: Success ✅

View logs ↗
No Preview URL. Enable ↗ d026fd1 2026-11-11T00:07:47.548Z View logs ↗
  • Build: Success ✅

View logs ↗
No Preview URL. Enable ↗ cd9e078 2026-11-10T20:54:00.122Z View logs ↗
  • Build: Success ✅

View logs ↗
No Preview URL. Enable ↗ 0b3700d 2026-11-09T11:04:25.825Z View logs ↗
  • Build: Success ✅

View logs ↗
No Preview URL. Enable ↗ be718cd 2026-11-09T10:38:04.653Z View logs ↗
  • Build: Success ✅

View logs ↗
No Preview URL. Enable ↗ cfe9347 2026-11-09T02:27:29.983Z View logs ↗
  • Build: Success ✅

View logs ↗
No Preview URL. Enable ↗ 3611a22 2026-11-06T19:44:32.673Z View logs ↗
  • Build: Success ✅

View logs ↗
No Preview URL. Enable ↗ b6f721b 2026-11-06T19:32:06.185Z View logs ↗
  • Build: Success ✅

View logs ↗
No Preview URL. Enable ↗ 084e275 2026-11-06T13:37:37.920Z View logs ↗

View all previews: View all previews ↗

kgeminicdev

This comment was marked as low quality.

Base automatically changed from perf/drain-results to main November 13, 2026 16:48
@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

api-links Generator

Performance estimate (single CI run)

  • Generation time: 35.7% faster (980.00 ms → 630.00 ms)
  • Peak memory: 4.3% higher (272.87 MB → 284.55 MB)

json Generator

Performance estimate (single CI run)

  • Generation time: 36.2% slower (6.80 s → 9.26 s)
  • Peak memory: 2.1% higher (910.91 MB → 929.93 MB)

legacy-html Generator

Performance estimate (single CI run)

  • Generation time: 20.0% slower (12.13 s → 14.56 s)
  • Peak memory: 1.3% higher (1.37 GB → 1.39 GB)

legacy-json Generator

Performance estimate (single CI run)

  • Generation time: 16.8% slower (5.18 s → 6.05 s)
  • Peak memory: 2.9% lower (980.96 MB → 952.89 MB)

llms-txt Generator

Performance estimate (single CI run)

  • Generation time: 0.2% faster (5.58 s → 5.57 s)
  • Peak memory: 2.7% lower (944.72 MB → 919.45 MB)

orama-db Generator

Performance estimate (single CI run)

  • Generation time: 58.5% slower (4.99 s → 7.91 s)
  • Peak memory: 8.2% lower (1.03 GB → 971.38 MB)

web Generator

Performance estimate (single CI run)

  • Generation time: 15.6% slower (15.07 s → 17.42 s)
  • Peak memory: 2.7% lower (1.51 GB → 1.47 GB)

ovflowd and others added 6 commits November 17, 2026 15:04
…t chunk

With more than one thread, metadata and section-pages now start on the
first chunk of their dependency instead of waiting for all of it, so
parsing, metadata and section pages overlap. The generated files are
unchanged.

section-pages passes each module's entries on as they arrive and adds
the chunk pages after the last of them, in module order, as if built
from the whole input at once. A module's chunk pages need all of its
entries, and a chunk need not hold whole modules, so groupWholeModules
regroups streamed entries into batches of whole modules, and rejects a
module whose entries come back after another's.

Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
Assisted-by: Claude Sonnet 5.5 <noreply@anthropic.com>
…Modules

worker.stream no longer reads its input with toBatches, so the helper
now comes with groupWholeModules, its first caller.

Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
…ules

From the review. A module is one file: parseApiDoc sets every entry's
api once per file, and worker.stream never splits an item, so each of
metadata's chunks already holds whole modules. section-pages reads its
input as it comes with toBatches instead of regrouping it, and the
helper and its tests go.

Co-authored-by: Aviv Keller <me@aviv.sh>
Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
The two tests comparing section-pages' batched output with its whole
input become one: both compared against the same output. core's
collectAsyncGenerator replaces a local copy of it, and html's test gives
section-pages its entries as an array, which it reads as one batch, so
it needs no stream of its own. toBatches' test no longer defines a
generator body it never runs. The comment on section-pages' chunk pages
says batches, as chunk already means a page there.

Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
Every streaming generator reads its input through toBatches, which
takes both the stream and the collected array one thread gives, but the
docs for writing one only said to accept both.

Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
…rable

From the review, which asked for it. With one thread the input is still
collected first, as interleaving the stages there would only keep
every stage's data alive at once, but it now arrives as its only chunk
instead of as the array. A generator that sets streamsInput reads its
input with one for await either way, so toBatches goes, and the types
take only the async iterable.

core's once, which the cache already used to stream a result that
doesn't, moves to utils/generators.mjs next to the other generator
helpers. The test of one thread reads a source of two chunks, so it
shows they arrive as one.

Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
@avivkeller
avivkeller merged commit ad826e4 into main Nov 18, 2026
25 of 26 checks passed
@avivkeller
avivkeller deleted the perf/stream-metadata branch November 18, 2026 03:37
@github-actions github-actions Bot mentioned this pull request Nov 18, 2026

This branch was successfully deployed

1 active deployment
Preview – api-docs-tooling — 7649f2d5 Deployed Nov 17, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants