Page Loading

Last updated 2026-10-06

When a page counts as loaded, how the scroll loop captures dynamically loaded content, and when it stops.

Settling

Before anything is extracted, the page settles in two steps:

WaitDOMStable with a window of min(400ms, IdleWait) and a 0.01 change threshold, capped at IdleWait overall
SettleJS, capped at IdleWait; the default listener.js resolves after one requestIdleCallback (250ms timeout) or after 1 second, whichever comes first

Only step 1 runs again: after each consent pass that dismisses something, and after each scroll. SettleJS runs once per load. Timeouts here are not errors: extraction continues with whatever the DOM holds.

Scroll loop

After consent handling, the first snapshot is the full page HTML. Then, up to ScrollCount times:

Step	Detail
Check	Stop when the document is no longer taller than the viewport plus 4px
Wait	150ms plus 0–300ms of random jitter; stops if the context ends
Scroll	Smooth scroll to the bottom over 300ms with `requestAnimationFrame`
Settle	DOM stability wait only (step 1); `SettleJS` is not re-run
Snapshot	Stop when the new HTML is byte-identical to the previous snapshot

ScrollCount defaults to 3; a negative value disables scrolling entirely, leaving only the first snapshot.

Why multiple snapshots

Infinite-scroll feeds and lazy-loaded articles replace or append content as you scroll. Each snapshot captures one state, and the output stage combines them: TypeHTML merges their <body> children, while TypeMarkdown and TypeJSON run readability on each snapshot and join the article contents. Repeated paragraphs introduced by overlapping snapshots are removed afterwards. See Output Formats.

Viewport

The page uses Option.Viewport, 1280 × 960 by default, with DeviceScaleFactor 0 treated as 1. A taller viewport loads more content per snapshot.