v0.3.2
Home GitHub

Architecture

Last updated

go-browser is four modules: routing picks a browser, the browser loads the page, the page becomes snapshots, the snapshots become output; the only loop back is one headed retry when blocked.

System overview

graph TB
    subgraph Entry
        A[Fetch]
    end
    subgraph Routing
        B{Option.Headless}
        C{requiresSession + hasDisplay}
        D[headless first]
        E{isBlocked + hasDisplay}
    end
    subgraph Launcher
        F{SameSession}
        G[launchWithSnapshot<br/>temp profile + cookie decryption]
        H[ensureBrowser<br/>cache key headless + UA]
    end
    subgraph Pipeline
        I[load: Navigate + Settle]
        J[handleConsent]
        K[Scroll + Snapshots]
    end
    subgraph Output
        L[Merge / Readability]
        M[HTMLToMarkdown]
        N[HTMLToNode]
    end
    A --> B
    B -->|true| F
    B -->|false| C
    C -->|yes: headed| F
    C -->|no| D --> F
    F -->|yes| G
    F -->|no| H
    G -->|ErrProfileNotFound| H
    G --> I
    H --> I --> J --> K --> L
    L --> M
    M -->|TypeJSON| N
    D -.->|result 403 / 429 / 503| E
    E -->|yes: one headed retry| F

Layers

Layer File Responsibility Exported
Entry and routing core/fetch.go Mode selection, blocked detection, retry decision, load pipeline Fetch, Option, Viewport, Result, Error, TypeMarkdown, TypeHTML, TypeJSON
Browser lifecycle core/launcher.go Instance cache, idle eviction, concurrency gate, temp-profile launch Close, SetMaxConcurrency, Browser, DefaultUserAgent, ErrProfileNotFound
Cookie decryption core/cookie.go (macOS / Linux), core/cookieOther.go (stub that returns an error; unreachable in practice because SameSession falls back to a regular browser on other platforms before reaching it) Keychain password, PBKDF2, AES-CBC decryption, CDP cookie assembly none (internal)
Content processing core/markdown.go Snapshot merge, time inlining, Markdown and tree output, deduplication Merge, HTMLToMarkdown, HTMLToNode, DedupTree, DedupMarkdownParagraphs, InlineTimeElements, Node

The three injected scripts are embedded from core/embed/ with go:embed: stealth.js for anti-detection, listener.js for load settling, consent.js for consent banners.

Cross-cutting rules

Rule Implementation
Context flows through The ctx given to Fetch reaches settling, consent evaluation and scroll waits; timeout wraps it in a child context
Concurrency gated by a semaphore acquireSem must grant a slot before a page is created, and load releases it on return
Resource release is paired launchWithSnapshot returns a cleanup that closes the browser and removes the temp directory; cached instances are reclaimed by the evictor or Close
Degradation does not abort A missing Chrome profile returns ErrProfileNotFound, and fetchWith continues on a regular browser
Failures stay distinguishable Site blocks return *Error{Status}, extraction failures return a plain error, and routing treats them differently

Further reading

The module-level expansion — per-module diagrams, sequences and state machines — lives in doc/architecture.md in the repository. For the detail of any single flow, see Core Concepts.

中文