Architecture

System architecture of go-browser — launcher, fetch pipeline, interactive tabs, cookie extraction, and content processing modules.

System Overview

graph TB
    A[Fetch / CreateTab] --> B[Launcher]
    B --> C{Headless?}
    C -->|Yes| D[Stealth JS]
    C -->|No| E[Cookie Session]
    D --> F[Page Navigate + Scroll]
    E --> F
    F --> G[Snapshot Merge]
    G --> H[Readability + Dedup]
    H --> I[Markdown / HTML / JSON]

Module Layers

Launcher

Manages two browser modes:

• Singleton headless — A single headless browser instance reused across requests, with a 5-minute idle eviction goroutine. Used when Headless: true or when no display is available.
• Profile-snapshot — A temporary browser launched with cookies extracted from the local Chrome profile. Used when SameSession: true or when a headed fetch fails and falls back.

The launcher sets Chrome flags including disable-blink-features=AutomationControlled, no-sandbox, disable-dev-shm-usage, and a custom user-agent. On macOS it locates Chrome at /Applications/Google Chrome.app; on Linux it searches PATH.

Fetch Pipeline

1. URL parsing — Validates scheme and hostname via parseHref.
2. Mode selection — Determines headless vs. session mode based on Option.Headless, Option.SameSession, and domain-based session requirements.
3. Page creation — Creates a page with viewport settings and injects stealth.js via EvalOnNewDocument.
4. Navigation — Navigates to the URL, waits for load, checks final URL for 404/403 patterns.
5. DOM stabilization — Waits for DOM stability and runs settle.js for lazy-loaded content.
6. Scroll loop — Captures initial snapshot, then scrolls smoothly with random delays, capturing snapshots after each scroll.
7. Content processing — Depending on Option.Type: merges HTML snapshots, runs readability extraction, converts to Markdown/HTML/JSON, and deduplicates.

Interactive Tabs

Stateful tab management for multi-step browser interactions:

• CreateTab — Launches or reuses an interactive browser, creates a page, navigates, and returns a tab ID.
• TabClick — Clicks an element via document.querySelector(s)?.click().
• TabType — Sets input value via prototype property descriptor and dispatches input/change events.
• TabScroll — Scrolls to bottom with random delays.
• TabEval — Executes custom JavaScript and returns the result.
• TabSnapshot — Captures the current page as Markdown/HTML/JSON.
• CloseTab — Closes the page and releases the semaphore; if no tabs remain, closes the interactive browser.

Cookie Extraction

Cross-platform Chrome cookie decryption (core/cookie.go, darwin+linux only):

1. Retrieves the Chrome Safe Storage password via OS keychain (security on macOS, secret-tool on Linux).
2. Derives an AES key using PBKDF2-SHA1 with 1003 iterations.
3. Queries the Chrome profile's SQLite Cookies database for host_key, name, value, encrypted_value, path, expires_utc, is_secure, is_httponly, samesite.
4. Decrypts v10-prefixed encrypted values using AES-CBC with a space-filled IV.
5. Returns NetworkCookieParam objects for injection into the temporary browser.

Content Processing

• Merge — Combines multiple HTML snapshots by appending body children from subsequent snapshots to the base document body.
• InlineTimeElements — Replaces <time> elements with their datetime attribute and inner text.
• HTMLToMarkdown — Walks the HTML tree and maps tags to Markdown syntax (headings, lists, links, code blocks, etc.).
• HTMLToNode — Converts HTML to a structured []*Node tree for JSON output.
• DedupMarkdownParagraphs — Removes duplicate paragraphs by hash.
• DedupTree — Removes duplicate nodes from the JSON tree by FNV-64a hash.