Skip to content
rewget GitHub

Home / Features

Everything rewget does

rewget keeps the wget surface unchanged and adds exactly the machinery you need when a site starts blocking a plain HTTP client — no more, no less. Here is the whole feature set, grouped by what it is for.

Drop-in compatibility

rewget is wget where wget works. Nothing to learn, nothing to rewrite.

Same command-line surface

rewget forwards every wget flag unchanged and only intercepts its own --rewget-* options. Alias wget=rewget and your existing scripts, Makefiles, and CI keep running.

Zero overhead on the happy path

When a site does not block, stage 1 is plain wget — same output, same speed, no daemon spin-up and no browser cost.

wget2 support

Point rewget at wget2 if you want its HTTP/2 and parallel features underneath the fallback machinery.

The three-stage fallback

Plain wget → browser-impersonating HTTP → headless Chromium. The first stage that works wins.

Stage 1 — plain wget

Runs wget with your exact flags. If the response is not a configured fallback code, you are done. Configure which codes escalate with --rewget-fallback-codes.

Stage 2 — browser TLS via rquest

On a block, rewgetd retries with a real Chrome, Firefox, Safari, or Edge TLS and HTTP/2 fingerprint through rquest. Most fingerprint-based blocks fold here without launching a browser.

Stage 3 — headless Chromium

For pages that genuinely require JavaScript, chromiumoxide drives real Chromium, solves the challenge, exports cookies and headers, and the download resumes through the passed session.

Caching

Learn once, stay fast. The slow path runs a single time per host.

Per-domain result cache

Once rewget learns which stage works for a host, subsequent downloads skip straight there instead of re-probing from stage 1.

7-day TTL

Domain-level results are cached for seven days, so repeated mirror jobs against the same host stay fast for a week before re-checking.

Security & profiles

Fingerprint profiles are signed, verified, and updated on your terms.

Ed25519-signed profiles

Browser fingerprint profiles are cryptographically signed and verified on update — no silent network fetches of unverified fingerprints.

Explicit profile updates

New Chrome and Firefox releases roll out through --rewget-update-profiles when you choose to run it. List what is installed with --rewget-list-profiles.

No access-control bypass

rewget handles fingerprint blocks and JavaScript challenges — the cases a real browser would pass. It does not defeat authentication or logins.

Architecture

A thin CLI shim over a daemon that owns the expensive resources.

CLI-first, daemon-backed

The rewget CLI stays a fast, cheap-to-start shim. The rewgetd daemon owns the browser pool and TLS sessions and starts on demand.

Written in Rust

A memory-safe, single-purpose implementation. Dual-licensed MIT / Apache-2.0.

CI-friendly controls

Use --rewget-no-fallback to fail fast on a block so CI flags a regression instead of silently escalating.

See it in your own pipeline

Install rewget, alias wget=rewget, and the fallback is there the next time a site says no.