Skip to content
Deepak Gupta markRecon Atlas

URL and endpoint discovery

getallurls (gau)

Pulls known historical URLs for a domain from the Wayback Machine, Common Crawl, AlienVault OTX and URLScan.

PassiveScope a pentestAudit your own exposureResearch a competitor

5.2k stars · last push October 12, 2026 · v2.2.4 · MIT

GitHub repository

What it finds

gau queries four third-party archives and threat-intelligence sources for URLs recorded under a target domain. It never needs to contact the domain itself. Filters narrow results by date range, status code, MIME type and file extension.

  • URLs
  • historical endpoints
  • parameters

Does it touch the target?

Reads third-party sources only. Sends no traffic to the target's systems.

README: "getallurls (gau) fetches known URLs from AlienVault's Open Threat Exchange, the Wayback Machine, Common Crawl, and URLScan for any given domain." Providers are selectable with --providers wayback,commoncrawl,otx,urlscan.

API keys: The example config file linked from the README has an empty [urlscan] apikey field; the other providers run without keys.

Install

Install with go
go install github.com/lc/gau/v2/cmd/gau@latest

From the project's documentation

Install with binary
tar xvf gau_2.0.6_linux_amd64.tar.gz
mv gau /usr/bin/gau

From the project's documentation

Install with docker
docker run --rm sxcurity/gau:latest --help

From the project's documentation

How to run it

Examples use reserved documentation hosts such as example.com. Replace them with a domain you own or are authorised to test.

Fetch URLs for one domain
printf example.com | gau

From the project's documentation

Write results to a file
gau --o example-urls.txt example.com

From the project's documentation

Skip image files
gau --blacklist png,jpg,gif example.com

From the project's documentation

Include subdomains
gau example.com --subs

From the project's documentation

Output formats: stdout, file (--o), JSON (--json)

Limits the project documents

  • Piping input (echo "example.com" | gau) does not work with the Docker container.
  • The ohmyzsh git plugin aliases gau to git add --update, which conflicts with the binary.
  • Without a config file it runs on defaults and prints a message to stderr.

Deepak's take

Analysis, not documentation

Use it when
Use it first on a domain you are assessing to see historical URLs and parameters without sending any traffic to the target.
Skip it when
Skip it when you need what the site serves today; archived URLs may be dead. A live crawler such as Katana fits that need.
What it will miss
It only knows URLs that one of its four sources recorded, so pages never archived or indexed are missing.

Facts read from the project's README and docs on November 10, 2026. Repository data as of November 11, 2026. Licence: MIT. getallurls (gau) is a trademark or project name of its owners; this library is not affiliated with it.