Website URL Extractor & Crawler

Max Depth controls how many link levels the crawler follows from your starting URL. A depth of 1 only visits pages linked directly from the start URL. A depth of 2 also visits pages linked from those pages, and so on up to a maximum of 4 levels deep.

No. By default the crawler only follows internal links that share the same origin as your starting URL. You can enable the 'Follow external links' checkbox to allow the crawler to branch out to other domains, but this may consume your URL limit faster.

If the starting URL is unreachable or returns a client error like 404 or 403, the crawl will stop immediately and display an error message. Make sure the URL is publicly accessible and spelled correctly before starting.

Yes. A search bar above the results table lets you filter discovered URLs in real time by any column value, including URL path, page title, status code, or depth level.

If the crawler hits a rate limit or temporary block from the target server mid-crawl, it will stop fetching new pages and display a warning alongside whatever results were collected up to that point. You can still export those partial results normally.