I made a very tiny tool called Spyyyder that grabs a page, tells me what links are on that page, and then allows me to drill down through each of them. For getting to grips with how a site is put together, it's a pretty useful tool.
It groups them by internal links, subdomains, links to external sites, and any
I find it useful when researching competitors. For example, sometimes you'll find versioned url slugs, and that gives you a thread to pull at. I'm surprised at what it's possible to learn just from looking at the links on a page.
Spyyyder is pretty dumb. There are a bunch of tools out there that do this, and show you a pretty chart or whatever - I wanted Spyyyder to be really simple, and as fast as possible.
Under the hood
I've been keen to get to grips with TanStack Query; this seemed like a good project for it. Each URL you search for is a query key, and this means that you're never refetching a URL that's been seen before. It makes for a pretty snappy experience as you explore.
A Supabase edge function takes care of the CORS limitations and compiles the list of links.
Give it a go: https://spyyyder.vercel.app →