#WEBSITE AUDITOR NOT SAVING INSTALL#

So any pricing package for a cloud crawler needs to cover both the potential bandwidth you will use crawling, but also the potential storage. And that's just for URL data, for a site with 100 links on each page, you could have another 10 million links, all of which needs storing - perhaps indefinitely - on your cloud architecture.

you're talking 30 million fields of data. Say you've crawled a site with 100,000 URLs. Sitebulb stores over 300 pieces of data for any one URL. The reason cloud crawlers are expensive is because crawling websites produces a LOT of data. We actually spent 3 months building the product as a cloud crawler, before realising there would be no way to keep the product reasonably priced. We had a goal in mind when we built our crawler, we really wanted to help small agencies and consultants to do amazing technical SEO work, at scale, without it costing hundreds or thousands of dollars a month. It might not be intuitively obvious why this happens, so let me explain. Since you don't need to wait 10 minutes for a cloud instance to spin up, Sitebulb can begin the crawl and start showing you data immediately.ĭesktop also means that it does not need to be prohibitively expensive, like all cloud crawlers inevitably end up.

#WEBSITE AUDITOR NOT SAVING INSTALL#

This means that you install it on your own computer (Windows or Mac) and it runs using your own local resources (CPU/RAM). Desktop vs SaaSįirst things first, Sitebulb is a desktop product (like Screaming Frog), and not a cloud/SaaS product (like DeepCrawl). TL DR - it's actually not a 'crawler.' It's a website auditor. I tried and failed to write my answer in 140 characters, so I gave up and went all in with a full scale blog post. I was recently asked the following question about Sitebulb on Twitter:

Website auditor not saving