There has been some confusion around how Google handles pages that might be more costly for Google Search to crawl, render, index and serve – i.e. JavaScript pages. Google does not have a monetary budget per site, in terms of it will spend $X of crawling budget on your site.
Yes, sites do have a crawl budget, but not in terms of cost, more in terms of resources. You can check Google’s official documentation on the topic.
That being said, Martin Splitt of Google said on LinkedIn, “We don’t keep track of “how expensive was this page for us?” or something.” He added, “You don’t need to worry about the fact that rendering is expensive, we got you covered.”
Martin goes on to explain that crawling is expensive and so are other parts of search. Google’s ultimate goal is to show the most relevant result, no matter how expensive that result may cost. “Google Search’s goal is to provide users with relevant content for their queries. We’re not perfect at doing this for all queries all the time, but let’s focus on JavaScript here for a moment,” he wrote.
“Google Search does lots of things that are complicated and expensive (storage, bandwith, teams around the globe to keep it all running 24/7), etc. – JavaScript is one tiny part of that,” he goes on to explain.
Martin then explained that JavaScript is part of the web and likely growing. “We know that a substantial part of the web uses JavaScript to add, remove, change content on web pages. We just have to render, to see it all. It doesn’t really matter if a page does or does not use JavaScript, because we can only be reasonably sure to see all content once it’s rendered,” he wrote.
So having more “expensive” pages to crawl does not mean Google won’t crawl it – they will.
Forum discussion at LinkedIn.
