Crawl budget is the number of pages Google is willing and able to crawl on your site in a given period. It comes up in almost every technical SEO conversation, and for most websites it is not the problem.
The short answer
Google’s own crawl budget guide is written for very large sites (around a million pages or more) and for medium-sized sites (around ten thousand pages or more) whose content changes every day. It also says that if your new pages are usually crawled the same day you publish them, you do not need to worry about it. A 40-page website for a manufacturing company in Chakan or a 200-page blog is nowhere near that.
When it does matter
- Large sites. E-commerce stores, property portals and job boards with tens of thousands of URLs or more.
- Sites that generate URLs automatically. Filters, sort orders, search results pages and calendar pages can create thousands of near-duplicate URLs.
- Sites with slow servers. If your server struggles, Google crawls more gently to avoid overloading it.
How to check
Open Google Search Console and go to Settings, then Crawl stats. You will see how many requests Googlebot made, how fast your server responded and which kinds of files it fetched. Then look at the Pages report: a large pile of “Discovered, currently not indexed” URLs on a big site can be a sign that Google is not getting round to them.
What actually helps
- Stop wasting crawls. Block endless filter and sort combinations with robots.txt or handle them with sensible URL design.
- Fix redirect chains and broken links. Every hop is a wasted request.
- Keep your XML sitemap clean. List only the pages you want indexed, with correct, final URLs.
- Speed up the server. Faster responses let Google crawl more without straining your hosting.
- Link to important pages. Pages with strong internal links get found and recrawled sooner.
What does not help
Adding a noindex tag does not save crawl budget, because Google still has to crawl the page to see the tag. And asking for indexing in Search Console page by page is not a strategy for a large site.
If you run a large catalogue or listings site and new pages are slow to appear, our technical SEO service starts with exactly this kind of crawl analysis.