Should I worry about crawl budget?
Search Off the Record
00:00
Crowl Budget
We put up a block post in two thousand, 13, fourten about what cruel budget means for gualbot. And then we defined it as number of urals gualbot can and is willing, or is instructed, to crawl for given side, for given sight. For us, that's roughly what crowl budget means. Because, if you think about it, we don't want to harm web sides because gugalbot has enough crawl capacity to bring down sights. We tried to come up with at least a few internal metrics that could map together into something that people externally define a scroll budget. It maybe that part of the confusion is, like, what is good crowl
Transcript
Play full episode
