.Google's John Mueller responded to an inquiry about why Google indexes pages that are actually forbidden from crawling through robots.txt as well as why the it is actually risk-free to dismiss the associated Explore Console files about those crawls.Bot Visitor Traffic To Inquiry Parameter URLs.The individual inquiring the concern recorded that crawlers were actually developing web links to non-existent concern specification URLs (? q= xyz) to webpages along with noindex meta tags that are additionally blocked out in robots.txt. What motivated the inquiry is actually that Google.com is actually creeping the hyperlinks to those web pages, obtaining blocked out by robots.txt (without envisioning a noindex robots meta tag) after that getting turned up in Google Look Console as "Indexed, though obstructed through robots.txt.".The person talked to the observing concern:." However listed below's the huge inquiry: why will Google index webpages when they can not even find the web content? What's the advantage in that?".Google.com's John Mueller validated that if they can not creep the page they can not see the noindex meta tag. He also helps make an appealing mention of the web site: hunt driver, recommending to disregard the end results because the "average" customers won't see those outcomes.He wrote:." Yes, you're right: if our experts can't creep the page, our team can not view the noindex. That pointed out, if our company can't creep the webpages, then there is actually certainly not a great deal for us to mark. So while you might observe several of those pages along with a targeted internet site:- inquiry, the common user will not see all of them, so I would not bother it. Noindex is likewise great (without robots.txt disallow), it simply indicates the URLs will certainly end up being crept (as well as wind up in the Look Console record for crawled/not indexed-- neither of these conditions trigger concerns to the rest of the internet site). The essential part is actually that you do not create them crawlable + indexable.".Takeaways:.1. Mueller's answer validates the restrictions in operation the Site: search advanced hunt driver for analysis main reasons. Some of those reasons is due to the fact that it is actually certainly not connected to the normal search index, it is actually a different factor entirely.Google's John Mueller commented on the internet site hunt driver in 2021:." The brief response is actually that a website: query is certainly not meant to become complete, nor used for diagnostics functions.A website concern is actually a details type of search that restricts the end results to a certain website. It is actually primarily just words internet site, a colon, and after that the website's domain.This question confines the results to a specific web site. It's certainly not meant to become a detailed selection of all the pages from that internet site.".2. Noindex tag without using a robots.txt is actually fine for these type of situations where a robot is connecting to non-existent web pages that are actually getting found out by Googlebot.3. Links with the noindex tag will definitely produce a "crawled/not listed" item in Browse Console and that those won't possess a damaging impact on the rest of the site.Go through the concern and address on LinkedIn:.Why would certainly Google mark webpages when they can not even view the material?Featured Image through Shutterstock/Krakenimages. com.