Thursday, January 3, 2019

Bing: We Ignore Default Robots Directives If There Is A Bingbot Section

Frédéric Dubut from Bing's search team said on Twitter that if you create a specific robots.txt directive for Bingbot, their crawler, then Bing will only look at that specific section. So you should make sure that when you do that, copy all the directives from the default to the Bingbot section that you want Bing to comply with.

No comments:

Post a Comment