The older, the simpler: robotstxt.org. robots.txt in a nutshell

To talk about everything
User avatar
axew3
w3all User
w3all User
Posts: 2677
Joined: Fri Jan 22, 2016 5:15 pm
Location: Italy
Contact:

The older, the simpler: robotstxt.org. robots.txt in a nutshell

Post by axew3 »

I fall on this today, after i know it since his creation: has been a surprise to see that, in all these years, it changed nothing at all!
http://www.robotstxt.org