Obsolete:robots.txt

The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

This page contains historical information. It may be outdated or unreliable.

The rewrite rule

RewriteRule ^/robots.txt$ /w/robots.php [L]

sends all requests for robots.txt through robots.php. This script checks whether a page Mediawiki:robots.txt exists on the wiki. If it exists, the content of that page will be sent first. The file /apache/common/robots.txt will be sent afterwards.

To edit the static file, do the following:

Edit /home/wikipedia/common/robots.txt
Run sync-common-file robots.txt

The source of robots.php is available here: <https://gerrit.wikimedia.org/r/gitweb?p=operations/mediawiki-config.git;a=blob;f=live-1.5/robots.php;hb=HEAD>.