url-downloader

The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

Please, never use this functionality to reach out to endpoints belonging to Wikimedia. You should be able to utilize the service mesh/services proxy to reach most endpoints internally in a safer, more reliable and more performing way. If you can't find out how, please reach out to SRE

url-downloader is a Squid non-caching proxy for HTTP requests. It was originally created for MediaWiki's upload-by-URL functionality, these days it is being used by any service that wants to access resources outside the boundaries of the wikimedia production realm. It is not a highly available service and as such no assumptions should be made in code regarding availability.

Service

See also url_downloader in Puppet.

For eqiad: Hosted on urldownloader100[34].wikimedia.org
For codfw: Hosted on urldownloader200[34].wikimedia.org

In both datacenters only one url-downloader is active at any time, and is pointed at via the DNS record for url-downloader.$DC.wikimedia.org, and exposed at url-downloader.$DC.wikimedia.org:8080.

url-downloader.wikimedia.org:8080 points to eqiad for backward-compatibility.

In MediaWiki configuration, it would be typically obtained via $wmgLocalServices['urldownloader'] (which also provides an appropriate proxy URL on beta).