Wikimedia servers

Other languages: de, eo, es, fr, it, ja, ko, zh, ua

Wikipedia and the other Wikimedia projects are run from several racks full of servers. See also Wikimedia's technical blog.

The Network topology is described in the article Server layout diagrams.

• Our DNS servers run PowerDNS. We use geographical DNS to distribute requests between our two main sites (US and Europe) depending on the location of the client.
• We use Linux Virtual Server (LVS) on commodity servers to load-balance incoming requests. LVS is also used as an internal load balancer to distribute MediaWiki and Lucene back-end requests. For back-end monitoring and failover, we have our own system called PyBal.
• For HTML we use Squid and Varnish caching proxy servers in front of Apache HTTP Server. For image files, we use Squid/Varnish in front of Sun Java System Web Server.
• All our servers run Ubuntu Server, except for the image file storage servers which run Solaris.
• For distributed object storage we use Ceph and Swift
• Our main web application is MediaWiki, which is written in PHP
• Our structured data is stored in MySQL. We group wikis into clusters, and each cluster is served by several MySQL servers, replicated in a single-master configuration.
• We use Memcached for caching of database query and computation results.
• For full-text search we use Lucene, with extensive customization contributed by Robert Stojnic.
• http://noc.wikimedia.org/conf/ - Wikimedia configuration files

As of late 2013, we have four colocation facilities (the name of each comes from acronym of the facility's company + three-letter acronym of closest airport):

eqiad

Equinix in Ashburn, Virginia.

pmtpa

Hostway (formerly PowerMedium) in Tampa, Florida.

sdtpa

Equinix (formerly Switch and Data) in Tampa, Florida (networking for the above).

esams

EvoSwitch in Amsterdam, the Netherlands (cache).

knams

Kennisnet in Amsterdam, the Netherlands (networking for the above).

ulsfo

United Layer in San Fransisco (cache; new, for USA West and East Asia).

The backend web and database servers are in Ashburn and a partial fallback is divided between our two Tampa facilities; as of July 2013, plans are known "to finally decommission" Tampa in 2013-14; this has been clarified to imply replacement with a new location to be selected in November 2013. At EvoSwitch, we have a Squid cache cluster and several miscellaneous servers. The Kennisnet location is now used only for network access and routing.

Ashburn (eqiad) became the primary data center in January 2013. In the past we've had other caching locations like Seoul (yaseo, Yahoo!) and Paris (lopar, Lost Oasis); the WMF 2010-2015 strategic plan reach target includes «additional caching centers in key locations to manage increased traffic from Latin America, Asia and the Middle East, as well as to ensure reasonable and consistent load times no matter where a reader is located».

EvoSwitch and Kennisnet are recognised as benefactors for their in-kind donations. See the current list of benefactors.

A list of servers and their functions used to be available at the server roles page; no such list is currently maintained publicly (perhaps the private racktables tool has one). It's however possible to see a compact table of all servers grouped by type on icinga (click to have a list of the servers in each group and then click on the names to see their machine details).

While Wikipedia is free in both the "free speech" and "no charge" senses of the word, running the web site does cost money. You can help with purchasing new server hardware by donating to the non-profit Wikimedia Foundation: https://wikimediafoundation.org/wiki/Donate

You can check one of the following sites if you want to know if the Wikimedia servers are overloaded, or if you just want to see how they are doing.

• Ganglia
• Icinga (formerly Nagios)
• Gdash

If you are seeing errors in real time, visit #wikimedia-tech ^connect on irc.freenode.net. Check the topic to see if someone is already looking into the problem you are having. If not, please report your problem to the channel. It would be helpful if you could report specific symptoms, including the exact text of any error messages, what you were doing right before the error, and what server(s) are generating the error, if you can tell.

• Technical FAQ - How about the hardware?
• Your donations at work: new servers for Wikipedia , by Brion Vibber, 02-12-2009
• wikitech:Clusters - technical and usually more up-to-date information on the Wikimedia clusters

• Server admin log - Documents server changes (especially software changes)

• Alexa traffic rank

• wikitech-l mailing list
• wikitech.wikimedia.org

• Cache strategy
• PHP caching and optimization
• Hardware orders up to 2007

• Evolution of LiveJournal systems:
  • 04/2004 MySQLCon 2004 PDF/SXI
  • 07/2004 OSCON 2004 PDF/SXI
  • 11/2004 LISA 2004 PDF/SXI
  • 04/2005 MySQLCon 2005 PDF/PPT/SXI
  • journals to watch for system details: Brad (Fitzpatrick) lj_backend lj_maintenance
• Google cluster architecture (PDF)
• MySQL User's Conference 2004 blog highlights

• Pages that need updating
• Wikimedia hardware
• Wikimedia servers administration