MobileRoles
There are several roles needed for a functioning Mobile system.
Terms
- RackApp = The main (currently) Merb Ruby application that makes up "WikimediaMobile"'s core code
- MobileArticle = A custom transformed version of a Wikipedia article in a reusable state
List of Roles
- MobileHTTP = A server ready to handle public-facing mobile requests
- MemcachedServer
- Mysql = A log database server... only used by the LogParse script
Basic Outline of Request Cycle
- An HTTP request comes in to MobileHTTP
- The Proxy Server on port 80 passes the request into the/an RackApp process
- The RackApp checks with MemcacheServer to see if the MobileArticle is cached
- If it is not found, then the RackApp uses libcurl to scrape a copy from #{lang}.wikipedia.org and transforms it with libxml2 into a cachable MobileArticle. This is then stored in the cache.
- The MobileArticle is put in a proper layout by the RackApp and is returned to the user
- The RackApp writes a log entry in JSON format in a file on MobileHTTP
- The Proxy Server sends back the document
Basic Outline of Log Parsing
- A LogParse script is run every 5 minutes
- The script moves the log file to a tmp location (RackApps are still writing to it)
- The script sends a USR1 signal to the RackApp cluster which causes them to write to a new file
- The script parses the JSON-formatted log file and merges the information into a single stat entry
- The stat entry (makes up a row in the db) is written to the Mysql database
- The temporary log file is deleted