grosley

**grosley**
	grosley
Location:	pmtpa
Status
Overall:	This device has been decommissioned.
Icinga?:	host status services status
Hardware
Software

The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

gosley was the failover host for aluminium. See Grosley-Aluminium-Migration for notes from switchover in November 2011.

Services

Various fundraising services, including:

civicrm.wikimedia.org (Apache, PHP & CiviCRM)
fundraising.wikimedia.org
Non-web accessible fundraising scripts can be found in /opt/fundraising_misc.

Hudson

Grosley has Hudson CI installed to facilitate the periodic running of maintenance scripts. It's flexibility, particularly for logging/capturing output, is why we are preferring it over cron. Hudson CI ver. 1.368 running. It is accessible at: http://grosley.wikimedia.org:8080/

Potential issues

Hitting ext3 subdir limit in builds directories

Every time a Hudson job runs, a new directory gets created on Hudson where information about that particular build gets stored. Build dirs get created in:

/var/lib/hudson/jobs/<job name>/builds

Particularly for frequently running jobs (eg queue consumption which runs every 5 minutes), we run the risk of hitting the ext3 limit on subirectories (32000). When this limit gets reached, build jobs will fail silently until the issue gets resolved

Prevention

We have prepared a Nagios script which we are hoping to have monitor the job directories. Warning thresholds should be set at 75% of subdirectory limit and critical at 85%.

Resolution

As the limit gets approached, build directories should be archived to storage3 and then cleaned out:

# tar czf hudson_builds_<build name>_<starting date>-<ending date>.tar.gz /var/lib/hudson/jobs/<build name>/builds
# scp hudson_builds_<build name>_<starting date>-<ending date>.tar.gz user@storage3.pmtpa.wmnet:/archive/fundraising/hudson_builds/
# rm -rf /var/lib/hudson/jobs/<build name>/builds/*

Mail setup

grosley runs a simple Exim setup, which can send out e-mail directly to the outside world using its dnslookup router and remote_smtp transport. It's also configured to accept mail for *@donate.wikimedia.org, and deliver this mail into a Maildir box in /var/mail/civimail, using the civimail router and civimail_delivery transport. This is basically a simpler version of the configuration described in the Mail article.

Dovecot is installed for IMAP access by CiviMail. Its configuration is almost default, but it's configured to listen using non-SSL IMAP on the loopback interface only, and to retrieve mail using Maildir from /var/mail/user. Authentication is done using PAM and system users, so a system user civimail has been added.

Configuration changes

localhost IPs (v4 and v6) added to relay_from_hosts in exim4.conf --David Strauss 02:37, 7 May 2009 (UTC)[reply]
Truncated /var/log/exim4/paniclog, which had an old configuration syntax error notice in it --David Strauss 02:38, 7 May 2009 (UTC)[reply]