The Mechanical Turk

Wow, some of the php scripts I wrote years ago are quite inefficient. I really should rewrite them. Well, when our clients want an upgrade, I'll do it then. For now, I'm going to install eaccelerator. Luckily, it's exceedingly easy.

wget http://bart.eaccelerator.net/source/0.9.5/eaccelerator-0.9.5-rc1.zip
unzip eaccelerator-0.9.5-rc1.zip
cd eaccelerator-0.9.5-rc1

Phpize is needed to configure eaccelerator, and phpize needs autoconf so make sure you've got that.

yum install autoconf

Now to configure eaccelerator:

phpize
./configure
make
make install

Take note of where the shared lib was installed (last line of output of the the above command).

cp eaccelerator.ini /etc/php.d

Edit the above (copied) file, locate this line:
zend_extension_ts="/usr/lib/php4/eaccelerator.so"
and replace it with:

zend_extension_ts="<location you took note of above>/eaccelerator.so"

Now all that's left is to create a tmp dir for eaccelerator's cache (/tmp/eaccelerator is the default, but you can change it in the above file), and restart apache.

mkdir /tmp/eaccelerator
chown apache /tmp/eaccelerator
chgrp apache /tmp/eaccelerator
chmod 644 /tmp/eaccelerator
/sbin/service httpd restart

Test by knocking up a phpinfo.php page. At the bottom of the first section it should look something like this:

WTF?!

I've been out of the php world for a while it seems. We've recently had to move some old project from a shared host to our server. I prepared myself for upgrading old php4 stuff to php5. Not fun, but not terrible. Time comes to deploy... and mysql is not installed in the apache php module...!?

So I check yum

$ yum list | grep php-mysql
php-mysql.i386                           5.0.4-10.5             installed

WTF!?

So I knock up a phpinfo page and check it, and sure enough it's been compiled --without-mysql. WTF!? Turn's out that since php5 automatic support for the mysql extension is turned off. I'm so over php these days that I couldn't even be bothered to rant.

Because I didn't want to waste any precious rails time on this stuff, I just went straight for the jugular and decided to recompile php --with-mysql--duh

Thanks to an article at fedoranews.org, it was a not so painful. In fact following the instructions over at php.net will get you there in the end.

But, systems differ, so if you're running FC4, you should be able to get things up and running with the following commands and config.

Plan and assumptions

I assume you've got http-2.0.54 (the current FC4 release) and mysql already installed (and the php-mysql.i386 package which is causing the problem). The plan is to configure php just like the one you get with yum, but with mysql, and without some other problematic dependencies which are not easy to get with yum.

Install required dependencies

yum install curl-devel.i386 \
            bzip2-devel.i386 \
            libxml2-devel.i386 \
            flex.i386 \
            gmp-devel.i386 \
            httpd-devel-2.0.54 \
            libc-client-devel

Get php, configure and make it

wget http://us2.php.net/get/php-5.1.6.tar.gz/from/www.php.net/mirror
tar -zxvf php-5.1.6.tar.gz
cd php-5.1.6

./configure \
  '--build=i386-redhat-linux' '--host=i386-redhat-linux' \
  '--target=i386-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr' \
  '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' \
  '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' \
  '--libdir=/usr/lib' '--libexecdir=/usr/libexec' '--localstatedir=/var' \
  '--sharedstatedir=/usr/com' '--mandir=/usr/share/man' \
  '--infodir=/usr/share/info' '--cache-file=../config.cache' \
  '--with-libdir=lib'  '--with-config-file-path=/etc' \
  '--with-config-file-scan-dir=/etc/php.d' '--disable-debug' '--with-pic' \
  '--disable-rpath' '--with-bz2' '--with-curl' '--with-exec-dir=/usr/bin' \
  '--with-freetype-dir=/usr' '--with-png-dir=/usr' '--enable-gd-native-ttf' \
  '--without-gdbm' '--with-gettext' '--with-gmp' '--with-iconv' \
  '--with-jpeg-dir=/usr' '--with-openssl' '--with-png' '--without-pspell' \
  '--with-expat-dir=/usr' '--with-pcre-regex=/usr' '--with-zlib' \
  '--with-layout=GNU' '--enable-exif' '--enable-ftp' '--enable-magic-quotes' \
  '--enable-sockets' '--enable-sysvsem' '--enable-sysvshm' '--enable-sysvmsg' \
  '--enable-track-vars' '--enable-trans-sid' '--enable-yp' '--enable-wddx' \
  '--with-pear=/usr/share/pear' '--with-kerberos' '--enable-ucd-snmp-hack' \
  '--without-unixODBC' '--enable-memory-limit' '--enable-shmop' \
  '--enable-calendar' '--enable-dbx' '--enable-dio' \
  '--with-mime-magic=/etc/httpd/conf/magic' '--without-sqlite' \
  '--with-libxml-dir=/usr' '--with-xml' '--with-apxs2=/usr/sbin/apxs' \
  '--with-mysql' '--without-gd' '--without-odbc' '--disable-dom' \
  '--disable-dba' '--with-imap-ssl'

make

(Ignore lots of pointer warnings)

Install php and configure Apache

/sbin/service httpd stop
make install

Now we need to make apache look for the new compiled module. On my dist the php config is in /etc/httpd/conf.d/php.conf, it may be in /etc/httpd/conf/httpd.conf on yours.

Open the conf file, locate the following line:LoadModule php5_module modules/libphp5.so and replace it with:

LoadModule php5_module /usr/lib/httpd/modules/libphp5.so

Let go of your attachments and desires

(like the idea of your server working) Now you're in the best state of mind to issue the final command.

/sbin/service httpd start

Write a phpinfo page and check that everything works.

I've been trying to get a nice, flexible, solution for automagically configuring vhosts and subdomains. Here's my latest effort:

Subversion: http://svn.ardes.com/ardes/srv/local/etc/lightttpd

The solution is to meet the following desiderata:

• Flexible: no enforcing of a particular lighty/rails setup for a particular vhost,
• Flexible: no enforcing of a particular directory layout, and I want it to work for vhosts and subdomains, or a mixture of the two,
• Flexible: I want a solution which is compatible with serving only some sites with lighty (via ProxyPass in Apache),
• Set and forget: I don't want to have to edit my main lighttpd.conf file every time I add or remove a site.
• Encapsulated: I want the config files for a site to reside in that site's directory.

How to

Assumptions

I'm using a plesk system, and wanted to make it compatible with that. But the following should work with any site layout you want. Here's an example site layout:

  /var/www/vhosts/example.com
  /var/www/vhosts/example.com/subdomains/blog

You want to serve both of the above using lighty, just static content from example.com, and a railsy blog from the blog.example.com subdomain.

Requirements and setup

First you need to get a copy of hosts.conf.rb. You should put this in your /etc/lighttpd directory and make it executable.

Then edit your lighttpd.conf file so that contains the line:

include_shell "/etc/lighttpd/hosts.conf.rb"

This will replace all of your host conditionals that configure your vhosts and subdomains. The configuration now goes into small files in the site directories (default is conf/lighttpd.conf, but this is configurable). Also, a var called host-root is set to the current path.

In /var/www/vhosts/example.com/conf/lighttpd.conf:

server.document-root = var.host-root + "/httpdocs"

In /var/www/vhosts/example.com/subdomains/blog/conf/lighttpd.conf:

var.rails-root = var.host-root + "/rails/typo"
var.fcgi-min-procs = 2
var.fcgi-max-procs = 4
  
include "rails.conf" # this script does all the rails stuff and should be
                     # placed in /etc/lighttpd

See rails.conf for the details of the above include script

Test

Now test this by running hosts.conf.rb from the command line. You should see the following output:

$HTTP["host"] =~ "(^|www\.)example\.com" {
  var.host-root = "/var/www/vhosts/example.com"
  server.document-root = var.host-root
}
$HTTP["host"] == "blog.example.com" {
  var.host-root = "/var/www/vhosts/ardes.com/subdomains/gtd"
  var.rails-root = var.host-root + "/rails/typo"
  var.fcgi-min-procs = 2
  var.fcgi-max-procs = 4

  include "rails.conf" # this script does all the rails stuff and should be
                       # placed in /etc/lighttpd
}

Notice the the vhost is configured to catch www.example.com and example.com. Notice also that you don't have to edit the main /etc/lighttpd.conf file. The idea being: you can add and remove conf files to vhost and subdomain paths, and when lighty is restarted, they'll automatically be picked up by the hosts.conf.rb script!

So once you've got the output of hosts.conf.rb working the way you like it, you should restart lighty.

A glitch

At this point, I encountered a small problem with lighttpd 1.4.11. It complained that SHELL wasn't set when I tried to use the 'include_shell' directive (kinda essential). This is no problem if you're manually starting lighty, but I'm using the init.d script. So I simply had to add the followng to /etc/init.d/lighttpd.

  export SHELL=/bin/sh

Restart lighty and test!

Other configurations

If you want to do this with a different layout, you just need to edit the config section of hosts.conf.rb. The syntax is quite flexible, here's the config for the above setup:

Config = {
  :vhost => {
    :path  => '/var/www/vhosts/:vhost:',
    :host  => '(^|www\.):vhost:'
  },
  :subdomain => {
    :path  => '/var/www/vhosts/:vhost:/subdomains/:subdomain:',
    :match => '==',
    :host  => ':subdomain:.:vhost:'
  }
}

The :path string can contain as many named variables as you like, they are surrounded by colons (e.g. :vhost:). The script simply searches the filesystem with these as glob wildcards. However the values are captured and stored by name for use in the host regexp specification. This is what makes it so easy to configure (the www example above is a good starting point).

Roundup

This is my first foray into lighty configuration. Please improve on this offering, or let me know if it doesn't work for you.

One of our clients just bought a new server and asked us to move a rails app onto it. Unfortunately, I found that the server software was Fedora Core 2, which is not the best playmate for rails (you can only get ruby 1.8.2 via yum for a start). And support for FC2 has just been dropped by the Fedora Legacy project. Methinks an upgrade. This looked like it could consume many days and involve much pain...

(you can skip straight to the upgrade)

After a bit of googling, it seemed that this was not a totally crazy plan. The basic idea being: upgrade yum first, and grab the target fedora-release. Then upgrade everything, removing any conflicts, reboot into FC3. Repeat for FC4. Ta-da. But I'd been here before...

Owing to a spectacular failure of imagination, I'm an optimist about pretty much anything that is more than 20 minutes into the future. Luckily my memory is in better shape than my imagination. So a little voice said 'you could totally tool yourself here, we're talking utterly spannered'

That voice was right. I needed to know more, and I needed to be rational about this. It was time to consult my handy tarot utility...

$ tarot --help
Gives a tarot reading for the specified path(s).
USAGE: tarot [options] path[s]
  --help       this text
  --ironic     invokes no real powers
  --port=PORT  Attempts to establish a connection to the nether world on
               the specified port (default = 777).  Connecting on port 666 is
               only recommended with --ironic.
               
               Many have reported good results when used in conjunction with 
               the 'pom' utility.
$
$ tarot --ironic /usr/local/bin/psa /usr/boot /var/www/vhosts 
  card 1: #13 La Mort
  card 2:
Segmentation Fault: Core dumped
$

La Mort, Death... then a segfault, holy crap!

But a few minutes on the interweb revealed that it's a schoolboy error to equate 'La Mort' with bad. No! It can mean rebirth, or starting anew. This was great news!

I had previously thought that this sort of thing was just guff. A handy way for the charlatans to avoid the unfortunate economic effects of their their arcane art's heritage. (Remember the scandal in the 1960s when the North Ulster Tarot Society proposed replacing La Mort with Le Lapin?).

But I could see that this reading was telling me, in some detail, exactly what the outcome of the update would be. Confidently, without even cross-referencing with pom, I plowed ahead.

Requirements

The server was running FC2 and Plesk 8.01, and very little else. I had to make sure that the existing data on the server (about 100 vhosts) remained a-o-k. I had to make sure that Plesk and Webmail would all work in the same way for the client.

I won't keep you in suspense; it turned out fine, but there was one heart stopping moment...

The Upgrade

A lot of the following info was gleaned from Fedora's yum upgrade faq

DISCLAIMER: that little voice is telling me to tell you "it's yer own damn fault if you tool yourself"

Backup

The first thing to do is backup the plesk client data. Make sure you transfer the resulting file off the server in case you tool it and can't reboot.

/usr/local/psa/bin/pleskbackup --all <filename>

FC2 to FC3

Next, upgrade yum and fedora-release to version 3 (make sure you're logged in as root). If you find that you can;t download the rpms from the example mirrors, just google the rpm name with 'Fedora'.

wget http://fedora.glorb.com/core/3/i386/os/Fedora/RPMS/fedora-release-3-8.i386.rpm
wget http://fedora.glorb.com/core/3/i386/os/Fedora/RPMS/yum-2.1.11-3.noarch.rpm
rpm -Uvh yum-2.1.11-3.noarch.rpm
rpm -Uvh fedora-release-3-8.i386.rpm
yum -y upgrade

Now, while the upgrade is churning away, prepare yourself for some dependency failures. I had a missing dependency, various plesk packages (psa-*) require libcurl v2, which is not in FC3. By far the best solution is to remove these packages, and start over. I only had to remove psa and php, which you will install later.

Once you've removed all the offending packages (with yum remove <packagename> by the way) and have gotten yum to upgrade you;re ready to set the kernel to the new version. To do this edit /boot/grub/grub.conf and set the value of default to the position in the list that 2.6.9 is at (the list starts from 0)

You're now ready to reboot into FC3! Do so with reboot

FC3 to FC4

Now you want to upgrade to FC4. Pretty much the same procedure as before, but easier because you're now using a better yum.

rpm -Uvh http://download.fedora.redhat.com/pub/fedora/linux/core/4/i386/os/Fedora/RPMS/fedora-release-4-2.noarch.rpm
yum -y update kernel

It's a good idea to remove any stale kernels at this stage

yum remove kernel-2.6.\*FC3\*

Now do the upgrade, and again just remove any offending packages which cause dependency failures

yum -y upgrade

Once you're updated check the /boot/grub/grub.conf file again, making sure the FC4 kernel is default, and reboot!

Congratulations! you're on FC4. But there still some stuff to do to get plesk back.

Reinstall Plesk

Find the version of plesk you want (make sure it's packaged for FC4) and grab the autoinstaller, make it executable, and execute it, and follow all the prompts.

wget http://download1.swsoft.com/Plesk/Plesk8.0/FedoraC4/psa_installer_v3.0.2_build060704.16_os_FedoraCore_4_i386
chmod +x psa_installer_v3.0.2_build060704.16_os_FedoraCore_4_i386
./psa_installer_v3.0.2_build060704.16_os_FedoraCore_4_i386

After a while you will have plesk back!

Firstly, the default firewall for FC4 won't allow access to the plesk admin server on port 8443. So you should either add some rules to iptables to allow this, or disable it with chkconfig --del iptables. Disabling it is fine, as Plesk comes with a firewall.

Now, check that no php4 cruft is hanging about in /etc/php.ini. I just replaced it with the rpm distro one.

Check your webmail and admin interface. Hopefully, it will all be there, and you won't have to restore from backup. But you'll probably find that plesk is running in 'Demo' mode.

At this stage I had to reinstall my license key for plesk, which I got from my reseller. You can apparently reinstall the license key by restoring from the plesk backup.

Which brings me to the heart stopping moment...

Restoring the Plesk Backup

I was pretty sure everything was still there, but I thought that restoring from the backup was prudent. First you must create a 'map' file of your current system, without the 'map' your backup is useless - you're spannered. Here goes...

$ /usr/local/psa/bin/pleskrestore --create-map my_plesk_backup -map backup.map
  ERROR: PleskException
  Can not start supervisor /root/restore: Traceback (most recent call last):

  ...
  ImportError: Bad magic number in /usr/local/psa/admin/lib/python/libxml2.pyc
  ---

D'oh!

Much fun ensued b/n me, my host company, and eventually SwSoft. SwSoft said it was a known issue, but was fixed in Plesk 8.01, but it wasn't. It was established that I did have the latest version. Eventually the plesk version of the libxml2.pyc library simply had to be removed and recompiled as follows.

rm /usr/local/psa/admin/lib/python/libxml2.pyc
/usr/local/psa/admin/sbin/supervisor 

Throughout all this our host company Idaq were extremely good (especially at not blaming me). SwSoft were also quick to acknowledge and diagnose the problem. But I did have to laugh at the closing statement of SwSoft.

"I contacted our developers regarding the issue and they surmised that it was a hardware problem during first compilation of libxml2.pyc, so the file was created with error."

Hmmm, spooky. I think they might need a copy of tarot.

When you're trying to marry a server admin interface with a server config that's not about 5 years old (e.g. lighty and rails), you tend to fight the system. That is, the various control panel interfaces do not seem to play nicely with your custom setup out of the box. If you're like me, you tend to end up having your client's files scattered all over the place, config files here and there, rails apps in /var/www/lighttpd, static conetnt in /var/www/httpd/vhosts/client/http_docs, etc, etc.

This should be avoided because; (a) If your client backs up their site using a control panel type interface, they wont get any of the lighty conf or their rails app, (b) In a couple of months, unless you've taken copious notes, which is a pain in itself, you won't have a clue where all the relevant data for a particular client is, and (c) when it comes to upgrading your system, or the client's application, you'll probably miss something.

In this post, and others (linked below), I detail my attempt to create a uniform way of hosting rails apps for our clients, using Plesk. Plesk is real easy to use for keeping track of various admin related things, setting up mail, dns stuff, and has a nice interface for the client. that you can use for website admin. I'd like to make it easy for my and my forgetful brain to add a new app in a month.

System setup

Our server is a Fedora Core 4 machine, and it's real easy to get the system up to speed. A quick google found a real nice howto by Digital Media Minute. If you're starting with Plesk v8, then you'll already have a lot of the software you need. Note: If you're running FC < 4, I'd strongly recommend upgrading.

Setup lighttpd

I'm keeping the same permissions system as plesk for the rails apps. This means that lighttpd must belong to the psaserv group to be able to read your rails project. To achieve this edit /etc/group and change:

psaserv:x:2523:apache,psaftp,psaadm

to:

psaserv:x:2523:lighttpd,apache,psaftp,psaadm

You should set lighttpd to run at startup:

chkconfig --add lighttpd 
chkconfig --level 345 lighttpd on

See my post about lighttpd config for a full-blown vhost and subdomain setup, or see the rails wiki for a simple example.

Configuring a rails site

So here's what I do when setting up a new rails app with plesk:

1. Set up the vhost or subdomain with plesk control panel.

This will create the required directories in /var/www/vhosts/<your domain> or /var/www/vhosts/<your domain>/subdomain/<your subdomain>

2. Set up proxy pass.

We're going to leave Apache running, and just get it to pass requests to and from lighty for our railsified sites. This is pretty easy to do with plesk. Just add the following in /var/www/vhosts/<your domain>/conf/vhost.conf:

ProxyRequests Off
ProxyPreserveHost On
ProxyPass / http://0.0.0.0:81/
ProxyPassReverse / http://0.0.0.0:81/

If you want to also pass SSL to lighty then simply copy vhost.conf to vhost_ssl.conf and add the following line at the end.

RequestHeader set X_FORWARDED_PROTO 'https'

If you're configuring a subdomain then you need to edit the subdomain's conf, no need to touch the vhosts's conf.

Finally, to get plesk to recognise the new vhost.conf file you need to do the following (irrespective of whether it's a domain or subdomain):

/usr/local/psa/admin/sbin/websrvmng --reconfigure-vhost --vhost-name=<your domain>

3. Set up a rails directory

Make a rails directory in the client's site (/var/www/vhosts/<your domain>), and make it belong to the psaserv group, and to the user who owns the site. Also set the sticky bit on group. You want the permissions to look like this:

chmod 750 rails
chmod g+s rails
ls -al | grep rails
drwxr-s---   2 client  psaserv 4096 Sep 26 09:15 rails

4. Deploy your rails app

Get your rails app, for example typo, into rails/ by whatever method you normally would. Make sure that lighttpd has write access to public/, log/ and tmp/.

chmod -R g+w tmp public log

5. Configure lighty for your app

Now you need to write the lighty conf for the app. If you're following my post about lighty config then you need to put the following in /var/www/vhosts/<your domain>/conf/lighttpd.conf:

var.rails-root = var.host-root + "/rails/typo"
var.fcgi-min-procs = 1 # or whatever you like
var.fcgi-max-procs = 2 # for these values
include "rails.conf"

Otherwise edit /etc/lighttpd/lighttpd.conf and add something like the following:

  $HTTP["host"] =~ "(^|www\.)domain\.com" {
    
    var.rails-root = "/var/www/vhosts/domain/rails/typo"
    
    server.document-root = var.rails-root + "/public"
    server.errorlog      = var.rails-root + "/log/lighttpd_error.log"
    accesslog.filename   = var.rails-root + "/log/lighttpd_access.log"

    url.rewrite = ( "^/$" => "index.html", "^([^.]+)$" => "$1.html" )

    server.error-handler-404 = "/dispatch.fcgi"

    fastcgi.server = ( ".fcgi" =>
      ( "localhost" =>
        ( "min-procs" => 1,
          "max-procs" => 2,
          "socket" => var.rails-root + "/tmp/sockets/fcgi.socket",
          "bin-path" => var.rails-root + "/public/dispatch.fcgi",
          "bin-environment" => ( "RAILS_ENV" => "production" )
        )
      )
    )
  }

Finally, restart your lighttpd server:

service lighttpd restart