Sunday, April 12, 2009

apache mpm-worker with php on low memory servers

I'm partly writing this because I think such valuable info is too hard to find, I'd read a lot that if you want php, you have to use mpm-prefork but it's not true!

I've been using slicehost for dev server for almost a year now. Since my 1 year deal is up, I decided to switch to their affiliate mosso. I really like slicehost, but Mosso "cloud servers" seem a good fit for a server that goes through spurts of development and use followed by weeks of non-use. So now, I can keep it as a 256MB instance at about $10/month and update to a larger instance when doing real dev.
I built it today as a 1024MB instance -- installed all my usual stuff, and updated my build script for ubuntu. That's here.
The machine I'm on is extremely fast, normally I set GDAL building and leave, but it finished before I had a chance. After all was built, I resized it to a 256MB server -- that took 12minutes, but my instance was accessible for at least 10 of those.
After that, I log right back in and all works fine. On this dev server, I have to run PHP (at least I dont have to code in it). I've been getting tired of having to use apache's mpm-prefork because ubuntu won't let me have php and mpm-worker. (For more from someone who actually knows what they are talking about, Graham Dumpleton, the author of mod_wsgi, has a good write up here.) So, every apache process takes up a good chunk of the available memory and things are sloooow. Even just running trac, it starts swapping.
I found a good post for using fcgi for php and followed it blindly and all works!
Using worker_mpm, Trac (via mod_wsgi) is so fast I doubled checked to make sure my instance was still at 256MB. After setting ThreadStackSize to 1.5MB trac takes up only 16% of the available memory. The relevant bits of my apache config look like:

<IfModule mpm_worker_module>
StartServers 2
MaxClients 40
MinSpareThreads 5
MaxSpareThreads 20
ThreadsPerChild 20
MaxRequestsPerChild 5000
</IfModule>

MaxKeepAliveRequests 1000
Timeout 30

# i keep this fairly high for serving image tiles
# from mapserver
KeepAliveTimeout 40

# default is 8 per child process, only use 1.5MB
ThreadStackSize 1500000

AddHandler fcgid-script .php
FCGIWrapper /usr/lib/cgi-bin/php5 .php

and then in 000-default, i added +ExecCGI to the DocumentRoot Directory Options.
If any of that config is not sane, let me know.

I should say that I have no idea how PHP will do under fast-cgi, if it uses more memory, or not, or if it will have any problems, on quick look, everything seems ok.

Apparently, you can also use mpm-worker with php if you build php from source. Since using fcgi worked so easily, I didn't try that.

2 comments:

Graham Dumpleton said...

The issue isn't really with the PHP core but the third party extensions for it. It is those which are not necessarily thread safe.

As to your Apache configuration, I would be careful running with such a high KeepAliveTimeout. If the site is extremely low traffic may be okay, but anything more significant and the keep alive connections can consume all the Apache threads across the worker processes and server will stall until they are released.

You may be better off putting static media on nginx front end with nginx proxying through to Apache for dynamic web applications. This is because nginx, not using threads, will cope with larger numbers of keep alive connections on a long time much better.

brentp said...

@graham
i will lower the KeepAliveTimeout as you suggest, that makes sense.

since everything is working so much faster than before, i dont yet feel motivated to set up nginx as well.

most of the images are not static, they're geographic data rendered on the fly by a wsgi script. and the images are requested in bunches because they are tiles in a mapping app. my thought was that this is where changing to a larger "cloud" server with more memory would be nice when it's actually in use--rather than trying to get so much from a server with 256MB of RAM.
given that use-case, do you see a better setup than what i've described?

thanks (for modwsgi and the feedback).