I’ve been spending a lot of time porting over some code from ASP.NET to mod_perl2. Along with that rewrite, I’ve been migrating a side business away from managed hosting running Windows 2003 to Linux. Between the porting and migrating, managing the clients and working to grow the business, I’ve had time for little else.

I’ve discovered some pretty cool things along the way, though.

Apache2’s prefork model seems to work much better than its worker model, specifically when interacting with mod_perl2. Gathering specific statistics is difficult so this is strictly empirical data. The worker model worked just fine during testing and initial testing. Once I started pushing some real traffic through the system (on the order of 1M requests/day) I began noticing some odd behavior. Internal to my application, I track how much time it took to complete each request, from start to finish. Those times remained consistent throughout the process, but when I tried going through Apache it took a minimum of 20 seconds.

Something was obviously wrong so I started eliminating possible bottlenecks, like network, processor, and memory. All of those checked out fine. I ran ngrep (one damn fine tool, btw) and watched the request. It was hitting Apache, hanging for a while, and then spitting back a response. So I tested against a static file — it was fast. I wrote a small mod_perl2 handler that did nothing but return OK. It hung. So, I figure that something with the request being handed off from Apache2 to mod_perl2 wasn’t working right.

I ran Apache2 through strace but I didn’t see anything enlightening. I googled, read, and googled some more. I tweaked all of the worker mpm settings with no success. There were plenty of processes waiting for connection but no apparent reason for the requests to be delayed. Then I remembered someone on IRC asked me about prefork vs. worker. I had thought, because it was the default when I apt-get installed mod_perl & Apache2, that the worker model was the one mod_perl2 preferred.

So on a hunch I removed the worker mpm in favor of the prefork one. I had to tune the prefork settings a bit from the default. Suddenly everything became responsive. No processes sitting in “sending reply” and lightning fast response times from Apache. Memory usage seems to be lower and almost nil processor usage. It’s working so well, in fact, that I’m afraid to go to bed for fear it’s just my imagination.