How to investigate slow LAMP site?

Hello

A friend of mine is running a LAMP (PHP, MySQL, Apache,
Debian) Web 2.0 site on a dedicated, hosted server with about 300
connected users when it's really busy.

The server is an Intel Celeron 2.4 GHz with 1GB of RAM, a 40GB IDE
drive, and a 10Mbps bandwith.

Here's what top says:

Tasks: 196 total, 6 running, 181 sleeping, 8 stopped, 1 zombie
Cpu(s): 71.6% us, 8.6% sy, 0.0% ni, 58.0% id, 0.8% wa, 0.3% hi, 0.7%
si
Mem: 1023656k total, 963348k used, 60308k free, 119612k buffers
Swap: 514040k total, 0k used, 514040k free, 484324k cached

The site is too slow in the evenings, but we don't have the skills to
check why and what can be done about it.

For all you LAMP experts out there: What things would you try to see
where the bottlenecks are? Is there some kind of check-list that we
could go through?

Thanks for any tip!

Re: How to investigate slow LAMP site?

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Gilles Ganault wrote:
> Hello
>
> A friend of mine is running a LAMP (PHP, MySQL, Apache,
> Debian) Web 2.0 site on a dedicated, hosted server with about 300
> connected users when it's really busy.
>
> The server is an Intel Celeron 2.4 GHz with 1GB of RAM, a 40GB IDE
> drive, and a 10Mbps bandwith.
>
> Here's what top says:
>
> Tasks: 196 total, 6 running, 181 sleeping, 8 stopped, 1 zombie
> Cpu(s): 71.6% us, 8.6% sy, 0.0% ni, 58.0% id, 0.8% wa, 0.3% hi, 0.7%
> si
> Mem: 1023656k total, 963348k used, 60308k free, 119612k buffers
> Swap: 514040k total, 0k used, 514040k free, 484324k cached
>
> The site is too slow in the evenings, but we don't have the skills to
> check why and what can be done about it.
>
> For all you LAMP experts out there: What things would you try to see
> where the bottlenecks are? Is there some kind of check-list that we
> could go through?
>
> Thanks for any tip!

In your code (I'm using PHP for example here, adjust to your language)
disable any output from it and insert echo time(); every now and then.
Once you've run that, consolidate the numbers into something like gnuplot
(I'm guessing you're using linux here) and make a graph to see where the
page takes most of the time.

- --
Brendan Gillatt
brendan {at} brendangillatt {dot} co {dot} uk
http://www.brendangillatt.co.uk
PGP Key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xBACD7433
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (MingW32)

iD8DBQFHC66NkA9dCbrNdDMRAoAkAKCcxc0CnYnoT9ib/2D4q5CdWpur3QCg lPVw
Fm7BxGhrGCMKYGJ7/jLm//4=
=T5Mw
-----END PGP SIGNATURE-----

Re: How to investigate slow LAMP site?

On Tue, 09 Oct 2007 17:38:37 +0100, Brendan Gillatt
wrote:
>In your code (I'm using PHP for example here, adjust to your language)
>disable any output from it and insert echo time(); every now and then.
>Once you've run that, consolidate the numbers into something like gnuplot
>(I'm guessing you're using linux here) and make a graph to see where the
>page takes most of the time.

Thanks for the tip. I'll give it a shot.

Re: How to investigate slow LAMP site?

On Oct 9, 9:10 am, Gilles Ganault wrote:
>
> A friend of mine is running a LAMP (PHP, MySQL, Apache,
> Debian) Web 2.0 site on a dedicated, hosted server with
> about 300 connected users when it's really busy.
>
> The server is an Intel Celeron 2.4 GHz with 1GB of RAM,
> a 40GB IDE drive, and a 10Mbps bandwith.
>
> Here's what top says:
>
> Tasks: 196 total, 6 running, 181 sleeping, 8 stopped, 1 zombie
> Cpu(s): 71.6% us, 8.6% sy, 0.0% ni, 58.0% id, 0.8% wa, 0.3% hi,
> 0.7% si
> Mem: 1023656k total, 963348k used, 60308k free, 119612k buffers
> Swap: 514040k total, 0k used, 514040k free, 484324k cached
>
> The site is too slow in the evenings, but we don't have
> the skills to check why and what can be done about it.

You should consider hiring someone with the skills to give you a
definite answer. Here in the newsgroup, we can hypothesize until we
turn blue, but only actual hands-on work with your application can
tell if the hypotheses we advance are anywhere near reality.

> What things would you try to see where the bottlenecks are?
> Is there some kind of check-list that we could go through?

Since your problems seem to arise during peak hours, and neither
memory nor CPU seems to be overloaded, a natural first guess is that
the holdup is with disk I/O, more specifically, in the database
interaction. What to do about it will depend on your database usage
patterns. If your application is write-intensive, you may be nearing
your hardware's capacity, and the only way to improve performance is
to upgrade the hardware (do check the UPDATE queries though; you may
be able to improve their performance by optimizing them). If your
application is not write-intensive, there are several possible avenues
to pursue: (1) optimize SELECT queries (adding an index here and there
may be necessary), (2) turn on query cache (which may help if you have
a news site, but not likely to improve things if you have a social
networking site), (3) increase mysql.max_links (which in turn will
increase your memory requirements). Do check your MySQL connection
method just in case; the best performance on a single server is
achieved by using socket connection rather than a TCP/IP connection.

Cheers,
NC

Re: How to investigate slow LAMP site?

On 9 Oct, 17:10, Gilles Ganault wrote:
> Hello
>
> A friend of mine is running a LAMP (PHP, MySQL, Apache,
> Debian) Web 2.0 site on a dedicated, hosted server with about 300
> connected users when it's really busy.
>
> The server is an Intel Celeron 2.4 GHz with 1GB of RAM, a 40GB IDE
> drive, and a 10Mbps bandwith.
>
> Here's what top says:
>
> Tasks: 196 total, 6 running, 181 sleeping, 8 stopped, 1 zombie
> Cpu(s): 71.6% us, 8.6% sy, 0.0% ni, 58.0% id, 0.8% wa, 0.3% hi, 0.7%
> si
> Mem: 1023656k total, 963348k used, 60308k free, 119612k buffers
> Swap: 514040k total, 0k used, 514040k free, 484324k cached
>
> The site is too slow in the evenings, but we don't have the skills to
> check why and what can be done about it.
>
> For all you LAMP experts out there: What things would you try to see
> where the bottlenecks are? Is there some kind of check-list that we
> could go through?
>
> Thanks for any tip!

Measuring bottlenecks on any system is kind of difficult. Its a lot
easier to fix the stuff which is running slowly (not the same thing).

1) set an appropriate value for the slow query log and see what's
causing the pain there (or implement you own logging for each SQL call
which will also simplify consolidation if you 'anonymize' the queries
- e.g. is query is 'SELECT * FROM widgets WHERE id={$requested_id}'
then you can log the non-interpolated version of the query)
2) fix the caching headers for static content. It's probably wrong.
3) check that you've got compression on output (mod_gzip for Apache/
static content, set_output_handler(...) for PHP which generates large
files).
3) identify PHP output which can be cached on browsers - fix the
headers (IIRC 'Vary:' breaks some MSIE)
4) identify PHP output which can be cached on the server and write
appropriate wrappers or set up a reverse proxy
4) snarf the output of 1, identify what is using up most database time
and see if it can be tuned (NB you will get more mileage from fixing a
query which takes 3 seconds and runs 10000 times a day than one which
takes 30 seconds and runs 10 times a day)
5) install a PHP accelerator if you don't already have one.


Maybe you really are reaching the limits of what you can do on one
box. But I'd be surprised if that's the case.

C.

Re: How to investigate slow LAMP site?

..oO(Gilles Ganault)

>For all you LAMP experts out there: What things would you try to see
>where the bottlenecks are? Is there some kind of check-list that we
>could go through?

Have a look at the Xdebug extension and profile the code.

http://xdebug.org/
http://xdebug.org/docs/profiler

Micha

Re: How to investigate slow LAMP site?

On Wed, 10 Oct 2007 20:01:43 +0200, Michael Fesser
wrote:
>Have a look at the Xdebug extension and profile the code.

Thanks everyone for the feedback. We'll go through this and see how it
goes.