MySQL daemons restarting every 7 minutes

Hi,

as of yesterday the MySQL Daemons keep restarting every 7 mins or
so on one of my FreeBSD servers. The only work carried out recently
related to MySQL on this server was to temporarily disable replication
(its a slave) of one DB, and then re-enable it (via restore of data
and updating the log file and pos).
Now I keep seeing this:

110907 18:03:58 mysqld_safe mysqld restarted
110907 18:03:58 [Note] Plugin 'FEDERATED' is disabled.
110907 18:03:58 InnoDB: Initializing buffer pool, size = 2.0G
110907 18:03:59 InnoDB: Completed initialization of buffer pool
110907 18:03:59 InnoDB: Started; log sequence number 0 44233
110907 18:03:59 [Note] Event Scheduler: Loaded 0 events
110907 18:03:59 [Note] Slave SQL thread initialized, starting
replication in log 'mysql-bin.002818' at position 46048, relay log
'./tau-relay-bin.000037' position: 251
110907 18:03:59 [Note] /usr/local/libexec/mysqld: ready for connections.
Version: '5.1.58' socket: '/tmp/mysql.sock' port: 3306 FreeBSD
port: mysql-server-5.1.58
110907 18:03:59 [Note] Slave I/O thread: connected to master
'slave@kappa:3306',replication started in log 'mysql-bin.002818' at
position 46048

Not seeing any other info, such as why the daemons stopped, just this
repeated over and over...
I didn't find much useful info searching on the internet, came up with this:

http://bugs.mysql.com/bug.php?id=26895

but its an unresolved bug.

I have tried, restoring all DBs from a working server and that didn't
work. So wouldnt seem to be related to the contents of the databases.

System is FreeBSD 8.2 amd64, MySQL 5.1.58 (I upgrade this from 5.1.51
as a try and fix it quick approach but no joy).

Any one chip in from experience what this may be?? I'm considering
upgrading to 5.5 to give that a try...

thanks in advance,

Andy.




--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org

Re: MySQL daemons restarting every 7 minutes

--e89a8f647823ac6b0304ac7ad2d8
Content-Type: text/plain; charset=ISO-8859-1

Can yo paste the complete error log, Ram memory size and configuration file
here and make sure the machine has enough memory to run the services.

Check the sys log for what is happening just before the service restart.

On Wed, Sep 7, 2011 at 10:51 PM, wrote:

> Hi,
>
> as of yesterday the MySQL Daemons keep restarting every 7 mins or so on
> one of my FreeBSD servers. The only work carried out recently related to
> MySQL on this server was to temporarily disable replication (its a slave) of
> one DB, and then re-enable it (via restore of data and updating the log file
> and pos).
> Now I keep seeing this:
>
> 110907 18:03:58 mysqld_safe mysqld restarted
> 110907 18:03:58 [Note] Plugin 'FEDERATED' is disabled.
> 110907 18:03:58 InnoDB: Initializing buffer pool, size = 2.0G
> 110907 18:03:59 InnoDB: Completed initialization of buffer pool
> 110907 18:03:59 InnoDB: Started; log sequence number 0 44233
> 110907 18:03:59 [Note] Event Scheduler: Loaded 0 events
> 110907 18:03:59 [Note] Slave SQL thread initialized, starting replication
> in log 'mysql-bin.002818' at position 46048, relay log
> './tau-relay-bin.000037' position: 251
> 110907 18:03:59 [Note] /usr/local/libexec/mysqld: ready for connections.
> Version: '5.1.58' socket: '/tmp/mysql.sock' port: 3306 FreeBSD port:
> mysql-server-5.1.58
> 110907 18:03:59 [Note] Slave I/O thread: connected to master 'slave@kappa:3306',replication
> started in log 'mysql-bin.002818' at position 46048
>
> Not seeing any other info, such as why the daemons stopped, just this
> repeated over and over...
> I didn't find much useful info searching on the internet, came up with
> this:
>
> http://bugs.mysql.com/bug.php?**id=26895
>
> but its an unresolved bug.
>
> I have tried, restoring all DBs from a working server and that didn't work.
> So wouldnt seem to be related to the contents of the databases.
>
> System is FreeBSD 8.2 amd64, MySQL 5.1.58 (I upgrade this from 5.1.51 as a
> try and fix it quick approach but no joy).
>
> Any one chip in from experience what this may be?? I'm considering
> upgrading to 5.5 to give that a try...
>
> thanks in advance,
>
> Andy.
>
>
>
>
> --
> MySQL General Mailing List
> For list archives: http://lists.mysql.com/mysql
> To unsubscribe: http://lists.mysql.com/mysql?**
> unsub=sureshkumarilu@gmail.com
>
>


--
Thanks
Suresh Kuna
MySQL DBA

--e89a8f647823ac6b0304ac7ad2d8--

Re: MySQL daemons restarting every 7 minutes

Hi,

that really is the complete error log, that exact same info gets
repeated over and over, there is zero in the syslog and I get this
behaviour when running with no my.cnf (I do obviously have one but I
tried without and it I still see the prob, so that probably makes
things easier from a troubleshooting perspective)...
The system has loads of free RAM (8GB total).
Really the system isn't giving me much to go in in terms of clues....

Andy.

Quoting Suresh Kuna :

> Can yo paste the complete error log, Ram memory size and configuration file
> here and make sure the machine has enough memory to run the services.
>
> Check the sys log for what is happening just before the service restart.
>




--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org

Re: MySQL daemons restarting every 7 minutes

--20cf303bf65c44526c04ac839ae9
Content-Type: text/plain; charset=ISO-8859-1

can you remove it from service and start it normally using mysqld_safe with
log warnings enabled in the cnf file.

On Fri, Sep 9, 2011 at 4:16 PM, wrote:

> Hi,
>
> that really is the complete error log, that exact same info gets repeated
> over and over, there is zero in the syslog and I get this behaviour when
> running with no my.cnf (I do obviously have one but I tried without and it I
> still see the prob, so that probably makes things easier from a
> troubleshooting perspective)...
> The system has loads of free RAM (8GB total).
> Really the system isn't giving me much to go in in terms of clues....
>
> Andy.
>
>
> Quoting Suresh Kuna :
>
> Can yo paste the complete error log, Ram memory size and configuration
>> file
>> here and make sure the machine has enough memory to run the services.
>>
>> Check the sys log for what is happening just before the service restart.
>>
>>
>
>
>


--
Thanks
Suresh Kuna
MySQL DBA

--20cf303bf65c44526c04ac839ae9--

Re: MySQL daemons restarting every 7 minutes

Yep, but its basically identical to the info in the logs when its
restarting itself. I have upgraded to MySQL 5.5 as of yesterday, so
the error info differs due to the version now. Here is the log output:

110909 17:27:35 mysqld_safe Starting mysqld daemon with databases from
/var/db/mysql
110909 17:27:35 InnoDB: The InnoDB memory heap is disabled
110909 17:27:35 InnoDB: Mutexes and rw_locks use GCC atomic builtins
110909 17:27:35 InnoDB: Compressed tables use zlib 1.2.3
110909 17:27:35 InnoDB: Initializing buffer pool, size = 2.0G
110909 17:27:36 InnoDB: Completed initialization of buffer pool
110909 17:27:36 InnoDB: highest supported file format is Barracuda.
110909 17:27:36 InnoDB: Waiting for the background threads to start
110909 17:27:37 InnoDB: 1.1.8 started; log sequence number 1589339
110909 17:27:37 [Note] Slave SQL thread initialized, starting
replication in log 'mysql-bin.002830' at position 293541, relay log
'./tau-relay-bin.000920' position: 253
110909 17:27:37 [Note] Event Scheduler: Loaded 0 events
110909 17:27:37 [Note] /usr/local/libexec/mysqld: ready for connections.
Version: '5.5.15' socket: '/tmp/mysql.sock' port: 3306 Source distribution
110909 17:27:37 [Note] Slave I/O thread: connected to master
'slave@kappa:3306',replication started in log 'mysql-bin.002830' at
position 293541

Just not giving any hit of a problem....

thanks Andy.

Quoting Suresh Kuna :

> can you remove it from service and start it normally using mysqld_safe with
> log warnings enabled in the cnf file.
>
> On Fri, Sep 9, 2011 at 4:16 PM, wrote:
>
>> Hi,
>>
>> that really is the complete error log, that exact same info gets repeated
>> over and over, there is zero in the syslog and I get this behaviour when
>> running with no my.cnf (I do obviously have one but I tried without and it I
>> still see the prob, so that probably makes things easier from a
>> troubleshooting perspective)...
>> The system has loads of free RAM (8GB total).
>> Really the system isn't giving me much to go in in terms of clues....
>>
>> Andy.





--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org

Re: MySQL daemons restarting every 7 minutes

--------------enigE6019954378DEA9458EABBD2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

jesus christ can you post a WHOLE log
this is only a normal start, there is no single line about stop

Am 09.09.2011 18:30, schrieb a.smith@ukgrid.net:
> Yep, but its basically identical to the info in the logs when its resta=
rting itself. I have upgraded to MySQL 5.5
> as of yesterday, so the error info differs due to the version now. Here=
is the log output:
>=20
> 110909 17:27:35 mysqld_safe Starting mysqld daemon with databases from =
/var/db/mysql
> 110909 17:27:35 InnoDB: The InnoDB memory heap is disabled
> 110909 17:27:35 InnoDB: Mutexes and rw_locks use GCC atomic builtins
> 110909 17:27:35 InnoDB: Compressed tables use zlib 1.2.3
> 110909 17:27:35 InnoDB: Initializing buffer pool, size =3D 2.0G
> 110909 17:27:36 InnoDB: Completed initialization of buffer pool
> 110909 17:27:36 InnoDB: highest supported file format is Barracuda.
> 110909 17:27:36 InnoDB: Waiting for the background threads to start
> 110909 17:27:37 InnoDB: 1.1.8 started; log sequence number 1589339
> 110909 17:27:37 [Note] Slave SQL thread initialized, starting replicati=
on in log 'mysql-bin.002830' at position
> 293541, relay log './tau-relay-bin.000920' position: 253
> 110909 17:27:37 [Note] Event Scheduler: Loaded 0 events
> 110909 17:27:37 [Note] /usr/local/libexec/mysqld: ready for connections=

> Version: '5.5.15' socket: '/tmp/mysql.sock' port: 3306 Source distri=
bution
> 110909 17:27:37 [Note] Slave I/O thread: connected to master 'slave@kap=
pa:3306',replication started in log
> 'mysql-bin.002830' at position 293541
>=20
> Just not giving any hit of a problem....
>=20
> thanks Andy.
>=20
> Quoting Suresh Kuna :
>=20
>> can you remove it from service and start it normally using mysqld_safe=
with
>> log warnings enabled in the cnf file.
>>
>> On Fri, Sep 9, 2011 at 4:16 PM, wrote:
>>
>>> Hi,
>>>
>>> that really is the complete error log, that exact same info gets rep=
eated
>>> over and over, there is zero in the syslog and I get this behaviour w=
hen
>>> running with no my.cnf (I do obviously have one but I tried without a=
nd it I
>>> still see the prob, so that probably makes things easier from a
>>> troubleshooting perspective)...
>>> The system has loads of free RAM (8GB total).
>>> Really the system isn't giving me much to go in in terms of clues....=

>>>
>>> Andy.
>=20
>=20
>=20
>=20
>=20

--=20

Mit besten Grüßen, Reindl Harald
the lounge interactive design GmbH
A-1060 Vienna, Hofmühlgasse 17
CTO / software-development / cms-solutions
p: +43 (1) 595 3999 33, m: +43 (676) 40 221 40
icq: 154546673, http://www.thelounge.net/

http://www.thelounge.net/signature.asc.what.htm


--------------enigE6019954378DEA9458EABBD2
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5qQ6gACgkQhmBjz394AnlObgCdG+JFzwwzKBKWQVN4uPTj +53D
4o8AoJUERFf1/3mHftWFaKOmS2EeBdyc
=4kp1
-----END PGP SIGNATURE-----

--------------enigE6019954378DEA9458EABBD2--

Re: MySQL daemons restarting every 7 minutes

No need for that really is there? I posted what was requested.
The part for the shutdown:

110909 17:27:31 InnoDB: Starting shutdown...
110909 17:27:32 InnoDB: Shutdown completed; log sequence number 1589339
110909 17:27:32 [Note] /usr/local/libexec/mysqld: Shutdown complete

110909 17:27:32 mysqld_safe mysqld from pid file /var/db/mysql/tau.pid ended
110909 17:27:35 mysqld_safe Starting mysqld daemon with databases from
/var/db/mysql
110909 17:27:35 InnoDB: The InnoDB memory heap is disabled
110909 17:27:35 InnoDB: Mutexes and rw_locks use GCC atomic builtins
110909 17:27:35 InnoDB: Compressed tables use zlib 1.2.3
110909 17:27:35 InnoDB: Initializing buffer pool, size = 2.0G
110909 17:27:36 InnoDB: Completed initialization of buffer pool
110909 17:27:36 InnoDB: highest supported file format is Barracuda.
110909 17:27:36 InnoDB: Waiting for the background threads to start
110909 17:27:37 InnoDB: 1.1.8 started; log sequence number 1589339
110909 17:27:37 [Note] Slave SQL thread initialized, starting
replication in log 'mysql-bin.002830' at position 293541, relay log
'./tau-relay-bin.000920' position: 253
110909 17:27:37 [Note] Event Scheduler: Loaded 0 events
110909 17:27:37 [Note] /usr/local/libexec/mysqld: ready for connections.
Version: '5.5.15' socket: '/tmp/mysql.sock' port: 3306 Source distribution
110909 17:27:37 [Note] Slave I/O thread: connected to master
'slave_user@kappa:3306',replication started in log 'mysql-bin.002830'
at position 293541






--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org

Re: MySQL daemons restarting every 7 minutes

--20cf305b0daa77e4f804ac8ea638
Content-Type: text/plain; charset=ISO-8859-1

can you check for any table crashes in the db by using mysqlcheck.

and enable the general log for the database.

On Fri, Sep 9, 2011 at 10:37 PM, wrote:

> No need for that really is there? I posted what was requested.
> The part for the shutdown:
>
> 110909 17:27:31 InnoDB: Starting shutdown...
> 110909 17:27:32 InnoDB: Shutdown completed; log sequence number 1589339
> 110909 17:27:32 [Note] /usr/local/libexec/mysqld: Shutdown complete
>
> 110909 17:27:32 mysqld_safe mysqld from pid file /var/db/mysql/tau.pid
> ended
>
> 110909 17:27:35 mysqld_safe Starting mysqld daemon with databases from
> /var/db/mysql
> 110909 17:27:35 InnoDB: The InnoDB memory heap is disabled
> 110909 17:27:35 InnoDB: Mutexes and rw_locks use GCC atomic builtins
> 110909 17:27:35 InnoDB: Compressed tables use zlib 1.2.3
> 110909 17:27:35 InnoDB: Initializing buffer pool, size = 2.0G
> 110909 17:27:36 InnoDB: Completed initialization of buffer pool
> 110909 17:27:36 InnoDB: highest supported file format is Barracuda.
> 110909 17:27:36 InnoDB: Waiting for the background threads to start
> 110909 17:27:37 InnoDB: 1.1.8 started; log sequence number 1589339
> 110909 17:27:37 [Note] Slave SQL thread initialized, starting replication
> in log 'mysql-bin.002830' at position 293541, relay log
> './tau-relay-bin.000920' position: 253
> 110909 17:27:37 [Note] Event Scheduler: Loaded 0 events
> 110909 17:27:37 [Note] /usr/local/libexec/mysqld: ready for connections.
> Version: '5.5.15' socket: '/tmp/mysql.sock' port: 3306 Source
> distribution
> 110909 17:27:37 [Note] Slave I/O thread: connected to master
> 'slave_user@kappa:3306',**replication started in log 'mysql-bin.002830' at
> position 293541
>
>
>
>
>
>
>
> --
> MySQL General Mailing List
> For list archives: http://lists.mysql.com/mysql
> To unsubscribe: http://lists.mysql.com/mysql?**
> unsub=sureshkumarilu@gmail.com
>
>


--
Thanks
Suresh Kuna
MySQL DBA

--20cf305b0daa77e4f804ac8ea638--

Re: MySQL daemons restarting every 7 minutes

This was me restarting MySQL as was requested by Suresh...

Quoting "Singer X.J. Wang" :

> This doesn't look like a MySQL issue. Verify that there's no rogue scripts
> that shutdowns MySQL...
>
>




--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org

Re: MySQL daemons restarting every 7 minutes

--------------enigDCB22E4DE9A3BECAF8BF7CDF
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

i bet if he stops crond the problem is going away

anyways i have enough of this thread after "the part for the shutdown"
because if peopole are way too stupid to provide full logs (normally
in the first post) even after requested multiple times they should
learn their lessons the hard way.......

Am 10.09.2011 06:33, schrieb Singer X.J. Wang:This doesn't look like a My=
SQL issue. Verify that there's no rogue
scripts that shutdowns MySQL...


On Fri, Sep 9, 2011 at 13:07, net>> wrote:

No need for that really is there? I posted what was requested.
The part for the shutdown:

110909 17:27:31 InnoDB: Starting shutdown...
110909 17:27:32 InnoDB: Shutdown completed; log sequence number 1589=
339
110909 17:27:32 [Note] /usr/local/libexec/mysqld: Shutdown complete

110909 17:27:32 mysqld_safe mysqld from pid file /var/db/mysql/tau.pi=
d ended

110909 17:27:35 mysqld_safe Starting mysqld daemon with databases fro=
m /var/db/mysql
110909 17:27:35 InnoDB: The InnoDB memory heap is disabled
110909 17:27:35 InnoDB: Mutexes and rw_locks use GCC atomic builtins
110909 17:27:35 InnoDB: Compressed tables use zlib 1.2.3
110909 17:27:35 InnoDB: Initializing buffer pool, size =3D 2.0G
110909 17:27:36 InnoDB: Completed initialization of buffer pool
110909 17:27:36 InnoDB: highest supported file format is Barracuda.
110909 17:27:36 InnoDB: Waiting for the background threads to start
110909 17:27:37 InnoDB: 1.1.8 started; log sequence number 1589339
110909 17:27:37 [Note] Slave SQL thread initialized, starting replica=
tion in log 'mysql-bin.002830' at position
293541, relay log './tau-relay-bin.000920' position: 253
110909 17:27:37 [Note] Event Scheduler: Loaded 0 events
110909 17:27:37 [Note] /usr/local/libexec/mysqld: ready for connectio=
ns.
Version: '5.5.15' socket: '/tmp/mysql.sock' port: 3306 Source dist=
ribution
110909 17:27:37 [Note] Slave I/O thread: connected to master 'slave_u=
ser@kappa:3306',replication started in log
'mysql-bin.002830' at position 293541







--=20
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=3Dwang@singerwa=
ng.com




--------------enigDCB22E4DE9A3BECAF8BF7CDF
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5rTvoACgkQhmBjz394Ank94QCeLA6HrlPYkTE2ND/ckGAD G/+7
IZwAn0/KsSIvIsyRCQAGSyLGBQpLX6Q5
=Myx5
-----END PGP SIGNATURE-----

--------------enigDCB22E4DE9A3BECAF8BF7CDF--

Re: MySQL daemons restarting every 7 minutes

Ok, this is pretty odd but I have found the problem.

Today I have repointed all applications to a different DB server, so I
have been free to do any testing on the problem server.

I started by dropping the databases one by one, dropped em all and the
issue persisted.
I stopped crond, even tho Id already looked in crontab and was
satisfied it wasn't the culprit. Problem persisted.
I then removed the mysql data dir, and my.cnf and restarted with a
blank config. Problem still persisted.
Turned on the general log, nothing happening (as nothing legit was
still pointing to the DB on this server). Problem persisted.

Then I thought, what if I have hosts.allow misconfigured and its wide
open maybe a remote system is connecting and messing with it. But
hosts.allow was correct (mysql not listed, so denied by the last
all:all). I tested connecting from a remote server, guess what? Mysql
daemons restarted in exactly the way I was seeing each 7 mins, each
time just by simply running:

mysql -h tau

Odd that, so I added a mysql specific line to the hosts.allow (which
is not necessary as all services are blocked on the last line anyway).
Tested that, no no more crashing. Odd!
The hosts.allow file is the stock FreeBSD updated to not allow all:all
and to use denyhosts (ssh blocked).

Additional info, using tcpdump I could see that the every 7 min
crashes were not in fact caused by any connections from other hosts,
MySQL was crashing regardless.

Still don't know why this would suddenly appear as an issue, or why I
don't see this issue on other identically configured servers... :S

thanks Andy.




--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org

Re: MySQL daemons restarting every 7 minutes

Am 10.09.2011 16:07, schrieb a.smith@ukgrid.net:
> Ok, this is pretty odd but I have found the problem.
>
> Today I have repointed all applications to a different DB server, so I
> have been free to do any testing on the problem server.
>
> I started by dropping the databases one by one, dropped em all and the
> issue persisted.
> I stopped crond, even tho Id already looked in crontab and was satisfied
> it wasn't the culprit. Problem persisted.
> I then removed the mysql data dir, and my.cnf and restarted with a blank
> config. Problem still persisted.
> Turned on the general log, nothing happening (as nothing legit was still
> pointing to the DB on this server). Problem persisted.
>
> Then I thought, what if I have hosts.allow misconfigured and its wide
> open maybe a remote system is connecting and messing with it. But
> hosts.allow was correct (mysql not listed, so denied by the last
> all:all). I tested connecting from a remote server, guess what? Mysql
> daemons restarted in exactly the way I was seeing each 7 mins, each time
> just by simply running:
>
> mysql -h tau
>
> Odd that, so I added a mysql specific line to the hosts.allow (which is
> not necessary as all services are blocked on the last line anyway).
> Tested that, no no more crashing. Odd!
> The hosts.allow file is the stock FreeBSD updated to not allow all:all
> and to use denyhosts (ssh blocked).
>
> Additional info, using tcpdump I could see that the every 7 min crashes
> were not in fact caused by any connections from other hosts, MySQL was
> crashing regardless.
>


What i found odd that your mysqld actualy restarts.
Do you have it in some runlevel ? if yes stop and see
what happens.
If this does not work simple move the mysqld out of he way
and replace it with a script like

#!/bin/sh
echo "mysqld ..." | logger -t TEST


see what happens in /var/log/syslog (you get the idea).

re,
wh




--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org

Re: MySQL daemons restarting every 7 minutes

Quoting walter harms :

>
> What i found odd that your mysqld actualy restarts.
> Do you have it in some runlevel ? if yes stop and see
> what happens.
> If this does not work simple move the mysqld out of he way
> and replace it with a script like
>
> #!/bin/sh
> echo "mysqld ..." | logger -t TEST
>
>
> see what happens in /var/log/syslog (you get the idea).
>

Hi,

its FreeBSD so no run levels other than single user and up/multi
user. And FreeBSD has no Solaris like SMF like monitoring of daemons
so I can be fairly certain that the RC script is not being called by
anything. I do use PSMON but this will advise me if its taken action,
and Ive tested with PSMON stopped also.
As I said I can get it to restart just by doing a remote connect from
another server...

Andy.




--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org

Re: MySQL daemons restarting every 7 minutes

--------------enig1A075D741977DE193A5CA26E
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable



Am 10.09.2011 16:07, schrieb a.smith@ukgrid.net:
> Then I thought, what if I have hosts.allow misconfigured and its wide o=
pen maybe a remote system is connecting and
> messing with it. But hosts.allow was correct (mysql not listed, so deni=
ed by the last all:all). I tested connecting
> from a remote server, guess what? Mysql daemons restarted in exactly th=
e way I was seeing each 7 mins, each time
> just by simply running:
>=20
> mysql -h tau
>=20
> Odd that, so I added a mysql specific line to the hosts.allow=20

who is using hosts.allow for protection instead a firewall in front of th=
e machine
or iptables (linux) / ipf (bsd)?

> As I said I can get it to restart just by doing a remote connect from a=
nother server...

reproduceable or was it a lucky hit once?

if mysqld crashs ypu see normally something in the mysqld-error-log or in=
the syslog
of the machine, until now ypu provided no silngle line about a crash from=
any log





--------------enig1A075D741977DE193A5CA26E
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5rd/4ACgkQhmBjz394AnlPIwCeKPAhAott3g4yxroYcngZ d7M4
RJsAnj5lgXSO82ASxHH+ISYMKlSntXi4
=59uw
-----END PGP SIGNATURE-----

--------------enig1A075D741977DE193A5CA26E--

Re: MySQL daemons restarting every 7 minutes

Am 10.09.2011 16:25, schrieb a.smith@ukgrid.net:
> Quoting walter harms :
>
>>
>> What i found odd that your mysqld actualy restarts.
>> Do you have it in some runlevel ? if yes stop and see
>> what happens.
>> If this does not work simple move the mysqld out of he way
>> and replace it with a script like
>>
>> #!/bin/sh
>> echo "mysqld ..." | logger -t TEST
>>
>>
>> see what happens in /var/log/syslog (you get the idea).
>>
>
> Hi,
>
> its FreeBSD so no run levels other than single user and up/multi user.
> And FreeBSD has no Solaris like SMF like monitoring of daemons so I can
> be fairly certain that the RC script is not being called by anything. I
> do use PSMON but this will advise me if its taken action, and Ive tested
> with PSMON stopped also.
> As I said I can get it to restart just by doing a remote connect from
> another server...
>
Sorry, I am a late starter in the thread ...

What i would like to understand is:
is it a "propper" shutdown ? (So far i understand yes, so the idea of a starter script)
Does it concern target mysqld only ? (therefore the script)
Since my first idea was that something was calling the starter script
we will modify the dummy a bit.

#!/bin/sh
while true
do
echo "mysqld ..." | logger -t TEST
sleep 1m
done

This will make sure that you see a msg every minute. If you trigger something it will try
to kill the script.

I still do not see why it is restarting ... there must be something watching is disappear.
Just to be sure, you do from a remote host: mysql -hHOST -ume -e "show tables" ?
long shot: Do you have LDAP, NIS or so enabled ?

re,
wh

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org

Re: MySQL daemons restarting every 7 minutes

Quoting Reindl Harald :

Odd that, so I added a mysql specific line to the hosts.allow
>
> who is using hosts.allow for protection instead a firewall in front
> of the machine
> or iptables (linux) / ipf (bsd)?
Its used for denyhosts as I mentioned.

>
>> As I said I can get it to restart just by doing a remote connect
>> from another server...
>
> reproduceable or was it a lucky hit once?
>
> if mysqld crashs ypu see normally something in the mysqld-error-log
> or in the syslog
> of the machine, until now ypu provided no silngle line about a crash
> from any log

Yes its reproduceable as I mentioned. I have provided all info from
the mysqld-error-log already, I haven't provided anything from syslog
as there is nothing there, as I mentioned. If you don't believe me I
can't provide it to you...




--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org

Re: MySQL daemons restarting every 7 minutes

Quoting walter harms :

> I still do not see why it is restarting ... there must be something
> watching is disappear.
> Just to be sure, you do from a remote host: mysql -hHOST -ume -e
> "show tables" ?
> long shot: Do you have LDAP, NIS or so enabled ?
>

Ok so made a script as you suggested, and it is called from
mysqld_safe. So that is what is restarting mysqld.

With regard to is it doing a proper shutdown, no it isn't. It restarts
instantly when triggered by a remote connection, and for example I see
no "InnoDB: Starting shutdown..." info logged by mysqld.

I'm not using LDAP or NIS.

Andy.




--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org

Re: MySQL daemons restarting every 7 minutes

Am 10.09.2011 17:32, schrieb a.smith@ukgrid.net:
> Quoting walter harms :
>
>> I still do not see why it is restarting ... there must be something
>> watching is disappear.
>> Just to be sure, you do from a remote host: mysql -hHOST -ume -e "show
>> tables" ?
>> long shot: Do you have LDAP, NIS or so enabled ?
>>
>
> Ok so made a script as you suggested, and it is called from mysqld_safe.
> So that is what is restarting mysqld.
>
when can establish that here is no interference from outside
we need to reduce more since you are running mysqld_safe - that is nice -
can you see the parameters who mysqld is started ?
restart it with the same parameter on the command line and see what happens
the server support a verbos option (never used) perhaps it will tell you more.

> With regard to is it doing a proper shutdown, no it isn't. It restarts
> instantly when triggered by a remote connection, and for example I see
> no "InnoDB: Starting shutdown..." info logged by mysqld.
>
> I'm not using LDAP or NIS.
So it must be a local problem .... (at least it has nothing todo with auth)

re,
wh
>
> Andy.
>
>
>
>

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org

Re: MySQL daemons restarting every 7 minutes

Quoting walter harms :

> restart it with the same parameter on the command line and see what happens
> the server support a verbos option (never used) perhaps it will tell
> you more.
>

I can start mysqld direct from the command line and reproduce the
problem. I checked and it seems the verbose option only works in
conjunction with --help.

Playing around with a couple of tools that are installed on FreeBSD, I
ran via truss and ktrace to see if there was anything obvious before
the process dies but not really. In ktrace you can see that it gets as
far as deciding that the remote connection should not be allowed then
rapidly just dies:


78353 echo GIO fd 1 wrote 44 bytes
"You are not welcome to use mysqld from tau.
"
78353 echo RET writev 44/0x2c
78353 echo CALL sigprocmask(SIG_BLOCK,0x800638400,0x7fffffffebd0)
78353 echo RET sigprocmask 0
78353 echo CALL sigprocmask(SIG_SETMASK,0x800638410,0)
78353 echo RET sigprocmask 0
78353 echo CALL sigprocmask(SIG_BLOCK,0x800638400,0x7fffffffeb80)
78353 echo RET sigprocmask 0
78353 echo CALL sigprocmask(SIG_SETMASK,0x800638410,0)
78353 echo RET sigprocmask 0
78353 echo CALL exit(0)


Interesting also to see it exits with status 0 :S. What would be the
best tool to debug this further?

thanks Andy.




--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org

Re: MySQL daemons restarting every 7 minutes

--------------enig08BA6DEB9480A3DAE08C77CB
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable


Am 10.09.2011 18:52, schrieb a.smith@ukgrid.net:
> Quoting walter harms :
>
>> restart it with the same parameter on the command line and see what ha=
ppens
>> the server support a verbos option (never used) perhaps it will tell y=
ou more.
>>
>
> I can start mysqld direct from the command line and reproduce the probl=
em. I checked and it seems the verbose
> option only works in conjunction with --help.
>
> Playing around with a couple of tools that are installed on FreeBSD, I =
ran via truss and ktrace to see if there
> was anything obvious before the process dies but not really. In ktrace =
you can see that it gets as far as
> deciding that the remote connection should not be allowed then rapidly =
just dies:
>
>
> 78353 echo GIO fd 1 wrote 44 bytes
> "You are not welcome to use mysqld from tau.
> "
> 78353 echo RET writev 44/0x2c
> 78353 echo CALL sigprocmask(SIG_BLOCK,0x800638400,0x7fffffffebd0)=

> 78353 echo RET sigprocmask 0
> 78353 echo CALL sigprocmask(SIG_SETMASK,0x800638410,0)
> 78353 echo RET sigprocmask 0
> 78353 echo CALL sigprocmask(SIG_BLOCK,0x800638400,0x7fffffffeb80)=

> 78353 echo RET sigprocmask 0
> 78353 echo CALL sigprocmask(SIG_SETMASK,0x800638410,0)
> 78353 echo RET sigprocmask 0
> 78353 echo CALL exit(0)
>
>
> Interesting also to see it exits with status 0 :S. What would be the be=
st tool to debug this further?

"You are not welcome to use mysqld from tau" is NOT from mysqld
remove your hosts.allow/hosts.deny crap and replace it with firewall-rule=
s
if the problem goes away make a bugreport on BSD side becahuse this is NO=
T a mysqld issue



--------------enig08BA6DEB9480A3DAE08C77CB
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5rluAACgkQhmBjz394AnlfQACdEHmwAlmoQiw7CHNMPHLM R9lD
4RwAn3RM267LNAkK9Z3sIlffORm21Pim
=+LUR
-----END PGP SIGNATURE-----

--------------enig08BA6DEB9480A3DAE08C77CB--

Re: MySQL daemons restarting every 7 minutes

Quoting Reindl Harald :

>
> "You are not welcome to use mysqld from tau" is NOT from mysqld
> remove your hosts.allow/hosts.deny crap and replace it with firewall-rules
> if the problem goes away make a bugreport on BSD side becahuse this
> is NOT a mysqld issue
>

I've already established that the issue is MySQL in conjuration with
certain config in hosts.allow. I have also resolve my problem. I only
continue looking at this for curiosity, because Walter asked me to do
some other test and to do my part for MySQL and the community. Also
given MySQL supports tcp wrappers, it should work or that code should
be remove from MySQL...




--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org

Re: MySQL daemons restarting every 7 minutes

--------------enig00803350547C026535F8DB91
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable



Am 10.09.2011 19:02, schrieb a.smith@ukgrid.net:
> Quoting Reindl Harald :
>=20
>>
>> "You are not welcome to use mysqld from tau" is NOT from mysqld
>> remove your hosts.allow/hosts.deny crap and replace it with firewall-r=
ules
>> if the problem goes away make a bugreport on BSD side becahuse this is=
NOT a mysqld issue
>>
>=20
> I've already established that the issue is MySQL in conjuration with ce=
rtain config in hosts.allow. I have also
> resolve my problem. I only continue looking at this for curiosity, beca=
use Walter asked me to do some other test
> and to do my part for MySQL and the community. Also given MySQL support=
s tcp wrappers, it should work or that code
> should be remove from MySQL...

REMOVE IT to see if this is the problem

"should be remove from MySQL" is another question and depends on where th=
e error
is triggered, if it is the tcp-wrapper from BSD and not happening on anot=
her OS
there has to be notihing removed from mysql because it is on BSD side

remove and disable all crap around mysqld (tcp-wrappers, crond, watchdogs=
..)
to locate the poblem and if mysqld runs normal after that enable one afte=
r
one to look what exactly is triggering the problem - this is the way debu=
gging
works since computers exists





--------------enig00803350547C026535F8DB91
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5rmmQACgkQhmBjz394AnnTKgCfZ7xPv1cGeMBRx8Im+IVH zkbL
IQIAni3wuYnAOjdzt3kASZitumKuqw0t
=TFFF
-----END PGP SIGNATURE-----

--------------enig00803350547C026535F8DB91--

Re: MySQL daemons restarting every 7 minutes

Hi Walter/all,

ok nailed it, the issue is the default hosts.allow installed on
FreeBSD, and specifically the last section that denies everything. By
default it looks like this:

# The rest of the daemons are protected.
ALL : ALL \
: severity auth.info \
: twist /bin/echo "You are not welcome to use %d from %h."

The twist command breaks it. In theory this is just meant to send a
custom message back to the application calling the tcp wrapper
library. I'm not sure if this should work in theory or not, but the
twist command is also meant to close the connection so possibly the
behaviour I see is normal and just not compatible with MySQL.

Anyway, its not a great default for FreeBSD given that MySQL also
installs by default with support for tcp wrappers. The two together
results in a broken configuration.

thanks Andy.




--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org

Re: MySQL daemons restarting every 7 minutes

--------------enigEE07D708ECF92F274E14FC73
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable



Am 10.09.2011 19:21, schrieb a.smith@ukgrid.net:
> Hi Walter/all,
>=20
> ok nailed it, the issue is the default hosts.allow installed on FreeB=
SD, and specifically the last section that
> denies everything. By default it looks like this:
>=20
> # The rest of the daemons are protected.
> ALL : ALL \
> : severity auth.info \
> : twist /bin/echo "You are not welcome to use %d from %h."
>=20
> The twist command breaks it. In theory this is just meant to send a cus=
tom message back to the application calling
> the tcp wrapper library. I'm not sure if this should work in theory or =
not, but the twist command is also meant to
> close the connection so possibly the behaviour I see is normal and just=
not compatible with MySQL.
>=20
> Anyway, its not a great default for FreeBSD given that MySQL also insta=
lls by default with support for tcp
> wrappers. The two together results in a broken configuration.

well, and that is why i said nobody is using hosts.allow in real life
if you want to protect anything use packet-filters

i have seen so many peopole typing something in hosts.allow and not
realizing that the service is not using tcp-wrappers which means
there is no protection - additionaly most peopole doe snot test
their configurations really well

the point of "not testing configurations" affects you too because
if you would have tested this the issue would have been happened
after the first connection long before go in production


--------------enigEE07D708ECF92F274E14FC73
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5rnqoACgkQhmBjz394Ankc/ACggr65JD5Ch/DXjAOWXATe r1AK
2JsAoJAyO7QXccH+HomdDiQ0cXLwam/g
=aNbW
-----END PGP SIGNATURE-----

--------------enigEE07D708ECF92F274E14FC73--