Slave log files going nuts...

Slave log files going nuts...

am 16.07.2009 22:18:02 von Gary Smith

I have a new slave I setup against a new master server. The master server =
has 4 log files in it, the most current being updated on the 16th. The sla=
ve server on the other hand has several files, many which seem to be blank.=
This slave is set to slave the master and act as a master for downstream =
slaves. Note, there is no master/master on this configuration, even though=
the master itself could do it. =20

Below is the my.cnf file and a listing of the files generated on the slave.=
This portion of the file is unique on both machines. Only differences in =
the file are the server-id and report-host. Both servers are running 5.1.3=
5

Any ideas?

#INNO DB settings
innodb_file_per_table
innodb_flush_log_at_trx_commit =3D 1
innodb_autoextend_increment =3D 2M
innodb_buffer_pool_size =3D 256M
innodb_additional_mem_pool_size =3D 48M
innodb_log_file_size =3D 10M=20
innodb_log_buffer_size =3D 64M
innodb_log_files_in_group =3D 1

log-slave-updates
sync_binlog =3D 1
replicate-same-server-id =3D 0
log_bin_trust_function_creators =3D 1
log-bin =3D /exports/mysql-log/log-repl
binlog-ignore-db =3D mysql=20
binlog-ignore-db =3D information_schema=20
replicate-ignore-db =3D mysql
replicate-ignore-db =3D information_schema
relay-log =3D hs-relay-bin
relay-log-index =3D hs-relay-index
relay-log-info-file =3D hs-relay-info
relay-log-purge =3D 1


-rw-rw---- 1 mysql mysql 125 Jul 11 23:43 log-repl.000001
-rw-rw---- 1 mysql mysql 125 Jul 11 23:43 log-repl.000002
-rw-rw---- 1 mysql mysql 125 Jul 11 23:43 log-repl.000003
-rw-rw---- 1 mysql mysql 2350918 Jul 14 08:33 log-repl.000004
-rw-rw---- 1 mysql mysql 148 Jul 14 08:33 log-repl.000005
-rw-rw---- 1 mysql mysql 148 Jul 14 08:33 log-repl.000006
-rw-rw---- 1 mysql mysql 148 Jul 14 08:33 log-repl.000007
-rw-rw---- 1 mysql mysql 148 Jul 14 08:33 log-repl.000008
-rw-rw---- 1 mysql mysql 148 Jul 14 08:33 log-repl.000009
-rw-rw---- 1 mysql mysql 148 Jul 14 08:33 log-repl.000010
-rw-rw---- 1 mysql mysql 148 Jul 14 08:33 log-repl.000011
-rw-rw---- 1 mysql mysql 148 Jul 14 08:33 log-repl.000012
-rw-rw---- 1 mysql mysql 148 Jul 14 08:33 log-repl.000013
-rw-rw---- 1 mysql mysql 148 Jul 14 08:33 log-repl.000014
-rw-rw---- 1 mysql mysql 117655 Jul 14 11:24 log-repl.000015
-rw-rw---- 1 mysql mysql 148 Jul 14 11:24 log-repl.000016
-rw-rw---- 1 mysql mysql 148 Jul 14 11:24 log-repl.000017
-rw-rw---- 1 mysql mysql 148 Jul 14 11:24 log-repl.000018
-rw-rw---- 1 mysql mysql 148 Jul 14 11:24 log-repl.000019
-rw-rw---- 1 mysql mysql 148 Jul 14 11:24 log-repl.000020
-rw-rw---- 1 mysql mysql 148 Jul 14 11:24 log-repl.000021
-rw-rw---- 1 mysql mysql 148 Jul 14 11:24 log-repl.000022
-rw-rw---- 1 mysql mysql 148 Jul 14 11:24 log-repl.000023
-rw-rw---- 1 mysql mysql 148 Jul 14 11:24 log-repl.000024
-rw-rw---- 1 mysql mysql 148 Jul 14 11:24 log-repl.000025
-rw-rw---- 1 mysql mysql 148 Jul 14 11:25 log-repl.000026
-rw-rw---- 1 mysql mysql 148 Jul 14 11:25 log-repl.000027
-rw-rw---- 1 mysql mysql 148 Jul 14 11:25 log-repl.000028
-rw-rw---- 1 mysql mysql 148 Jul 14 11:25 log-repl.000029
-rw-rw---- 1 mysql mysql 148 Jul 14 11:25 log-repl.000030
-rw-rw---- 1 mysql mysql 148 Jul 14 11:25 log-repl.000031
-rw-rw---- 1 mysql mysql 148 Jul 14 11:25 log-repl.000032
-rw-rw---- 1 mysql mysql 148 Jul 14 11:25 log-repl.000033
-rw-rw---- 1 mysql mysql 148 Jul 14 11:25 log-repl.000034
-rw-rw---- 1 mysql mysql 148 Jul 14 11:25 log-repl.000035
-rw-rw---- 1 mysql mysql 148 Jul 14 11:25 log-repl.000036
-rw-rw---- 1 mysql mysql 148 Jul 14 11:26 log-repl.000037
-rw-rw---- 1 mysql mysql 148 Jul 14 11:26 log-repl.000038
-rw-rw---- 1 mysql mysql 148 Jul 14 11:26 log-repl.000039
-rw-rw---- 1 mysql mysql 148 Jul 14 11:26 log-repl.000040
-rw-rw---- 1 mysql mysql 148 Jul 14 11:26 log-repl.000041
-rw-rw---- 1 mysql mysql 148 Jul 14 11:26 log-repl.000042
-rw-rw---- 1 mysql mysql 148 Jul 14 11:26 log-repl.000043
-rw-rw---- 1 mysql mysql 148 Jul 14 11:26 log-repl.000044
-rw-rw---- 1 mysql mysql 148 Jul 14 11:26 log-repl.000045
-rw-rw---- 1 mysql mysql 148 Jul 14 11:26 log-repl.000046
-rw-rw---- 1 mysql mysql 148 Jul 14 11:26 log-repl.000047
-rw-rw---- 1 mysql mysql 148 Jul 14 11:26 log-repl.000048
-rw-rw---- 1 mysql mysql 148 Jul 14 11:26 log-repl.000049
-rw-rw---- 1 mysql mysql 148 Jul 14 11:26 log-repl.000050
-rw-rw---- 1 mysql mysql 148 Jul 14 11:26 log-repl.000051
-rw-rw---- 1 mysql mysql 148 Jul 14 11:26 log-repl.000052
-rw-rw---- 1 mysql mysql 148 Jul 14 11:26 log-repl.000053
-rw-rw---- 1 mysql mysql 148 Jul 14 11:26 log-repl.000054
-rw-rw---- 1 mysql mysql 148 Jul 14 11:26 log-repl.000055
-rw-rw---- 1 mysql mysql 148 Jul 14 11:26 log-repl.000056
-rw-rw---- 1 mysql mysql 148 Jul 14 11:26 log-repl.000057
-rw-rw---- 1 mysql mysql 148 Jul 14 11:26 log-repl.000058
-rw-rw---- 1 mysql mysql 148 Jul 14 11:27 log-repl.000059
-rw-rw---- 1 mysql mysql 148 Jul 14 11:27 log-repl.000060
-rw-rw---- 1 mysql mysql 148 Jul 14 11:27 log-repl.000061
-rw-rw---- 1 mysql mysql 148 Jul 14 11:27 log-repl.000062
-rw-rw---- 1 mysql mysql 148 Jul 14 11:27 log-repl.000063
-rw-rw---- 1 mysql mysql 148 Jul 14 11:27 log-repl.000064
-rw-rw---- 1 mysql mysql 148 Jul 14 11:27 log-repl.000065
-rw-rw---- 1 mysql mysql 148 Jul 14 11:27 log-repl.000066
-rw-rw---- 1 mysql mysql 148 Jul 14 11:27 log-repl.000067
-rw-rw---- 1 mysql mysql 148 Jul 14 11:27 log-repl.000068
-rw-rw---- 1 mysql mysql 148 Jul 14 11:27 log-repl.000069
-rw-rw---- 1 mysql mysql 128433 Jul 15 01:02 log-repl.000070
-rw-rw---- 1 mysql mysql 148 Jul 15 01:02 log-repl.000071
-rw-rw---- 1 mysql mysql 148 Jul 15 01:02 log-repl.000072
-rw-rw---- 1 mysql mysql 148 Jul 15 01:02 log-repl.000073
-rw-rw---- 1 mysql mysql 148 Jul 15 01:02 log-repl.000074
-rw-rw---- 1 mysql mysql 148 Jul 15 01:02 log-repl.000075
-rw-rw---- 1 mysql mysql 148 Jul 15 01:02 log-repl.000076
-rw-rw---- 1 mysql mysql 148 Jul 15 01:02 log-repl.000077
-rw-rw---- 1 mysql mysql 148 Jul 15 01:02 log-repl.000078
-rw-rw---- 1 mysql mysql 148 Jul 15 01:02 log-repl.000079
-rw-rw---- 1 mysql mysql 148 Jul 15 01:02 log-repl.000080
-rw-rw---- 1 mysql mysql 148 Jul 16 01:02 log-repl.000081
-rw-rw---- 1 mysql mysql 148 Jul 16 01:02 log-repl.000082
-rw-rw---- 1 mysql mysql 148 Jul 16 01:02 log-repl.000083
-rw-rw---- 1 mysql mysql 148 Jul 16 01:02 log-repl.000084
-rw-rw---- 1 mysql mysql 148 Jul 16 01:02 log-repl.000085
-rw-rw---- 1 mysql mysql 148 Jul 16 01:02 log-repl.000086
-rw-rw---- 1 mysql mysql 148 Jul 16 01:02 log-repl.000087
-rw-rw---- 1 mysql mysql 148 Jul 16 01:02 log-repl.000088
-rw-rw---- 1 mysql mysql 148 Jul 16 01:02 log-repl.000089
-rw-rw---- 1 mysql mysql 148 Jul 16 01:02 log-repl.000090
-rw-rw---- 1 mysql mysql 148 Jul 16 01:02 log-repl.000091
-rw-rw---- 1 mysql mysql 27956 Jul 16 13:07 log-repl.000092
-rw-rw---- 1 mysql mysql 3220 Jul 16 01:02 log-repl.index=

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=3Dgcdmg-mysql-2@m.gmane.o rg

Re: Slave log files going nuts...

am 16.07.2009 23:41:15 von Todd Lyons

On Thu, Jul 16, 2009 at 1:18 PM, Gary Smith wrote:
> I have a new slave I setup against a new master server. =A0The master ser=
ver has 4 log files in it, the most current being updated on the 16th. =A0T=
he slave server on the other hand has several files, many which seem to be =
blank. =A0This slave is set to slave the master and act as a master for dow=
nstream slaves. =A0Note, there is no master/master on this configuration, e=
ven though the master itself could do it.
>
> Any ideas?

Something is doing several 'mysqladmin refresh' or a related command,
all sequentially in a row in short order. Look at your cron jobs that
start or end around the time that all those "empty" binlogs are being
created.

--=20
Regards... Todd

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=3Dgcdmg-mysql-2@m.gmane.o rg

RE: Slave log files going nuts...

am 17.07.2009 00:15:52 von Gary Smith

I'll have to look into that. We have a single cronjob that just does a mys=
qldump daily but not during the time of the log file generation, but that's=
all I can think of off the top of my head.

Gary

________________________________________
From: tlyons@sitehelp.org [tlyons@sitehelp.org] On Behalf Of Todd Lyons [tl=
yons@ivenue.com]
Sent: Thursday, July 16, 2009 2:41 PM
To: Gary Smith
Cc: mysql@lists.mysql.com
Subject: Re: Slave log files going nuts...

On Thu, Jul 16, 2009 at 1:18 PM, Gary Smith wrote:
> I have a new slave I setup against a new master server. The master serve=
r has 4 log files in it, the most current being updated on the 16th. The s=
lave server on the other hand has several files, many which seem to be blan=
k. This slave is set to slave the master and act as a master for downstrea=
m slaves. Note, there is no master/master on this configuration, even thou=
gh the master itself could do it.
>
> Any ideas?

Something is doing several 'mysqladmin refresh' or a related command,
all sequentially in a row in short order. Look at your cron jobs that
start or end around the time that all those "empty" binlogs are being
created.

--
Regards... Todd

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=3Dgary@primeexalia.co=
m=

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=3Dgcdmg-mysql-2@m.gmane.o rg

RE: Slave log files going nuts...

am 17.07.2009 23:01:39 von Gavin Towey

The binlogs are closed and reopened every time you do a FLUSH LOGS; command=
, or when the server restarts. Is your server crashing continuously? Take=
a look at your error log as well.

Regards,
Gavin Towey

-----Original Message-----
From: Gary Smith [mailto:Gary@primeexalia.com]
Sent: Thursday, July 16, 2009 3:16 PM
To: Todd Lyons
Cc: mysql@lists.mysql.com
Subject: RE: Slave log files going nuts...

I'll have to look into that. We have a single cronjob that just does a mys=
qldump daily but not during the time of the log file generation, but that's=
all I can think of off the top of my head.

Gary

________________________________________
From: tlyons@sitehelp.org [tlyons@sitehelp.org] On Behalf Of Todd Lyons [tl=
yons@ivenue.com]
Sent: Thursday, July 16, 2009 2:41 PM
To: Gary Smith
Cc: mysql@lists.mysql.com
Subject: Re: Slave log files going nuts...

On Thu, Jul 16, 2009 at 1:18 PM, Gary Smith wrote:
> I have a new slave I setup against a new master server. The master serve=
r has 4 log files in it, the most current being updated on the 16th. The s=
lave server on the other hand has several files, many which seem to be blan=
k. This slave is set to slave the master and act as a master for downstrea=
m slaves. Note, there is no master/master on this configuration, even thou=
gh the master itself could do it.
>
> Any ideas?

Something is doing several 'mysqladmin refresh' or a related command,
all sequentially in a row in short order. Look at your cron jobs that
start or end around the time that all those "empty" binlogs are being
created.

--
Regards... Todd

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=3Dgary@primeexalia.co=
m
--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=3Dgtowey@ffn.com


The information contained in this transmission may contain privileged and c=
onfidential information. It is intended only for the use of the person(s) n=
amed above. If you are not the intended recipient, you are hereby notified =
that any review, dissemination, distribution or duplication of this communi=
cation is strictly prohibited. If you are not the intended recipient, pleas=
e contact the sender by reply email and destroy all copies of the original =
message.

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=3Dgcdmg-mysql-2@m.gmane.o rg

RE: Slave log files going nuts...

am 19.07.2009 21:48:35 von Gary Smith

> -----Original Message-----
> From: Gavin Towey [mailto:gtowey@ffn.com]
> Sent: Friday, July 17, 2009 2:02 PM
> To: Gary Smith; Todd Lyons
> Cc: mysql@lists.mysql.com
> Subject: RE: Slave log files going nuts...
>=20
> The binlogs are closed and reopened every time you do a FLUSH LOGS;
> command, or when the server restarts. Is your server crashing
> continuously? Take a look at your error log as well.
>=20
> Regards,
> Gavin Towey

Gavin/Todd,=20

Tracking it down, the timestamp correlates with a mysqldump that I have a c=
rontab for which of course, has flush-logs.

/usr/bin/mysqldump --opt --flush-logs --all-databases --result-file=3D$FILE=
NAME

I'm not sure if we want to flush the log files on this machine at this poin=
t. It's a tossup since there are multiple slaves hitting this guy replicat=
ing only specific tables in some cases. I would guess that we could always=
restore the entire backup and dump what we want across to a new server bef=
ore starting the incremental replication, but I'm still working that plan u=
p right now.

Anyway, thanks for the pointers to indentify the problem. =20

Gary=20



--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=3Dgcdmg-mysql-2@m.gmane.o rg