Replication recovery

I have 2 - 5.0.51a mysql databases setup in a dual master scenario. A is
master of B and vise versa...
In Linux 2.6.26 (if that matters).
Everything is great while all is running normally. But, when I am
testing the system by creating disasterous scenarios, I find some
challenges I hope to get overcome.

Let's say 'A' machine's plug gets kicked out of the wall and so when
mysql restarts it starts fresh bin-log and the slave 'B' does not
realize this change and we are now out of sync. 'A', however will simply
catch up to 'B' and there MAY not be a problem.

Even worse, 'A' dies and no one does anything about it, then later 'B'
dies. Now Someone finally comes along and restarts both machines at the
same time and neither are on the 'same page' and are totally out of sync.

How, without re-copying the datafiles and starting over (after
determining the most up to date machine to use), can I bring both 'A'
and 'B' to the same point so I can move forward?

Thanks,

Bryancan

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org

RE: Replication recovery

In the case that one machine has a power failure, then starts a new binlog,=
you just have to set the slave to start replicating from the beginning of =
that binlog. That's easy to detect and repair with a daemon script. Even =
if both machines die, it'll be a similar scenario.

Regards,
Gavin Towey

-----Original Message-----
From: Bryan Cantwell [mailto:bcantwell@firescope.com]
Sent: Wednesday, October 07, 2009 10:47 AM
To: mysql@lists.mysql.com
Subject: Replication recovery

I have 2 - 5.0.51a mysql databases setup in a dual master scenario. A is
master of B and vise versa...
In Linux 2.6.26 (if that matters).
Everything is great while all is running normally. But, when I am
testing the system by creating disasterous scenarios, I find some
challenges I hope to get overcome.

Let's say 'A' machine's plug gets kicked out of the wall and so when
mysql restarts it starts fresh bin-log and the slave 'B' does not
realize this change and we are now out of sync. 'A', however will simply
catch up to 'B' and there MAY not be a problem.

Even worse, 'A' dies and no one does anything about it, then later 'B'
dies. Now Someone finally comes along and restarts both machines at the
same time and neither are on the 'same page' and are totally out of sync.

How, without re-copying the datafiles and starting over (after
determining the most up to date machine to use), can I bring both 'A'
and 'B' to the same point so I can move forward?

Thanks,

Bryancan

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=3Dgtowey@ffn.com


The information contained in this transmission may contain privileged and c=
onfidential information. It is intended only for the use of the person(s) n=
amed above. If you are not the intended recipient, you are hereby notified =
that any review, dissemination, distribution or duplication of this communi=
cation is strictly prohibited. If you are not the intended recipient, pleas=
e contact the sender by reply email and destroy all copies of the original =
message.

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=3Dgcdmg-mysql-2@m.gmane.o rg

Re: Replication recovery

When only one machine dies I do send the new master position info to the
still running slave, and yes, it does the trick.
My main challenge is when A dies and is dead for 24 hours and then B
dies too. Now A is already out of synch with B and now B has a new log
position... Doesnt this make A now have a huge gap in data? How do I get
A up to date with B?

thx,
Bryancan

On 10/07/2009 12:53 PM, Gavin Towey wrote:
> In the case that one machine has a power failure, then starts a new binlog, you just have to set the slave to start replicating from the beginning of that binlog. That's easy to detect and repair with a daemon script. Even if both machines die, it'll be a similar scenario.
>
> Regards,
> Gavin Towey
>
> -----Original Message-----
> From: Bryan Cantwell [mailto:bcantwell@firescope.com]
> Sent: Wednesday, October 07, 2009 10:47 AM
> To: mysql@lists.mysql.com
> Subject: Replication recovery
>
> I have 2 - 5.0.51a mysql databases setup in a dual master scenario. A is
> master of B and vise versa...
> In Linux 2.6.26 (if that matters).
> Everything is great while all is running normally. But, when I am
> testing the system by creating disasterous scenarios, I find some
> challenges I hope to get overcome.
>
> Let's say 'A' machine's plug gets kicked out of the wall and so when
> mysql restarts it starts fresh bin-log and the slave 'B' does not
> realize this change and we are now out of sync. 'A', however will simply
> catch up to 'B' and there MAY not be a problem.
>
> Even worse, 'A' dies and no one does anything about it, then later 'B'
> dies. Now Someone finally comes along and restarts both machines at the
> same time and neither are on the 'same page' and are totally out of sync.
>
> How, without re-copying the datafiles and starting over (after
> determining the most up to date machine to use), can I bring both 'A'
> and 'B' to the same point so I can move forward?
>
> Thanks,
>
> Bryancan
>
> --
> MySQL General Mailing List
> For list archives: http://lists.mysql.com/mysql
> To unsubscribe: http://lists.mysql.com/mysql?unsub=gtowey@ffn.com
>
>
> The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.
>


--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org

RE: Replication recovery

B should be the only one with a bad replication position, since it was repl=
icating when A crashed. So just adjust B, and A should catch up as normal =
(provided you have the last 24 hours of binlogs on B for A to read )

Regards,
Gavin Towey

-----Original Message-----
From: Bryan Cantwell [mailto:bcantwell@firescope.com]
Sent: Wednesday, October 07, 2009 11:12 AM
To: Gavin Towey
Cc: mysql@lists.mysql.com
Subject: Re: Replication recovery

When only one machine dies I do send the new master position info to the
still running slave, and yes, it does the trick.
My main challenge is when A dies and is dead for 24 hours and then B
dies too. Now A is already out of synch with B and now B has a new log
position... Doesnt this make A now have a huge gap in data? How do I get
A up to date with B?

thx,
Bryancan

On 10/07/2009 12:53 PM, Gavin Towey wrote:
> In the case that one machine has a power failure, then starts a new binlo=
g, you just have to set the slave to start replicating from the beginning o=
f that binlog. That's easy to detect and repair with a daemon script. Eve=
n if both machines die, it'll be a similar scenario.
>
> Regards,
> Gavin Towey
>
> -----Original Message-----
> From: Bryan Cantwell [mailto:bcantwell@firescope.com]
> Sent: Wednesday, October 07, 2009 10:47 AM
> To: mysql@lists.mysql.com
> Subject: Replication recovery
>
> I have 2 - 5.0.51a mysql databases setup in a dual master scenario. A is
> master of B and vise versa...
> In Linux 2.6.26 (if that matters).
> Everything is great while all is running normally. But, when I am
> testing the system by creating disasterous scenarios, I find some
> challenges I hope to get overcome.
>
> Let's say 'A' machine's plug gets kicked out of the wall and so when
> mysql restarts it starts fresh bin-log and the slave 'B' does not
> realize this change and we are now out of sync. 'A', however will simply
> catch up to 'B' and there MAY not be a problem.
>
> Even worse, 'A' dies and no one does anything about it, then later 'B'
> dies. Now Someone finally comes along and restarts both machines at the
> same time and neither are on the 'same page' and are totally out of sync.
>
> How, without re-copying the datafiles and starting over (after
> determining the most up to date machine to use), can I bring both 'A'
> and 'B' to the same point so I can move forward?
>
> Thanks,
>
> Bryancan
>
> --
> MySQL General Mailing List
> For list archives: http://lists.mysql.com/mysql
> To unsubscribe: http://lists.mysql.com/mysql?unsub=3Dgtowey@ffn.com
>
>
> The information contained in this transmission may contain privileged and=
confidential information. It is intended only for the use of the person(s)=
named above. If you are not the intended recipient, you are hereby notifie=
d that any review, dissemination, distribution or duplication of this commu=
nication is strictly prohibited. If you are not the intended recipient, ple=
ase contact the sender by reply email and destroy all copies of the origina=
l message.
>


The information contained in this transmission may contain privileged and c=
onfidential information. It is intended only for the use of the person(s) n=
amed above. If you are not the intended recipient, you are hereby notified =
that any review, dissemination, distribution or duplication of this communi=
cation is strictly prohibited. If you are not the intended recipient, pleas=
e contact the sender by reply email and destroy all copies of the original =
message.

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=3Dgcdmg-mysql-2@m.gmane.o rg