raid1 with rotating offsite disks for backup

I'm planning a backup system for my home server and have run into a question I can't find answered in the mailing list archives or the wiki. Here's the plan:

1. Install system and valuable data on a 3-disk raid1 array (call the disks A, B, and C).
2. Remove disk C, put it offsite. ("offsite" is moderately time-consuming to get to.)
3a. Periodically, remove disk B, take it offsite, and retrieve disk C
3b. Insert disk C, which will be re-synced to gain any changes made since it was removed.
4. Repeat steps 3a and 3b indefinitely, alternating the roles of disks B and C.

Thus I hope to get continuous protection against a single drive failure and protection back to the last offsite swap for corrupted or deleted data.

My questions are:

In step 3b, when a disk that was a member of the array in the past but has been removed for a while is re-inserted into the 3-disk array, how does the raid system know to update C with A's contents and not A with C's contents? Is there a timestamp involved, and if so, how can I examine it before syncing?

Is it important to always rotate disks B and C, leaving one that never leaves the computer, or does it make no difference which of the two live disks I pluck out to swap with the offsite disk when I make the trip? Can all three disks take turns offsite, so that they all have the same duty cycle?

I saw in another list message the advice to use two stacked raid1s for this application: http://marc.info/?l=linux-raid&m=126761399008775&w=2
> Also, if you want two rotating backups I would create two stacked raid1s.
>
> mdadm -C /dev/md0 -l1 -n2 -b internal /dev/main-device /dev/first-backup
> mdadm -C /dev/md1 -l1 -n2 -b internal /dev/md0 /dev/second-backup
> mkfs -j /dev/md1


Are there important differences between the single 3-disk raid1 array I'm planning to use and this stacked configuration?

Thanks for any help you can offer.

Jeff


--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: raid1 with rotating offsite disks for backup

On Mon, 7 Feb 2011 15:53:46 -0800 Jeff Klingner wrote:

> I'm planning a backup system for my home server and have run into a question I can't find answered in the mailing list archives or the wiki. Here's the plan:
>
> 1. Install system and valuable data on a 3-disk raid1 array (call the disks A, B, and C).
> 2. Remove disk C, put it offsite. ("offsite" is moderately time-consuming to get to.)
> 3a. Periodically, remove disk B, take it offsite, and retrieve disk C
> 3b. Insert disk C, which will be re-synced to gain any changes made since it was removed.
> 4. Repeat steps 3a and 3b indefinitely, alternating the roles of disks B and C.
>
> Thus I hope to get continuous protection against a single drive failure and protection back to the last offsite swap for corrupted or deleted data.
>
> My questions are:
>
> In step 3b, when a disk that was a member of the array in the past but has been removed for a while is re-inserted into the 3-disk array, how does the raid system know to update C with A's contents and not A with C's contents? Is there a timestamp involved, and if so, how can I examine it before syncing?
>
> Is it important to always rotate disks B and C, leaving one that never leaves the computer, or does it make no difference which of the two live disks I pluck out to swap with the offsite disk when I make the trip? Can all three disks take turns offsite, so that they all have the same duty cycle?
>
> I saw in another list message the advice to use two stacked raid1s for this application: http://marc.info/?l=linux-raid&m=126761399008775&w=2
> > Also, if you want two rotating backups I would create two stacked raid1s.
> >
> > mdadm -C /dev/md0 -l1 -n2 -b internal /dev/main-device /dev/first-backup
> > mdadm -C /dev/md1 -l1 -n2 -b internal /dev/md0 /dev/second-backup
> > mkfs -j /dev/md1
>
>
> Are there important differences between the single 3-disk raid1 array I'm planning to use and this stacked configuration?

Yes. The single 3-disk RAID1 array won't work, the stacked configuration
will.


md can resync 'just the changed blocks' by using the 'write intent bitmap'
and event counters.
However it only clears the bits in the bitmap when the array is not degraded.
In your suggestion the 3-drive RAID1 is always degraded so bits are never
cleared, so each resync is effectively a complete resync.

In the stacked configuration there are two bitmaps so md can track of
different differences on two different devices.

You can see the event counts on each device with

mdadm --examine /dev/A

The bitmap knows whether a device is new enough to be safely recovered based
on the bitmap by storing the event count of the time a bit was cleared
'Events Cleared'.

Hope that helps.

NeilBrown

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: raid1 with rotating offsite disks for backup

On Feb 7, 2011, at 4:17 PM, NeilBrown wrote:

> On Mon, 7 Feb 2011 15:53:46 -0800 Jeff Klingner wrote:
>
>> I'm planning a backup system for my home server and have run into a question I can't find answered in the mailing list archives or the wiki. Here's the plan:
>>
>> 1. Install system and valuable data on a 3-disk raid1 array (call the disks A, B, and C).
>> 2. Remove disk C, put it offsite. ("offsite" is moderately time-consuming to get to.)
>> 3a. Periodically, remove disk B, take it offsite, and retrieve disk C
>> 3b. Insert disk C, which will be re-synced to gain any changes made since it was removed.
>> 4. Repeat steps 3a and 3b indefinitely, alternating the roles of disks B and C.
>>
>> Thus I hope to get continuous protection against a single drive failure and protection back to the last offsite swap for corrupted or deleted data.
>>
>> I saw in another list message the advice to use two stacked raid1s for this application: http://marc.info/?l=linux-raid&m=126761399008775&w=2
>>> Also, if you want two rotating backups I would create two stacked raid1s.
>>>
>>> mdadm -C /dev/md0 -l1 -n2 -b internal /dev/main-device /dev/first-backup
>>> mdadm -C /dev/md1 -l1 -n2 -b internal /dev/md0 /dev/second-backup
>>> mkfs -j /dev/md1
>>
>>
>> Are there important differences between the single 3-disk raid1 array I'm planning to use and this stacked configuration?
>
> Yes. The single 3-disk RAID1 array won't work, the stacked configuration
> will.
>

Goodness, I'm glad I asked! Thank you! I've already done step 1. Do you know if I can transition to the stacked configuration without erasing the disks? That is, generally speaking, can I change the logical arrangement of the devices in raid1 array(s), then re-sync and keep the current (lvm) storage system built on top of the raid intact?

Jeff



--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: raid1 with rotating offsite disks for backup

On Mon, 7 Feb 2011 17:03:55 -0800 Jeff Klingner wrote:

>
> On Feb 7, 2011, at 4:17 PM, NeilBrown wrote:
>
> > On Mon, 7 Feb 2011 15:53:46 -0800 Jeff Klingner wrote:
> >
> >> I'm planning a backup system for my home server and have run into a question I can't find answered in the mailing list archives or the wiki. Here's the plan:
> >>
> >> 1. Install system and valuable data on a 3-disk raid1 array (call the disks A, B, and C).
> >> 2. Remove disk C, put it offsite. ("offsite" is moderately time-consuming to get to.)
> >> 3a. Periodically, remove disk B, take it offsite, and retrieve disk C
> >> 3b. Insert disk C, which will be re-synced to gain any changes made since it was removed.
> >> 4. Repeat steps 3a and 3b indefinitely, alternating the roles of disks B and C.
> >>
> >> Thus I hope to get continuous protection against a single drive failure and protection back to the last offsite swap for corrupted or deleted data.
> >>
> >> I saw in another list message the advice to use two stacked raid1s for this application: http://marc.info/?l=linux-raid&m=126761399008775&w=2
> >>> Also, if you want two rotating backups I would create two stacked raid1s.
> >>>
> >>> mdadm -C /dev/md0 -l1 -n2 -b internal /dev/main-device /dev/first-backup
> >>> mdadm -C /dev/md1 -l1 -n2 -b internal /dev/md0 /dev/second-backup
> >>> mkfs -j /dev/md1
> >>
> >>
> >> Are there important differences between the single 3-disk raid1 array I'm planning to use and this stacked configuration?
> >
> > Yes. The single 3-disk RAID1 array won't work, the stacked configuration
> > will.
> >
>
> Goodness, I'm glad I asked! Thank you! I've already done step 1. Do you know if I can transition to the stacked configuration without erasing the disks? That is, generally speaking, can I change the logical arrangement of the devices in raid1 array(s), then re-sync and keep the current (lvm) storage system built on top of the raid intact?
>
> Jeff
>
>

I'm surprised that you found the correct advise, and chose not to follow
it .....


The array you will end up installing on is (slightly) smaller than than the
array you have created. So no: you cannot re-arrange things - you have to
start again.


(well, to be honest, it is possible that you could resize whatever is on the
array and then build the secondary array and make it work. But that path is
error-prone and this margin is not large enough to properly describe the
process).

NeilBrown
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: raid1 with rotating offsite disks for backup

>>>> Also, if you want two rotating backups I would create two stacked raid1s.
>>>>
>>>> mdadm -C /dev/md0 -l1 -n2 -b internal /dev/main-device /dev/first-backup
>>>> mdadm -C /dev/md1 -l1 -n2 -b internal /dev/md0 /dev/second-backup
>>>> mkfs -j /dev/md1



On Feb 7, 2011, at 5:19 PM, NeilBrown wrote:
> I'm surprised that you found the correct advise, and chose not to follow
> it .....

Thanks again for your advice. I didn't search the list until after completing my initial setup; my mistake and my lesson learned: research first.

> The array you will end up installing on is (slightly) smaller than than the
> array you have created. So no: you cannot re-arrange things - you have to
> start again.
>
>
> (well, to be honest, it is possible that you could resize whatever is on the
> array and then build the secondary array and make it work. But that path is
> error-prone and this margin is not large enough to properly describe the
> process).

Reckless fool that I am, I'm going to try it anyway. Here's my rough plan:

1. Degrade the 3-disk raid1 array by removing two disks, leaving only one, and leaving the array running.
2. Set up the first 2-disk array (md0 in your description) de novo on the two freed disks
3. Move my data from the original raid1 array (with only 1 of three disks present) to the new 2-disk md0 via lvm mirroring
4. Free the third disk by finally taking down the original raid1 array
5. create the second 2-device array (md1) using md0 and the freed third disk. Somehow ensure that the md0 component has primacy for syncing.

Jeff

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: raid1 with rotating offsite disks for backup

On Mon, 7 Feb 2011 17:32:48 -0800 Jeff Klingner wrote:

> >>>> Also, if you want two rotating backups I would create two stacked raid1s.
> >>>>
> >>>> mdadm -C /dev/md0 -l1 -n2 -b internal /dev/main-device /dev/first-backup
> >>>> mdadm -C /dev/md1 -l1 -n2 -b internal /dev/md0 /dev/second-backup
> >>>> mkfs -j /dev/md1
>
>
>
> On Feb 7, 2011, at 5:19 PM, NeilBrown wrote:
> > I'm surprised that you found the correct advise, and chose not to follow
> > it .....
>
> Thanks again for your advice. I didn't search the list until after completing my initial setup; my mistake and my lesson learned: research first.
>
> > The array you will end up installing on is (slightly) smaller than than the
> > array you have created. So no: you cannot re-arrange things - you have to
> > start again.
> >
> >
> > (well, to be honest, it is possible that you could resize whatever is on the
> > array and then build the secondary array and make it work. But that path is
> > error-prone and this margin is not large enough to properly describe the
> > process).
>
> Reckless fool that I am, I'm going to try it anyway. Here's my rough plan:
>
> 1. Degrade the 3-disk raid1 array by removing two disks, leaving only one, and leaving the array running.
> 2. Set up the first 2-disk array (md0 in your description) de novo on the two freed disks
> 3. Move my data from the original raid1 array (with only 1 of three disks present) to the new 2-disk md0 via lvm mirroring

Nope. That would be wrong. You need to create both mirrors before moving
any data.
So:

3. Set up the second 2-disk array (md1) as a degraded array containing
only md0 as a member:

mdadm --create /dev/md1 -l1 -n2 -b internal /dev/md0 missing

3a. move my data from the original raid1 to the new md1 using pvmove

> 4. Free the third disk by finally taking down the original raid1 array
> 5. create the second 2-device array (md1) using md0 and the freed third disk. Somehow ensure that the md0 component has primacy for syncing.

Then step 5 becomes:
5. Add this newly freed disk to the degraded md1:
mdadm /dev/md1 --add /dev/whatever

NeilBrown



>
> Jeff

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: raid1 with rotating offsite disks for backup

Jeff Klingner wrote on Mon, Feb 07, 2011 at 03:53:46PM -0800:
> I'm planning a backup system for my home server and have run into a question I can't find answered in the mailing list archives or the wiki. Here's the plan:
>
> 1. Install system and valuable data on a 3-disk raid1 array (call the disks A, B, and C).
> 2. Remove disk C, put it offsite. ("offsite" is moderately time-consuming to get to.)
> 3a. Periodically, remove disk B, take it offsite, and retrieve disk C
> 3b. Insert disk C, which will be re-synced to gain any changes made since it was removed.
> 4. Repeat steps 3a and 3b indefinitely, alternating the roles of disks B and C.
>
> Thus I hope to get continuous protection against a single drive failure and protection back to the last offsite swap for corrupted or deleted data.

You are aware that this will only work reliably if at the point of
time when you remove the disk the filesystem(s) on it are one of:
- mounted readonly
- unmounted
- machine is off

Linux doesn't really have a `umount -f`, so the first two options only
work if you can get rid of all processes that might want to hold on to
the filesystem at the time when you want to remove your disk. A
possible hack is going through a NFS mount which does support forceful
operations on the filesystem in Linux.

As has been pointed out, you don't gain much from the added
complexity. If you would just rsync to one of the spare drives you
would only copy over what actually changed, and not do a full re-sync
of all blocks. And that works fine with the source filesystem being
mounted read-write.

Another problem is that you are temporarily screwed if disk A dies
while re-syncing B, since C isn't with you, A is hosed and B is
half-synced.

What you do lose is that the raid1 based solution would keep the new
disk up-to-date with then-new file disk writes. But the problem with
filesystem status is hard to solve. Overall going 4 disks with raid1
local and having two disks that are rsynced on demand is what I would
do.

Martin
--
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%
Martin Cracauer http://www.cons.org/cracauer/
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: raid1 with rotating offsite disks for backup

i think you could use snapshot at filesystem or lvm level
use a backup disk (it's don't have many operations)
on raid use a write-mostly and the other just for write
the first disk to crash is the read/write (more operations)
the second: write-mostly
the third the snapshot/rsync/backup

it's a probability not a real situation... but it's nice =3D]
if you don't want performace :)

2011/2/8 Martin Cracauer :
> Jeff Klingner wrote on Mon, Feb 07, 2011 at 03:53:46PM -0800:
>> I'm planning a backup system for my home server and have run into a =
question I can't find answered in the mailing list archives or the wiki=
=A0Here's the plan:
>>
>> 1. Install system and valuable data on a 3-disk raid1 array (call th=
e disks A, B, and C).
>> 2. Remove disk C, put it offsite. =A0("offsite" is moderately time-c=
onsuming to get to.)
>> 3a. Periodically, remove disk B, take it offsite, and retrieve disk =
C
>> 3b. Insert disk C, which will be re-synced to gain any changes made =
since it was removed.
>> 4. Repeat steps 3a and 3b indefinitely, alternating the roles of dis=
ks B and C.
>>
>> Thus I hope to get continuous protection against a single drive fail=
ure and protection back to the last offsite swap for corrupted or delet=
ed data.
>
> You are aware that this will only work reliably if at the point of
> time when you remove the disk the filesystem(s) on it are one of:
> - mounted readonly
> - unmounted
> - machine is off
>
> Linux doesn't really have a `umount -f`, so the first two options onl=
y
> work if you can get rid of all processes that might want to hold on t=
o
> the filesystem at the time when you want to remove your disk. =A0A
> possible hack is going through a NFS mount which does support forcefu=
l
> operations on the filesystem in Linux.
>
> As has been pointed out, you don't gain much from the added
> complexity. =A0If you would just rsync to one of the spare drives you
> would only copy over what actually changed, and not do a full re-sync
> of all blocks. =A0And that works fine with the source filesystem bein=
g
> mounted read-write.
>
> Another problem is that you are temporarily screwed if disk A dies
> while re-syncing B, since C isn't with you, A is hosed and B is
> half-synced.
>
> What you do lose is that the raid1 based solution would keep the new
> disk up-to-date with then-new file disk writes. =A0But the problem wi=
th
> filesystem status is hard to solve. =A0Overall going 4 disks with rai=
d1
> local and having two disks that are rsynced on demand is what I would
> do.
>
> Martin
> --
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%
> Martin Cracauer =A0 http://www.cons.org/cracauer/
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid"=
in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
>



--=20
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

RE: raid1 with rotating offsite disks for backup

> -----Original Message-----
> From: linux-raid-owner@vger.kernel.org [mailto:linux-raid-
> owner@vger.kernel.org] On Behalf Of NeilBrown
> Sent: Monday, February 07, 2011 6:18 PM
> To: Jeff Klingner
> Cc: linux-raid@vger.kernel.org
> Subject: Re: raid1 with rotating offsite disks for backup
>
> On Mon, 7 Feb 2011 15:53:46 -0800 Jeff Klingner
> wrote:
>
> > I'm planning a backup system for my home server and have run into a
> question I can't find answered in the mailing list archives or the wiki.
> Here's the plan:
> >
> > 1. Install system and valuable data on a 3-disk raid1 array (call the
> disks A, B, and C).
> > 2. Remove disk C, put it offsite. ("offsite" is moderately time-
> consuming to get to.)
> > 3a. Periodically, remove disk B, take it offsite, and retrieve disk C
> > 3b. Insert disk C, which will be re-synced to gain any changes made
> since it was removed.
> > 4. Repeat steps 3a and 3b indefinitely, alternating the roles of disks B
> and C.
> >
> > Thus I hope to get continuous protection against a single drive failure
> and protection back to the last offsite swap for corrupted or deleted
> data.
> >
> > My questions are:
> >
> > In step 3b, when a disk that was a member of the array in the past but
> has been removed for a while is re-inserted into the 3-disk array, how
> does the raid system know to update C with A's contents and not A with C's
> contents? Is there a timestamp involved, and if so, how can I examine it
> before syncing?
> >
> > Is it important to always rotate disks B and C, leaving one that never
> leaves the computer, or does it make no difference which of the two live
> disks I pluck out to swap with the offsite disk when I make the trip? Can
> all three disks take turns offsite, so that they all have the same duty
> cycle?
> >
> > I saw in another list message the advice to use two stacked raid1s for
> this application: http://marc.info/?l=linux-raid&m=126761399008775&w=2
> > > Also, if you want two rotating backups I would create two stacked
> raid1s.
> > >
> > > mdadm -C /dev/md0 -l1 -n2 -b internal /dev/main-device /dev/first-
> backup
> > > mdadm -C /dev/md1 -l1 -n2 -b internal /dev/md0 /dev/second-backup
> > > mkfs -j /dev/md1
> >
> >
> > Are there important differences between the single 3-disk raid1 array
> I'm planning to use and this stacked configuration?
>
> Yes. The single 3-disk RAID1 array won't work, the stacked configuration
> will.

Oh. I think I mis-read his original post. When I read it the first
time, I inferred he was attempting this to do a full backup of the array.
Reading again, I think you are correct. If he wants to just update the data
on the array, I think rsync (or maybe dar) would be a better solution. If
he wants a full backup from scratch, I don't see why a RAID1 solution would
not work, do you?

> md can resync 'just the changed blocks' by using the 'write intent bitmap'
> and event counters.
> However it only clears the bits in the bitmap when the array is not
> degraded.
> In your suggestion the 3-drive RAID1 is always degraded so bits are never
> cleared, so each resync is effectively a complete resync.

Yeah, to do a full backup from scratch, I think I would set the
array up as a 2 drive array, take the array off-line, remove one drive,
assemble the array, and then add the spare. 'Clumsy, though. I still think
he would be better off with rsync or dar.

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: raid1 with rotating offsite disks for backup

i think that a 4 disks array using spare disks is better
first... if you remove 1 mirror (2 mirrors maybe 2 disks...), you have
a protection problem (you don't have redundat array with only 1 mirror
working)
with 4 disks... 2 mirrors, to start backup put a spare online, wait it
to sync, remove it
if you have problem when resync, you have a second spare (the new
backup), and consider the resync disk, as the new array disk (not more
a backup disk)

it's like a online backup... i think we can implement it as snapshot
(on filesystem level) but since we can make it at lower level, it's a
nice feature... i think 4 disks is more secure... 3 disks is just more
cheap..

2011/2/8 Leslie Rhorer :
>
>
>> -----Original Message-----
>> From: linux-raid-owner@vger.kernel.org [mailto:linux-raid-
>> owner@vger.kernel.org] On Behalf Of NeilBrown
>> Sent: Monday, February 07, 2011 6:18 PM
>> To: Jeff Klingner
>> Cc: linux-raid@vger.kernel.org
>> Subject: Re: raid1 with rotating offsite disks for backup
>>
>> On Mon, 7 Feb 2011 15:53:46 -0800 Jeff Klingner du>
>> wrote:
>>
>> > I'm planning a backup system for my home server and have run into =
a
>> question I can't find answered in the mailing list archives or the w=
iki.
>> Here's the plan:
>> >
>> > 1. Install system and valuable data on a 3-disk raid1 array (call =
the
>> disks A, B, and C).
>> > 2. Remove disk C, put it offsite. =A0("offsite" is moderately time=
-
>> consuming to get to.)
>> > 3a. Periodically, remove disk B, take it offsite, and retrieve dis=
k C
>> > 3b. Insert disk C, which will be re-synced to gain any changes mad=
e
>> since it was removed.
>> > 4. Repeat steps 3a and 3b indefinitely, alternating the roles of d=
isks B
>> and C.
>> >
>> > Thus I hope to get continuous protection against a single drive fa=
ilure
>> and protection back to the last offsite swap for corrupted or delete=
d
>> data.
>> >
>> > My questions are:
>> >
>> > In step 3b, when a disk that was a member of the array in the past=
but
>> has been removed for a while is re-inserted into the 3-disk array, h=
ow
>> does the raid system know to update C with A's contents and not A wi=
th C's
>> contents? =A0Is there a timestamp involved, and if so, how can I exa=
mine it
>> before syncing?
>> >
>> > Is it important to always rotate disks B and C, leaving one that n=
ever
>> leaves the computer, or does it make no difference which of the two =
live
>> disks I pluck out to swap with the offsite disk when I make the trip=
? =A0Can
>> all three disks take turns offsite, so that they all have the same d=
uty
>> cycle?
>> >
>> > I saw in another list message the advice to use two stacked raid1s=
for
>> this application: http://marc.info/?l=3Dlinux-raid&m=3D1267613990087=
75&w=3D2
>> > > Also, if you want two rotating backups I would create two stacke=
d
>> raid1s.
>> > >
>> > > mdadm -C /dev/md0 -l1 -n2 -b internal =A0/dev/main-device /dev/f=
irst-
>> backup
>> > > mdadm -C /dev/md1 -l1 -n2 -b internal /dev/md0 /dev/second-backu=
p
>> > > mkfs -j /dev/md1
>> >
>> >
>> > Are there important differences between the single 3-disk raid1 ar=
ray
>> I'm planning to use and this stacked configuration?
>>
>> Yes. =A0The single 3-disk RAID1 array won't work, the stacked config=
uration
>> will.
>
> =A0 =A0 =A0 =A0Oh. =A0I think I mis-read his original post. =A0When I=
read it the first
> time, I inferred he was attempting this to do a full backup of the ar=
ray.
> Reading again, I think you are correct. =A0If he wants to just update=
the data
> on the array, I think rsync (or maybe dar) would be a better solution=
=A0If
> he wants a full backup from scratch, I don't see why a RAID1 solution=
would
> not work, do you?
>
>> md can resync 'just the changed blocks' by using the 'write intent b=
itmap'
>> and event counters.
>> However it only clears the bits in the bitmap when the array is not
>> degraded.
>> In your suggestion the 3-drive RAID1 is always degraded so bits are =
never
>> cleared, so each resync is effectively a complete resync.
>
> =A0 =A0 =A0 =A0Yeah, to do a full backup from scratch, I think I woul=
d set the
> array up as a 2 drive array, take the array off-line, remove one driv=
e,
> assemble the array, and then add the spare. =A0'Clumsy, though. =A0I =
still think
> he would be better off with rsync or dar.
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid"=
in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
>



--=20
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: raid1 with rotating offsite disks for backup

On Tue, 8 Feb 2011 03:37:40 -0200 Roberto Spadim >
wrote:

> i think that a 4 disks array using spare disks is better

If all you are saying is that 4 disks are better than 3 disks - then ye=
s:
obviously.

If you want to rotate two of them out independently it would still be b=
est to
have 2 stacked RAID1 arrays as then the resync time will be lots short=
er.

Or where you saying something else?

NeilBrown


> first... if you remove 1 mirror (2 mirrors maybe 2 disks...), you hav=
e
> a protection problem (you don't have redundat array with only 1 mirro=
r
> working)
> with 4 disks... 2 mirrors, to start backup put a spare online, wait i=
t
> to sync, remove it
> if you have problem when resync, you have a second spare (the new
> backup), and consider the resync disk, as the new array disk (not mor=
e
> a backup disk)
>=20
> it's like a online backup... i think we can implement it as snapshot
> (on filesystem level) but since we can make it at lower level, it's a
> nice feature... i think 4 disks is more secure... 3 disks is just mor=
e
> cheap..
>=20
> 2011/2/8 Leslie Rhorer :
> >
> >
> >> -----Original Message-----
> >> From: linux-raid-owner@vger.kernel.org [mailto:linux-raid-
> >> owner@vger.kernel.org] On Behalf Of NeilBrown
> >> Sent: Monday, February 07, 2011 6:18 PM
> >> To: Jeff Klingner
> >> Cc: linux-raid@vger.kernel.org
> >> Subject: Re: raid1 with rotating offsite disks for backup
> >>
> >> On Mon, 7 Feb 2011 15:53:46 -0800 Jeff Klingner edu>
> >> wrote:
> >>
> >> > I'm planning a backup system for my home server and have run int=
o a
> >> question I can't find answered in the mailing list archives or the=
wiki.
> >> Here's the plan:
> >> >
> >> > 1. Install system and valuable data on a 3-disk raid1 array (cal=
l the
> >> disks A, B, and C).
> >> > 2. Remove disk C, put it offsite. =A0("offsite" is moderately ti=
me-
> >> consuming to get to.)
> >> > 3a. Periodically, remove disk B, take it offsite, and retrieve d=
isk C
> >> > 3b. Insert disk C, which will be re-synced to gain any changes m=
ade
> >> since it was removed.
> >> > 4. Repeat steps 3a and 3b indefinitely, alternating the roles of=
disks B
> >> and C.
> >> >
> >> > Thus I hope to get continuous protection against a single drive =
failure
> >> and protection back to the last offsite swap for corrupted or dele=
ted
> >> data.
> >> >
> >> > My questions are:
> >> >
> >> > In step 3b, when a disk that was a member of the array in the pa=
st but
> >> has been removed for a while is re-inserted into the 3-disk array,=
how
> >> does the raid system know to update C with A's contents and not A =
with C's
> >> contents? =A0Is there a timestamp involved, and if so, how can I e=
xamine it
> >> before syncing?
> >> >
> >> > Is it important to always rotate disks B and C, leaving one that=
never
> >> leaves the computer, or does it make no difference which of the tw=
o live
> >> disks I pluck out to swap with the offsite disk when I make the tr=
ip? =A0Can
> >> all three disks take turns offsite, so that they all have the same=
duty
> >> cycle?
> >> >
> >> > I saw in another list message the advice to use two stacked raid=
1s for
> >> this application: http://marc.info/?l=3Dlinux-raid&m=3D12676139900=
8775&w=3D2
> >> > > Also, if you want two rotating backups I would create two stac=
ked
> >> raid1s.
> >> > >
> >> > > mdadm -C /dev/md0 -l1 -n2 -b internal =A0/dev/main-device /dev=
/first-
> >> backup
> >> > > mdadm -C /dev/md1 -l1 -n2 -b internal /dev/md0 /dev/second-bac=
kup
> >> > > mkfs -j /dev/md1
> >> >
> >> >
> >> > Are there important differences between the single 3-disk raid1 =
array
> >> I'm planning to use and this stacked configuration?
> >>
> >> Yes. =A0The single 3-disk RAID1 array won't work, the stacked conf=
iguration
> >> will.
> >
> > =A0 =A0 =A0 =A0Oh. =A0I think I mis-read his original post. =A0When=
I read it the first
> > time, I inferred he was attempting this to do a full backup of the =
array.
> > Reading again, I think you are correct. =A0If he wants to just upda=
te the data
> > on the array, I think rsync (or maybe dar) would be a better soluti=
on. =A0If
> > he wants a full backup from scratch, I don't see why a RAID1 soluti=
on would
> > not work, do you?
> >
> >> md can resync 'just the changed blocks' by using the 'write intent=
bitmap'
> >> and event counters.
> >> However it only clears the bits in the bitmap when the array is no=
t
> >> degraded.
> >> In your suggestion the 3-drive RAID1 is always degraded so bits ar=
e never
> >> cleared, so each resync is effectively a complete resync.
> >
> > =A0 =A0 =A0 =A0Yeah, to do a full backup from scratch, I think I wo=
uld set the
> > array up as a 2 drive array, take the array off-line, remove one dr=
ive,
> > assemble the array, and then add the spare. =A0'Clumsy, though. =A0=
I still think
> > he would be better off with rsync or dar.
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-rai=
d" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at =A0http://vger.kernel.org/majordomo-info.htm=
l
> >
>=20
>=20
>=20

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: raid1 with rotating offsite disks for backup

that's right, i think the point is...
with hardware raid, to 'make a backup' you just need to put disk on
array, wait the led to flash red, and get green after sync, and remove
the disk... i think he is trying to make this with md (software raid)
since we are not hardware raid, software is very better we can make
many more commands and know what's in each disk, i think rsync is
better, maybe a online backup with md is good, but i don't know if
it's really good better than a rsync, or a snapshot at filesystem
level (not md problem)
just to help with ideas =3D]

2011/2/8 NeilBrown :
> On Tue, 8 Feb 2011 03:37:40 -0200 Roberto Spadim br>
> wrote:
>
>> i think that a 4 disks array using spare disks is better
>
> If all you are saying is that 4 disks are better than 3 disks - then =
yes:
> obviously.
>
> If you want to rotate two of them out independently it would still be=
best to
> have 2 stacked RAID1 arrays as then =A0the resync time will be lots s=
horter.
>
> Or where you saying something else?
>
> NeilBrown
>
>
>> first... if you remove 1 mirror (2 mirrors maybe 2 disks...), you ha=
ve
>> a protection problem (you don't have redundat array with only 1 mirr=
or
>> working)
>> with 4 disks... 2 mirrors, to start backup put a spare online, wait =
it
>> to sync, remove it
>> if you have problem when resync, you have a second spare (the new
>> backup), and consider the resync disk, as the new array disk (not mo=
re
>> a backup disk)
>>
>> it's like a online backup... i think we can implement it as snapshot
>> (on filesystem level) but since we can make it at lower level, it's =
a
>> nice feature... i think 4 disks is more secure... 3 disks is just mo=
re
>> cheap..
>>
>> 2011/2/8 Leslie Rhorer :
>> >
>> >
>> >> -----Original Message-----
>> >> From: linux-raid-owner@vger.kernel.org [mailto:linux-raid-
>> >> owner@vger.kernel.org] On Behalf Of NeilBrown
>> >> Sent: Monday, February 07, 2011 6:18 PM
>> >> To: Jeff Klingner
>> >> Cc: linux-raid@vger.kernel.org
>> >> Subject: Re: raid1 with rotating offsite disks for backup
>> >>
>> >> On Mon, 7 Feb 2011 15:53:46 -0800 Jeff Klingner d.edu>
>> >> wrote:
>> >>
>> >> > I'm planning a backup system for my home server and have run in=
to a
>> >> question I can't find answered in the mailing list archives or th=
e wiki.
>> >> Here's the plan:
>> >> >
>> >> > 1. Install system and valuable data on a 3-disk raid1 array (ca=
ll the
>> >> disks A, B, and C).
>> >> > 2. Remove disk C, put it offsite. =A0("offsite" is moderately t=
ime-
>> >> consuming to get to.)
>> >> > 3a. Periodically, remove disk B, take it offsite, and retrieve =
disk C
>> >> > 3b. Insert disk C, which will be re-synced to gain any changes =
made
>> >> since it was removed.
>> >> > 4. Repeat steps 3a and 3b indefinitely, alternating the roles o=
f disks B
>> >> and C.
>> >> >
>> >> > Thus I hope to get continuous protection against a single drive=
failure
>> >> and protection back to the last offsite swap for corrupted or del=
eted
>> >> data.
>> >> >
>> >> > My questions are:
>> >> >
>> >> > In step 3b, when a disk that was a member of the array in the p=
ast but
>> >> has been removed for a while is re-inserted into the 3-disk array=
, how
>> >> does the raid system know to update C with A's contents and not A=
with C's
>> >> contents? =A0Is there a timestamp involved, and if so, how can I =
examine it
>> >> before syncing?
>> >> >
>> >> > Is it important to always rotate disks B and C, leaving one tha=
t never
>> >> leaves the computer, or does it make no difference which of the t=
wo live
>> >> disks I pluck out to swap with the offsite disk when I make the t=
rip? =A0Can
>> >> all three disks take turns offsite, so that they all have the sam=
e duty
>> >> cycle?
>> >> >
>> >> > I saw in another list message the advice to use two stacked rai=
d1s for
>> >> this application: http://marc.info/?l=3Dlinux-raid&m=3D1267613990=
08775&w=3D2
>> >> > > Also, if you want two rotating backups I would create two sta=
cked
>> >> raid1s.
>> >> > >
>> >> > > mdadm -C /dev/md0 -l1 -n2 -b internal =A0/dev/main-device /de=
v/first-
>> >> backup
>> >> > > mdadm -C /dev/md1 -l1 -n2 -b internal /dev/md0 /dev/second-ba=
ckup
>> >> > > mkfs -j /dev/md1
>> >> >
>> >> >
>> >> > Are there important differences between the single 3-disk raid1=
array
>> >> I'm planning to use and this stacked configuration?
>> >>
>> >> Yes. =A0The single 3-disk RAID1 array won't work, the stacked con=
figuration
>> >> will.
>> >
>> > =A0 =A0 =A0 =A0Oh. =A0I think I mis-read his original post. =A0Whe=
n I read it the first
>> > time, I inferred he was attempting this to do a full backup of the=
array.
>> > Reading again, I think you are correct. =A0If he wants to just upd=
ate the data
>> > on the array, I think rsync (or maybe dar) would be a better solut=
ion. =A0If
>> > he wants a full backup from scratch, I don't see why a RAID1 solut=
ion would
>> > not work, do you?
>> >
>> >> md can resync 'just the changed blocks' by using the 'write inten=
t bitmap'
>> >> and event counters.
>> >> However it only clears the bits in the bitmap when the array is n=
ot
>> >> degraded.
>> >> In your suggestion the 3-drive RAID1 is always degraded so bits a=
re never
>> >> cleared, so each resync is effectively a complete resync.
>> >
>> > =A0 =A0 =A0 =A0Yeah, to do a full backup from scratch, I think I w=
ould set the
>> > array up as a 2 drive array, take the array off-line, remove one d=
rive,
>> > assemble the array, and then add the spare. =A0'Clumsy, though. =A0=
I still think
>> > he would be better off with rsync or dar.
>> >
>> > --
>> > To unsubscribe from this list: send the line "unsubscribe linux-ra=
id" in
>> > the body of a message to majordomo@vger.kernel.org
>> > More majordomo info at =A0http://vger.kernel.org/majordomo-info.ht=
ml
>> >
>>
>>
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid"=
in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
>



--=20
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: raid1 with rotating offsite disks for backup

On Tue, Feb 8, 2011 at 6:53 AM, Jeff Klingner wrote:
> I'm planning a backup system for my home server and have run into a question I can't find answered in the mailing list archives or the wiki. Here's the plan:
>
> 1. Install system and valuable data on a 3-disk raid1 array (call the disks A, B, and C).
> 2. Remove disk C, put it offsite. ("offsite" is moderately time-consuming to get to.)
> 3a. Periodically, remove disk B, take it offsite, and retrieve disk C
> 3b. Insert disk C, which will be re-synced to gain any changes made since it was removed.
> 4. Repeat steps 3a and 3b indefinitely, alternating the roles of disks B and C.
>
> Thus I hope to get continuous protection against a single drive failure and protection back to the last offsite swap for corrupted or deleted data.


I went through a (probably much too) long discussion here on similar
(but more complex) ideas.


http://www.issociate.de/board/goto/2050886/Q:_LVM_over_RAID, _or_plain_disks?_A:%22Yes%22_=_best_of_both_worlds?.html

Bottom line conclusions
- mdraid isn't designed for this, and may not be as resilient as you'd expect
- there is extra overhead involved in the resulting filesystem on
the member, that in this case doesn't actually help fault-tolerance,
makes it more difficult to recover your data from the offsite drive,
and wastes time
- better to just mount a plain-partitioned disk and rsync for your
removable offsite backups

In my case, some of the target filesystems (backup targets with
millions of hardlinks, created by tools like rdiff-backup and
BackupPC) can't be duplicated at the file-level; I need to copy the
filesystem using block-level tools, so that's why the end of the
thread talks about COTS cloning tools as well as FOSS enhanced
versions of dd.

For regular filesystems, IMO rsync's the way to go.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: raid1 with rotating offsite disks for backup

i agree with rsync
rsync is more filesystem related feature
if you want copy files, you should use rsync
if you want to copy the device, may be a dd with devices?
dd if=3D/dev/md0 of=3D/dev/sda

if you want a snapshot copy, you should first remount your filesystem
to readonly (there=B4s no flock() for /dev/md0 since filesystem don=B4t
use flock() on /dev/md0) or another way to block writes to filesystem
while you is read from device, some filesystems have online backup
features..
i don=B4t know lvm very well, but maybe they implement online backup th=
ere...
i don=B4t think it=B4s a problem to solve at md level (it could work, b=
ut
the raid1 purpose is make a fail safe device (or partially safe), not
a online backup device)


2011/2/9 :
> On Tue, Feb 8, 2011 at 6:53 AM, Jeff Klingner =
wrote:
>> I'm planning a backup system for my home server and have run into a =
question I can't find answered in the mailing list archives or the wiki=
=A0Here's the plan:
>>
>> 1. Install system and valuable data on a 3-disk raid1 array (call th=
e disks A, B, and C).
>> 2. Remove disk C, put it offsite. =A0("offsite" is moderately time-c=
onsuming to get to.)
>> 3a. Periodically, remove disk B, take it offsite, and retrieve disk =
C
>> 3b. Insert disk C, which will be re-synced to gain any changes made =
since it was removed.
>> 4. Repeat steps 3a and 3b indefinitely, alternating the roles of dis=
ks B and C.
>>
>> Thus I hope to get continuous protection against a single drive fail=
ure and protection back to the last offsite swap for corrupted or delet=
ed data.
>
>
> I went through a (probably much too) long discussion here on similar
> (but more complex) ideas.
>
>
> http://www.issociate.de/board/goto/2050886/Q:_LVM_over_RAID, _or_plain=
_disks?_A:%22Yes%22_=3D_best_of_both_worlds?.html
>
> Bottom line conclusions
> =A0- mdraid isn't designed for this, and may not be as resilient as y=
ou'd expect
> =A0- there is extra overhead involved in the resulting filesystem on
> the member, that in this case doesn't actually help fault-tolerance,
> makes it more difficult to recover your data from the offsite drive,
> and wastes time
> =A0- better to just mount a plain-partitioned disk and rsync for your
> removable offsite backups
>
> In my case, some of the target filesystems (backup targets with
> millions of hardlinks, created by tools like rdiff-backup and
> BackupPC) can't be duplicated at the file-level; I need to copy the
> filesystem using block-level tools, so that's why the end of the
> thread talks about COTS cloning tools as well as FOSS enhanced
> versions of dd.
>
> For regular filesystems, IMO rsync's the way to go.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid"=
in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
>



--=20
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: raid1 with rotating offsite disks for backup

On 09/02/2011 19:53, Roberto Spadim wrote:
> i agree with rsync
> rsync is more filesystem related feature
> if you want copy files, you should use rsync
> if you want to copy the device, may be a dd with devices?
> dd if=3D/dev/md0 of=3D/dev/sda
>
> if you want a snapshot copy, you should first remount your filesystem
> to readonly (there=B4s no flock() for /dev/md0 since filesystem don=B4=
t
> use flock() on /dev/md0) or another way to block writes to filesystem
> while you is read from device, some filesystems have online backup
> features..
> i don=B4t know lvm very well, but maybe they implement online backup =
there...
> i don=B4t think it=B4s a problem to solve at md level (it could work,=
but
> the raid1 purpose is make a fail safe device (or partially safe), not
> a online backup device)

I do both with several of my customers: md array with LVM on top,=20
suspend system services (mailstore), take snapshot, start system=20
services, then rsync diff the snapshot to another drive with luks=20
crypto, which gets rotated off-site once a week. Generally the backup=20
drives are bigger than the system array, so I can keep at least a week'=
s=20
worth of nightlies and several months' worth of weeklies on each one.

Yes, I have my own separate securely-stored copies of their crypto keys=


Cheers,

John.

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html