Best strategy to incrementally replace smaller HDDs

--=-McANE+GaDtAgVgMlO7t3
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi all, given the configuration below:
* 8 x 1TB HDDs
* 2 x 2TB HDDs

On which I currently have:
* (10 x 1TB) RAID6 - persistent storage
* (2 x 1TB) system / temporary storage etc.

I want to replace the 1TB drives for 2TB ones on an as-needed basis,
what strategy would you recommend?

1. If I moved to 2TB RAID6 members (using RAID0 on the 1TB drives),
I would need to replace two of the drives just to match current
capacity. Each next 2TB drive would get me 1TB additional
capacity (but actually I'd need to replace two to gain
anything). That sounds to be most future-proof, but most
expensive.
2. If I moved to 2TB RAID5 members, that would reduce redundancy
but replacing just two would gain me 2TB capacity. Most of the
above still applies.
3. If I kept to 1TB RAID6 (two on the 2TB drives), I would reduce
the redundancy to just one drive if it was the 2TB drive that
failed, but each new drive would gain me 1TB capacity and I only
need to replace one-by-one. This sounds like the cheapest, but
worst possible approach.

So, am I missing something? What do you think?
--=20
Michał (Saviq) Sawicz

--=-McANE+GaDtAgVgMlO7t3
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQIcBAABAgAGBQJOae7hAAoJEGnv7NPGHSZFpM0QAJyydq2ISQTIrdVk2lvk Wys8
k+YERjAU6AJsXxMmGKYuOfQDswfRe7aiq/+0qVy3NQhrQlGCUtaYL4Cm6TJc WnEf
MYrpPlcJdf/fv4+K+mKXWoyJJ32qokhlO1It1OpndeynNX7gJzeUp0QllVbh 34vE
VmBpBU4J9nV9x+w4V995I+y0NV1CJsX35LF1siT6XEnyzBdoumwQpZbNdZLO tnzx
Ib1sCTN78KJ2I6o+q1vfj/NLdSCzs5wXHRgZ1Xq51TUMo+ibX84MtiOuT/Y6 vnsV
ycMd6JM8/OzOkl7PgI+2y9Qqrackr+ZiYIUpXilzzv6r+VMcG8EgrSDga3Yx yCaX
wlE51VPtY9NaULYgdmM57Ld+tCPylQNELAhrbH8lddkjTMtYFuVQuNmA0sSf tDai
zfGKrdcyV9sbNBEOgD7VswfjHwrNcMV/plzd7NlaMw4+sSnOQzSTilGUsaDP VoP8
ku17eQdWGsv67QTnIR6c6FvdgvfGU9htMMc65AXGyUpqampT9wWJ1C5olRoY m0Yo
i7aoZdN6XKgZRUttb/j3wOaysYGdrn8QQvnexxK4ooJ4qKYTgQOJp9nV8KwF 74ow
TOUFh36WhBlMBoMeMnhvSoOCdZVPeDmgu+elp+aODZd+dCUXKuqVxENYGe7h fhXX
QmDwsCAE7rzSqWQv+T3R
=NEbE
-----END PGP SIGNATURE-----

--=-McANE+GaDtAgVgMlO7t3--

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Best strategy to incrementally replace smaller HDDs

On 09/09/2011 12:48, Michał Sawicz wrote:
> Hi all, given the configuration below:
> * 8 x 1TB HDDs
> * 2 x 2TB HDDs
>
> On which I currently have:
> * (10 x 1TB) RAID6 - persistent storage
> * (2 x 1TB) system / temporary storage etc.
>

By this do you mean that you have 8 x 1TB drives with a 1 TB partition=20
on each, and 2 x 2T drives with 2 x 1TB partition on each? So that the=
=20
two big disks are shared with both raids?

> I want to replace the 1TB drives for 2TB ones on an as-needed basis,
> what strategy would you recommend?

This sounds like you are thinking that you can replace a single disk in=
=20
your RAID6 array and get more storage - changing 10 x 1 TB raid6 =3D 8T=
B=20
into 9 x 1TB + 1 x 2TB raid6 =3D 9 TB. It doesn't work like that. You=
=20
will have to replace /all/ your 1 TB devices with 2 TB devices (and mov=
e=20
the second raid off the two existing 2TB devices) - all members of the=20
raid6 must be the same size.

To help you plan your upgrades, it is also useful to know your=20
partitioning scheme (for example, do you use LVM?), whether you have th=
e=20
space to put lots more drives in the system or must do it one drive at =
a=20
time, whether you can take the system off-line during the process, and=20
whether you need to do the upgrade quickly or can spend a week or so at=
=20
it (some of these are conflicting requirements).

Before you think about upgrading, however, make sure you have a solid=20
backup. Then make sure you have a backup of that backup - and a plan=20
for how to restore everything if something goes horribly wrong.


>
> 1. If I moved to 2TB RAID6 members (using RAID0 on the 1TB driv=
es),
> I would need to replace two of the drives just to match curr=
ent
> capacity. Each next 2TB drive would get me 1TB additional
> capacity (but actually I'd need to replace two to gain
> anything). That sounds to be most future-proof, but most
> expensive.
> 2. If I moved to 2TB RAID5 members, that would reduce redundanc=
y
> but replacing just two would gain me 2TB capacity. Most of t=
he
> above still applies.
> 3. If I kept to 1TB RAID6 (two on the 2TB drives), I would redu=
ce
> the redundancy to just one drive if it was the 2TB drive tha=
t
> failed, but each new drive would gain me 1TB capacity and I =
only
> need to replace one-by-one. This sounds like the cheapest, b=
ut
> worst possible approach.
>
> So, am I missing something? What do you think?



--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Best strategy to incrementally replace smaller HDDs

--X1bOJ3K7DJ5YkBrT
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Fri Sep 09, 2011 at 12:48:01PM +0200, Michał Sawicz wrote:

> Hi all, given the configuration below:
> * 8 x 1TB HDDs
> * 2 x 2TB HDDs
>=20
=46rom what you cover below, it sounds like you're limited to 10 drives in
total. Is this the case?

> On which I currently have:
> * (10 x 1TB) RAID6 - persistent storage
> * (2 x 1TB) system / temporary storage etc.
>=20
> I want to replace the 1TB drives for 2TB ones on an as-needed basis,
> what strategy would you recommend?
>=20
> 1. If I moved to 2TB RAID6 members (using RAID0 on the 1TB drives),
> I would need to replace two of the drives just to match current
> capacity. Each next 2TB drive would get me 1TB additional
> capacity (but actually I'd need to replace two to gain
> anything). That sounds to be most future-proof, but most
> expensive.
> 2. If I moved to 2TB RAID5 members, that would reduce redundancy
> but replacing just two would gain me 2TB capacity. Most of the
> above still applies.

So, for both of these, you're planning on changing to:
* (2 x 1TB) system
* (2 x 2TB) + (3 x 1+1TB) persistent storage

This gives you 6TB persistent storage in RAID6, or 8TB in RAID5. As you
say, replacing 2 1TB drives for 2TB drives (or adding a 2TB drive, if
possible) would give you 8TB in RAID6 again.

You'll also need to do a backup, rebuild & restore for either of these
solutions.

In either case, replacing a single drive (assuming you're limited in
total drive count) would not gain any capacity, whereas a pair of drive
replacements would give you a 2TB increase.

I'd steer clear of RAID5 with 2TB drives - rebuilding a 2TB drive will
present a very large window for total array failure.

> 3. If I kept to 1TB RAID6 (two on the 2TB drives), I would reduce
> the redundancy to just one drive if it was the 2TB drive that
> failed, but each new drive would gain me 1TB capacity and I only
> need to replace one-by-one. This sounds like the cheapest, but
> worst possible approach.
>=20
Losing a single 2TB drive would lose 2 partitions, wiping out all
redundancy. You're only needing to rebuild 1TB before you're back to
partial redundancy though, so you're less prone to read failures than
with RAID5 above. A second 2TB drive failure during recovery of either
partition will cause total array failure though.

Replacing a single 1TB drive would gain you 1TB in space.

There's also no need to do a complete wipe & restore - you can just
replace the drive and recover onto the new one.

Performance will be worse (particularly during recovery) as you're
always seeking between the two partitions.

Personally, I'd go with RAID6 on 2TB partitions. It is more expensive,
but I think the performance and redundancy benefits outweigh the costs.

Cheers,
Robin
--=20
___ =20
( ' } | Robin Hill |
/ / ) | Little Jim says .... |
// !! | "He fallen in de water !!" |

--X1bOJ3K7DJ5YkBrT
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)

iEYEARECAAYFAk5qEQ8ACgkQShxCyD40xBLN5wCcCP8wce9XCISDBqz9mcnQ eS2q
NAwAoLVOD0tkQzK9RZoiX8jYNqNchK1u
=NuYa
-----END PGP SIGNATURE-----

--X1bOJ3K7DJ5YkBrT--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Best strategy to incrementally replace smaller HDDs

--=-zeZJsAMiTGEAhi+i1KZZ
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Dnia 2011-09-09, pią o godzinie 14:34 +0200, David Brown pisze:
> On 09/09/2011 12:48, Michał Sawicz wrote:
> > Hi all, given the configuration below:
> > * 8 x 1TB HDDs
> > * 2 x 2TB HDDs
> >
> > On which I currently have:
> > * (10 x 1TB) RAID6 - persistent storage
> > * (2 x 1TB) system / temporary storage etc.
> >
>=20
> By this do you mean that you have 8 x 1TB drives with a 1 TB partition=
=20
> on each, and 2 x 2T drives with 2 x 1TB partition on each? So that the=
=20
> two big disks are shared with both raids?
Yes, that wasn't like that before, but that's a result of exchanging two
1TB drives to 2TB ones.=20

> > I want to replace the 1TB drives for 2TB ones on an as-needed basis,
> > what strategy would you recommend?
>=20
> This sounds like you are thinking that you can replace a single disk in=
=20
> your RAID6 array and get more storage - changing 10 x 1 TB raid6 =3D 8TB=
=20
> into 9 x 1TB + 1 x 2TB raid6 =3D 9 TB. It doesn't work like that. You=
=20
> will have to replace /all/ your 1 TB devices with 2 TB devices (and move=
=20
> the second raid off the two existing 2TB devices) - all members of the=
=20
> raid6 must be the same size.
Actually no, I know how RAIDs work, so the three schemes I described
below account for that.

> To help you plan your upgrades, it is also useful to know your=20
> partitioning scheme (for example, do you use LVM?), whether you have the=
=20
> space to put lots more drives in the system or must do it one drive at a=
=20
> time, whether you can take the system off-line during the process, and=
=20
> whether you need to do the upgrade quickly or can spend a week or so at=
=20
> it (some of these are conflicting requirements).
No LVM here. The most important - big RAID6 - is a single partition. The
remaining 2 x 1TB on the big drives are split into RAID1 for boot,
RAID10 for root, RAID0 for temporary storage and swap space. I can
temporarily put more drives in there. I can take it off-line for some
periods and I can do the upgrade over an extended period of time.=20

> Before you think about upgrading, however, make sure you have a solid=20
> backup. Then make sure you have a backup of that backup - and a plan=20
> for how to restore everything if something goes horribly wrong.
Yeah... I plan to rely on RAID's redundancy by moving data around,
growing / shrinking the filesystems and then shuffling the HDDs. I know
that's asking for trouble, but I haven't yet found a place to rent 8TBs
of storage for a week... And that's worked for me in the past, btw.
Twice, even, IIRC.

--=20
Michał (Saviq) Sawicz

--=-zeZJsAMiTGEAhi+i1KZZ
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQIcBAABAgAGBQJOah7UAAoJEGnv7NPGHSZFr4gP/R1SMqdQ5tvAIoaAvfii UTUh
t+63PUdaooAe6fncGEnQ56jDf0CwnbexQugaFE1OctWHzvzBK1sDAqabg7Kr 3Nu8
HWpUVI3ELKdDep3dS0GXkvX1C3STuHY+uZYPSjr3Wl/9K+Gc8r8huHcMbwXa ANK7
D5c1GmmGjf/XpONl+gzkHVl+v0XEBjm2HtTBmhZ6jniAsRkymFl5m6j5JwM/ W3SJ
LweygFetcigarVPwyaFsieB4zyksL8aaBscBsL4XhhcDI7FUq4Ur5wrXo3Dv 0zU6
8B1jP0k1CQM37twuuxdHxcW9Uu9DSC28O/C9SdnX4xe1RgcWjsveolZecLL6 up3X
haSYSDl+vy5JAKQOSyYZAtjV9FtgJiJcBF5owFcmlI/h4H75swOj+X5WrGpb U4RX
bNEig6v0Ihwo1W4dcyK6ggH02LL8h2iLImaCUSn3npqwLvSJPN2gAObrC5Zv d1pG
nYxEzOHmWfWoXOHLQhkSM4GtIEf6plyxh5kMylnMTTiBaEsE0YFvbvRb0nTM a885
7pt8BR2W9GsgeHA6lTG7l0sJUaFkf+mhRnnEOr4IBPA3XPBY6fAhGw3Jrfur g1vc
a9A+Y9d1wBPfK7ziKcsRjZXshgKjuyrRO3LRzooS+WsXd5dKzUeI0nbvAT7k RkxZ
v7ZGK5jsTjpA6WghLUim
=km5Q
-----END PGP SIGNATURE-----

--=-zeZJsAMiTGEAhi+i1KZZ--

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Best strategy to incrementally replace smaller HDDs

--=-mH7/mfCPlN9BsdfrxIC7
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Dnia 2011-09-09, pią o godzinie 14:13 +0100, Robin Hill pisze:
> On Fri Sep 09, 2011 at 12:48:01PM +0200, Michał Sawicz wrote:
>=20
> > Hi all, given the configuration below:
> > * 8 x 1TB HDDs
> > * 2 x 2TB HDDs
> >=20
> From what you cover below, it sounds like you're limited to 10 drives in
> total. Is this the case?
In the long run - yes, but I can have more temporarily. It's just a
physical issue, not controller issue.

> > On which I currently have:
> > * (10 x 1TB) RAID6 - persistent storage
> > * (2 x 1TB) system / temporary storage etc.
> >=20
> > I want to replace the 1TB drives for 2TB ones on an as-needed basis,
> > what strategy would you recommend?
> >=20
> > 1. If I moved to 2TB RAID6 members (using RAID0 on the 1TB drives)=
,
> > I would need to replace two of the drives just to match current
> > capacity. Each next 2TB drive would get me 1TB additional
> > capacity (but actually I'd need to replace two to gain
> > anything). That sounds to be most future-proof, but most
> > expensive.
> > 2. If I moved to 2TB RAID5 members, that would reduce redundancy
> > but replacing just two would gain me 2TB capacity. Most of the
> > above still applies.
>=20
> So, for both of these, you're planning on changing to:
> * (2 x 1TB) system
> * (2 x 2TB) + (3 x 1+1TB) persistent storage
Actually no, currently I'm planning to go:
* (2 x 1TB) system,
* (4 x 2TB) + (2 x 1+1TB)

So that's 8TB in RAID6, 10TB in RAID5. I already have 8TB and the only
way is up. With RAID6 I'd have to replace at least 4x1TB for 3x2TB
initially (one less drive total).

> This gives you 6TB persistent storage in RAID6, or 8TB in RAID5. As you
> say, replacing 2 1TB drives for 2TB drives (or adding a 2TB drive, if
> possible) would give you 8TB in RAID6 again.
>=20
> You'll also need to do a backup, rebuild & restore for either of these
> solutions.
As I mentioned in my previous reply to David, I'm planning to
shrink/grow as needed.

> In either case, replacing a single drive (assuming you're limited in
> total drive count) would not gain any capacity, whereas a pair of drive
> replacements would give you a 2TB increase.
>=20
> I'd steer clear of RAID5 with 2TB drives - rebuilding a 2TB drive will
> present a very large window for total array failure.
Yeah, that's why I switched to RAID6 in the first place...

> > 3. If I kept to 1TB RAID6 (two on the 2TB drives), I would reduce
> > the redundancy to just one drive if it was the 2TB drive that
> > failed, but each new drive would gain me 1TB capacity and I onl=
y
> > need to replace one-by-one. This sounds like the cheapest, but
> > worst possible approach.
> >=20
> Losing a single 2TB drive would lose 2 partitions, wiping out all
> redundancy. You're only needing to rebuild 1TB before you're back to
> partial redundancy though, so you're less prone to read failures than
> with RAID5 above. A second 2TB drive failure during recovery of either
> partition will cause total array failure though.
>=20
> Replacing a single 1TB drive would gain you 1TB in space.
>=20
> There's also no need to do a complete wipe & restore - you can just
> replace the drive and recover onto the new one.
>=20
> Performance will be worse (particularly during recovery) as you're
> always seeking between the two partitions.
>=20
> Personally, I'd go with RAID6 on 2TB partitions. It is more expensive,
> but I think the performance and redundancy benefits outweigh the costs.
That's the direction I'm leaning to as well.

Thanks a lot for the comments!
--=20
Michał (Saviq) Sawicz

--=-mH7/mfCPlN9BsdfrxIC7
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQIcBAABAgAGBQJOaiCkAAoJEGnv7NPGHSZFNOAQAK0ON40k6dADMoYxdAnX uW8v
aAqYuZAi1+nGdHTWdIO38/VOUkcGViVGVKyGehPBZvg5g+8jYI2Q5++ETcRf id/7
lEVrSNno3AxeBTqPtv8u+tfhuPBOZ+Y4NVD8+Z5pW2RhVKZw25PSul+Epscz C/Y4
KCmCKQK7CiGhH1KF8OHa8h+rYwTVZuyH/IR6pnq4R6kyf6Nj50ez2OXdXwfd ekL/
6hCUSeSMNpB1z/yF5gUOzAiLzmiAg5UDG8Zf5xbzmKLBPsS5GBRl6hv/VOsz pOaI
RjLlsN9qOLUGhG1UBwF/clUkkULixegJKbBKD3He4j/LzD0+ThqFcZZQpBOa iPUj
nXG3fRTm+ldIQvFCAgLUNl0kplZZK4j499sD/8O6uNZhMoafBcxB6Z5LWluP gu+3
gBNIPMYxOc834h00rRdOh39/DVfcOtqSPPXE+FQE0vpbCXT5nI4EWSATZQDW N+65
WPGNEn23ilAKb7mQh30XvEfHd9VGOt0843UmeOcKdiaiUgH4qxn81vZD4y8F 1pft
MrjGa+6qWYqn+52kGCCwvx8KpaQpjwFsGKoF9zkOidrwtzwpcVuPY316tpUy sRME
MSt24nccxoCxX2XqKd2kT1W3wiG1lXmXjRcqJUdiXPA+M2vZnwkByMZKAXrb vugF
rq6rOqy0drycVL+Jsi+z
=+mGN
-----END PGP SIGNATURE-----

--=-mH7/mfCPlN9BsdfrxIC7--

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Best strategy to incrementally replace smaller HDDs [successstory]

--=-p6p61j9fxM4kK0CBInvv
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Thanks again for all the pointers, I will now describe my workflow that
got me going with the new drives!

I'll try and do some ASCII-art [===3D used space, --- space (not drives=
)
used for parity].


1. initial situation
-----------------
~~1TB~~
sda |===========3D|
sdb |===========3D|
sdc |=====3D|
sdd |=====3D|
sde |=====3D|
sdf |=====3D|
sdg |=====3D|
sdh |=====3D|
sdi |-----|
sdj |-----|

md1 |=====3D| - 2 x 1TB RAID0 == 2TB
md2 |=====3D| - 10 x 1TB RAID6 == 8TB


2. added 4 x 2TB drives
--------------------
sdk | |
sdl | |
sdm | |
sdn |-----------|

md3 | | - 5 x 2TB RAID6 == 6TB (degraded)


3. moved part of md2 data onto md3
----------------------------------
sda | ======|
sdb | ======|
sdc |=====3D|
sdd |=====3D|
sde |=3D |
sdf | |
sdg | |
sdh | |
sdi |-----|
sdj |-----|
sdk |===========3D|
sdl |===========3D|
sdm |===========3D|
sdn |-----------|

md1 |=====3D| - 2 x 1TB RAID0 == 2TB
md2 |== | - 10 x 1TB RAID6 == 8TB
md3 |===========3D| - 5 x 2TB RAID6 ===
6TB (degraded)


4. shrunk and removed drives from md2
----------------------------------
sda | ======|
sdb | ======|
sdc |=====3D|
sdd |=====3D|
sde |=3D |
sdf |-----|
sdg | |
sdh | |
sdi | |
sdj | |
sdk |===========3D|
sdl |===========3D|
sdm |===========3D|
sdn |-----------|

md1 |=====3D| - 2 x 1TB RAID0 == 2TB
md2 |==== | - 5 x 1TB RAID6 == 3TB (degraded=
)
md3 |===========3D| - 5 x 2TB RAID6 ===
6TB (degraded)


5. new arrays to free the old 2TB drives
and create an additional member for md4
---------------------------------------
sda |===========3D|
sdb |===========3D|
sdc |=====3D|
sdd |=====3D|
sde |=3D |
sdf |-----|
sdg |=====3D|
sdh |=====3D|
sdi |=====3D|
sdj |=====3D|
sdk | |
sdl | |
sdm |-----------|
sdn |-----------|

md4 | | - 4 x 1TB RAID10 == 2TB (degraded)
md1 |=====3D| - 2 x 1TB RAID0 == 2TB
md2 |==== | - 5 x 1TB RAID6 == 3TB (degraded=
)
md3 |=======3D | - 7 x 2TB RAID6 == 10TB


6. Moved rest of the data from md2,
dropped it and replaced a disk in md4
-------------------------------------
sda |===========3D|
sdb |===========3D|
sdc |=====3D|
sdg |=====3D|
sdh |=====3D|
sdj |=====3D|
sdk |===========3D|
sdl | |
sdm |-----------|
sdn |-----------|

md4 | | - 4 x 1TB RAID10 == 2TB (degraded)
md1 |=====3D| - 2 x 1TB RAID0 == 2TB
md3 |=========3D | - 7 x 2TB RAID6 == 10T=
B


And it only took two weeks! ;)

And again, thanks for all the pointers I got on the list, it all
happened without any data loss.
--=20
Michał (Saviq) Sawicz

--=-p6p61j9fxM4kK0CBInvv
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQIcBAABAgAGBQJOgYtcAAoJEGnv7NPGHSZFgYEQAI5be5Ov0BYQvR6+FdJJ ny8M
10TemC95FmGL7NPuA1taYSMiynTncFgUH8gb6lnmTSQ6JDuOm3ec52fecmpp SIY3
sY+id924JAl8LCBiEYTbcHmHQl7EdxhAm2LaALxhxjpnib5qVT5Ph6cmgDEY R9Al
uPE/bl05zmnLQdPh5QGokI7nvQBvCVeLW6G/r2cdYAt/4nLrKiJ5mpyTNSSG 1EDs
vZdN43ibTio4bjyarcRbUtjgrp8D+z3JwHPlR4uam7Rt79kYOoFT0pNjiXeT qb7B
U65lkFZy/+Kv5uQeZ2ZhjCT+btAQ9iq7feX/iaqR5/qaPwaHLbpUz+MtW95H fJbG
UETJTkSMJcTNh7DOjtp12jLh00soD3GZE9oH0vnpHhjTfZxU2uqHD547Yqsn 50aA
kNL7ncu1MVRljzKe6s8SPd0oqqsTm86oKbUkTXLSiDPqBNtjGiTCSCtjkXFz yo52
iykymokhSjiOYm0gvFNByguyvAYqsumrYyas+GiOsgbZhBf9gNhS7BrN8ayf VIem
zcqQGjLiSUaIGhVQWe6xZC46eUVWBjbmrMB0Z4a23sG7kjSC7Ufkz3105aKL ecBp
COpNYoHCwHIIyQ58b318EANQwwa0bKThFulrjSZK10LTCmJPFnAkvTV0Nn47 wi9J
zp47Vsze6LdZlt/R/kEE
=JoZD
-----END PGP SIGNATURE-----

--=-p6p61j9fxM4kK0CBInvv--

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html