Re: Bug#624343: linux-image-2.6.38-2-amd64: frequent message "biotoo big device md0 (248 > 2

--=-3SaXzk4AdGrlfyejgzFU
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sun, 2011-05-01 at 20:42 -0400, Daniel Kahn Gillmor wrote:
> On 05/01/2011 08:00 PM, Ben Hutchings wrote:
> > On Sun, 2011-05-01 at 15:06 -0700, Jameson Graef Rollins wrote:
> >> Hi, Ben. Can you explain why this is not expected to work? Which par=
t
> >> exactly is not expected to work and why?
> >=20
> > Adding another type of disk controller (USB storage versus whatever the
> > SSD interface is) to a RAID that is already in use.
> >=20
> [...]
> > The normal state of a RAID set is that all disks are online. You have
> > deliberately turned this on its head; the normal state of your RAID set
> > is that one disk is missing. This is such a basic principle that most
> > documentation won't mention it.
>=20
> This is somewhat worrisome to me. Consider a fileserver with
> non-hotswap disks. One disk fails in the morning, but the machine is in
> production use, and the admin's goals are:
>=20
> * minimize downtime,
> * reboot only during off-hours, and
> * minimize the amount of time that the array is spent de-synced.
>=20
> A responsible admin might reasonably expect to attach a disk via a
> well-tested USB or ieee1394 adapter, bring the array back into sync,
> announce to the rest of the organization that there will be a scheduled
> reboot later in the evening.
>=20
> Then, at the scheduled reboot, move the disk from the USB/ieee1394
> adapter to the direct ATA interface on the machine.
>=20
> If this sequence of operations is likely (or even possible) to cause
> data loss, it should be spelled out in BIG RED LETTERS someplace.

So far as I'm aware, the RAID may stop working, but without loss of data
that's already on disk.

> I don't think any of the above steps seem unreasonable, and the set of
> goals the admin is attempting to meet are certainly commonplace goals.
>=20
> > The error is that you changed the I/O capabilities of the RAID while it
> > was already in use. But what I was describing as 'correct' was that an
> > error code was returned, rather than the error condition only being
> > logged. If the error condition is not properly propagated then it coul=
d
> > lead to data loss.
>=20
> How is an admin to know which I/O capabilities to check before adding a
> device to a RAID array? When is it acceptable to mix I/O capabilities?
> Can a RAID array which is not currently being used as a backing store
> for a filesystem be assembled of unlike disks? What if it is then
> (later) used as a backing store for a filesystem?
[...]

I think the answers are:
- Not easily
- When the RAID does not have another device on top
- Yes
- Yes
but Neil can correct me on this.

Ben.

--=20
Ben Hutchings
Once a job is fouled up, anything done to improve it makes it worse.

--=-3SaXzk4AdGrlfyejgzFU
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQIVAwUATb4C/ue/yOyVhhEJAQpxdg//TwR5SE0cJde2Rk5NV7Fw8vEeo0K3 zRbW
TzXS97A3S52X5Ay6dUI2WiAtI6N48/xEjRjOmhIqx30yiD8pBcODF3hNjVuI hl8Y
RnmJeHfv3F/5GHSwI1NGHa5NisviUSAAPNaZM05MkCaJE2vPucoGrxLiVrXG 0pzy
m1klLJpPcsWEf6iGOV+ydmiTH2/mr8E9N0cUIWwNpJ5CTTZlY4/oxVrrUPWU S8sA
3lNDbyJrqLzB5JmOIHG+fb7fXeOoEPycICpj824OHFMCwjD+uWnz9Nma7PXX v/qC
/XoXM//M6V82krmVLUyBxRJQMXRgtaC4mxFXaVvSzXptjvm6+MKeEDYynKa7 IwW9
5IJpT403Ixa3qNt58vmOvrF0U3Wtm3mNXSiGTetdTH0OKVaY7zWUOtwhASdt x66j
CsXiSDAWNam3KqXt4a4MILA5ZvKnuBWNzE8fc97ZTX8DGc5jbJzGtcfKfrqt LG2Y
wITuvZZCEGigjvmjHfnPRB4Lp6rnVEbgHEIXcFUgaXPyBaOewrZO9LXxzhyE 5cUy
XB5u+Ixh67VDLW7/WufGF0RcNREsL6RiKDmNIjWL57fpwrptKZGAhAdrtFMO Sgp1
0BgN/TGiWEyxZATQGyGfnX7jc0sh9ftptbB3eRrrWbUpfipc3IJCSFcXale0 Sv79
/YG01UMA3tc=
=6GU5
-----END PGP SIGNATURE-----

--=-3SaXzk4AdGrlfyejgzFU--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Bug#624343: linux-image-2.6.38-2-amd64: frequent message "bio too big device md0 (248 >

--=-=-=
Content-Transfer-Encoding: quoted-printable

On Mon, 02 May 2011 02:04:18 +0100, Ben Hutchings wro=
te:
> On Sun, 2011-05-01 at 20:42 -0400, Daniel Kahn Gillmor wrote:
> So far as I'm aware, the RAID may stop working, but without loss of data
> that's already on disk.

What exactly does "RAID may stop working mean"? Do you mean that this
bug will be triggered? The raid will refuse to do further syncs? Or do
you mean something else?

> > How is an admin to know which I/O capabilities to check before adding a
> > device to a RAID array? When is it acceptable to mix I/O capabilities?
> > Can a RAID array which is not currently being used as a backing store
> > for a filesystem be assembled of unlike disks? What if it is then
> > (later) used as a backing store for a filesystem?
> [...]
>=20
> I think the answers are:
> - Not easily
> - When the RAID does not have another device on top

This is very upsetting to me, if it's true. It completely undermines
all of my assumptions about how software raid works.

Are you really saying that md with mixed disks is not possible/supported
when the md device has *any* other device on top of it? This is a in
fact a *very* common setup. *ALL* of my raid devices have other devices
on top of them (lvm at least). In fact, the debian installer supports
putting dm and/or lvm on top of md on mixed disks. If what you're
saying is true then the debian installer is in big trouble.

jamie.

--=-=-=
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQIcBAEBCAAGBQJNvgYSAAoJEO00zqvie6q86lAP/3LtkkpHPdE6kY9kBQVl EPb+
iR+V58tBq4mKAzc8Lta3HfJCZMEiZoQveUJiJYuMtwsI9eo9ED5XDapme5GU ocgN
fliyUHHCwMjvwzzBwYRZxOxMPJo8VfTovf3cfJdcma3legERCb7qE3tO8daa TkSP
9e9g5BWu0RNtHapk+IkC+TFSLMzd+q9CSLmgI1jL9+GJLjaRSj8vyTdpy9Z9 PlVz
uXUm/cv6AF5H/+lPlKvH+dUT6diuaN1eg5ixSdkeaV5t3S2ndxC4qPUsdOqi TJOn
DbLPL+DKvYwqcvEH/muAPOxQAS/dEH8qrB1hyLRODbUvXcwGmdhr3Z8GVO7n Cm1U
pu8VfGLyZrwx2CyDYVJK6yW7BqBWT4gBWIM/zjE4yAxJQlI2YWRv/wiWUhQ7 tdrZ
Y3MdOfYegTWgRLV6nmhVvNYLA1yOiOcjLTj67PWTJG/z5LiSUY9n1l8Akdgp IHLu
YQhHUcO01JdkPgQapZJc+U6KY2JJWjElmMc/mzEDeIFVObMBPN7xm4ehhgsY D2yV
2QC2dJ/Vx66xvRxMv5dGALH3gqCoUs8h4CQtbUD0dFf8y2eRZjN36BA+fyRx 5tBB
TpQkDtqL50xPii41eHyDETyBARJNoDi5e8cxYa5XFePg+P0ASCfKn7DKsLDD p0kY
qjxCc+gFIZ/WJb1WJ6Cb
=gdgF
-----END PGP SIGNATURE-----
--=-=-=--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Bug#624343: linux-image-2.6.38-2-amd64: frequent message "biotoo big device md0 (248 > 2

On 02/05/2011 03:17, Jameson Graef Rollins wrote:
> On Mon, 02 May 2011 02:04:18 +0100, Ben Hutchings wrote:
>> On Sun, 2011-05-01 at 20:42 -0400, Daniel Kahn Gillmor wrote:
>> So far as I'm aware, the RAID may stop working, but without loss of data
>> that's already on disk.
>
> What exactly does "RAID may stop working mean"? Do you mean that this
> bug will be triggered? The raid will refuse to do further syncs? Or do
> you mean something else?
>
>>> How is an admin to know which I/O capabilities to check before adding a
>>> device to a RAID array? When is it acceptable to mix I/O capabilities?
>>> Can a RAID array which is not currently being used as a backing store
>>> for a filesystem be assembled of unlike disks? What if it is then
>>> (later) used as a backing store for a filesystem?
>> [...]
>>
>> I think the answers are:
>> - Not easily
>> - When the RAID does not have another device on top
>
> This is very upsetting to me, if it's true. It completely undermines
> all of my assumptions about how software raid works.
>
> Are you really saying that md with mixed disks is not possible/supported
> when the md device has *any* other device on top of it? This is a in
> fact a *very* common setup. *ALL* of my raid devices have other devices
> on top of them (lvm at least). In fact, the debian installer supports
> putting dm and/or lvm on top of md on mixed disks. If what you're
> saying is true then the debian installer is in big trouble.
>
> jamie.

I can't imagine that this is the case - layered setups are perfectly
standard. While the dm-layer might be less used, it is normal practice
to have lvm on top of md raids, and it is not uncommon to have more than
one md layer (such as raid50 setups). It is also perfectly reasonable
to through USB media into the mix (though depending on the
kernel/distro, there may be boot issues if the USB disk is not stable
fast enough during booting - I had such problems with a USB disk in an
LVM setup without md raid).

As far as I understand it, there are two sorts of communication between
the layers of the block devices. There is the block device access
itself - the ability to read and write blocks of data. And there is the
metadata, covering things like sizes, stripe information, etc. Only the
block access is actually needed to get everything working - the other
information is used for things like resizing, filesystem layout
optimisation, etc.

The whole point of the layered block system is that the block access
layers are independent and isolated. So if you have a dm layer on top
of an md layer, then the dm layer should not care how the md layer is
implemented - it just sees a /dev/mdX device. It doesn't matter if it's
a degraded raid1 or anything else. As long as the /dev/mdX device stays
up, it should not matter that you add or remove devices, or what type of
underlying device is used.

Similarly, the md raid1 layer is mainly interested in the block access -
it will work with any block devices. It will use the metadata to
improve things like resizes, and perhaps to optimise accesses, but it
should work /correctly/ (though perhaps slower than optimal) regardless
of the mix of disks.


I have used layered setups and odd block devices (such as loopback
devices on files on a tmpfs mount and multiple md layers) - getting
resizing to work properly involved a little more effort, but it all
worked perfectly. I haven't tried such a mix as the OP has been describing.


If my understanding of the block layers is wrong, then I too would like
to know - running lvm on top of md raid is essential capability, as is
using USB disks as temporary additions to an array.


--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html