Re: Software RAID and Fakeraid

On Mon, Jan 31, 2011 at 11:44:07AM -0500, Phillip Susi wrote:
> I know you went on vacation over the new year, but it has been about two
> months now so I figured I would try to revive this conversation.
>
> Having collected my thoughts, this is what I propose:
>
> I believe that right now, grub recognizes an md device that is built on
> partitions, and installs to the underlying disks. When the device is
> built on the whole disks, it needs to divide into two categories:
>
> 1) Native md metadata. Grub should ask mdadm for a suitable location
> to embed the core image on each component disk. This can be satisfied
> with a 32kb area at the end ( or maybe the beginning ) of the disk that
> is not used by the array ( or the metadata ). This will work for all
> native formats except for 1.1, for the obvious reason that it is using
> the boot block.
>
> 2) Other metadata formats. Either assume or explicitly be told by
> mdadm that these formats are fakeraid and thus understood by the bios
> int 13. Grub installs to the array as a whole, just like it does today
> when they are handled by dmraid. If it is known that it is not bios
> supported, then error out. In the future this could be supported by
> creating additional grub modules that understand those metadata formats.
>
> One question that needs answered for method 1 is what should the
> partition table look like? Should grub, or maybe mdadm, create a
> protective mbr when using the whole disk, similar to the one used with GPT?

I think in the case of BIOS raid, the bios will boot from the raid
device, so installing to the boot area of the raid device makes sense
(/dev/mapper/whatever or similar).

In the case of software md raid on partitions, the BIOS boots from the
boot area of the disks directly, so installing to all member disks makes
sense (/dev/sda, /dev/sdb, etc).

If you do software md raid on whole disks, I am not even sure if the
BIOS could boot from that, since there won't be a partition table, no
partition marked bootable (which some BIOSs requrie), sector 0 may not
even contain boot code. Not sure there is anyway whole device software
raid makes sense for a bootable drive at all. It makes sense for data
drives perhaps. I just don't see this as a candidate for booting at all.

Now that's all assuming x86 BIOS based booting. For other systems
supported by grub, things could be different perhaps.

--
Len Sorensen

Re: Software RAID and Fakeraid

On 1/31/2011 12:03 PM, Lennart Sorensen wrote:
> If you do software md raid on whole disks, I am not even sure if the
> BIOS could boot from that, since there won't be a partition table, no
> partition marked bootable (which some BIOSs requrie), sector 0 may not
> even contain boot code. Not sure there is anyway whole device software
> raid makes sense for a bootable drive at all. It makes sense for data
> drives perhaps. I just don't see this as a candidate for booting at all.

That is why I asked what the partition table should look like. When
grub is installed there will be one ( and boot code ), the question is
whether it should be empty, or have a protective partition that claims
the entire disk, like EFI does.

Normally with format 0.9, the raid array starts on sector 0, so the same
MBR appears both at the start of the first physical disk, and at the
start of the array. This is not possible with 1.1 or 1.2, so where do
they start the array? I must be at some point further away from sector
0, leaving room for a protective MBR on the component disks that would
be entirely separate from any MBR inside the array. This leaves room
for grub to be installed on each component disk, rather than only the
first. It also means that it is incompatible with formats 0.9, 1.1, and
possibly 1.0, depending on whether it includes the first sector in the
array like 0.9 does.

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Software RAID and Fakeraid

On Mon, Jan 31, 2011 at 02:21:51PM -0500, Phillip Susi wrote:
> That is why I asked what the partition table should look like. When
> grub is installed there will be one ( and boot code ), the question is
> whether it should be empty, or have a protective partition that claims
> the entire disk, like EFI does.
>
> Normally with format 0.9, the raid array starts on sector 0, so the same
> MBR appears both at the start of the first physical disk, and at the
> start of the array. This is not possible with 1.1 or 1.2, so where do
> they start the array? I must be at some point further away from sector
> 0, leaving room for a protective MBR on the component disks that would
> be entirely separate from any MBR inside the array. This leaves room
> for grub to be installed on each component disk, rather than only the
> first. It also means that it is incompatible with formats 0.9, 1.1, and
> possibly 1.0, depending on whether it includes the first sector in the
> array like 0.9 does.

Well one of them stores the raid info at the end of the device, so that
one offers no help for the bootloader at all. I think the other stores
it at 4KB in to the device if my memory serves me right. Not sure if
the first 4KB are left empty for other uses or not.

It seems to me the simplest thing to do is to simply state in
documentation that grub can not be installed on md raid if there is no
partition table in use. This doesn't mean people can't do it for purely
data oriented drives, but for the drives with the boot laoder there
are limitations.

--
Len Sorensen
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Software RAID and Fakeraid

On 01/31/2011 05:12 PM, Lennart Sorensen wrote:
> Well one of them stores the raid info at the end of the device, so that
> one offers no help for the bootloader at all. I think the other stores

What do you mean help?

> it at 4KB in to the device if my memory serves me right. Not sure if
> the first 4KB are left empty for other uses or not.

It is left empty afaik, which is the whole point of shifting it over to 4kb.

> It seems to me the simplest thing to do is to simply state in
> documentation that grub can not be installed on md raid if there is no
> partition table in use. This doesn't mean people can't do it for purely
> data oriented drives, but for the drives with the boot laoder there
> are limitations.

Obviously doing nothing is the simplest thing one can do. Instead I
would like to do something and end up with a better, more functional
system. Hence my suggestion on WHAT to do and the existence of this
entire thread.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Software RAID and Fakeraid

On 1 February 2011 02:31, Phillip Susi wrote:
> On 01/31/2011 05:12 PM, Lennart Sorensen wrote:
>>
>> Well one of them stores the raid info at the end of the device, so t=
hat
>> one offers no help for the bootloader at all.  I think the othe=
r stores
>
> What do you mean help?
>


I guess that the point of this thread is to determine under what
conditions and where Grub can be installed on a raid array.

Since Grub is somewhat bloated compared to other bootloaders it has
its specific problems. However, the point is that even bootloaders
that do fit into a single sector do need to know where that sector can
be placed, if anywhere, and it is currently not possible to know with
mdraid unless the bootloader reimplements the full mdadm logic itself.

To avoid that mdadm should at the very least provide the information
whether the raid is bios-bootable or not. Ideally it should also
provide a list of sectors unused by the raid format in question.
=46ailing that at least a document clearly summarizing this for various
raid format versions and mdadm reporting the version would be helpful.

Thanks

Michal
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Software RAID and Fakeraid

On Mon, Jan 31, 2011 at 08:31:38PM -0500, Phillip Susi wrote:
> What do you mean help?

If the raid stores the raid info at the end, then the data starts at
sector 0. So no space for a bootloader at all.

> It is left empty afaik, which is the whole point of shifting it over to 4kb.

Certainly makes sense. Now is 4K enough for a boot loader? Not sure.

> Obviously doing nothing is the simplest thing one can do. Instead I
> would like to do something and end up with a better, more functional
> system. Hence my suggestion on WHAT to do and the existence of this
> entire thread.

I personally consider soft raid on raw devices so convluted that I
have never done it. I would rather have something I know works with my
bootloader and other tools, than gain that extra 1MB (at most) that not
having partitions gives. Also given many PCs won't boot from a drive
without a partition table, it isn't even an option then.

Really I think it comes down to only being able to handle the case
where there is a 4K gap, if you can fit grub in there in a format that
a system would be able to boot reading just sector 0 of the device.
The raid info at the start and raid info at the end both prevent you
from having any boot code in sector 0 so can't be supported at all.

--
Len Sorensen
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Software RAID and Fakeraid

On 02/01/2011 11:26 AM, Lennart Sorensen wrote:
> If the raid stores the raid info at the end, then the data starts at
> sector 0. So no space for a bootloader at all.

I know that is how it works with 0.9, but are you sure it is for 1.0?
If so, then for anything but raid-1 we will just have to try to install
only to the first device if it has an MBR.

> Certainly makes sense. Now is 4K enough for a boot loader? Not sure.

It is enough for the MBR. The core image will need to go elsewhere,
hence the proposal to ask mdadm for a suitable location.

> I personally consider soft raid on raw devices so convluted that I
> have never done it. I would rather have something I know works with my
> bootloader and other tools, than gain that extra 1MB (at most) that not
> having partitions gives. Also given many PCs won't boot from a drive
> without a partition table, it isn't even an option then.

That is why I like the idea of 1.2 since you could still have a bootable
MBR when using the whole disk. Though now that you mention it, I can't
think of a good reason to use the whole disk instead of a partition either.

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html