additional feature for linear

Good day,

I have an itch I'm hoping somebody can provide me information with to
scratch it. Please be patient, I'm not very well versed in all the
technical information regarding RAID, so I'm still finding my way
around.

Lets start with the itch.

I'm looking for a way to take 3 drives and create one pool. This is
similar to linear mode, but in this case if a drive in the linear mode
fail the rest of the pool is intact and just the missing files are
removed from the "device"

From my reading it looks like linear mode is the most likely
candidate, but there is no guarantee that the remaining data will be
accessible.

"If one disk crashes you will most probably lose all your data. You
can however be lucky to recover some data, since the filesystem will
just be missing one large consecutive chunk of data"

Would it not be possible to add functions to linear mode to mark this
missing chunks as "bad blacks" and let the FS deal with it as such
thereby allowing you to mount the linear device as per normal and
still access the remaining data ?

Id this is not possible, is there not some other way to implement this ?

I see there is some mails regarding adding additional drives to linear
as well and that the correct way is to stop the linear array and
recreate it with additional drives. Is this correct ?

I also see there was work being done on extending the array while
online. Was this ever done ?

Regards
Henti
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: additional feature for linear

On 17/09/11 09:12, Henti Smith wrote:
> Good day,
>
> I have an itch I'm hoping somebody can provide me information with to
> scratch it. Please be patient, I'm not very well versed in all the
> technical information regarding RAID, so I'm still finding my way
> around.
>
> Lets start with the itch.
>
> I'm looking for a way to take 3 drives and create one pool. This is
> similar to linear mode, but in this case if a drive in the linear mode
> fail the rest of the pool is intact and just the missing files are
> removed from the "device"
>
> From my reading it looks like linear mode is the most likely
> candidate, but there is no guarantee that the remaining data will be
> accessible.
>
> "If one disk crashes you will most probably lose all your data. You
> can however be lucky to recover some data, since the filesystem will
> just be missing one large consecutive chunk of data"
>
> Would it not be possible to add functions to linear mode to mark this
> missing chunks as "bad blacks" and let the FS deal with it as such
> thereby allowing you to mount the linear device as per normal and
> still access the remaining data ?
>
> Id this is not possible, is there not some other way to implement this ?
>
> I see there is some mails regarding adding additional drives to linear
> as well and that the correct way is to stop the linear array and
> recreate it with additional drives. Is this correct ?
>
> I also see there was work being done on extending the array while
> online. Was this ever done ?
>
> Regards
> Henti

All this would require a filesystem that can cope with suddenly losing
large numbers of disk blocks. Most file systems can't - so even if the
raid layer simply marked the missing chunks as bad, the filesystem would
die.

What you are asking for here is a filesystem that understands the
separate disks, and is careful to put individual files and related
metadata and directories onto individual disks (so that when a disk
dies, the file is either intact or completely lost), as well as
duplicating its critical metadata so that it will survive a disk loss.
All of this is the direct antithesis of raid, which aims to make
multiple disks look like a single block device.

I believe at the moment, your only answer (other than to re-think your
requirements) is ZFS. It may be that btrfs has the features you need -
they are certainly planned, as far as I know - but you'd have to be sure
of using a recent kernel and utilities.


--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: additional feature for linear

On 2011-09-17, at 03:14, Henti Smith wrote:

> Would it not be possible to add functions to linear mode to mark this
> missing chunks as "bad blacks" and let the FS deal with it as such.

Linux RAID can already deal with such a problem IIRC but if the
filesystem superblock is missing or directory entries, you ain't going
no where. This kind of feature would require filesystem knowledge of
where the RAID split the device and would be better suited to a FS
like BTRFS which mirror superblock and metadata without any Linux
RAID.

Envoyé de mon appareil mobile.

Jérôme Poulin
Solutions G.A.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: additional feature for linear

On Sat, Sep 17, 2011 at 5:20 PM, J=E9r=F4me Poulin com> wrote:
> On 2011-09-17, at 03:14, Henti Smith wrote:
>
>> Would it not be possible to add functions to linear mode to mark thi=
s
>> missing chunks as "bad blacks" and let the FS deal with it as such.
>
> Linux RAID can already deal with such a problem IIRC but if the
> filesystem superblock is missing or directory entries, you ain't goin=
g
> no where. This kind of feature would require filesystem knowledge of
> where the RAID split the device and would be better suited to a FS
> like BTRFS which mirror superblock and metadata without any Linux
> RAID.

Hi J=E9r=F4me

Thanks for the reply. I though BTRFS would be a good option as well,
but they don't seem to understand what I'm looking for :)

I have found a fuse base FS that seems to do this, so will play with th=
at.

It's called http://romanrm.ru/en/mhddfs and seems to do what I need it
to do so will test :)

Thanks again for replying.

Henti
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: additional feature for linear

On 17/09/11 17:32, Henti Smith wrote:
> On Sat, Sep 17, 2011 at 5:20 PM, J=E9r=F4me Poulin com> wrote:
>> On 2011-09-17, at 03:14, Henti Smith wrote:
>>
>>> Would it not be possible to add functions to linear mode to mark th=
is
>>> missing chunks as "bad blacks" and let the FS deal with it as such.
>>
>> Linux RAID can already deal with such a problem IIRC but if the
>> filesystem superblock is missing or directory entries, you ain't goi=
ng
>> no where. This kind of feature would require filesystem knowledge of
>> where the RAID split the device and would be better suited to a FS
>> like BTRFS which mirror superblock and metadata without any Linux
>> RAID.
>
> Hi J=E9r=F4me
>
> Thanks for the reply. I though BTRFS would be a good option as well,
> but they don't seem to understand what I'm looking for :)
>
> I have found a fuse base FS that seems to do this, so will play with =
that.
>
> It's called http://romanrm.ru/en/mhddfs and seems to do what I need i=
t
> to do so will test :)
>
> Thanks again for replying.
>
> Henti


That looks like an interesting idea, and would be particularly useful i=
f=20
you've already got several disks with filesystems and want to join them=
=20
together. However, I would never consider it for a /new/ setup. The=20
price of disk space is so low that I never again expect to set up a=20
computer without raid, unless it is a laptop with only space for one=20
disk, or a machine using some nameless inferior operating system that=20
doesn't support proper raid. Certainly if you are thinking of putting=20
three disks in a machine then raid10,far would be the natural choice, o=
r=20
raid5 if you want a little more space for your pennies.


--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: RAID1 with MBR and GPT fails to auto-assemble during boot

I have a RAID1 array with 2 partitions. One partition is on an MBR drive and one is on a GPT drive.

When I reboot, only the MBR drive has been assembled.

Using

mdadm /dev/md1 --add /dev/sdc1

mdadm reports that the drive is "readded" and syncing proceeds without difficulty. Reboot, and the array again
shows up with only one drive.

This array contains an LVM containing the system partition. mdadm.conf contains
--------------------------------------------------------
#
# PLEASE DO NOT MODIFY THIS CONFIGURATION FILE!
# This configuration file was auto-generated
# by Openfiler. Please do not modify it.
#
# Generated at: Sun Nov 1 19:28:17 MST 2009
#

DEVICE partitions
ARRAY /dev/md8 UUID=066bd3ee:b35a0f59:d111a4d2:91b551e6
ARRAY /dev/md7 UUID=8245e0c8:aeb40213:7d63f449:7e79d5c0
ARRAY /dev/md6 UUID=78d87cc4:4e8dfa53:6da2eef9:a1dfa4c3
ARRAY /dev/md5 UUID=b282fd35:d6ce1cb8:9747d521:9d1e8e97
ARRAY /dev/md1 UUID=21b91d6f:e97bf1e3:ef6fdcec:d6c4a58e
PROGRAM /opt/openfiler/bin/mdalert
------------------------------------------------------------ --

dmesg shows

[ 5.415275] md: Autodetecting RAID arrays.
[ 5.415509] md: Scanned 1 and added 1 devices.
[ 5.415514] md: autorun ...
[ 5.415519] md: considering sda1 ...
[ 5.415535] md: adding sda1 ...
[ 5.415541] md: created md1
[ 5.415546] md: bind
[ 5.415572] md: running:
[ 5.416220] raid1: raid set md1 active with 1 out of 2 mirrors
[ 5.416395] md: ... autorun DONE.
[ 5.416861] md: Autodetecting RAID arrays.
[ 5.416867] md: Scanned 0 and added 0 devices.
[ 5.416872] md: autorun ...
[ 5.416876] md: ... autorun DONE.
[ 5.417349] md: Autodetecting RAID arrays.
[ 5.417357] md: Scanned 0 and added 0 devices.
[ 5.417362] md: autorun ...
[ 5.417365] md: ... autorun DONE.
[ 5.417829] md: Autodetecting RAID arrays.
[ 5.417836] md: Scanned 0 and added 0 devices.
[ 5.417840] md: autorun ...
[ 5.417844] md: ... autorun DONE.
[ 5.418296] md: Autodetecting RAID arrays.
[ 5.418303] md: Scanned 0 and added 0 devices.
[ 5.418307] md: autorun ...
[ 5.418311] md: ... autorun DONE.


Later on, it assembles /dev/md5 to /dev/md8 without difficulty, even though one of the disk partitions from each is on the GPT drive.

This is Openfiler with kernel 2.6.29.6 and mdadm version 2.6.4.

Previously, when /dev/sdc was partitioned MBR, there was no problem with assembling the RAID1 array.

Output of mdadm -E for /dev/sda1 and /dev/sdc1 is

/dev/sda1:
Magic : a92b4efc
Version : 00.90.03
UUID : 21b91d6f:e97bf1e3:ef6fdcec:d6c4a58e
Creation Time : Sat Oct 31 14:27:53 2009
Raid Level : raid1
Used Dev Size : 9767424 (9.31 GiB 10.00 GB)
Array Size : 9767424 (9.31 GiB 10.00 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 1

Update Time : Mon Sep 19 18:22:14 2011
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Checksum : 148e96a2 - correct
Events : 0.6224


Number Major Minor RaidDevice State
this 0 8 1 0 active sync /dev/sda1

0 0 8 1 0 active sync /dev/sda1
1 1 8 33 1 active sync /dev/sdc1




/dev/sdc1:
Magic : a92b4efc
Version : 00.90.03
UUID : 21b91d6f:e97bf1e3:ef6fdcec:d6c4a58e
Creation Time : Sat Oct 31 14:27:53 2009
Raid Level : raid1
Used Dev Size : 9767424 (9.31 GiB 10.00 GB)
Array Size : 9767424 (9.31 GiB 10.00 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 1

Update Time : Mon Sep 19 18:24:14 2011
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Checksum : 148e973c - correct
Events : 0.6224


Number Major Minor RaidDevice State
this 1 8 33 1 active sync /dev/sdc1

0 0 8 1 0 active sync /dev/sda1
1 1 8 33 1 active sync /dev/sdc1


Why does Linux fail to auto-assemble /dev/md1 during boot and how do I fix the problem?


Thanks!

Jim



--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html