Linux software RAID assistance

Hi all

I use a 3ware 9500-12 port sata card (JBOD) which will not work without=
a
128mb sodimm. The sodimm socket is flakey and the result is that the
machine occasionally crashes. Yesterday I finally gave in and put=20
together another
machine so that I can rsync between them. When I turned the machine
on today to set up rync, the RAID array was not gone, but corrupted.=20
Typical...

I built the array in Aug 2010 using the following command:

mdadm --create --verbose /dev/md0 --metadata=3D1.1 --level=3D5
--raid-devices=3D10 /dev/sd{b,c,d,e,f,g,h,i,j,k}1 --chunk=3D64

Using LVM, I did the following:
pvscan
pvcreate -M2 /dev/md0
vgcreate lvm-raid /dev/md0
vgdisplay lvm-raid
vgscan
lvscan
lvcreate -v -l 100%VG -n RAID lvm-raid
lvdisplay /dev/lvm-raid/lvm0

I then formatted using:
mkfs -t ext4 -v -m .1 -b 4096 -E stride=3D16,stripe-width=3D144=20
/dev/lvm-raid/RAID

This worked perfectly since I created the array. Now mdadm is coming u=
p=20
with

proxmox:/dev/md# mdadm --assemble --scan --verbose
mdadm: looking for devices for further assembly
mdadm: no recogniseable superblock on /dev/md/ubuntu:0
mdadm: cannot open device /dev/dm-2: Device or resource busy
mdadm: cannot open device /dev/dm-1: Device or resource busy
mdadm: cannot open device /dev/dm-0: Device or resource busy
mdadm: cannot open device /dev/sdm1: Device or resource busy
mdadm: cannot open device /dev/sdm: Device or resource busy
mdadm: cannot open device /dev/sdl1: Device or resource busy
mdadm: cannot open device /dev/sdl: Device or resource busy
mdadm: cannot open device /dev/sdk1: Device or resource busy
mdadm: cannot open device /dev/sdk: Device or resource busy
mdadm: cannot open device /dev/sdj1: Device or resource busy
mdadm: cannot open device /dev/sdj: Device or resource busy
mdadm: cannot open device /dev/sdh1: Device or resource busy
mdadm: cannot open device /dev/sdh: Device or resource busy
mdadm: cannot open device /dev/sdi1: Device or resource busy
mdadm: cannot open device /dev/sdi: Device or resource busy
mdadm: cannot open device /dev/sdg1: Device or resource busy
mdadm: cannot open device /dev/sdg: Device or resource busy
mdadm: cannot open device /dev/sdf1: Device or resource busy
mdadm: cannot open device /dev/sdf: Device or resource busy
mdadm: cannot open device /dev/sde1: Device or resource busy
mdadm: cannot open device /dev/sde: Device or resource busy
mdadm: no RAID superblock on /dev/sdd
mdadm: cannot open device /dev/sda2: Device or resource busy
mdadm: cannot open device /dev/sda1: Device or resource busy
mdadm: cannot open device /dev/sda: Device or resource busy
mdadm: /dev/sdd1 is identified as a member of /dev/md/proï¿=BDlox:0=
, slot 0.
mdadm: no uptodate device for slot 1 of /dev/md/proï¿=BDlox:0
mdadm: no uptodate device for slot 2 of /dev/md/proï¿=BDlox:0
mdadm: no uptodate device for slot 3 of /dev/md/proï¿=BDlox:0
mdadm: no uptodate device for slot 4 of /dev/md/proï¿=BDlox:0
mdadm: no uptodate device for slot 5 of /dev/md/proï¿=BDlox:0
mdadm: no uptodate device for slot 6 of /dev/md/proï¿=BDlox:0
mdadm: no uptodate device for slot 7 of /dev/md/proï¿=BDlox:0
mdadm: no uptodate device for slot 8 of /dev/md/proï¿=BDlox:0
mdadm: no uptodate device for slot 9 of /dev/md/proï¿=BDlox:0
mdadm: failed to add /dev/sdd1 to /dev/md/proï¿=BDlox:0: Invalid ar=
gument
mdadm: /dev/md/proï¿=BDlox:0 assembled from 0 drives - not enough t=
o start
the array.
mdadm: looking for devices for further assembly
mdadm: no recogniseable superblock on /dev/sdd
mdadm: No arrays found in config file or automatically

pvscan and vgscan show nothing.

So I tried running mdadm --create --verbose /dev/md0 --metadata=3D1.1
--level=3D5 --raid-devices=3D10 missing /dev/sde1 /dev/sdf1 /dev/sdg1
/dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 --chunk=3D6=
4

as it seemed that /dev/sdd1 failed to be added to the array. This did=20
nothing.

dmesg contains:

md: invalid superblock checksum on sdd1
md: sdd1 does not have a valid v1.1 superblock, not importing!
md: md_import_device returned -22

The output of mdadm -E is as follows:

proxmox:~# mdadm -E /dev/sd[d-m]1
/dev/sdd1:
Magic : a92b4efc
Version : 1.1
Feature Map : 0x0
Array UUID : c4f62f32:4244a1db:b7746203:f10b5227
Name : proølox:0
Creation Time : Sat Aug 21 19:16:38 2010
Raid Level : raid5
Raid Devices : 10

Avail Dev Size : 1953099512 (931.31 GiB 999.99 GB)
Array Size : 17577894528 (8381.79 GiB 8999.88 GB)
Used Dev Size : 1953099392 (931.31 GiB 999.99 GB)
Data Offset : 264 sectors
Super Offset : 0 sectors
State : clean
Device UUID : b0ffd0a1:866676af:3ae4c03c:d219676e

Update Time : Mon Feb 7 21:08:29 2011
Checksum : 13aa9685 - expected 93aa9672
Events : 60802

Layout : left-symmetric
Chunk Size : 64K

Device Role : Active device 0
Array State : AAAAAAAAAA ('A' == active, '.' == missing)
/dev/sde1:
Magic : a92b4efc
Version : 1.1
Feature Map : 0x0
Array UUID : 7d55c29e:273c35da:f6438197:0365f95f
Name : ubuntu:0
Creation Time : Thu Feb 10 11:59:48 2011
Raid Level : raid5
Raid Devices : 10

Avail Dev Size : 1953099512 (931.31 GiB 999.99 GB)
Array Size : 17577894528 (8381.79 GiB 8999.88 GB)
Used Dev Size : 1953099392 (931.31 GiB 999.99 GB)
Data Offset : 264 sectors
Super Offset : 0 sectors
State : clean
Device UUID : 4121fa31:7543218c:f42a937d:fddf04e8

Update Time : Thu Feb 10 12:17:59 2011
Checksum : 5f965c84 - correct
Events : 4

Layout : left-symmetric
Chunk Size : 64K

Device Role : Active device 1
Array State : .AAAAAAAAA ('A' == active, '.' == missing)
/dev/sdf1:
Magic : a92b4efc
Version : 1.1
Feature Map : 0x0
Array UUID : 7d55c29e:273c35da:f6438197:0365f95f
Name : ubuntu:0
Creation Time : Thu Feb 10 11:59:48 2011
Raid Level : raid5
Raid Devices : 10

Avail Dev Size : 1953099512 (931.31 GiB 999.99 GB)
Array Size : 17577894528 (8381.79 GiB 8999.88 GB)
Used Dev Size : 1953099392 (931.31 GiB 999.99 GB)
Data Offset : 264 sectors
Super Offset : 0 sectors
State : clean
Device UUID : e70da8ed:c80e9533:7d8e200e:dc285255

Update Time : Thu Feb 10 12:17:59 2011
Checksum : c092c0e5 - correct
Events : 4

Layout : left-symmetric
Chunk Size : 64K

Device Role : Active device 2
Array State : .AAAAAAAAA ('A' == active, '.' == missing)
/dev/sdg1:
Magic : a92b4efc
Version : 1.1
Feature Map : 0x0
Array UUID : 7d55c29e:273c35da:f6438197:0365f95f
Name : ubuntu:0
Creation Time : Thu Feb 10 11:59:48 2011
Raid Level : raid5
Raid Devices : 10

Avail Dev Size : 1953099512 (931.31 GiB 999.99 GB)
Array Size : 17577894528 (8381.79 GiB 8999.88 GB)
Used Dev Size : 1953099392 (931.31 GiB 999.99 GB)
Data Offset : 264 sectors
Super Offset : 0 sectors
State : clean
Device UUID : 5878fdbb:d0c7c892:40c8fa6c:0c5e257b

Update Time : Thu Feb 10 12:17:59 2011
Checksum : 72c95353 - correct
Events : 4

Layout : left-symmetric
Chunk Size : 64K

Device Role : Active device 3
Array State : .AAAAAAAAA ('A' == active, '.' == missing)
/dev/sdh1:
Magic : a92b4efc
Version : 1.1
Feature Map : 0x0
Array UUID : 7d55c29e:273c35da:f6438197:0365f95f
Name : ubuntu:0
Creation Time : Thu Feb 10 11:59:48 2011
Raid Level : raid5
Raid Devices : 10

Avail Dev Size : 1953099512 (931.31 GiB 999.99 GB)
Array Size : 17577894528 (8381.79 GiB 8999.88 GB)
Used Dev Size : 1953099392 (931.31 GiB 999.99 GB)
Data Offset : 264 sectors
Super Offset : 0 sectors
State : clean
Device UUID : 7bced3d0:5ebac414:02effc8b:ac1ad69e

Update Time : Thu Feb 10 12:17:59 2011
Checksum : 4c4e7f67 - correct
Events : 4

Layout : left-symmetric
Chunk Size : 64K

Device Role : Active device 4
Array State : .AAAAAAAAA ('A' == active, '.' == missing)
/dev/sdi1:
Magic : a92b4efc
Version : 1.1
Feature Map : 0x0
Array UUID : 7d55c29e:273c35da:f6438197:0365f95f
Name : ubuntu:0
Creation Time : Thu Feb 10 11:59:48 2011
Raid Level : raid5
Raid Devices : 10

Avail Dev Size : 1953099512 (931.31 GiB 999.99 GB)
Array Size : 17577894528 (8381.79 GiB 8999.88 GB)
Used Dev Size : 1953099392 (931.31 GiB 999.99 GB)
Data Offset : 264 sectors
Super Offset : 0 sectors
State : clean
Device UUID : 56513558:a25025bb:11e8b563:ae42f8d4

Update Time : Thu Feb 10 12:17:59 2011
Checksum : 87ebb998 - correct
Events : 4

Layout : left-symmetric
Chunk Size : 64K

Device Role : Active device 5
Array State : .AAAAAAAAA ('A' == active, '.' == missing)
/dev/sdj1:
Magic : a92b4efc
Version : 1.1
Feature Map : 0x0
Array UUID : 7d55c29e:273c35da:f6438197:0365f95f
Name : ubuntu:0
Creation Time : Thu Feb 10 11:59:48 2011
Raid Level : raid5
Raid Devices : 10

Avail Dev Size : 1953099512 (931.31 GiB 999.99 GB)
Array Size : 17577894528 (8381.79 GiB 8999.88 GB)
Used Dev Size : 1953099392 (931.31 GiB 999.99 GB)
Data Offset : 264 sectors
Super Offset : 0 sectors
State : clean
Device UUID : 90621937:23cff36c:c55b1581:ed28b433

Update Time : Thu Feb 10 12:17:59 2011
Checksum : 94b9a346 - correct
Events : 4

Layout : left-symmetric
Chunk Size : 64K

Device Role : Active device 6
Array State : .AAAAAAAAA ('A' == active, '.' == missing)
/dev/sdk1:
Magic : a92b4efc
Version : 1.1
Feature Map : 0x0
Array UUID : 7d55c29e:273c35da:f6438197:0365f95f
Name : ubuntu:0
Creation Time : Thu Feb 10 11:59:48 2011
Raid Level : raid5
Raid Devices : 10

Avail Dev Size : 1953099512 (931.31 GiB 999.99 GB)
Array Size : 17577894528 (8381.79 GiB 8999.88 GB)
Used Dev Size : 1953099392 (931.31 GiB 999.99 GB)
Data Offset : 264 sectors
Super Offset : 0 sectors
State : clean
Device UUID : 72d5ce06:855aa34c:3a354abc:0ddb6e80

Update Time : Thu Feb 10 12:17:59 2011
Checksum : cc0e2d20 - correct
Events : 4

Layout : left-symmetric
Chunk Size : 64K

Device Role : Active device 7
Array State : .AAAAAAAAA ('A' == active, '.' == missing)
/dev/sdl1:
Magic : a92b4efc
Version : 1.1
Feature Map : 0x0
Array UUID : 7d55c29e:273c35da:f6438197:0365f95f
Name : ubuntu:0
Creation Time : Thu Feb 10 11:59:48 2011
Raid Level : raid5
Raid Devices : 10

Avail Dev Size : 1953099512 (931.31 GiB 999.99 GB)
Array Size : 17577894528 (8381.79 GiB 8999.88 GB)
Used Dev Size : 1953099392 (931.31 GiB 999.99 GB)
Data Offset : 264 sectors
Super Offset : 0 sectors
State : clean
Device UUID : 8792e04a:ee298ace:fec79850:ddd3167b

Update Time : Thu Feb 10 12:17:59 2011
Checksum : 20fd4534 - correct
Events : 4

Layout : left-symmetric
Chunk Size : 64K

Device Role : Active device 8
Array State : .AAAAAAAAA ('A' == active, '.' == missing)
/dev/sdm1:
Magic : a92b4efc
Version : 1.1
Feature Map : 0x0
Array UUID : 7d55c29e:273c35da:f6438197:0365f95f
Name : ubuntu:0
Creation Time : Thu Feb 10 11:59:48 2011
Raid Level : raid5
Raid Devices : 10

Avail Dev Size : 1953099512 (931.31 GiB 999.99 GB)
Array Size : 17577894528 (8381.79 GiB 8999.88 GB)
Used Dev Size : 1953099392 (931.31 GiB 999.99 GB)
Data Offset : 264 sectors
Super Offset : 0 sectors
State : clean
Device UUID : 8916d09c:448a5791:c2923182:ab99eb92

Update Time : Thu Feb 10 12:17:59 2011
Checksum : 7f27ba1f - correct
Events : 4

Layout : left-symmetric
Chunk Size : 64K

Device Role : Active device 9
Array State : .AAAAAAAAA ('A' == active, '.' == missing)

I'm confident the data is on there, I just need to get a backup of it o=
ff !

Yours desperately.

regards
Simon McNair


--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

Hi Simon,

On 02/10/2011 11:16 AM, Simon McNair wrote:
>=20
[trim /]
>=20
> So I tried running mdadm --create --verbose /dev/md0 --metadata=3D1.1
> --level=3D5 --raid-devices=3D10 missing /dev/sde1 /dev/sdf1 /dev/sdg1
> /dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 --chunk=3D=
64

Uh-Oh. You told mdadm to create an array with a missing member, but yo=
u didn't tell it to assume-clean.

> as it seemed that /dev/sdd1 failed to be added to the array. This di=
d nothing.

According to your mdadm -E output below, it did what you asked: it cre=
ated a new array from the devices given (note the new array UUID on the=
nine devices). Since you have one device missing, it can't rebuild, s=
o you might not have lost your data yet.

That mdadm -E /dev/sdd1 reports its array assembled and clean is odd.

Please show us the output of /proc/mdstat and then "mdadm -D /dev/md*" =
or "mdadm -D /dev/md/*"

Phil

>=20
> dmesg contains:
>=20
> md: invalid superblock checksum on sdd1
> md: sdd1 does not have a valid v1.1 superblock, not importing!
> md: md_import_device returned -22
>=20
> The output of mdadm -E is as follows:
>=20
> proxmox:~# mdadm -E /dev/sd[d-m]1
> /dev/sdd1:
> Magic : a92b4efc
> Version : 1.1
> Feature Map : 0x0
> Array UUID : c4f62f32:4244a1db:b7746203:f10b5227
> Name : proølox:0
> Creation Time : Sat Aug 21 19:16:38 2010
> Raid Level : raid5
> Raid Devices : 10
>=20
> Avail Dev Size : 1953099512 (931.31 GiB 999.99 GB)
> Array Size : 17577894528 (8381.79 GiB 8999.88 GB)
> Used Dev Size : 1953099392 (931.31 GiB 999.99 GB)
> Data Offset : 264 sectors
> Super Offset : 0 sectors
> State : clean
> Device UUID : b0ffd0a1:866676af:3ae4c03c:d219676e
>=20
> Update Time : Mon Feb 7 21:08:29 2011
> Checksum : 13aa9685 - expected 93aa9672
> Events : 60802
>=20
> Layout : left-symmetric
> Chunk Size : 64K
>=20
> Device Role : Active device 0
> Array State : AAAAAAAAAA ('A' == active, '.' == missing)
> /dev/sde1:
> Magic : a92b4efc
> Version : 1.1
> Feature Map : 0x0
> Array UUID : 7d55c29e:273c35da:f6438197:0365f95f
> Name : ubuntu:0
> Creation Time : Thu Feb 10 11:59:48 2011
> Raid Level : raid5
> Raid Devices : 10
>=20
> Avail Dev Size : 1953099512 (931.31 GiB 999.99 GB)
> Array Size : 17577894528 (8381.79 GiB 8999.88 GB)
> Used Dev Size : 1953099392 (931.31 GiB 999.99 GB)
> Data Offset : 264 sectors
> Super Offset : 0 sectors
> State : clean
> Device UUID : 4121fa31:7543218c:f42a937d:fddf04e8
>=20
> Update Time : Thu Feb 10 12:17:59 2011
> Checksum : 5f965c84 - correct
> Events : 4
>=20
> Layout : left-symmetric
> Chunk Size : 64K
>=20
> Device Role : Active device 1
> Array State : .AAAAAAAAA ('A' == active, '.' == missing)
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

On Thu, 10 Feb 2011 16:16:44 +0000 Simon McNair =
wrote:

>=20
> Hi all
>=20
> I use a 3ware 9500-12 port sata card (JBOD) which will not work witho=
ut a
> 128mb sodimm. The sodimm socket is flakey and the result is that the
> machine occasionally crashes. Yesterday I finally gave in and put=20
> together another
> machine so that I can rsync between them. When I turned the machine
> on today to set up rync, the RAID array was not gone, but corrupted.=20
> Typical...

Presumably the old machine was called 'ubuntu' and the new machine 'pro=
ølox'


>=20
> I built the array in Aug 2010 using the following command:
>=20
> mdadm --create --verbose /dev/md0 --metadata=3D1.1 --level=3D5
> --raid-devices=3D10 /dev/sd{b,c,d,e,f,g,h,i,j,k}1 --chunk=3D64
>=20
> Using LVM, I did the following:
> pvscan
> pvcreate -M2 /dev/md0
> vgcreate lvm-raid /dev/md0
> vgdisplay lvm-raid
> vgscan
> lvscan
> lvcreate -v -l 100%VG -n RAID lvm-raid
> lvdisplay /dev/lvm-raid/lvm0
>=20
> I then formatted using:
> mkfs -t ext4 -v -m .1 -b 4096 -E stride=3D16,stripe-width=3D144=20
> /dev/lvm-raid/RAID
>=20
> This worked perfectly since I created the array. Now mdadm is coming=
up=20
> with
>=20
> proxmox:/dev/md# mdadm --assemble --scan --verbose
> mdadm: looking for devices for further assembly
> mdadm: no recogniseable superblock on /dev/md/ubuntu:0

And it seems that ubuntu:0 have been successfully assembled.
It is missing one device for some reason (sdd1) but RAID can cope with =
that.



> mdadm: cannot open device /dev/dm-2: Device or resource busy
> mdadm: cannot open device /dev/dm-1: Device or resource busy
> mdadm: cannot open device /dev/dm-0: Device or resource busy
> mdadm: cannot open device /dev/sdm1: Device or resource busy
> mdadm: cannot open device /dev/sdm: Device or resource busy
> mdadm: cannot open device /dev/sdl1: Device or resource busy
> mdadm: cannot open device /dev/sdl: Device or resource busy
> mdadm: cannot open device /dev/sdk1: Device or resource busy
> mdadm: cannot open device /dev/sdk: Device or resource busy
> mdadm: cannot open device /dev/sdj1: Device or resource busy
> mdadm: cannot open device /dev/sdj: Device or resource busy
> mdadm: cannot open device /dev/sdh1: Device or resource busy
> mdadm: cannot open device /dev/sdh: Device or resource busy
> mdadm: cannot open device /dev/sdi1: Device or resource busy
> mdadm: cannot open device /dev/sdi: Device or resource busy
> mdadm: cannot open device /dev/sdg1: Device or resource busy
> mdadm: cannot open device /dev/sdg: Device or resource busy
> mdadm: cannot open device /dev/sdf1: Device or resource busy
> mdadm: cannot open device /dev/sdf: Device or resource busy
> mdadm: cannot open device /dev/sde1: Device or resource busy
> mdadm: cannot open device /dev/sde: Device or resource busy
> mdadm: no RAID superblock on /dev/sdd
> mdadm: cannot open device /dev/sda2: Device or resource busy
> mdadm: cannot open device /dev/sda1: Device or resource busy
> mdadm: cannot open device /dev/sda: Device or resource busy
> mdadm: /dev/sdd1 is identified as a member of /dev/md/proï¿=BDlox=
:0, slot 0.
> mdadm: no uptodate device for slot 1 of /dev/md/proï¿=BDlox:0
> mdadm: no uptodate device for slot 2 of /dev/md/proï¿=BDlox:0
> mdadm: no uptodate device for slot 3 of /dev/md/proï¿=BDlox:0
> mdadm: no uptodate device for slot 4 of /dev/md/proï¿=BDlox:0
> mdadm: no uptodate device for slot 5 of /dev/md/proï¿=BDlox:0
> mdadm: no uptodate device for slot 6 of /dev/md/proï¿=BDlox:0
> mdadm: no uptodate device for slot 7 of /dev/md/proï¿=BDlox:0
> mdadm: no uptodate device for slot 8 of /dev/md/proï¿=BDlox:0
> mdadm: no uptodate device for slot 9 of /dev/md/proï¿=BDlox:0
> mdadm: failed to add /dev/sdd1 to /dev/md/proï¿=BDlox:0: Invalid =
argument
> mdadm: /dev/md/proï¿=BDlox:0 assembled from 0 drives - not enough=
to start
> the array.

This looks like it is *after* to trying the --create command you give
below.. It is best to report things in the order they happen, else you=
can
confuse people (or get caught out!).


> mdadm: looking for devices for further assembly
> mdadm: no recogniseable superblock on /dev/sdd
> mdadm: No arrays found in config file or automatically
>=20
> pvscan and vgscan show nothing.
>=20
> So I tried running mdadm --create --verbose /dev/md0 --metadata=3D1.1
> --level=3D5 --raid-devices=3D10 missing /dev/sde1 /dev/sdf1 /dev/sdg1
> /dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 --chunk=3D=
64
>=20
> as it seemed that /dev/sdd1 failed to be added to the array. This di=
d=20
> nothing.

It did not to nothing. It wrote a superblock to /dev/sdd1 and complain=
ed
that it couldn't write to all the others --- didn't it?


>=20
> dmesg contains:
>=20
> md: invalid superblock checksum on sdd1

I guess that is why sdd1 was missing from 'ubuntu:0'. Though as I cann=
ot
tell if this happened before or after any of the various things reporte=
d
above, it is hard to be sure.


The real mystery is why 'pvscan' reports nothing.

What about
pvscan --verbose

or

blkid -p /dev/md/ubuntu:0

or even

dd of=3D/dev/md/ubuntu:0 count=3D8 | od -c=20

??

NeilBrown


--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

Hi Neal,
Thanks for the response. Phil Turmel has been giving me some kind=20
assistance. After some investigation we have found that the 3ware card=
=20
I'm using as a controller (as JBOD) is making the whole thing far too=20
flakey and providing some data corruption (hopefully not for too long).=
=20
Thankfully this is mainly a data archive so the data changes very=20
infrequently. I have ordered a Supermicro 8 port sata controller and=20
intend to use that, in conjunction with the 6 onboard sata ports to try=
=20
and get the array back in shape. Some seriously bad stuff has happened=
,=20
so I'm hoping the data is still retrievable.

I'll cc the latest update to the list. Any input you can provide would=
=20
be seriously welcome.

regards
Simon

On 15/02/2011 04:53, NeilBrown wrote:
> On Thu, 10 Feb 2011 16:16:44 +0000 Simon McNair > wrote:
>
>> Hi all
>>
>> I use a 3ware 9500-12 port sata card (JBOD) which will not work with=
out a
>> 128mb sodimm. The sodimm socket is flakey and the result is that th=
e
>> machine occasionally crashes. Yesterday I finally gave in and put
>> together another
>> machine so that I can rsync between them. When I turned the machine
>> on today to set up rync, the RAID array was not gone, but corrupted.
>> Typical...
> Presumably the old machine was called 'ubuntu' and the new machine 'p=
roølox'
>
>
>> I built the array in Aug 2010 using the following command:
>>
>> mdadm --create --verbose /dev/md0 --metadata=3D1.1 --level=3D5
>> --raid-devices=3D10 /dev/sd{b,c,d,e,f,g,h,i,j,k}1 --chunk=3D64
>>
>> Using LVM, I did the following:
>> pvscan
>> pvcreate -M2 /dev/md0
>> vgcreate lvm-raid /dev/md0
>> vgdisplay lvm-raid
>> vgscan
>> lvscan
>> lvcreate -v -l 100%VG -n RAID lvm-raid
>> lvdisplay /dev/lvm-raid/lvm0
>>
>> I then formatted using:
>> mkfs -t ext4 -v -m .1 -b 4096 -E stride=3D16,stripe-width=3D144
>> /dev/lvm-raid/RAID
>>
>> This worked perfectly since I created the array. Now mdadm is comin=
g up
>> with
>>
>> proxmox:/dev/md# mdadm --assemble --scan --verbose
>> mdadm: looking for devices for further assembly
>> mdadm: no recogniseable superblock on /dev/md/ubuntu:0
> And it seems that ubuntu:0 have been successfully assembled.
> It is missing one device for some reason (sdd1) but RAID can cope wit=
h that.
>
>
>
>> mdadm: cannot open device /dev/dm-2: Device or resource busy
>> mdadm: cannot open device /dev/dm-1: Device or resource busy
>> mdadm: cannot open device /dev/dm-0: Device or resource busy
>> mdadm: cannot open device /dev/sdm1: Device or resource busy
>> mdadm: cannot open device /dev/sdm: Device or resource busy
>> mdadm: cannot open device /dev/sdl1: Device or resource busy
>> mdadm: cannot open device /dev/sdl: Device or resource busy
>> mdadm: cannot open device /dev/sdk1: Device or resource busy
>> mdadm: cannot open device /dev/sdk: Device or resource busy
>> mdadm: cannot open device /dev/sdj1: Device or resource busy
>> mdadm: cannot open device /dev/sdj: Device or resource busy
>> mdadm: cannot open device /dev/sdh1: Device or resource busy
>> mdadm: cannot open device /dev/sdh: Device or resource busy
>> mdadm: cannot open device /dev/sdi1: Device or resource busy
>> mdadm: cannot open device /dev/sdi: Device or resource busy
>> mdadm: cannot open device /dev/sdg1: Device or resource busy
>> mdadm: cannot open device /dev/sdg: Device or resource busy
>> mdadm: cannot open device /dev/sdf1: Device or resource busy
>> mdadm: cannot open device /dev/sdf: Device or resource busy
>> mdadm: cannot open device /dev/sde1: Device or resource busy
>> mdadm: cannot open device /dev/sde: Device or resource busy
>> mdadm: no RAID superblock on /dev/sdd
>> mdadm: cannot open device /dev/sda2: Device or resource busy
>> mdadm: cannot open device /dev/sda1: Device or resource busy
>> mdadm: cannot open device /dev/sda: Device or resource busy
>> mdadm: /dev/sdd1 is identified as a member of /dev/md/proï¿=BDlo=
x:0, slot 0.
>> mdadm: no uptodate device for slot 1 of /dev/md/proï¿=BDlox:0
>> mdadm: no uptodate device for slot 2 of /dev/md/proï¿=BDlox:0
>> mdadm: no uptodate device for slot 3 of /dev/md/proï¿=BDlox:0
>> mdadm: no uptodate device for slot 4 of /dev/md/proï¿=BDlox:0
>> mdadm: no uptodate device for slot 5 of /dev/md/proï¿=BDlox:0
>> mdadm: no uptodate device for slot 6 of /dev/md/proï¿=BDlox:0
>> mdadm: no uptodate device for slot 7 of /dev/md/proï¿=BDlox:0
>> mdadm: no uptodate device for slot 8 of /dev/md/proï¿=BDlox:0
>> mdadm: no uptodate device for slot 9 of /dev/md/proï¿=BDlox:0
>> mdadm: failed to add /dev/sdd1 to /dev/md/proï¿=BDlox:0: Invalid=
argument
>> mdadm: /dev/md/proï¿=BDlox:0 assembled from 0 drives - not enoug=
h to start
>> the array.
> This looks like it is *after* to trying the --create command you give
> below.. It is best to report things in the order they happen, else y=
ou can
> confuse people (or get caught out!).
>
>
>> mdadm: looking for devices for further assembly
>> mdadm: no recogniseable superblock on /dev/sdd
>> mdadm: No arrays found in config file or automatically
>>
>> pvscan and vgscan show nothing.
>>
>> So I tried running mdadm --create --verbose /dev/md0 --metadata=3D1.=
1
>> --level=3D5 --raid-devices=3D10 missing /dev/sde1 /dev/sdf1 /dev/sdg=
1
>> /dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 --chunk=3D=
64
>>
>> as it seemed that /dev/sdd1 failed to be added to the array. This d=
id
>> nothing.
> It did not to nothing. It wrote a superblock to /dev/sdd1 and compla=
ined
> that it couldn't write to all the others --- didn't it?
>
>
>> dmesg contains:
>>
>> md: invalid superblock checksum on sdd1
> I guess that is why sdd1 was missing from 'ubuntu:0'. Though as I ca=
nnot
> tell if this happened before or after any of the various things repor=
ted
> above, it is hard to be sure.
>
>
> The real mystery is why 'pvscan' reports nothing.
>
> What about
> pvscan --verbose
>
> or
>
> blkid -p /dev/md/ubuntu:0
>
> or even
>
> dd of=3D/dev/md/ubuntu:0 count=3D8 | od -c
>
> ??
>
> NeilBrown
>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

Hi Neil,

Since Simon has responded, let me summarize the assistance I provided p=
er his off-list request:

On 02/14/2011 11:53 PM, NeilBrown wrote:
> On Thu, 10 Feb 2011 16:16:44 +0000 Simon McNair m> wrote:
>=20
>>
>> Hi all
>>
>> I use a 3ware 9500-12 port sata card (JBOD) which will not work with=
out a
>> 128mb sodimm. The sodimm socket is flakey and the result is that th=
e
>> machine occasionally crashes. Yesterday I finally gave in and put=20
>> together another
>> machine so that I can rsync between them. When I turned the machine
>> on today to set up rync, the RAID array was not gone, but corrupted.=
=20
>> Typical...
>=20
> Presumably the old machine was called 'ubuntu' and the new machine 'p=
roølox'
>=20
>=20
>>
>> I built the array in Aug 2010 using the following command:
>>
>> mdadm --create --verbose /dev/md0 --metadata=3D1.1 --level=3D5
>> --raid-devices=3D10 /dev/sd{b,c,d,e,f,g,h,i,j,k}1 --chunk=3D64
>>
>> Using LVM, I did the following:
>> pvscan
>> pvcreate -M2 /dev/md0
>> vgcreate lvm-raid /dev/md0
>> vgdisplay lvm-raid
>> vgscan
>> lvscan
>> lvcreate -v -l 100%VG -n RAID lvm-raid
>> lvdisplay /dev/lvm-raid/lvm0
>>
>> I then formatted using:
>> mkfs -t ext4 -v -m .1 -b 4096 -E stride=3D16,stripe-width=3D144=20
>> /dev/lvm-raid/RAID
>>
>> This worked perfectly since I created the array. Now mdadm is comin=
g up=20
>> with
>>
>> proxmox:/dev/md# mdadm --assemble --scan --verbose
>> mdadm: looking for devices for further assembly
>> mdadm: no recogniseable superblock on /dev/md/ubuntu:0
>=20
> And it seems that ubuntu:0 have been successfully assembled.
> It is missing one device for some reason (sdd1) but RAID can cope wit=
h that.

3ware card is compromised, with a loose buffer memory dimm. Some of it=
s ECC errors were caught and reported in dmesg. Its likely, based on t=
he loose memory socket, that many multiple-bit errors got through.

[trim /]

>> mdadm: no uptodate device for slot 8 of /dev/md/proï¿=BDlox:0
>> mdadm: no uptodate device for slot 9 of /dev/md/proï¿=BDlox:0
>> mdadm: failed to add /dev/sdd1 to /dev/md/proï¿=BDlox:0: Invalid=
argument
>> mdadm: /dev/md/proï¿=BDlox:0 assembled from 0 drives - not enoug=
h to start
>> the array.
>=20
> This looks like it is *after* to trying the --create command you give
> below.. It is best to report things in the order they happen, else y=
ou can
> confuse people (or get caught out!).

Yes, this was after.

>> mdadm: looking for devices for further assembly
>> mdadm: no recogniseable superblock on /dev/sdd
>> mdadm: No arrays found in config file or automatically
>>
>> pvscan and vgscan show nothing.
>>
>> So I tried running mdadm --create --verbose /dev/md0 --metadata=3D1.=
1
>> --level=3D5 --raid-devices=3D10 missing /dev/sde1 /dev/sdf1 /dev/sdg=
1
>> /dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 --chunk=3D=
64
>>
>> as it seemed that /dev/sdd1 failed to be added to the array. This d=
id=20
>> nothing.
>=20
> It did not to nothing. It wrote a superblock to /dev/sdd1 and compla=
ined
> that it couldn't write to all the others --- didn't it?

There were multiple attempts to create. One wrote to just sdd1, anothe=
r succeeded with all but sdd1.

>> dmesg contains:
>>
>> md: invalid superblock checksum on sdd1
>=20
> I guess that is why sdd1 was missing from 'ubuntu:0'. Though as I ca=
nnot
> tell if this happened before or after any of the various things repor=
ted
> above, it is hard to be sure.
>=20
>=20
> The real mystery is why 'pvscan' reports nothing.

The original array was created with mdadm v2.6.7, and had a data offset=
of 264 sectors. After Simon's various attempts to --create, he ended =
up with data offset of 2048, using mdadm v3.1.4. The mdadm -E reports =
he posted to the list showed the 264 offset. We didn't realize the off=
set had been updated until somewhat later in our troubleshooting effort=
s.

In any case, pvscan couldn't see the LVM signature because it wasn't th=
ere (at offset 2048).

> What about
> pvscan --verbose
>=20
> or
>=20
> blkid -p /dev/md/ubuntu:0
>=20
> or even
>=20
> dd of=3D/dev/md/ubuntu:0 count=3D8 | od -c=20

=46ortunately, Simon did have a copy of his LVM configuration. With th=
e help of dd, strings, and grep, we did locate his LVM sig at the corre=
ct location on sdd1 (for data offset 264). After a number of attempts =
to bypass LVM and access his single LV with dmsetup (based on his backe=
d up configuration, on the assembled new array less sdd1), I realized t=
hat the data offset was wrong on the recreated array, and went looking =
for the cause. I found your git commit that changed that logic last sp=
ring, and recommended that Simon revert to the default package for his =
ubuntu install, which is v2.6.7.

Simon has now attempted to recreate the array with v2.6.7, but the cont=
roller is throwing too many errors to succeed, and I suggested it was t=
oo flakey to trust any further. Based on the existence of the LVM sig =
on sdd1, I believe Simon's data is (mostly) intact, and only needs a su=
ccessful create operation with a properly functioning controller. (He =
might also need to perform an lvm vgcfgrestore, but he has the necessar=
y backup file.)

A new controller is on order.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

Phil,
Thanks for filling in the gaps, I had forgotton quite how much help and=
=20
assistance you had provided up to now. You're a real godsend.

To fill in some of the other pieces of info:
The original machine was called proxmox (KVM virtualisation machine) an=
d=20
I used an Ubuntu live cd to see if it was the software which was=20
preventing progress or not. It does seem like there is data corruption=
=20
in the machine name for some reason.

The original company I ordered the controller through sent me an email=20
at 4pm saying that they could not fulfill the order, 4pm was too late=20
for them to pick & pack so there was another 24hr turn around delay. =20
The supermicro card should arrive here tomorrow and hopefully I'll be=20
able to get a dd of each of the drives prior to Phil coming online.

=46or some reason blkid doesn't exist on my machine even though I have=20
e2fs-utils installed. V weird.

Simon


On 15/02/2011 14:51, Phil Turmel wrote:
> Hi Neil,
>
> Since Simon has responded, let me summarize the assistance I provided=
per his off-list request:
>
> On 02/14/2011 11:53 PM, NeilBrown wrote:
>> On Thu, 10 Feb 2011 16:16:44 +0000 Simon McNair m> wrote:
>>
>>> Hi all
>>>
>>> I use a 3ware 9500-12 port sata card (JBOD) which will not work wit=
hout a
>>> 128mb sodimm. The sodimm socket is flakey and the result is that t=
he
>>> machine occasionally crashes. Yesterday I finally gave in and put
>>> together another
>>> machine so that I can rsync between them. When I turned the machin=
e
>>> on today to set up rync, the RAID array was not gone, but corrupted=

>>> Typical...
>> Presumably the old machine was called 'ubuntu' and the new machine '=
proølox'
>>
>>
>>> I built the array in Aug 2010 using the following command:
>>>
>>> mdadm --create --verbose /dev/md0 --metadata=3D1.1 --level=3D5
>>> --raid-devices=3D10 /dev/sd{b,c,d,e,f,g,h,i,j,k}1 --chunk=3D64
>>>
>>> Using LVM, I did the following:
>>> pvscan
>>> pvcreate -M2 /dev/md0
>>> vgcreate lvm-raid /dev/md0
>>> vgdisplay lvm-raid
>>> vgscan
>>> lvscan
>>> lvcreate -v -l 100%VG -n RAID lvm-raid
>>> lvdisplay /dev/lvm-raid/lvm0
>>>
>>> I then formatted using:
>>> mkfs -t ext4 -v -m .1 -b 4096 -E stride=3D16,stripe-width=3D144
>>> /dev/lvm-raid/RAID
>>>
>>> This worked perfectly since I created the array. Now mdadm is comi=
ng up
>>> with
>>>
>>> proxmox:/dev/md# mdadm --assemble --scan --verbose
>>> mdadm: looking for devices for further assembly
>>> mdadm: no recogniseable superblock on /dev/md/ubuntu:0
>> And it seems that ubuntu:0 have been successfully assembled.
>> It is missing one device for some reason (sdd1) but RAID can cope wi=
th that.
> 3ware card is compromised, with a loose buffer memory dimm. Some of =
its ECC errors were caught and reported in dmesg. Its likely, based on=
the loose memory socket, that many multiple-bit errors got through.
>
> [trim /]
>
>>> mdadm: no uptodate device for slot 8 of /dev/md/proï¿=BDlox:0
>>> mdadm: no uptodate device for slot 9 of /dev/md/proï¿=BDlox:0
>>> mdadm: failed to add /dev/sdd1 to /dev/md/proï¿=BDlox:0: Invali=
d argument
>>> mdadm: /dev/md/proï¿=BDlox:0 assembled from 0 drives - not enou=
gh to start
>>> the array.
>> This looks like it is *after* to trying the --create command you giv=
e
>> below.. It is best to report things in the order they happen, else =
you can
>> confuse people (or get caught out!).
> Yes, this was after.
>
>>> mdadm: looking for devices for further assembly
>>> mdadm: no recogniseable superblock on /dev/sdd
>>> mdadm: No arrays found in config file or automatically
>>>
>>> pvscan and vgscan show nothing.
>>>
>>> So I tried running mdadm --create --verbose /dev/md0 --metadata=3D1=
1
>>> --level=3D5 --raid-devices=3D10 missing /dev/sde1 /dev/sdf1 /dev/sd=
g1
>>> /dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 --chunk=
=3D64
>>>
>>> as it seemed that /dev/sdd1 failed to be added to the array. This =
did
>>> nothing.
>> It did not to nothing. It wrote a superblock to /dev/sdd1 and compl=
ained
>> that it couldn't write to all the others --- didn't it?
> There were multiple attempts to create. One wrote to just sdd1, anot=
her succeeded with all but sdd1.
>
>>> dmesg contains:
>>>
>>> md: invalid superblock checksum on sdd1
>> I guess that is why sdd1 was missing from 'ubuntu:0'. Though as I c=
annot
>> tell if this happened before or after any of the various things repo=
rted
>> above, it is hard to be sure.
>>
>>
>> The real mystery is why 'pvscan' reports nothing.
> The original array was created with mdadm v2.6.7, and had a data offs=
et of 264 sectors. After Simon's various attempts to --create, he ende=
d up with data offset of 2048, using mdadm v3.1.4. The mdadm -E report=
s he posted to the list showed the 264 offset. We didn't realize the o=
ffset had been updated until somewhat later in our troubleshooting effo=
rts.
>
> In any case, pvscan couldn't see the LVM signature because it wasn't =
there (at offset 2048).
>
>> What about
>> pvscan --verbose
>>
>> or
>>
>> blkid -p /dev/md/ubuntu:0
>>
>> or even
>>
>> dd of=3D/dev/md/ubuntu:0 count=3D8 | od -c
> Fortunately, Simon did have a copy of his LVM configuration. With th=
e help of dd, strings, and grep, we did locate his LVM sig at the corre=
ct location on sdd1 (for data offset 264). After a number of attempts =
to bypass LVM and access his single LV with dmsetup (based on his backe=
d up configuration, on the assembled new array less sdd1), I realized t=
hat the data offset was wrong on the recreated array, and went looking =
for the cause. I found your git commit that changed that logic last sp=
ring, and recommended that Simon revert to the default package for his =
ubuntu install, which is v2.6.7.
>
> Simon has now attempted to recreate the array with v2.6.7, but the co=
ntroller is throwing too many errors to succeed, and I suggested it was=
too flakey to trust any further. Based on the existence of the LVM si=
g on sdd1, I believe Simon's data is (mostly) intact, and only needs a =
successful create operation with a properly functioning controller. (H=
e might also need to perform an lvm vgcfgrestore, but he has the necess=
ary backup file.)
>
> A new controller is on order.
>
> Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

On 02/15/2011 02:04 PM, Simon McNair wrote:
> Phil,
> Thanks for filling in the gaps, I had forgotton quite how much help and assistance you had provided up to now. You're a real godsend.
>
> To fill in some of the other pieces of info:
> The original machine was called proxmox (KVM virtualisation machine) and I used an Ubuntu live cd to see if it was the software which was preventing progress or not. It does seem like there is data corruption in the machine name for some reason.
>
> The original company I ordered the controller through sent me an email at 4pm saying that they could not fulfill the order, 4pm was too late for them to pick & pack so there was another 24hr turn around delay. The supermicro card should arrive here tomorrow and hopefully I'll be able to get a dd of each of the drives prior to Phil coming online.
>
> For some reason blkid doesn't exist on my machine even though I have e2fs-utils installed. V weird.

FWIW, on my VPS running Ubuntu Server 10.10, the blkid executable is part of util-linux, and installed in /sbin. You might need to re-install it.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

--Sig_/fUQ2CcW+MQ_xtLnDIIS+KHg
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

On Tue, 15 Feb 2011 14:37:55 -0500
Phil Turmel wrote:

> > For some reason blkid doesn't exist on my machine even though I have
> > e2fs-utils installed. V weird.

Weird is that you think it is somehow a "e2fs-utils" program.

dpkg -S blkid
....
util-linux: /sbin/blkid
....

>=20
> FWIW, on my VPS running Ubuntu Server 10.10, the blkid executable is part=
of
> util-linux, and installed in /sbin. You might need to re-install it.

--=20
With respect,
Roman

--Sig_/fUQ2CcW+MQ_xtLnDIIS+KHg
Content-Type: application/pgp-signature; name=signature.asc
Content-Disposition: attachment; filename=signature.asc

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAk1a19QACgkQTLKSvz+PZwgzRgCggc5PkkLeGXlGvKlLiAL/ iXvc
JRYAnjvEhUbHENECZNV0QHpsZV+MOR+o
=WXj6
-----END PGP SIGNATURE-----

--Sig_/fUQ2CcW+MQ_xtLnDIIS+KHg--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

I got my info from:

http://linux.die.net/man/8/blkid

"*blkid* is part the e2fsprogs package since version 1.26 and is
available from http://e2fsprogs.sourceforge.net
. "

I also did apt-file search blkid and it showed up as e2fsprogs or was it
e2fs-utils (I forget). The system I was/am running was Debian Lenny x64.

This just shows how little I know about Linux (I've only been dabbling
for 6 months or so). I'll install util-linux next time the machine
start up. Thanks for the pointer.

regards
Simon

On 15/02/2011 19:45, Roman Mamedov wrote:
> On Tue, 15 Feb 2011 14:37:55 -0500
> Phil Turmel wrote:
>
>>> For some reason blkid doesn't exist on my machine even though I have
>>> e2fs-utils installed. V weird.
> Weird is that you think it is somehow a "e2fs-utils" program.
>
> dpkg -S blkid
> ...
> util-linux: /sbin/blkid
> ...
>
>> FWIW, on my VPS running Ubuntu Server 10.10, the blkid executable is part of
>> util-linux, and installed in /sbin. You might need to re-install it.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

latest update. A bit long I'm afraid...

hi, card installed and devices plugged in. Now have
sda sda2 sdb1 sdc1 sde sdf1 sdg1 sdi sdj1 sdk1 sdl1 sdm1
sda1 sdb sdc sdd sdf sdg sdh sdj sdk sdl sdm


proxmox:/home/simon# ./lsdrv.sh
Controller device @ pci0000:00/0000:00:01.0/0000:01:00.0 [mvsas]
SCSI storage controller: Marvell Technology Group Ltd.=20
MV64460/64461/64462 Sys tem=20
Controller, Revision B (rev 01)
host4: /dev/sdf ATA Hitachi HDS72101 {SN: GTA000PAGABXRA}
host4: /dev/sdg ATA Hitachi HDS72101 {SN: GTA000PAGAA5DA}
host4: /dev/sdh ATA Hitachi HDS72101 {SN: GTA000PAG9NL9A}
host4: /dev/sdi ATA Hitachi HDS72101 {SN: GTA000PAGA8V4A}
host4: /dev/sdj ATA Hitachi HDS72101 {SN: GTD000PAGMT9GD}
host4: /dev/sdk ATA Hitachi HDS72101 {SN: GTG000PAG18BJC}
host4: /dev/sdl ATA Hitachi HDS72101 {SN: GTG000PAG1DPLC}
host4: /dev/sdm ATA Hitachi HDS72101 {SN: GTA000PAG7WMEA}
Controller device @=20
pci0000:00/0000:00:1a.7/usb1/1-4/1-4.1/1-4.1.1/1-4.1.1:1.0=2 0
[ usb-storage]
Bus 001 Device 007: ID 0424:2228 Standard Microsystems Corp. 9-in-2=20
Card Reade r {SN: 08050920003A}
host9: /dev/sdd Generic Flash HS-CF
host9: /dev/sde Generic Flash HS-COMBO
Controller device @ pci0000:00/0000:00:1c.4/0000:04:00.0 [ahci]
SATA controller: JMicron Technology Corp. JMB362/JMB363 Serial ATA=20
Controller (rev 03)
host7: [Empty]
host8: [Empty]
Controller device @ pci0000:00/0000:00:1c.4/0000:04:00.1 [pata_jmicron]
IDE interface: JMicron Technology Corp. JMB362/JMB363 Serial ATA=20
Controller (r ev 03)
host5: [Empty]
host6: [Empty]
Controller device @ pci0000:00/0000:00:1f.2 [ata_piix]
IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port SATA=20
IDE Contro ller #1
host0: /dev/sda ATA STM3500418AS {SN: 9VM3QJ5C}
host1: /dev/sr0 Optiarc DVD RW AD-5240S
Controller device @ pci0000:00/0000:00:1f.5 [ata_piix]
IDE interface: Intel Corporation 82801JI (ICH10 Family) 2 port SATA=20
IDE Contro ller #2
host2: /dev/sdb ATA Hitachi HDS72101 {SN: GTD000PAGMT8DD}
host3: /dev/sdc ATA Hitachi HDS72101 {SN: GTG000PAG04V0C}


proxmox:/home/simon# parted -l
Model: ATA STM3500418AS (scsi)
Disk /dev/sda: 500GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number Start End Size Type File system Flags
1 32.8kB 537MB 537MB primary ext3 boot
2 537MB 500GB 500GB primary lvm


Error: The backup GPT table is not at the end of the disk, as it should=
=20
be. This might mean that another operating
system believes the disk is smaller. Fix, by moving the backup to the=20
end (and removing the old backup)?
=46ix/Cancel? c

Error: The backup GPT table is not at the end of the disk, as it should=
=20
be. This might mean that another operating
system believes the disk is smaller. Fix, by moving the backup to the=20
end (and removing the old backup)?
=46ix/Cancel? c

Error: The backup GPT table is not at the end of the disk, as it should=
=20
be. This might mean that another operating
system believes the disk is smaller. Fix, by moving the backup to the=20
end (and removing the old backup)?
=46ix/Cancel? c

Model: Linux device-mapper (linear) (dm)
Disk /dev/mapper/pve-data: 380GB
Sector size (logical/physical): 512B/512B
Partition Table: loop

Number Start End Size File system Flags
1 0.00B 380GB 380GB ext3


cModel: Linux device-mapper (linear) (dm)
Disk /dev/mapper/pve-root: 103GB
Sector size (logical/physical): 512B/512B
Partition Table: loop

Number Start End Size File system Flags
1 0.00B 103GB 103GB ext3


Model: Linux device-mapper (linear) (dm)
Disk /dev/mapper/pve-swap: 11.8GB
Sector size (logical/physical): 512B/512B
Partition Table: loop

Number Start End Size File system Flags
1 0.00B 11.8GB 11.8GB linux-swap


Error: The backup GPT table is not at the end of the disk, as it should=
=20
be. This might mean that another operating
system believes the disk is smaller. Fix, by moving the backup to the=20
end (and removing the old backup)?
=46ix/Cancel? c

Error: /dev/sdh: unrecognised disk label

Error: /dev/sdi: unrecognised disk label

Error: The backup GPT table is not at the end of the disk, as it should=
=20
be. This might mean that another operating
system believes the disk is smaller. Fix, by moving the backup to the=20
end (and removing the old backup)?
=46ix/Cancel? c

Error: The backup GPT table is not at the end of the disk, as it should=
=20
be. This might mean that another operating
system believes the disk is smaller. Fix, by moving the backup to the=20
end (and removing the old backup)?
=46ix/Cancel? c

Error: The backup GPT table is not at the end of the disk, as it should=
=20
be. This might mean that another operating
system believes the disk is smaller. Fix, by moving the backup to the=20
end (and removing the old backup)?
=46ix/Cancel? c

Error: The backup GPT table is not at the end of the disk, as it should=
=20
be. This might mean that another operating
system believes the disk is smaller. Fix, by moving the backup to the=20
end (and removing the old backup)?
=46ix/Cancel? c

Error: /dev/md0: unrecognised disk label


proxmox:/home/simon# mdadm --assemble --scan --verbose
mdadm: looking for devices for /dev/md/0
mdadm: cannot open device /dev/dm-2: Device or resource busy
mdadm: /dev/dm-2 has wrong uuid.
mdadm: cannot open device /dev/dm-1: Device or resource busy
mdadm: /dev/dm-1 has wrong uuid.
mdadm: cannot open device /dev/dm-0: Device or resource busy
mdadm: /dev/dm-0 has wrong uuid.
mdadm: no RAID superblock on /dev/sdf1
mdadm: /dev/sdf1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdf
mdadm: /dev/sdf has wrong uuid.
mdadm: no RAID superblock on /dev/sdm1
mdadm: /dev/sdm1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdm
mdadm: /dev/sdm has wrong uuid.
mdadm: no RAID superblock on /dev/sdl1
mdadm: /dev/sdl1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdl
mdadm: /dev/sdl has wrong uuid.
mdadm: no RAID superblock on /dev/sdk1
mdadm: /dev/sdk1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdk
mdadm: /dev/sdk has wrong uuid.
mdadm: no RAID superblock on /dev/sdj1
mdadm: /dev/sdj1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdj
mdadm: /dev/sdj has wrong uuid.
mdadm: /dev/sdi has wrong uuid.
mdadm: /dev/sdh has wrong uuid.
mdadm: no RAID superblock on /dev/sdg1
mdadm: /dev/sdg1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdg
mdadm: /dev/sdg has wrong uuid.
mdadm: no RAID superblock on /dev/sdc1
mdadm: /dev/sdc1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdc
mdadm: /dev/sdc has wrong uuid.
mdadm: no RAID superblock on /dev/sdb1
mdadm: /dev/sdb1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdb
mdadm: /dev/sdb has wrong uuid.
mdadm: cannot open device /dev/sda2: Device or resource busy
mdadm: /dev/sda2 has wrong uuid.
mdadm: cannot open device /dev/sda1: Device or resource busy
mdadm: /dev/sda1 has wrong uuid.
mdadm: cannot open device /dev/sda: Device or resource busy
mdadm: /dev/sda has wrong uuid.
mdadm: no devices found for /dev/md/0
mdadm: looking for devices for further assembly
mdadm: cannot open device /dev/dm-2: Device or resource busy
mdadm: cannot open device /dev/dm-1: Device or resource busy
mdadm: cannot open device /dev/dm-0: Device or resource busy
mdadm: no recogniseable superblock on /dev/sdf1
mdadm: no recogniseable superblock on /dev/sdf
mdadm: no recogniseable superblock on /dev/sdm1
mdadm: no recogniseable superblock on /dev/sdm
mdadm: no recogniseable superblock on /dev/sdl1
mdadm: no recogniseable superblock on /dev/sdl
mdadm: no recogniseable superblock on /dev/sdk1
mdadm: no recogniseable superblock on /dev/sdk
mdadm: no recogniseable superblock on /dev/sdj1
mdadm: no recogniseable superblock on /dev/sdj
mdadm: /dev/sdi is not built for host proxmox.
mdadm: /dev/sdh is not built for host proxmox.
mdadm: no recogniseable superblock on /dev/sdg1
mdadm: no recogniseable superblock on /dev/sdg
mdadm: no recogniseable superblock on /dev/sdc1
mdadm: no recogniseable superblock on /dev/sdc
mdadm: no recogniseable superblock on /dev/sdb1
mdadm: no recogniseable superblock on /dev/sdb
mdadm: cannot open device /dev/sda2: Device or resource busy
mdadm: cannot open device /dev/sda1: Device or resource busy
mdadm: cannot open device /dev/sda: Device or resource busy

proxmox:/home/simon# apt-show-versions -a mdadm
mdadm 2.6.7.2-3 install ok installed
mdadm 2.6.7.2-3 lenny ftp.uk.debian.org
No stable version
No testing version
mdadm 3.1.4-1+8efb9d1 sid ftp.uk.debian.org
mdadm/lenny uptodate 2.6.7.2-3

anything else you want ?

Simon

On 15/02/2011 14:51, Phil Turmel wrote:
> Hi Neil,
>
> Since Simon has responded, let me summarize the assistance I provided=
per his off-list request:
>
> On 02/14/2011 11:53 PM, NeilBrown wrote:
>> On Thu, 10 Feb 2011 16:16:44 +0000 Simon McNair m> wrote:
>>
>>> Hi all
>>>
>>> I use a 3ware 9500-12 port sata card (JBOD) which will not work wit=
hout a
>>> 128mb sodimm. The sodimm socket is flakey and the result is that t=
he
>>> machine occasionally crashes. Yesterday I finally gave in and put
>>> together another
>>> machine so that I can rsync between them. When I turned the machin=
e
>>> on today to set up rync, the RAID array was not gone, but corrupted=

>>> Typical...
>> Presumably the old machine was called 'ubuntu' and the new machine '=
proølox'
>>
>>
>>> I built the array in Aug 2010 using the following command:
>>>
>>> mdadm --create --verbose /dev/md0 --metadata=3D1.1 --level=3D5
>>> --raid-devices=3D10 /dev/sd{b,c,d,e,f,g,h,i,j,k}1 --chunk=3D64
>>>
>>> Using LVM, I did the following:
>>> pvscan
>>> pvcreate -M2 /dev/md0
>>> vgcreate lvm-raid /dev/md0
>>> vgdisplay lvm-raid
>>> vgscan
>>> lvscan
>>> lvcreate -v -l 100%VG -n RAID lvm-raid
>>> lvdisplay /dev/lvm-raid/lvm0
>>>
>>> I then formatted using:
>>> mkfs -t ext4 -v -m .1 -b 4096 -E stride=3D16,stripe-width=3D144
>>> /dev/lvm-raid/RAID
>>>
>>> This worked perfectly since I created the array. Now mdadm is comi=
ng up
>>> with
>>>
>>> proxmox:/dev/md# mdadm --assemble --scan --verbose
>>> mdadm: looking for devices for further assembly
>>> mdadm: no recogniseable superblock on /dev/md/ubuntu:0
>> And it seems that ubuntu:0 have been successfully assembled.
>> It is missing one device for some reason (sdd1) but RAID can cope wi=
th that.
> 3ware card is compromised, with a loose buffer memory dimm. Some of =
its ECC errors were caught and reported in dmesg. Its likely, based on=
the loose memory socket, that many multiple-bit errors got through.
>
> [trim /]
>
>>> mdadm: no uptodate device for slot 8 of /dev/md/proï¿=BDlox:0
>>> mdadm: no uptodate device for slot 9 of /dev/md/proï¿=BDlox:0
>>> mdadm: failed to add /dev/sdd1 to /dev/md/proï¿=BDlox:0: Invali=
d argument
>>> mdadm: /dev/md/proï¿=BDlox:0 assembled from 0 drives - not enou=
gh to start
>>> the array.
>> This looks like it is *after* to trying the --create command you giv=
e
>> below.. It is best to report things in the order they happen, else =
you can
>> confuse people (or get caught out!).
> Yes, this was after.
>
>>> mdadm: looking for devices for further assembly
>>> mdadm: no recogniseable superblock on /dev/sdd
>>> mdadm: No arrays found in config file or automatically
>>>
>>> pvscan and vgscan show nothing.
>>>
>>> So I tried running mdadm --create --verbose /dev/md0 --metadata=3D1=
1
>>> --level=3D5 --raid-devices=3D10 missing /dev/sde1 /dev/sdf1 /dev/sd=
g1
>>> /dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 --chunk=
=3D64
>>>
>>> as it seemed that /dev/sdd1 failed to be added to the array. This =
did
>>> nothing.
>> It did not to nothing. It wrote a superblock to /dev/sdd1 and compl=
ained
>> that it couldn't write to all the others --- didn't it?
> There were multiple attempts to create. One wrote to just sdd1, anot=
her succeeded with all but sdd1.
>
>>> dmesg contains:
>>>
>>> md: invalid superblock checksum on sdd1
>> I guess that is why sdd1 was missing from 'ubuntu:0'. Though as I c=
annot
>> tell if this happened before or after any of the various things repo=
rted
>> above, it is hard to be sure.
>>
>>
>> The real mystery is why 'pvscan' reports nothing.
> The original array was created with mdadm v2.6.7, and had a data offs=
et of 264 sectors. After Simon's various attempts to --create, he ende=
d up with data offset of 2048, using mdadm v3.1.4. The mdadm -E report=
s he posted to the list showed the 264 offset. We didn't realize the o=
ffset had been updated until somewhat later in our troubleshooting effo=
rts.
>
> In any case, pvscan couldn't see the LVM signature because it wasn't =
there (at offset 2048).
>
>> What about
>> pvscan --verbose
>>
>> or
>>
>> blkid -p /dev/md/ubuntu:0
>>
>> or even
>>
>> dd of=3D/dev/md/ubuntu:0 count=3D8 | od -c
> Fortunately, Simon did have a copy of his LVM configuration. With th=
e help of dd, strings, and grep, we did locate his LVM sig at the corre=
ct location on sdd1 (for data offset 264). After a number of attempts =
to bypass LVM and access his single LV with dmsetup (based on his backe=
d up configuration, on the assembled new array less sdd1), I realized t=
hat the data offset was wrong on the recreated array, and went looking =
for the cause. I found your git commit that changed that logic last sp=
ring, and recommended that Simon revert to the default package for his =
ubuntu install, which is v2.6.7.
>
> Simon has now attempted to recreate the array with v2.6.7, but the co=
ntroller is throwing too many errors to succeed, and I suggested it was=
too flakey to trust any further. Based on the existence of the LVM si=
g on sdd1, I believe Simon's data is (mostly) intact, and only needs a =
successful create operation with a properly functioning controller. (H=
e might also need to perform an lvm vgcfgrestore, but he has the necess=
ary backup file.)
>
> A new controller is on order.
>
> Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

one other snippet:

proxmox:/home/simon# for x in /dev/sd{d..m} ; do echo $x ; dd if=3D$x=20
skip=3D2312 count=3D128 2>/dev/null |strings |grep=20
9fAJEz-HcaP-RQ51-fV8b-nxrN-Uqwb-PPnOLJ ; done
/dev/sdd
/dev/sde
/dev/sdf
/dev/sdg
/dev/sdh
/dev/sdi
id =3D "9fAJEz-HcaP-RQ51-fV8b-nxrN-Uqwb-PPnOLJ"
id =3D "9fAJEz-HcaP-RQ51-fV8b-nxrN-Uqwb-PPnOLJ"
/dev/sdj
/dev/sdk
/dev/sdl
/dev/sdm



On 15/02/2011 14:51, Phil Turmel wrote:
> Hi Neil,
>
> Since Simon has responded, let me summarize the assistance I provided=
per his off-list request:
>
> On 02/14/2011 11:53 PM, NeilBrown wrote:
>> On Thu, 10 Feb 2011 16:16:44 +0000 Simon McNair m> wrote:
>>
>>> Hi all
>>>
>>> I use a 3ware 9500-12 port sata card (JBOD) which will not work wit=
hout a
>>> 128mb sodimm. The sodimm socket is flakey and the result is that t=
he
>>> machine occasionally crashes. Yesterday I finally gave in and put
>>> together another
>>> machine so that I can rsync between them. When I turned the machin=
e
>>> on today to set up rync, the RAID array was not gone, but corrupted=

>>> Typical...
>> Presumably the old machine was called 'ubuntu' and the new machine '=
proølox'
>>
>>
>>> I built the array in Aug 2010 using the following command:
>>>
>>> mdadm --create --verbose /dev/md0 --metadata=3D1.1 --level=3D5
>>> --raid-devices=3D10 /dev/sd{b,c,d,e,f,g,h,i,j,k}1 --chunk=3D64
>>>
>>> Using LVM, I did the following:
>>> pvscan
>>> pvcreate -M2 /dev/md0
>>> vgcreate lvm-raid /dev/md0
>>> vgdisplay lvm-raid
>>> vgscan
>>> lvscan
>>> lvcreate -v -l 100%VG -n RAID lvm-raid
>>> lvdisplay /dev/lvm-raid/lvm0
>>>
>>> I then formatted using:
>>> mkfs -t ext4 -v -m .1 -b 4096 -E stride=3D16,stripe-width=3D144
>>> /dev/lvm-raid/RAID
>>>
>>> This worked perfectly since I created the array. Now mdadm is comi=
ng up
>>> with
>>>
>>> proxmox:/dev/md# mdadm --assemble --scan --verbose
>>> mdadm: looking for devices for further assembly
>>> mdadm: no recogniseable superblock on /dev/md/ubuntu:0
>> And it seems that ubuntu:0 have been successfully assembled.
>> It is missing one device for some reason (sdd1) but RAID can cope wi=
th that.
> 3ware card is compromised, with a loose buffer memory dimm. Some of =
its ECC errors were caught and reported in dmesg. Its likely, based on=
the loose memory socket, that many multiple-bit errors got through.
>
> [trim /]
>
>>> mdadm: no uptodate device for slot 8 of /dev/md/proï¿=BDlox:0
>>> mdadm: no uptodate device for slot 9 of /dev/md/proï¿=BDlox:0
>>> mdadm: failed to add /dev/sdd1 to /dev/md/proï¿=BDlox:0: Invali=
d argument
>>> mdadm: /dev/md/proï¿=BDlox:0 assembled from 0 drives - not enou=
gh to start
>>> the array.
>> This looks like it is *after* to trying the --create command you giv=
e
>> below.. It is best to report things in the order they happen, else =
you can
>> confuse people (or get caught out!).
> Yes, this was after.
>
>>> mdadm: looking for devices for further assembly
>>> mdadm: no recogniseable superblock on /dev/sdd
>>> mdadm: No arrays found in config file or automatically
>>>
>>> pvscan and vgscan show nothing.
>>>
>>> So I tried running mdadm --create --verbose /dev/md0 --metadata=3D1=
1
>>> --level=3D5 --raid-devices=3D10 missing /dev/sde1 /dev/sdf1 /dev/sd=
g1
>>> /dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 --chunk=
=3D64
>>>
>>> as it seemed that /dev/sdd1 failed to be added to the array. This =
did
>>> nothing.
>> It did not to nothing. It wrote a superblock to /dev/sdd1 and compl=
ained
>> that it couldn't write to all the others --- didn't it?
> There were multiple attempts to create. One wrote to just sdd1, anot=
her succeeded with all but sdd1.
>
>>> dmesg contains:
>>>
>>> md: invalid superblock checksum on sdd1
>> I guess that is why sdd1 was missing from 'ubuntu:0'. Though as I c=
annot
>> tell if this happened before or after any of the various things repo=
rted
>> above, it is hard to be sure.
>>
>>
>> The real mystery is why 'pvscan' reports nothing.
> The original array was created with mdadm v2.6.7, and had a data offs=
et of 264 sectors. After Simon's various attempts to --create, he ende=
d up with data offset of 2048, using mdadm v3.1.4. The mdadm -E report=
s he posted to the list showed the 264 offset. We didn't realize the o=
ffset had been updated until somewhat later in our troubleshooting effo=
rts.
>
> In any case, pvscan couldn't see the LVM signature because it wasn't =
there (at offset 2048).
>
>> What about
>> pvscan --verbose
>>
>> or
>>
>> blkid -p /dev/md/ubuntu:0
>>
>> or even
>>
>> dd of=3D/dev/md/ubuntu:0 count=3D8 | od -c
> Fortunately, Simon did have a copy of his LVM configuration. With th=
e help of dd, strings, and grep, we did locate his LVM sig at the corre=
ct location on sdd1 (for data offset 264). After a number of attempts =
to bypass LVM and access his single LV with dmsetup (based on his backe=
d up configuration, on the assembled new array less sdd1), I realized t=
hat the data offset was wrong on the recreated array, and went looking =
for the cause. I found your git commit that changed that logic last sp=
ring, and recommended that Simon revert to the default package for his =
ubuntu install, which is v2.6.7.
>
> Simon has now attempted to recreate the array with v2.6.7, but the co=
ntroller is throwing too many errors to succeed, and I suggested it was=
too flakey to trust any further. Based on the existence of the LVM si=
g on sdd1, I believe Simon's data is (mostly) intact, and only needs a =
successful create operation with a properly functioning controller. (H=
e might also need to perform an lvm vgcfgrestore, but he has the necess=
ary backup file.)
>
> A new controller is on order.
>
> Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

Good morning, Simon,

On 02/16/2011 08:51 AM, Simon McNair wrote:
> latest update. A bit long I'm afraid...
>
> hi, card installed and devices plugged in. Now have
> sda sda2 sdb1 sdc1 sde sdf1 sdg1 sdi sdj1 sdk1 sdl1 sdm1
> sda1 sdb sdc sdd sdf sdg sdh sdj sdk sdl sdm

Hmmm.

> proxmox:/home/simon# ./lsdrv.sh
> Controller device @ pci0000:00/0000:00:01.0/0000:01:00.0 [mvsas]
> SCSI storage controller: Marvell Technology Group Ltd. MV64460/64461/64462 Sys tem Controller, Revision B (rev 01)
> host4: /dev/sdf ATA Hitachi HDS72101 {SN: GTA000PAGABXRA}
> host4: /dev/sdg ATA Hitachi HDS72101 {SN: GTA000PAGAA5DA}
> host4: /dev/sdh ATA Hitachi HDS72101 {SN: GTA000PAG9NL9A}
> host4: /dev/sdi ATA Hitachi HDS72101 {SN: GTA000PAGA8V4A}
> host4: /dev/sdj ATA Hitachi HDS72101 {SN: GTD000PAGMT9GD}
> host4: /dev/sdk ATA Hitachi HDS72101 {SN: GTG000PAG18BJC}
> host4: /dev/sdl ATA Hitachi HDS72101 {SN: GTG000PAG1DPLC}
> host4: /dev/sdm ATA Hitachi HDS72101 {SN: GTA000PAG7WMEA}
> Controller device @ pci0000:00/0000:00:1a.7/usb1/1-4/1-4.1/1-4.1.1/1-4.1.1:1.0 [ usb-storage]
> Bus 001 Device 007: ID 0424:2228 Standard Microsystems Corp. 9-in-2 Card Reade r {SN: 08050920003A}
> host9: /dev/sdd Generic Flash HS-CF
> host9: /dev/sde Generic Flash HS-COMBO
> Controller device @ pci0000:00/0000:00:1c.4/0000:04:00.0 [ahci]
> SATA controller: JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (rev 03)
> host7: [Empty]
> host8: [Empty]
> Controller device @ pci0000:00/0000:00:1c.4/0000:04:00.1 [pata_jmicron]
> IDE interface: JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (r ev 03)
> host5: [Empty]
> host6: [Empty]
> Controller device @ pci0000:00/0000:00:1f.2 [ata_piix]
> IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port SATA IDE Contro ller #1
> host0: /dev/sda ATA STM3500418AS {SN: 9VM3QJ5C}
> host1: /dev/sr0 Optiarc DVD RW AD-5240S
> Controller device @ pci0000:00/0000:00:1f.5 [ata_piix]
> IDE interface: Intel Corporation 82801JI (ICH10 Family) 2 port SATA IDE Contro ller #2
> host2: /dev/sdb ATA Hitachi HDS72101 {SN: GTD000PAGMT8DD}
> host3: /dev/sdc ATA Hitachi HDS72101 {SN: GTG000PAG04V0C}

Good thing we recorded the serial numbers. From an earlier run of "lsdrv":

> Phil,
> sg3-utils did the job :-)
> Sorry for doubting you.
>
> host6: /dev/sdd AMCC 9500S-12 DISK {SN: PAGA8V4A3B8378002254}
> host6: /dev/sde AMCC 9500S-12 DISK {SN: PAG9NL9A3B8387004C88}
> host6: /dev/sdf AMCC 9500S-12 DISK {SN: PAGAA5DA3B8396001AE0}
> host6: /dev/sdg AMCC 9500S-12 DISK {SN: PAGABXRA3B83A0005EFB}
> host6: /dev/sdh AMCC 9500S-12 DISK {SN: PAG7WMEA3B83AA003E4F}
> host6: /dev/sdi AMCC 9500S-12 DISK {SN: PAG1DPLC3B83B40026E6}
> host6: /dev/sdj AMCC 9500S-12 DISK {SN: PAG18BJC3B83C3004760}
> host6: /dev/sdk AMCC 9500S-12 DISK {SN: PAGMT9GD3B83CD004B18}
> host6: /dev/sdl AMCC 9500S-12 DISK {SN: PAGMT8DD3B83D70021FF}
> host6: /dev/sdm AMCC 9500S-12 DISK {SN: PAG04V0C3B83E100ACA0}
>
> Simon

I don't know why the serial numbers are formatted differently, but we can still tell them apart (the eight characters starting with "PAG").

So, our device order in your new setup is: [ihgfmlkjbc], where /dev/sdi corresponds to the original report's /dev/sdd, which matches the sig grep in your other note.

Another note: The controller for sd[abc] is still showing ata_piix as its controller. That means you cannot hot-plug those ports. If you change your BIOS to AHCI mode instead of "Compatibility" or "Emulation", the full-featured ahci driver will run those ports. Not urgent, but I highly recommend it.

> proxmox:/home/simon# parted -l
> Model: ATA STM3500418AS (scsi)
> Disk /dev/sda: 500GB
> Sector size (logical/physical): 512B/512B
> Partition Table: msdos
>
> Number Start End Size Type File system Flags
> 1 32.8kB 537MB 537MB primary ext3 boot
> 2 537MB 500GB 500GB primary lvm
>
>
> Error: The backup GPT table is not at the end of the disk, as it should be. This might mean that another operating
> system believes the disk is smaller. Fix, by moving the backup to the end (and removing the old backup)?
> Fix/Cancel? c

The 3ware controller must have reserved some space at the end of each drive for its own use. Didn't know it'd do that. You will have to fix that.

[trim /]

> Error: /dev/sdh: unrecognised disk label
>
> Error: /dev/sdi: unrecognised disk label

It seems the flaky controller took out these partition tables. Hope that's all it got. They'll have to be re-created with parted.

Please run parted on each of the ten drives, and make sure they end up like so:

> proxmox:/home/simon# for x in sd{d..m} ; do parted -s /dev/$x unit s
> print ; done
> Model: AMCC 9500S-12 DISK (scsi)
> Disk /dev/sdd: 1953103872s
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
>
> Number Start End Size File system Name Flags
> 1 2048s 1953101823s 1953099776s primary raid

Make sure you request "unit s" before your other commands so we can make sure it matches.

When you think you are done, check them all:

for x in /dev/sd{i,h,g,f,m,l,k,j,b,c} ; do parted -s $x unit s print ; done

After that, create:

mdadm --create --verbose --assume-clean /dev/md0 --metadata=1.1 --level=5 --raid-devices=10 /dev/sd{i,h,g,f,m,l,k,j,b,c}1 --chunk=64

And finally:

pvscan --verbose

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

Hi Phil,
A couple of questions please.

On 16/02/2011 14:37, Phil Turmel wrote:
> Good morning, Simon,
>
> On 02/16/2011 08:51 AM, Simon McNair wrote:
>> latest update. A bit long I'm afraid...
>>
>> hi, card installed and devices plugged in. Now have
>> sda sda2 sdb1 sdc1 sde sdf1 sdg1 sdi sdj1 sdk1 sdl1 sdm1
>> sda1 sdb sdc sdd sdf sdg sdh sdj sdk sdl sdm
> Hmmm.
>
>> proxmox:/home/simon# ./lsdrv.sh
>> Controller device @ pci0000:00/0000:00:01.0/0000:01:00.0 [mvsas]
>> SCSI storage controller: Marvell Technology Group Ltd. MV64460/64461/64462 Sys tem Controller, Revision B (rev 01)
>> host4: /dev/sdf ATA Hitachi HDS72101 {SN: GTA000PAGABXRA}
>> host4: /dev/sdg ATA Hitachi HDS72101 {SN: GTA000PAGAA5DA}
>> host4: /dev/sdh ATA Hitachi HDS72101 {SN: GTA000PAG9NL9A}
>> host4: /dev/sdi ATA Hitachi HDS72101 {SN: GTA000PAGA8V4A}
>> host4: /dev/sdj ATA Hitachi HDS72101 {SN: GTD000PAGMT9GD}
>> host4: /dev/sdk ATA Hitachi HDS72101 {SN: GTG000PAG18BJC}
>> host4: /dev/sdl ATA Hitachi HDS72101 {SN: GTG000PAG1DPLC}
>> host4: /dev/sdm ATA Hitachi HDS72101 {SN: GTA000PAG7WMEA}
>> Controller device @ pci0000:00/0000:00:1a.7/usb1/1-4/1-4.1/1-4.1.1/1-4.1.1:1.0 [ usb-storage]
>> Bus 001 Device 007: ID 0424:2228 Standard Microsystems Corp. 9-in-2 Card Reade r {SN: 08050920003A}
>> host9: /dev/sdd Generic Flash HS-CF
>> host9: /dev/sde Generic Flash HS-COMBO
>> Controller device @ pci0000:00/0000:00:1c.4/0000:04:00.0 [ahci]
>> SATA controller: JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (rev 03)
>> host7: [Empty]
>> host8: [Empty]
>> Controller device @ pci0000:00/0000:00:1c.4/0000:04:00.1 [pata_jmicron]
>> IDE interface: JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (r ev 03)
>> host5: [Empty]
>> host6: [Empty]
>> Controller device @ pci0000:00/0000:00:1f.2 [ata_piix]
>> IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port SATA IDE Contro ller #1
>> host0: /dev/sda ATA STM3500418AS {SN: 9VM3QJ5C}
>> host1: /dev/sr0 Optiarc DVD RW AD-5240S
>> Controller device @ pci0000:00/0000:00:1f.5 [ata_piix]
>> IDE interface: Intel Corporation 82801JI (ICH10 Family) 2 port SATA IDE Contro ller #2
>> host2: /dev/sdb ATA Hitachi HDS72101 {SN: GTD000PAGMT8DD}
>> host3: /dev/sdc ATA Hitachi HDS72101 {SN: GTG000PAG04V0C}
> Good thing we recorded the serial numbers. From an earlier run of "lsdrv":
>
>> Phil,
>> sg3-utils did the job :-)
>> Sorry for doubting you.
>>
>> host6: /dev/sdd AMCC 9500S-12 DISK {SN: PAGA8V4A3B8378002254}
>> host6: /dev/sde AMCC 9500S-12 DISK {SN: PAG9NL9A3B8387004C88}
>> host6: /dev/sdf AMCC 9500S-12 DISK {SN: PAGAA5DA3B8396001AE0}
>> host6: /dev/sdg AMCC 9500S-12 DISK {SN: PAGABXRA3B83A0005EFB}
>> host6: /dev/sdh AMCC 9500S-12 DISK {SN: PAG7WMEA3B83AA003E4F}
>> host6: /dev/sdi AMCC 9500S-12 DISK {SN: PAG1DPLC3B83B40026E6}
>> host6: /dev/sdj AMCC 9500S-12 DISK {SN: PAG18BJC3B83C3004760}
>> host6: /dev/sdk AMCC 9500S-12 DISK {SN: PAGMT9GD3B83CD004B18}
>> host6: /dev/sdl AMCC 9500S-12 DISK {SN: PAGMT8DD3B83D70021FF}
>> host6: /dev/sdm AMCC 9500S-12 DISK {SN: PAG04V0C3B83E100ACA0}
>>
>> Simon
> I don't know why the serial numbers are formatted differently, but we can still tell them apart (the eight characters starting with "PAG").
>
> So, our device order in your new setup is: [ihgfmlkjbc], where /dev/sdi corresponds to the original report's /dev/sdd, which matches the sig grep in your other note.
>
> Another note: The controller for sd[abc] is still showing ata_piix as its controller. That means you cannot hot-plug those ports. If you change your BIOS to AHCI mode instead of "Compatibility" or "Emulation", the full-featured ahci driver will run those ports. Not urgent, but I highly recommend it.
>
Will do that now, before I forget

>> proxmox:/home/simon# parted -l
>> Model: ATA STM3500418AS (scsi)
>> Disk /dev/sda: 500GB
>> Sector size (logical/physical): 512B/512B
>> Partition Table: msdos
>>
>> Number Start End Size Type File system Flags
>> 1 32.8kB 537MB 537MB primary ext3 boot
>> 2 537MB 500GB 500GB primary lvm
>>
>>
>> Error: The backup GPT table is not at the end of the disk, as it should be. This might mean that another operating
>> system believes the disk is smaller. Fix, by moving the backup to the end (and removing the old backup)?
>> Fix/Cancel? c
> The 3ware controller must have reserved some space at the end of each drive for its own use. Didn't know it'd do that. You will have to fix that.
>
> [trim /]
>
Do you have any suggestions on how I can fix that ? I don't have a clue
>> Error: /dev/sdh: unrecognised disk label
>>
>> Error: /dev/sdi: unrecognised disk label
> It seems the flaky controller took out these partition tables. Hope that's all it got. They'll have to be re-created with parted.
>
> Please run parted on each of the ten drives, and make sure they end up like so:
>
>> proxmox:/home/simon# for x in sd{d..m} ; do parted -s /dev/$x unit s
>> print ; done
>> Model: AMCC 9500S-12 DISK (scsi)
>> Disk /dev/sdd: 1953103872s
>> Sector size (logical/physical): 512B/512B
>> Partition Table: gpt
>>
>> Number Start End Size File system Name Flags
>> 1 2048s 1953101823s 1953099776s primary raid
> Make sure you request "unit s" before your other commands so we can make sure it matches.
>
> When you think you are done, check them all:
>

when I was trying to figure out the command for this using 'man parted'
I came across this:
" rescue start end
Rescue a lost partition that was located
somewhere between start and end. If a partition is
found, parted will ask if you want to create an
entry for it in the partition table."
Is it worth trying ?

I originally created the partitions like so:
parted -s /dev/sdb rm 1
parted -s /dev/sdb mklabel gpt
parted -s --align optimal /dev/sdb mkpart primary ext4 .512 100%
parted -s /dev/sdb set 1 raid on
parted -s /dev/sdb align-check optimal 1

so to recreate the above I would do:
parted -s /dev/sdb mkpart primary ext4 2048s 1953101823s
parted -s /dev/sdc mkpart primary ext4 2048s 1953101823s
parted -s /dev/sdf mkpart primary ext4 2048s 1953101823s
parted -s /dev/sdg mkpart primary ext4 2048s 1953101823s
parted -s /dev/sdh mkpart primary ext4 2048s 1953101823s
parted -s /dev/sdi mkpart primary ext4 2048s 1953101823s
parted -s /dev/sdj mkpart primary ext4 2048s 1953101823s
parted -s /dev/sdk mkpart primary ext4 2048s 1953101823s
parted -s /dev/sdl mkpart primary ext4 2048s 1953101823s
parted -s /dev/sdm mkpart primary ext4 2048s 1953101823s

I'm guessing the backups that I want to do can wait until any potential
fsck ?

sorry if the questions are dumb but I'm not sure what I'm doing and I'd
rather ask more questions than fewer and understand the implications of
what I'm doing.

thanks
Simon

> for x in /dev/sd{i,h,g,f,m,l,k,j,b,c} ; do parted -s $x unit s print ; done
>
> After that, create:
>
> mdadm --create --verbose --assume-clean /dev/md0 --metadata=1.1 --level=5 --raid-devices=10 /dev/sd{i,h,g,f,m,l,k,j,b,c}1 --chunk=64
>
> And finally:
>
> pvscan --verbose
>
> Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

On 02/16/2011 12:49 PM, Simon McNair wrote:
> Hi Phil,
> A couple of questions please.

[trim /]

>>> Simon
>> I don't know why the serial numbers are formatted differently, but we can still tell them apart (the eight characters starting with "PAG").
>>
>> So, our device order in your new setup is: [ihgfmlkjbc], where /dev/sdi corresponds to the original report's /dev/sdd, which matches the sig grep in your other note.
>>
>> Another note: The controller for sd[abc] is still showing ata_piix as its controller. That means you cannot hot-plug those ports. If you change your BIOS to AHCI mode instead of "Compatibility" or "Emulation", the full-featured ahci driver will run those ports. Not urgent, but I highly recommend it.
>>
> Will do that now, before I forget

Hot-pluggability with suitable trays is very handy! :)

[trim /]

>>> Error: The backup GPT table is not at the end of the disk, as it should be. This might mean that another operating
>>> system believes the disk is smaller. Fix, by moving the backup to the end (and removing the old backup)?
>>> Fix/Cancel? c
>> The 3ware controller must have reserved some space at the end of each drive for its own use. Didn't know it'd do that. You will have to fix that.
>>
>> [trim /]
>>
> Do you have any suggestions on how I can fix that ? I don't have a clue

Just do 'parted /dev/sd?' and on the ones it offers to fix, say yes. Then request 'unit s' and 'print' to verify that it is correct.

[trim /]

> when I was trying to figure out the command for this using 'man parted' I came across this:
> " rescue start end
> Rescue a lost partition that was located somewhere between start and end. If a partition is
> found, parted will ask if you want to create an entry for it in the partition table."
> Is it worth trying ?

Nah. That's for when you don't know exactly where the partition is. We know.

> I originally created the partitions like so:
> parted -s /dev/sdb rm 1
> parted -s /dev/sdb mklabel gpt
> parted -s --align optimal /dev/sdb mkpart primary ext4 .512 100%
> parted -s /dev/sdb set 1 raid on
> parted -s /dev/sdb align-check optimal 1
>
> so to recreate the above I would do:
> parted -s /dev/sdb mkpart primary ext4 2048s 1953101823s
> parted -s /dev/sdc mkpart primary ext4 2048s 1953101823s
> parted -s /dev/sdf mkpart primary ext4 2048s 1953101823s
> parted -s /dev/sdg mkpart primary ext4 2048s 1953101823s
> parted -s /dev/sdh mkpart primary ext4 2048s 1953101823s
> parted -s /dev/sdi mkpart primary ext4 2048s 1953101823s
> parted -s /dev/sdj mkpart primary ext4 2048s 1953101823s
> parted -s /dev/sdk mkpart primary ext4 2048s 1953101823s
> parted -s /dev/sdl mkpart primary ext4 2048s 1953101823s
> parted -s /dev/sdm mkpart primary ext4 2048s 1953101823s

Only recreate the partition tables where you have to, i.e., the 'Fix' option above didn't work. And don't specify a filesystem.

Probably just /dev/sdh and /dev/sdi. Like so, though:

parted -s /dev/sdh mklabel gpt mkpart primary 2048s 1953101823s set 1 raid on
parted -s /dev/sdi mklabel gpt mkpart primary 2048s 1953101823s set 1 raid on

> I'm guessing the backups that I want to do can wait until any potential fsck ?

Do an 'fsck -N' first, and if it passes, or has few errors, mount the filesystem readonly and grab your backup. Then let fsck have at it for real. If anything gets fixed, compare your backup from the read-only fs to the fixed fs.

Given your flaky old controller, I expect there'll be *some* problems.

> sorry if the questions are dumb but I'm not sure what I'm doing and I'd rather ask more questions than fewer and understand the implications of what I'm doing.

Oh, no. You are right to be paranoid. If anything looks funny, stop.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

parted -l gives

Error: The backup GPT table is not at the end of the disk, as it should
be. This might mean that another operating
system believes the disk is smaller. Fix, by moving the backup to the
end (and removing the old backup)?
Fix/Cancel? fix
Warning: Not all of the space available to /dev/sdb appears to be used,
you can fix the GPT to use all of the space (an
extra 421296 blocks) or continue with the current setting?
Fix/Ignore?

I'm guessing ignore ?

Simon

On 16/02/2011 18:14, Phil Turmel wrote:
> On 02/16/2011 12:49 PM, Simon McNair wrote:
>> Hi Phil,
>> A couple of questions please.
> [trim /]
>
>>>> Simon
>>> I don't know why the serial numbers are formatted differently, but we can still tell them apart (the eight characters starting with "PAG").
>>>
>>> So, our device order in your new setup is: [ihgfmlkjbc], where /dev/sdi corresponds to the original report's /dev/sdd, which matches the sig grep in your other note.
>>>
>>> Another note: The controller for sd[abc] is still showing ata_piix as its controller. That means you cannot hot-plug those ports. If you change your BIOS to AHCI mode instead of "Compatibility" or "Emulation", the full-featured ahci driver will run those ports. Not urgent, but I highly recommend it.
>>>
>> Will do that now, before I forget
> Hot-pluggability with suitable trays is very handy! :)
>
> [trim /]
>
>>>> Error: The backup GPT table is not at the end of the disk, as it should be. This might mean that another operating
>>>> system believes the disk is smaller. Fix, by moving the backup to the end (and removing the old backup)?
>>>> Fix/Cancel? c
>>> The 3ware controller must have reserved some space at the end of each drive for its own use. Didn't know it'd do that. You will have to fix that.
>>>
>>> [trim /]
>>>
>> Do you have any suggestions on how I can fix that ? I don't have a clue
> Just do 'parted /dev/sd?' and on the ones it offers to fix, say yes. Then request 'unit s' and 'print' to verify that it is correct.
>
> [trim /]
>
>> when I was trying to figure out the command for this using 'man parted' I came across this:
>> " rescue start end
>> Rescue a lost partition that was located somewhere between start and end. If a partition is
>> found, parted will ask if you want to create an entry for it in the partition table."
>> Is it worth trying ?
> Nah. That's for when you don't know exactly where the partition is. We know.
>
>> I originally created the partitions like so:
>> parted -s /dev/sdb rm 1
>> parted -s /dev/sdb mklabel gpt
>> parted -s --align optimal /dev/sdb mkpart primary ext4 .512 100%
>> parted -s /dev/sdb set 1 raid on
>> parted -s /dev/sdb align-check optimal 1
>>
>> so to recreate the above I would do:
>> parted -s /dev/sdb mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdc mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdf mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdg mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdh mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdi mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdj mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdk mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdl mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdm mkpart primary ext4 2048s 1953101823s
> Only recreate the partition tables where you have to, i.e., the 'Fix' option above didn't work. And don't specify a filesystem.
>
> Probably just /dev/sdh and /dev/sdi. Like so, though:
>
> parted -s /dev/sdh mklabel gpt mkpart primary 2048s 1953101823s set 1 raid on
> parted -s /dev/sdi mklabel gpt mkpart primary 2048s 1953101823s set 1 raid on
>
>> I'm guessing the backups that I want to do can wait until any potential fsck ?
> Do an 'fsck -N' first, and if it passes, or has few errors, mount the filesystem readonly and grab your backup. Then let fsck have at it for real. If anything gets fixed, compare your backup from the read-only fs to the fixed fs.
>
> Given your flaky old controller, I expect there'll be *some* problems.
>
>> sorry if the questions are dumb but I'm not sure what I'm doing and I'd rather ask more questions than fewer and understand the implications of what I'm doing.
> Oh, no. You are right to be paranoid. If anything looks funny, stop.
>
> Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

On 02/16/2011 01:18 PM, Simon McNair wrote:
> parted -l gives
>
> Error: The backup GPT table is not at the end of the disk, as it should be. This might mean that another operating
> system believes the disk is smaller. Fix, by moving the backup to the end (and removing the old backup)?
> Fix/Cancel? fix
> Warning: Not all of the space available to /dev/sdb appears to be used, you can fix the GPT to use all of the space (an
> extra 421296 blocks) or continue with the current setting?
> Fix/Ignore?
>
> I'm guessing ignore ?

Either should work. If you 'Fix' for all of the drives, you could later 'grow' your array to include the extra space.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

On 02/16/2011 01:22 PM, Phil Turmel wrote:
> On 02/16/2011 01:18 PM, Simon McNair wrote:
>> parted -l gives
>>
>> Error: The backup GPT table is not at the end of the disk, as it should be. This might mean that another operating
>> system believes the disk is smaller. Fix, by moving the backup to the end (and removing the old backup)?
>> Fix/Cancel? fix
>> Warning: Not all of the space available to /dev/sdb appears to be used, you can fix the GPT to use all of the space (an
>> extra 421296 blocks) or continue with the current setting?
>> Fix/Ignore?
>>
>> I'm guessing ignore ?
>
> Either should work. If you 'Fix' for all of the drives, you could later 'grow' your array to include the extra space.

It's vital that the start sector be 2048, though. Please do recheck after all are done.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

have done this.... Results are:

proxmox:/home/simon# ./lsdrv.sh
Controller device @ pci0000:00/0000:00:01.0/0000:01:00.0 [mvsas]
SCSI storage controller: Marvell Technology Group Ltd.
MV64460/64461/64462 System Controller, Revision B (rev 01)
host0: /dev/sdf ATA Hitachi HDS72101 {SN: GTA000PAGABXRA}
host0: /dev/sdg ATA Hitachi HDS72101 {SN: GTA000PAGAA5DA}
host0: /dev/sdh ATA Hitachi HDS72101 {SN: GTA000PAG9NL9A}
host0: /dev/sdi ATA Hitachi HDS72101 {SN: GTA000PAGA8V4A}
host0: /dev/sdj ATA Hitachi HDS72101 {SN: GTD000PAGMT9GD}
host0: /dev/sdk ATA Hitachi HDS72101 {SN: GTG000PAG18BJC}
host0: /dev/sdl ATA Hitachi HDS72101 {SN: GTG000PAG1DPLC}
host0: /dev/sdm ATA Hitachi HDS72101 {SN: GTA000PAG7WMEA}
Controller device @
pci0000:00/0000:00:1a.7/usb1/1-4/1-4.1/1-4.1.1/1-4.1.1:1.0 [usb-storage]
Bus 001 Device 007: ID 0424:2228 Standard Microsystems Corp. 9-in-2
Card Reader {SN: 08050920003A}
host11: /dev/sdb Generic Flash HS-CF
host11: /dev/sdc Generic Flash HS-COMBO
Controller device @ pci0000:00/0000:00:1c.4/0000:04:00.0 [ahci]
SATA controller: JMicron Technology Corp. JMB362/JMB363 Serial ATA
Controller (rev 03)
host9: [Empty]
host10: [Empty]
Controller device @ pci0000:00/0000:00:1c.4/0000:04:00.1 [pata_jmicron]
IDE interface: JMicron Technology Corp. JMB362/JMB363 Serial ATA
Controller (rev 03)
host1: [Empty]
host2: [Empty]
Controller device @ pci0000:00/0000:00:1f.2 [ahci]
SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI
Controller
host3: /dev/sda ATA STM3500418AS {SN: 9VM3QJ5C}
host4: /dev/sr0 Optiarc DVD RW AD-5240S
host5: [Empty]
host6: [Empty]
host7: /dev/sdd ATA Hitachi HDS72101 {SN: GTD000PAGMT8DD}
host8: /dev/sde ATA Hitachi HDS72101 {SN: GTG000PAG04V0C}

enabling achi has switched the drive letters around for a couple of
drives again

parted list is:

proxmox:/home/simon# for x in /dev/sd{i,h,g,f,m,l,k,j,d,e} ; do parted
-s $x unit s print ; done
Model: ATA Hitachi HDS72101 (scsi)
Disk /dev/sdi: 1953525168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 2048s 1953101823s 1953099776s primary raid

Model: ATA Hitachi HDS72101 (scsi)
Disk /dev/sdh: 1953525168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 2048s 1953101823s 1953099776s primary raid

Model: ATA Hitachi HDS72101 (scsi)
Disk /dev/sdg: 1953525168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 2048s 1953101823s 1953099776s primary raid

Model: ATA Hitachi HDS72101 (scsi)
Disk /dev/sdf: 1953525168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 2048s 1953101823s 1953099776s primary raid

Model: ATA Hitachi HDS72101 (scsi)
Disk /dev/sdm: 1953525168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 2048s 1953101823s 1953099776s primary raid

Model: ATA Hitachi HDS72101 (scsi)
Disk /dev/sdl: 1953525168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 2048s 1953101823s 1953099776s primary raid

Model: ATA Hitachi HDS72101 (scsi)
Disk /dev/sdk: 1953525168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 2048s 1953101823s 1953099776s primary raid

Model: ATA Hitachi HDS72101 (scsi)
Disk /dev/sdj: 1953525168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 2048s 1953101823s 1953099776s primary raid

Model: ATA Hitachi HDS72101 (scsi)
Disk /dev/sdd: 1953525168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 2048s 1953101823s 1953099776s primary raid

Model: ATA Hitachi HDS72101 (scsi)
Disk /dev/sde: 1953525168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 2048s 1953101823s 1953099776s primary raid

is it now:

mdadm --create --verbose --assume-clean /dev/md0 --metadata=1.1
--level=5 --raid-devices=10 /dev/sd{i,h,g,f,m,l,k,j,d,e}1 --chunk=64

?
regards
Simon



On 16/02/2011 18:25, Phil Turmel wrote:
> On 02/16/2011 01:22 PM, Phil Turmel wrote:
>> On 02/16/2011 01:18 PM, Simon McNair wrote:
>>> parted -l gives
>>>
>>> Error: The backup GPT table is not at the end of the disk, as it should be. This might mean that another operating
>>> system believes the disk is smaller. Fix, by moving the backup to the end (and removing the old backup)?
>>> Fix/Cancel? fix
>>> Warning: Not all of the space available to /dev/sdb appears to be used, you can fix the GPT to use all of the space (an
>>> extra 421296 blocks) or continue with the current setting?
>>> Fix/Ignore?
>>>
>>> I'm guessing ignore ?
>> Either should work. If you 'Fix' for all of the drives, you could later 'grow' your array to include the extra space.
> It's vital that the start sector be 2048, though. Please do recheck after all are done.
>
> Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

On 02/16/2011 01:52 PM, Simon McNair wrote:
> have done this.... Results are:
>
> proxmox:/home/simon# ./lsdrv.sh
> Controller device @ pci0000:00/0000:00:01.0/0000:01:00.0 [mvsas]
> SCSI storage controller: Marvell Technology Group Ltd. MV64460/64461/64462 System Controller, Revision B (rev 01)
> host0: /dev/sdf ATA Hitachi HDS72101 {SN: GTA000PAGABXRA}
> host0: /dev/sdg ATA Hitachi HDS72101 {SN: GTA000PAGAA5DA}
> host0: /dev/sdh ATA Hitachi HDS72101 {SN: GTA000PAG9NL9A}
> host0: /dev/sdi ATA Hitachi HDS72101 {SN: GTA000PAGA8V4A}
> host0: /dev/sdj ATA Hitachi HDS72101 {SN: GTD000PAGMT9GD}
> host0: /dev/sdk ATA Hitachi HDS72101 {SN: GTG000PAG18BJC}
> host0: /dev/sdl ATA Hitachi HDS72101 {SN: GTG000PAG1DPLC}
> host0: /dev/sdm ATA Hitachi HDS72101 {SN: GTA000PAG7WMEA}
> Controller device @ pci0000:00/0000:00:1a.7/usb1/1-4/1-4.1/1-4.1.1/1-4.1.1:1.0 [usb-storage]
> Bus 001 Device 007: ID 0424:2228 Standard Microsystems Corp. 9-in-2 Card Reader {SN: 08050920003A}
> host11: /dev/sdb Generic Flash HS-CF
> host11: /dev/sdc Generic Flash HS-COMBO
> Controller device @ pci0000:00/0000:00:1c.4/0000:04:00.0 [ahci]
> SATA controller: JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (rev 03)
> host9: [Empty]
> host10: [Empty]
> Controller device @ pci0000:00/0000:00:1c.4/0000:04:00.1 [pata_jmicron]
> IDE interface: JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (rev 03)
> host1: [Empty]
> host2: [Empty]
> Controller device @ pci0000:00/0000:00:1f.2 [ahci]
> SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller
> host3: /dev/sda ATA STM3500418AS {SN: 9VM3QJ5C}
> host4: /dev/sr0 Optiarc DVD RW AD-5240S
> host5: [Empty]
> host6: [Empty]
> host7: /dev/sdd ATA Hitachi HDS72101 {SN: GTD000PAGMT8DD}
> host8: /dev/sde ATA Hitachi HDS72101 {SN: GTG000PAG04V0C}
>
> enabling achi has switched the drive letters around for a couple of drives again
>
> parted list is:
>
> proxmox:/home/simon# for x in /dev/sd{i,h,g,f,m,l,k,j,d,e} ; do parted -s $x unit s print ; done
> Model: ATA Hitachi HDS72101 (scsi)
> Disk /dev/sdi: 1953525168s
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
>
> Number Start End Size File system Name Flags
> 1 2048s 1953101823s 1953099776s primary raid
>
> Model: ATA Hitachi HDS72101 (scsi)
> Disk /dev/sdh: 1953525168s
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
>
> Number Start End Size File system Name Flags
> 1 2048s 1953101823s 1953099776s primary raid
>
> Model: ATA Hitachi HDS72101 (scsi)
> Disk /dev/sdg: 1953525168s
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
>
> Number Start End Size File system Name Flags
> 1 2048s 1953101823s 1953099776s primary raid
>
> Model: ATA Hitachi HDS72101 (scsi)
> Disk /dev/sdf: 1953525168s
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
>
> Number Start End Size File system Name Flags
> 1 2048s 1953101823s 1953099776s primary raid
>
> Model: ATA Hitachi HDS72101 (scsi)
> Disk /dev/sdm: 1953525168s
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
>
> Number Start End Size File system Name Flags
> 1 2048s 1953101823s 1953099776s primary raid
>
> Model: ATA Hitachi HDS72101 (scsi)
> Disk /dev/sdl: 1953525168s
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
>
> Number Start End Size File system Name Flags
> 1 2048s 1953101823s 1953099776s primary raid
>
> Model: ATA Hitachi HDS72101 (scsi)
> Disk /dev/sdk: 1953525168s
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
>
> Number Start End Size File system Name Flags
> 1 2048s 1953101823s 1953099776s primary raid
>
> Model: ATA Hitachi HDS72101 (scsi)
> Disk /dev/sdj: 1953525168s
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
>
> Number Start End Size File system Name Flags
> 1 2048s 1953101823s 1953099776s primary raid
>
> Model: ATA Hitachi HDS72101 (scsi)
> Disk /dev/sdd: 1953525168s
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
>
> Number Start End Size File system Name Flags
> 1 2048s 1953101823s 1953099776s primary raid
>
> Model: ATA Hitachi HDS72101 (scsi)
> Disk /dev/sde: 1953525168s
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
>
> Number Start End Size File system Name Flags
> 1 2048s 1953101823s 1953099776s primary raid

All looks good.

> is it now:
>
> mdadm --create --verbose --assume-clean /dev/md0 --metadata=1.1 --level=5 --raid-devices=10 /dev/sd{i,h,g,f,m,l,k,j,d,e}1 --chunk=64

Yes.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

pvscan is:
proxmox:/home/simon# pvscan --verbose
Wiping cache of LVM-capable devices
Wiping internal VG cache
Walking through all physical volumes
PV /dev/sda2 VG pve lvm2 [465.26 GB / 4.00 GB free]
PV /dev/md0 VG lvm-raid lvm2 [8.19 TB / 0 free]
Total: 2 [655.05 GB] / in use: 2 [655.05 GB] / in no VG: 0 [0 ]

lvm-raid is there ....

proxmox:/home/simon# fsck.ext4 -n /dev/md0
e2fsck 1.41.3 (12-Oct-2008)
fsck.ext4: Superblock invalid, trying backup blocks...
Superblock has an invalid ext3 journal (inode 8).
Clear? no

fsck.ext4: Illegal inode number while checking ext3 journal for /dev/md0

I thought I needed to run fsck against the lvm group ? I used to do...
fsck.ext4 /dev/lvm-raid/RAID so do I need to mount it or something ?

cheers
Simon


On 16/02/2011 18:57, Phil Turmel wrote:
> On 02/16/2011 01:52 PM, Simon McNair wrote:
>> have done this.... Results are:
>>
>> proxmox:/home/simon# ./lsdrv.sh
>> Controller device @ pci0000:00/0000:00:01.0/0000:01:00.0 [mvsas]
>> SCSI storage controller: Marvell Technology Group Ltd. MV64460/64461/64462 System Controller, Revision B (rev 01)
>> host0: /dev/sdf ATA Hitachi HDS72101 {SN: GTA000PAGABXRA}
>> host0: /dev/sdg ATA Hitachi HDS72101 {SN: GTA000PAGAA5DA}
>> host0: /dev/sdh ATA Hitachi HDS72101 {SN: GTA000PAG9NL9A}
>> host0: /dev/sdi ATA Hitachi HDS72101 {SN: GTA000PAGA8V4A}
>> host0: /dev/sdj ATA Hitachi HDS72101 {SN: GTD000PAGMT9GD}
>> host0: /dev/sdk ATA Hitachi HDS72101 {SN: GTG000PAG18BJC}
>> host0: /dev/sdl ATA Hitachi HDS72101 {SN: GTG000PAG1DPLC}
>> host0: /dev/sdm ATA Hitachi HDS72101 {SN: GTA000PAG7WMEA}
>> Controller device @ pci0000:00/0000:00:1a.7/usb1/1-4/1-4.1/1-4.1.1/1-4.1.1:1.0 [usb-storage]
>> Bus 001 Device 007: ID 0424:2228 Standard Microsystems Corp. 9-in-2 Card Reader {SN: 08050920003A}
>> host11: /dev/sdb Generic Flash HS-CF
>> host11: /dev/sdc Generic Flash HS-COMBO
>> Controller device @ pci0000:00/0000:00:1c.4/0000:04:00.0 [ahci]
>> SATA controller: JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (rev 03)
>> host9: [Empty]
>> host10: [Empty]
>> Controller device @ pci0000:00/0000:00:1c.4/0000:04:00.1 [pata_jmicron]
>> IDE interface: JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (rev 03)
>> host1: [Empty]
>> host2: [Empty]
>> Controller device @ pci0000:00/0000:00:1f.2 [ahci]
>> SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller
>> host3: /dev/sda ATA STM3500418AS {SN: 9VM3QJ5C}
>> host4: /dev/sr0 Optiarc DVD RW AD-5240S
>> host5: [Empty]
>> host6: [Empty]
>> host7: /dev/sdd ATA Hitachi HDS72101 {SN: GTD000PAGMT8DD}
>> host8: /dev/sde ATA Hitachi HDS72101 {SN: GTG000PAG04V0C}
>>
>> enabling achi has switched the drive letters around for a couple of drives again
>>
>> parted list is:
>>
>> proxmox:/home/simon# for x in /dev/sd{i,h,g,f,m,l,k,j,d,e} ; do parted -s $x unit s print ; done
>> Model: ATA Hitachi HDS72101 (scsi)
>> Disk /dev/sdi: 1953525168s
>> Sector size (logical/physical): 512B/512B
>> Partition Table: gpt
>>
>> Number Start End Size File system Name Flags
>> 1 2048s 1953101823s 1953099776s primary raid
>>
>> Model: ATA Hitachi HDS72101 (scsi)
>> Disk /dev/sdh: 1953525168s
>> Sector size (logical/physical): 512B/512B
>> Partition Table: gpt
>>
>> Number Start End Size File system Name Flags
>> 1 2048s 1953101823s 1953099776s primary raid
>>
>> Model: ATA Hitachi HDS72101 (scsi)
>> Disk /dev/sdg: 1953525168s
>> Sector size (logical/physical): 512B/512B
>> Partition Table: gpt
>>
>> Number Start End Size File system Name Flags
>> 1 2048s 1953101823s 1953099776s primary raid
>>
>> Model: ATA Hitachi HDS72101 (scsi)
>> Disk /dev/sdf: 1953525168s
>> Sector size (logical/physical): 512B/512B
>> Partition Table: gpt
>>
>> Number Start End Size File system Name Flags
>> 1 2048s 1953101823s 1953099776s primary raid
>>
>> Model: ATA Hitachi HDS72101 (scsi)
>> Disk /dev/sdm: 1953525168s
>> Sector size (logical/physical): 512B/512B
>> Partition Table: gpt
>>
>> Number Start End Size File system Name Flags
>> 1 2048s 1953101823s 1953099776s primary raid
>>
>> Model: ATA Hitachi HDS72101 (scsi)
>> Disk /dev/sdl: 1953525168s
>> Sector size (logical/physical): 512B/512B
>> Partition Table: gpt
>>
>> Number Start End Size File system Name Flags
>> 1 2048s 1953101823s 1953099776s primary raid
>>
>> Model: ATA Hitachi HDS72101 (scsi)
>> Disk /dev/sdk: 1953525168s
>> Sector size (logical/physical): 512B/512B
>> Partition Table: gpt
>>
>> Number Start End Size File system Name Flags
>> 1 2048s 1953101823s 1953099776s primary raid
>>
>> Model: ATA Hitachi HDS72101 (scsi)
>> Disk /dev/sdj: 1953525168s
>> Sector size (logical/physical): 512B/512B
>> Partition Table: gpt
>>
>> Number Start End Size File system Name Flags
>> 1 2048s 1953101823s 1953099776s primary raid
>>
>> Model: ATA Hitachi HDS72101 (scsi)
>> Disk /dev/sdd: 1953525168s
>> Sector size (logical/physical): 512B/512B
>> Partition Table: gpt
>>
>> Number Start End Size File system Name Flags
>> 1 2048s 1953101823s 1953099776s primary raid
>>
>> Model: ATA Hitachi HDS72101 (scsi)
>> Disk /dev/sde: 1953525168s
>> Sector size (logical/physical): 512B/512B
>> Partition Table: gpt
>>
>> Number Start End Size File system Name Flags
>> 1 2048s 1953101823s 1953099776s primary raid
> All looks good.
>
>> is it now:
>>
>> mdadm --create --verbose --assume-clean /dev/md0 --metadata=1.1 --level=5 --raid-devices=10 /dev/sd{i,h,g,f,m,l,k,j,d,e}1 --chunk=64
> Yes.
>
> Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

On 02/16/2011 02:07 PM, Simon McNair wrote:
> pvscan is:
> proxmox:/home/simon# pvscan --verbose
> Wiping cache of LVM-capable devices
> Wiping internal VG cache
> Walking through all physical volumes
> PV /dev/sda2 VG pve lvm2 [465.26 GB / 4.00 GB free]
> PV /dev/md0 VG lvm-raid lvm2 [8.19 TB / 0 free]
> Total: 2 [655.05 GB] / in use: 2 [655.05 GB] / in no VG: 0 [0 ]
>
> lvm-raid is there ....

Very good.

> proxmox:/home/simon# fsck.ext4 -n /dev/md0

Uh, no.

> e2fsck 1.41.3 (12-Oct-2008)
> fsck.ext4: Superblock invalid, trying backup blocks...
> Superblock has an invalid ext3 journal (inode 8).
> Clear? no
>
> fsck.ext4: Illegal inode number while checking ext3 journal for /dev/md0


> I thought I needed to run fsck against the lvm group ? I used to do... fsck.ext4 /dev/lvm-raid/RAID so do I need to mount it or something ?

vgscan --verbose

lvscan --verbose


Then either:

fsck -N /dev/mapper/lvm-raid-RAID

or:

fsck -N /dev/lvm-raid/RAID

depends on what udev is doing.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

proxmox:/home/simon# vgscan --verbose
Wiping cache of LVM-capable devices
Wiping internal VG cache
Reading all physical volumes. This may take a while...
Finding all volume groups
Finding volume group "pve"
Found volume group "pve" using metadata type lvm2
Finding volume group "lvm-raid"
Found volume group "lvm-raid" using metadata type lvm2
proxmox:/home/simon#
proxmox:/home/simon# lvscan --verbose
Finding all logical volumes
ACTIVE '/dev/pve/swap' [11.00 GB] inherit
ACTIVE '/dev/pve/root' [96.00 GB] inherit
ACTIVE '/dev/pve/data' [354.26 GB] inherit
inactive '/dev/lvm-raid/RAID' [8.19 TB] inherit

proxmox:/home/simon# vgchange -ay
3 logical volume(s) in volume group "pve" now active
1 logical volume(s) in volume group "lvm-raid" now active

proxmox:/home/simon# fsck.ext4 -n /dev/mapper/lvm-raid-RAID
e2fsck 1.41.3 (12-Oct-2008)
fsck.ext4: No such file or directory while trying to open
/dev/mapper/lvm-raid-RAID

The superblock could not be read or does not describe a correct ext2
filesystem. If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsck -b 8193

proxmox:/home/simon# fsck.ext4 -n /dev/mapper/
control lvm--raid-RAID pve-data pve-root pve-swap
proxmox:/home/simon# fsck.ext4 -n /dev/mapper/lvm--raid-RAID
e2fsck 1.41.3 (12-Oct-2008)
/dev/mapper/lvm--raid-RAID has unsupported feature(s): FEATURE_I31
e2fsck: Get a newer version of e2fsck!

my version of e2fsck always worked before ?

Simon

On 16/02/2011 19:10, Phil Turmel wrote:
> On 02/16/2011 02:07 PM, Simon McNair wrote:
>> pvscan is:
>> proxmox:/home/simon# pvscan --verbose
>> Wiping cache of LVM-capable devices
>> Wiping internal VG cache
>> Walking through all physical volumes
>> PV /dev/sda2 VG pve lvm2 [465.26 GB / 4.00 GB free]
>> PV /dev/md0 VG lvm-raid lvm2 [8.19 TB / 0 free]
>> Total: 2 [655.05 GB] / in use: 2 [655.05 GB] / in no VG: 0 [0 ]
>>
>> lvm-raid is there ....
> Very good.
>
>> proxmox:/home/simon# fsck.ext4 -n /dev/md0
> Uh, no.
>
>> e2fsck 1.41.3 (12-Oct-2008)
>> fsck.ext4: Superblock invalid, trying backup blocks...
>> Superblock has an invalid ext3 journal (inode 8).
>> Clear? no
>>
>> fsck.ext4: Illegal inode number while checking ext3 journal for /dev/md0
>
>> I thought I needed to run fsck against the lvm group ? I used to do... fsck.ext4 /dev/lvm-raid/RAID so do I need to mount it or something ?
> vgscan --verbose
>
> lvscan --verbose
>
>
> Then either:
>
> fsck -N /dev/mapper/lvm-raid-RAID
>
> or:
>
> fsck -N /dev/lvm-raid/RAID
>
> depends on what udev is doing.
>
> Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

On 02/16/2011 02:15 PM, Simon McNair wrote:
> proxmox:/home/simon# vgscan --verbose
> Wiping cache of LVM-capable devices
> Wiping internal VG cache
> Reading all physical volumes. This may take a while...
> Finding all volume groups
> Finding volume group "pve"
> Found volume group "pve" using metadata type lvm2
> Finding volume group "lvm-raid"
> Found volume group "lvm-raid" using metadata type lvm2
> proxmox:/home/simon#
> proxmox:/home/simon# lvscan --verbose
> Finding all logical volumes
> ACTIVE '/dev/pve/swap' [11.00 GB] inherit
> ACTIVE '/dev/pve/root' [96.00 GB] inherit
> ACTIVE '/dev/pve/data' [354.26 GB] inherit
> inactive '/dev/lvm-raid/RAID' [8.19 TB] inherit
>
> proxmox:/home/simon# vgchange -ay
> 3 logical volume(s) in volume group "pve" now active
> 1 logical volume(s) in volume group "lvm-raid" now active

Heh. Figures.

> proxmox:/home/simon# fsck.ext4 -n /dev/mapper/lvm-raid-RAID

Actually, I wanted you to try with a capital N. Lower case 'n' is similar, but not quite the same.

> e2fsck 1.41.3 (12-Oct-2008)
> fsck.ext4: No such file or directory while trying to open /dev/mapper/lvm-raid-RAID
>
> The superblock could not be read or does not describe a correct ext2
> filesystem. If the device is valid and it really contains an ext2
> filesystem (and not swap or ufs or something else), then the superblock
> is corrupt, and you might try running e2fsck with an alternate superblock:
> e2fsck -b 8193
>
> proxmox:/home/simon# fsck.ext4 -n /dev/mapper/
> control lvm--raid-RAID pve-data pve-root pve-swap

Strange. I guess it does that to distinguish dashes in the VG name from dashes between VG and LV names.

> proxmox:/home/simon# fsck.ext4 -n /dev/mapper/lvm--raid-RAID
> e2fsck 1.41.3 (12-Oct-2008)
> /dev/mapper/lvm--raid-RAID has unsupported feature(s): FEATURE_I31
> e2fsck: Get a newer version of e2fsck!
>
> my version of e2fsck always worked before ?

v1.41.14 was release 7 weeks ago. But, I suspect there's corruption in the superblock. Do you still have your disk images tucked away somewhere safe?

If so, try:

1) The '-b' option to e2fsck. We need to experiment with '-n -b offset' to find the alternate superblock. Trying 'offset' = to 8193, 16384, and 32768, per the man-page.

2) A newer e2fsprogs.

Finally,
3) mount -r /dev/lvm-raid/RAID /mnt/whatever

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

Phil,
Jeez I'm having a bad week (my windows 7x64 machine has just started
randomly crashing, my Thecus n5200 is playing up and the weather has
been dire so I've not been able to put my new shed up...oh and I have
ongoing 'other' issues as you're well aware ;-)) The thecus n5200 has
5x2TB hdd's. I wiped out the existing raid5 array to create a jbod span
of 10tb in order to hold my 9tb of backups. The Thecus has had a hissy
fit and I've had to set the process off again, so you can bet it'll be a
day or two before it get's the drives formatted (it's not a very
powerful nas), then I'll do the backups, then I'll try as you suggested.

Thanks for the ongoing assistance.

Simon

On 16/02/2011 19:36, Phil Turmel wrote:
> On 02/16/2011 02:15 PM, Simon McNair wrote:
>> proxmox:/home/simon# vgscan --verbose
>> Wiping cache of LVM-capable devices
>> Wiping internal VG cache
>> Reading all physical volumes. This may take a while...
>> Finding all volume groups
>> Finding volume group "pve"
>> Found volume group "pve" using metadata type lvm2
>> Finding volume group "lvm-raid"
>> Found volume group "lvm-raid" using metadata type lvm2
>> proxmox:/home/simon#
>> proxmox:/home/simon# lvscan --verbose
>> Finding all logical volumes
>> ACTIVE '/dev/pve/swap' [11.00 GB] inherit
>> ACTIVE '/dev/pve/root' [96.00 GB] inherit
>> ACTIVE '/dev/pve/data' [354.26 GB] inherit
>> inactive '/dev/lvm-raid/RAID' [8.19 TB] inherit
>>
>> proxmox:/home/simon# vgchange -ay
>> 3 logical volume(s) in volume group "pve" now active
>> 1 logical volume(s) in volume group "lvm-raid" now active
> Heh. Figures.
>
>> proxmox:/home/simon# fsck.ext4 -n /dev/mapper/lvm-raid-RAID
> Actually, I wanted you to try with a capital N. Lower case 'n' is similar, but not quite the same.
>
>> e2fsck 1.41.3 (12-Oct-2008)
>> fsck.ext4: No such file or directory while trying to open /dev/mapper/lvm-raid-RAID
>>
>> The superblock could not be read or does not describe a correct ext2
>> filesystem. If the device is valid and it really contains an ext2
>> filesystem (and not swap or ufs or something else), then the superblock
>> is corrupt, and you might try running e2fsck with an alternate superblock:
>> e2fsck -b 8193
>>
>> proxmox:/home/simon# fsck.ext4 -n /dev/mapper/
>> control lvm--raid-RAID pve-data pve-root pve-swap
> Strange. I guess it does that to distinguish dashes in the VG name from dashes between VG and LV names.
>
>> proxmox:/home/simon# fsck.ext4 -n /dev/mapper/lvm--raid-RAID
>> e2fsck 1.41.3 (12-Oct-2008)
>> /dev/mapper/lvm--raid-RAID has unsupported feature(s): FEATURE_I31
>> e2fsck: Get a newer version of e2fsck!
>>
>> my version of e2fsck always worked before ?
> v1.41.14 was release 7 weeks ago. But, I suspect there's corruption in the superblock. Do you still have your disk images tucked away somewhere safe?
>
> If so, try:
>
> 1) The '-b' option to e2fsck. We need to experiment with '-n -b offset' to find the alternate superblock. Trying 'offset' = to 8193, 16384, and 32768, per the man-page.
>
> 2) A newer e2fsprogs.
>
> Finally,
> 3) mount -r /dev/lvm-raid/RAID /mnt/whatever
>
> Phil
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

On 02/16/2011 04:28 PM, Simon McNair wrote:
> Phil,
> Jeez I'm having a bad week (my windows 7x64 machine has just started randomly crashing, my Thecus n5200 is playing up and the weather has been dire so I've not been able to put my new shed up...oh and I have ongoing 'other' issues as you're well aware ;-)) The thecus n5200 has 5x2TB hdd's. I wiped out the existing raid5 array to create a jbod span of 10tb in order to hold my 9tb of backups. The Thecus has had a hissy fit and I've had to set the process off again, so you can bet it'll be a day or two before it get's the drives formatted (it's not a very powerful nas), then I'll do the backups, then I'll try as you suggested.

That's fine.
>
> Thanks for the ongoing assistance.

No problem.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

I just went to bed and one last question popped in to my mind. Since
there is a fair timezone gap I thought I'd be presumptuous and ask it
in the hope I can turn it around a bit quicker in the morning.

My suspicion is that once I have 5 formatted 2tb drives I may be lucky
to get 10x 1tb dd images on to it. Can I feed the dd process in to
tar, bzip2, zip or something else which will give me enough space to
fit the images on ?

Will I get more usable space from 5x2tb partitions or from 1xspanned
volume ? (the thecus pretty much only allows you to create raid
volumes so a jbod needs to be 5x1tb arrays or 1 spanned volume or
stripe).

TIA
Simon

On 16 Feb 2011, at 21:30, Phil Turmel wrote:

> On 02/16/2011 04:28 PM, Simon McNair wrote:
>> Phil,
>> Jeez I'm having a bad week (my windows 7x64 machine has just started randomly crashing, my Thecus n5200 is playing up and the weather has been dire so I've not been able to put my new shed up...oh and I have ongoing 'other' issues as you're well aware ;-)) The thecus n5200 has 5x2TB hdd's. I wiped out the existing raid5 array to create a jbod span of 10tb in order to hold my 9tb of backups. The Thecus has had a hissy fit and I've had to set the process off again, so you can bet it'll be a day or two before it get's the drives formatted (it's not a very powerful nas), then I'll do the backups, then I'll try as you suggested.
>
> That's fine.
>>
>> Thanks for the ongoing assistance.
>
> No problem.
>
> Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

On 02/16/2011 05:44 PM, Simon Mcnair wrote:
> I just went to bed and one last question popped in to my mind. Since
> there is a fair timezone gap I thought I'd be presumptuous and ask it
> in the hope I can turn it around a bit quicker in the morning.
>
> My suspicion is that once I have 5 formatted 2tb drives I may be lucky
> to get 10x 1tb dd images on to it. Can I feed the dd process in to
> tar, bzip2, zip or something else which will give me enough space to
> fit the images on ?
>
> Will I get more usable space from 5x2tb partitions or from 1xspanned
> volume ? (the thecus pretty much only allows you to create raid
> volumes so a jbod needs to be 5x1tb arrays or 1 spanned volume or
> stripe).

I'd use one spanned volume, and gzip. I'd simultaneously generate an md5sum while streaming to your thecus. A script like so:

#! /bin/bash
#
function usage() {
printf "Usage:\n\t%s devname\n\n" "`basename \"$0\"`"
printf "'devname' must be a relative path in /dev/ to the desired block device.\n"
exit 1
}

# Verify the supplied name is a device
test -b "/dev/$1" || usage

# Convert path separators and spaces into dashes
outfile="`echo \"$1\" |sed -r -e 's:[ /]+:-:g'`"

# Create a side-stream for computing the MD5 of the data read
fifo=`mktemp -u`
mkfifo $fifo || exit
md5sum -b <$fifo >/mnt/thecus/$outfile.md5 &

# Read the device and compress it
dd if="/dev/$1" bs=1M | tee 2>$fifo | gzip >/mnt/thecus/$outfile.gz

# Wait for the background task to close
wait
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

Phil,
After a couple of attempts I realised that all I needed to do was
/backup.sh sdi rather than specifying ./backup.sh /dev/sdi schoolbo=
y
error huh, rtfm.

I tried modifying the code, so I could kick them all off at once, but
I wanted to check that this would work from a multithreaded/sane
perspective (yes I know it's a bit of IO, but iotop seems to infer I'm
only getting 10M/s throughput from the disk anyway).

is this code kosha ?

#! /bin/bash
#
function usage() {
printf "Usage:\n\t%s devname\n\n" "`basename \"$0\"`"
printf "'devname' must be a relative path in /dev/ to the
desired block device.\n"
exit 1
}

# Verify the supplied name is a device
test -b "/dev/$1" || usage

# Convert path separators and spaces into dashes
outfile=3D"`echo \"$1\" |sed -r -e 's:[ /]+:-:g'`"

# Create a side-stream for computing the MD5 of the data read
fifo=3D`mktemp -u`
mkfifo $fifo || exit
md5sum -b <$fifo >$2/$outfile.md5 &

# Read the device and compress it
dd if=3D"/dev/$1" bs=3D1M | tee 2>$fifo | gzip >$2/$outfile.gz

# Wait for the background task to close
wait

I was a little concerned as I didn't see the hdd drive LED light up
for my attempt even though the file was growing nicely on my CIFS
share. I'm dumping it on the 5x2TB disks (JBOD) in my windows 7 box
as my Thecus is not a happy chappy and I need time to mount the DOM
module in another machine to find out why.

cheers
Simon


On 16 February 2011 23:39, Phil Turmel wrote:
> On 02/16/2011 05:44 PM, Simon Mcnair wrote:
>> I just went to bed and one last question popped in to my mind. Since
>> there is a fair timezone gap I thought I'd be presumptuous and ask i=
t
>> in the hope I can turn it around a bit quicker in the morning.
>>
>> My suspicion is that once I have 5 formatted 2tb drives I may be luc=
ky
>> to get 10x 1tb dd images on to it. Can I feed the dd process in to
>> tar, bzip2, zip or something else which will give me enough space to
>> fit the images on ?
>>
>> Will I get more usable space from 5x2tb partitions or from 1xspanned
>> volume ? (the thecus pretty much only allows you to create raid
>> volumes so a jbod needs to be 5x1tb arrays or 1 spanned volume or
>> stripe).
>
> I'd use one spanned volume, and gzip. =A0I'd simultaneously generate =
an md5sum while streaming to your thecus. =A0A script like so:
>
> #! /bin/bash
> #
> function usage() {
> =A0 =A0 =A0 =A0printf "Usage:\n\t%s devname\n\n" "`basename \"$0\"`"
> =A0 =A0 =A0 =A0printf "'devname' must be a relative path in /dev/ to =
the desired block device.\n"
> =A0 =A0 =A0 =A0exit 1
> }
>
> # Verify the supplied name is a device
> test -b "/dev/$1" || usage
>
> # Convert path separators and spaces into dashes
> outfile=3D"`echo \"$1\" |sed -r -e 's:[ /]+:-:g'`"
>
> # Create a side-stream for computing the MD5 of the data read
> fifo=3D`mktemp -u`
> mkfifo $fifo || exit
> md5sum -b <$fifo >/mnt/thecus/$outfile.md5 &
>
> # Read the device and compress it
> dd if=3D"/dev/$1" bs=3D1M | tee 2>$fifo | gzip >/mnt/thecus/$outfile.=
gz
>
> # Wait for the background task to close
> wait
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

This is a multi-part message in MIME format.
--------------060200030107030205010800
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

On 02/17/2011 08:26 AM, Simon Mcnair wrote:
> Phil,
> After a couple of attempts I realised that all I needed to do was
> ./backup.sh sdi rather than specifying ./backup.sh /dev/sdi schoolboy
> error huh, rtfm.
>
> I tried modifying the code, so I could kick them all off at once, but
> I wanted to check that this would work from a multithreaded/sane
> perspective (yes I know it's a bit of IO, but iotop seems to infer I'm
> only getting 10M/s throughput from the disk anyway).

That sounds suspiciously like a 100 meg ethernet bottleneck, so parallel operation won't help. If so, this'll take a very long time.

> is this code kosha ?

Looks pretty good. I've attached a slight update with your changes and a bugfix.

> #! /bin/bash
> #
> function usage() {
> printf "Usage:\n\t%s devname\n\n" "`basename \"$0\"`"
> printf "'devname' must be a relative path in /dev/ to the
> desired block device.\n"
> exit 1
> }
>
> # Verify the supplied name is a device
> test -b "/dev/$1" || usage
>
> # Convert path separators and spaces into dashes
> outfile="`echo \"$1\" |sed -r -e 's:[ /]+:-:g'`"
>
> # Create a side-stream for computing the MD5 of the data read
> fifo=`mktemp -u`
> mkfifo $fifo || exit
> md5sum -b <$fifo >$2/$outfile.md5 &
>
> # Read the device and compress it
> dd if="/dev/$1" bs=1M | tee 2>$fifo | gzip >$2/$outfile.gz
>
> # Wait for the background task to close
> wait
>
> I was a little concerned as I didn't see the hdd drive LED light up
> for my attempt even though the file was growing nicely on my CIFS
> share. I'm dumping it on the 5x2TB disks (JBOD) in my windows 7 box
> as my Thecus is not a happy chappy and I need time to mount the DOM
> module in another machine to find out why.

If you're willing to dismantle the thecus, directly connecting its drives to your crippled system will make things go much faster. You've got four empty motherboard SATA ports. You just have to watch out for the power load.

Phil

--------------060200030107030205010800
Content-Type: application/x-sh;
name="block2gz.sh"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="block2gz.sh"

#! /bin/bash
#
function usage() {
printf "Usage:\n\t%s devname foldername\n\n" "`basename \"$0\"`"
printf "'devname' must be a relative path in /dev/ to the desired block device.\n"
printf "'foldername' is the directory that will receive the image and md5 files.\n"
exit 1
}

# Verify the supplied name is a device
test -b "/dev/$1" || usage

# Verify the supplied folder is a directory
test -d "$2" || usage

# Convert path separators and spaces into dashes
outfile="`echo \"$1\" |sed -r -e 's:[ /]+:-:g'`"

# Create a side-stream for computing the MD5 of the data read
fifo=`mktemp -u`
mkfifo $fifo || exit
md5sum -b <$fifo >$2/$outfile.md5 &

# Read the device and compress it
dd if="/dev/$1" bs=1M | tee 2>$fifo | gzip >$2/$outfile.gz

# Wait for the background task to close
wait
rm -f "$fifo"

--------------060200030107030205010800--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

Phil, I think I need to connect them to the crippled machine directly,
I was just trying to leave it alone, I know that doesn't make much
sense but I'm more familiar with windows than Linux. The cards that I
have are all connected at gigabit and bonded. That machine has two
ports, so does the other. I'm running two asus p6ts (1 deluxe and 1
se) with i7 920's

I cant understand why it's so slow though I have seen some reports
that the supermicro card is quite slow.
Simon
Ps thanks for the bugfixes and updates
Simon

On 17 Feb 2011, at 13:48, Phil Turmel wrote:

> On 02/17/2011 08:26 AM, Simon Mcnair wrote:
>> Phil,
>> After a couple of attempts I realised that all I needed to do was
>> ./backup.sh sdi rather than specifying ./backup.sh /dev/sdi schoolboy
>> error huh, rtfm.
>>
>> I tried modifying the code, so I could kick them all off at once, but
>> I wanted to check that this would work from a multithreaded/sane
>> perspective (yes I know it's a bit of IO, but iotop seems to infer I'm
>> only getting 10M/s throughput from the disk anyway).
>
> That sounds suspiciously like a 100 meg ethernet bottleneck, so parallel operation won't help. If so, this'll take a very long time.
>
>> is this code kosha ?
>
> Looks pretty good. I've attached a slight update with your changes and a bugfix.
>
>> #! /bin/bash
>> #
>> function usage() {
>> printf "Usage:\n\t%s devname\n\n" "`basename \"$0\"`"
>> printf "'devname' must be a relative path in /dev/ to the
>> desired block device.\n"
>> exit 1
>> }
>>
>> # Verify the supplied name is a device
>> test -b "/dev/$1" || usage
>>
>> # Convert path separators and spaces into dashes
>> outfile="`echo \"$1\" |sed -r -e 's:[ /]+:-:g'`"
>>
>> # Create a side-stream for computing the MD5 of the data read
>> fifo=`mktemp -u`
>> mkfifo $fifo || exit
>> md5sum -b <$fifo >$2/$outfile.md5 &
>>
>> # Read the device and compress it
>> dd if="/dev/$1" bs=1M | tee 2>$fifo | gzip >$2/$outfile.gz
>>
>> # Wait for the background task to close
>> wait
>>
>> I was a little concerned as I didn't see the hdd drive LED light up
>> for my attempt even though the file was growing nicely on my CIFS
>> share. I'm dumping it on the 5x2TB disks (JBOD) in my windows 7 box
>> as my Thecus is not a happy chappy and I need time to mount the DOM
>> module in another machine to find out why.
>
> If you're willing to dismantle the thecus, directly connecting its drives to your crippled system will make things go much faster. You've got four empty motherboard SATA ports. You just have to watch out for the power load.
>
> Phil
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

19390 root 17.49 M/s 0 B/s 0.00 % 21.36 % dd if /dev/sde b=
s 1M
19333 root 16.52 M/s 0 B/s 0.00 % 17.07 % dd if /dev/sdd b=
s 1M
19503 root 15.79 M/s 0 B/s 0.00 % 13.91 % dd if /dev/sdf b=
s 1M
18896 root 0 B/s 16.10 M/s 0.00 % 0.00 % ntfs-3g
/dev/sdb1 /media/ntfs3g/sdb
18909 root 0 B/s 13.49 M/s 0.00 % 0.00 % ntfs-3g
/dev/sdc1 /media/ntfs3g/sdc
18920 root 0 B/s 15.73 M/s 0.00 % 0.00 % ntfs-3g
/dev/sdn1 /media/ntfs3g/sdn

this will certainly be quicker :-)

On 17 February 2011 13:56, Simon Mcnair wrote:
> Phil, I think I need to connect them to the crippled machine directly=
,
> I was just trying to leave it alone, I know that doesn't make much
> sense but I'm more familiar with windows than Linux. The cards that I
> have are all connected at gigabit and bonded. That machine has two
> ports, so does the other. I'm running two asus p6ts (1 deluxe and 1
> se) with i7 920's
>
> I cant understand why it's so slow though I have seen some reports
> that the supermicro card is quite slow.
> Simon
> Ps thanks for the bugfixes and updates
> Simon
>
> On 17 Feb 2011, at 13:48, Phil Turmel wrote:
>
>> On 02/17/2011 08:26 AM, Simon Mcnair wrote:
>>> Phil,
>>> After a couple of attempts I realised that all I needed to do was
>>> ./backup.sh sdi rather than specifying ./backup.sh /dev/sdi =A0scho=
olboy
>>> error huh, rtfm.
>>>
>>> I tried modifying the code, so I could kick them all off at once, b=
ut
>>> I wanted to check that this would work from a multithreaded/sane
>>> perspective (yes I know it's a bit of IO, but iotop seems to infer =
I'm
>>> only getting 10M/s throughput from the disk anyway).
>>
>> That sounds suspiciously like a 100 meg ethernet bottleneck, so para=
llel operation won't help. =A0If so, this'll take a very long time.
>>
>>> is this code kosha ?
>>
>> Looks pretty good. =A0I've attached a slight update with your change=
s and a bugfix.
>>
>>> #! /bin/bash
>>> #
>>> function usage() {
>>> =A0 =A0 =A0 printf "Usage:\n\t%s devname\n\n" "`basename \"$0\"`"
>>> =A0 =A0 =A0 printf "'devname' must be a relative path in /dev/ to t=
he
>>> desired block device.\n"
>>> =A0 =A0 =A0 exit 1
>>> }
>>>
>>> # Verify the supplied name is a device
>>> test -b "/dev/$1" || usage
>>>
>>> # Convert path separators and spaces into dashes
>>> outfile=3D"`echo \"$1\" |sed -r -e 's:[ /]+:-:g'`"
>>>
>>> # Create a side-stream for computing the MD5 of the data read
>>> fifo=3D`mktemp -u`
>>> mkfifo $fifo || exit
>>> md5sum -b <$fifo >$2/$outfile.md5 &
>>>
>>> # Read the device and compress it
>>> dd if=3D"/dev/$1" bs=3D1M | tee 2>$fifo | gzip >$2/$outfile.gz
>>>
>>> # Wait for the background task to close
>>> wait
>>>
>>> I was a little concerned as I didn't see the hdd drive LED light up
>>> for my attempt even though the file was growing nicely on my CIFS
>>> share. =A0I'm dumping it on the 5x2TB disks (JBOD) in my windows 7 =
box
>>> as my Thecus is not a happy chappy and I need time to mount the DOM
>>> module in another machine to find out why.
>>
>> If you're willing to dismantle the thecus, directly connecting its d=
rives to your crippled system will make things go much faster. =A0You'v=
e got four empty motherboard SATA ports. =A0You just have to watch out =
for the power load.
>>
>> Phil
>>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

proxmox:~# dpkg -S blkid
libblkid1: /lib/libblkid.so.1.0
libblkid1: /lib/libblkid.so.1
libblkid1: /usr/share/doc/libblkid1/changelog.Debian.gz
libblkid1: /usr/share/doc/libblkid1/copyright
libblkid1: /usr/share/doc/libblkid1
e2fsprogs-dbg: /usr/lib/debug/sbin/blkid

proxmox:~# apt-get install e2fsprogs-dbg
Reading package lists... Done
Building dependency tree
Reading state information... Done
e2fsprogs-dbg is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.


I still can't blkid installed, even though I have e2fsprogs-dbg install=
ed...

On 15 February 2011 19:45, Roman Mamedov wrote:
> On Tue, 15 Feb 2011 14:37:55 -0500
> Phil Turmel wrote:
>
>> > For some reason blkid doesn't exist on my machine even though I ha=
ve
>> > e2fs-utils installed. =A0V weird.
>
> Weird is that you think it is somehow a "e2fs-utils" program.
>
> dpkg -S blkid
> ...
> util-linux: /sbin/blkid
> ...
>
>>
>> FWIW, on my VPS running Ubuntu Server 10.10, the blkid executable is=
part of
>> util-linux, and installed in /sbin. =A0You might need to re-install =
it.
>
> --
> With respect,
> Roman
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

--Sig_/_+hBX8RuWkGbV/G3w3j_Tac
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

On Thu, 17 Feb 2011 15:10:32 +0000
Simon Mcnair wrote:

> proxmox:~# dpkg -S blkid
> libblkid1: /lib/libblkid.so.1.0
> libblkid1: /lib/libblkid.so.1
> libblkid1: /usr/share/doc/libblkid1/changelog.Debian.gz
> libblkid1: /usr/share/doc/libblkid1/copyright
> libblkid1: /usr/share/doc/libblkid1
> e2fsprogs-dbg: /usr/lib/debug/sbin/blkid
>=20
> proxmox:~# apt-get install e2fsprogs-dbg
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> e2fsprogs-dbg is already the newest version.
> 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
>=20
>=20
> I still can't blkid installed, even though I have e2fsprogs-dbg installed=
....

/usr/lib/debug/sbin/ is not in system 'path', so you won't be able to run t=
he
program just by entering 'blkid' into the shell prompt.
Either use the complete file name and path, e.g.=20

# /usr/lib/debug/sbin/blkid --help

or symlink /usr/lib/debug/sbin/blkid to something like /usr/local/sbin/blki=
d.

--=20
With respect,
Roman

--Sig_/_+hBX8RuWkGbV/G3w3j_Tac
Content-Type: application/pgp-signature; name=signature.asc
Content-Disposition: attachment; filename=signature.asc

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAk1dQd0ACgkQTLKSvz+PZwi34ACferMv2HLit5FOWLHVKKBh rIMj
ZFYAnRRdNLho7QsjqbIXtEjwArdb6fi5
=WojE
-----END PGP SIGNATURE-----

--Sig_/_+hBX8RuWkGbV/G3w3j_Tac--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

On 02/17/2011 09:34 AM, Simon Mcnair wrote:
> 19390 root 17.49 M/s 0 B/s 0.00 % 21.36 % dd if /dev/sde bs 1M
> 19333 root 16.52 M/s 0 B/s 0.00 % 17.07 % dd if /dev/sdd bs 1M
> 19503 root 15.79 M/s 0 B/s 0.00 % 13.91 % dd if /dev/sdf bs 1M
> 18896 root 0 B/s 16.10 M/s 0.00 % 0.00 % ntfs-3g
> /dev/sdb1 /media/ntfs3g/sdb
> 18909 root 0 B/s 13.49 M/s 0.00 % 0.00 % ntfs-3g
> /dev/sdc1 /media/ntfs3g/sdc
> 18920 root 0 B/s 15.73 M/s 0.00 % 0.00 % ntfs-3g
> /dev/sdn1 /media/ntfs3g/sdn
>
> this will certainly be quicker :-)

But still a long time...

Let me know.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

Hi Roman,
Sorry to be so persistent on something that is not Linux raid specific,
but I still can't get blkid to run, the message is:

proxmox:~# ls -la /usr/lib/debug/sbin/blkid
-rwx------ 1 root root 19887 2008-10-13 04:54 /usr/lib/debug/sbin/blkid

proxmox:~# /usr/lib/debug/sbin/blkid --help
-bash: /usr/lib/debug/sbin/blkid: cannot execute binary file

If I can't run it directly I'm pretty sure symlinking it will not make
any difference. It's weird, normally when I apt-get things it modifies
the path etc and does everything required to make it work, I would have
thought if I installed a debug version of something it should add the
debug path as well ?

blkid always works on my Ubuntu boxes, I don't know why this is any
different (apart from being a different distribution, it's still a
mature program, package and platform).

:-)

Simon

On 17/02/2011 15:42, Roman Mamedov wrote:
> On Thu, 17 Feb 2011 15:10:32 +0000
> Simon Mcnair wrote:
>
>> proxmox:~# dpkg -S blkid
>> libblkid1: /lib/libblkid.so.1.0
>> libblkid1: /lib/libblkid.so.1
>> libblkid1: /usr/share/doc/libblkid1/changelog.Debian.gz
>> libblkid1: /usr/share/doc/libblkid1/copyright
>> libblkid1: /usr/share/doc/libblkid1
>> e2fsprogs-dbg: /usr/lib/debug/sbin/blkid
>>
>> proxmox:~# apt-get install e2fsprogs-dbg
>> Reading package lists... Done
>> Building dependency tree
>> Reading state information... Done
>> e2fsprogs-dbg is already the newest version.
>> 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
>>
>>
>> I still can't blkid installed, even though I have e2fsprogs-dbg installed...
> /usr/lib/debug/sbin/ is not in system 'path', so you won't be able to run the
> program just by entering 'blkid' into the shell prompt.
> Either use the complete file name and path, e.g.
>
> # /usr/lib/debug/sbin/blkid --help
>
> or symlink /usr/lib/debug/sbin/blkid to something like /usr/local/sbin/blkid.
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

time passes. You are eaten by a Grue. Sheesh this is taking a long ti=
me.

simon@proxmox:~$ top

top - 09:26:19 up 20:36, 11 users, load average: 2.46, 2.49, 2.39
Tasks: 334 total, 9 running, 325 sleeping, 0 stopped, 0 zombie
Cpu(s): 52.8%us, 9.4%sy, 0.0%ni, 35.2%id, 2.5%wa, 0.0%hi, 0.2%si, =
0.0%st
Mem: 12299244k total, 12227584k used, 71660k free, 11042868k buffer=
s
Swap: 11534332k total, 0k used, 11534332k free, 208204k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
18896 root 20 0 12288 1072 460 S 63 0.0 431:16.55 ntfs-3g
18909 root 20 0 12288 1064 460 R 62 0.0 419:17.11 ntfs-3g
18920 root 20 0 12288 1100 492 R 62 0.0 428:17.33 ntfs-3g
9210 root 20 0 4068 520 328 S 54 0.0 661:59.92 gzip
9138 root 20 0 4068 524 328 R 53 0.0 647:29.07 gzip
9247 root 20 0 4068 524 328 S 53 0.0 651:28.36 gzip
25678 root 20 0 4068 524 328 R 52 0.0 439:49.20 gzip
24957 root 20 0 4068 524 328 R 51 0.0 437:44.01 gzip
25792 root 20 0 4068 524 328 S 48 0.0 433:28.14 gzip

This is hardly touching the CPU on the box (in my opinion), any advice
on using renice ? I've never used it before, but now seems like a
good time ?

tia
Simon

On 16 February 2011 19:36, Phil Turmel wrote:
> On 02/16/2011 02:15 PM, Simon McNair wrote:
>> proxmox:/home/simon# vgscan --verbose
>> =A0 =A0 Wiping cache of LVM-capable devices
>> =A0 =A0 Wiping internal VG cache
>> =A0 Reading all physical volumes. =A0This may take a while...
>> =A0 =A0 Finding all volume groups
>> =A0 =A0 Finding volume group "pve"
>> =A0 Found volume group "pve" using metadata type lvm2
>> =A0 =A0 Finding volume group "lvm-raid"
>> =A0 Found volume group "lvm-raid" using metadata type lvm2
>> proxmox:/home/simon#
>> proxmox:/home/simon# lvscan --verbose
>> =A0 =A0 Finding all logical volumes
>> =A0 ACTIVE =A0 =A0 =A0 =A0 =A0 =A0'/dev/pve/swap' [11.00 GB] inherit
>> =A0 ACTIVE =A0 =A0 =A0 =A0 =A0 =A0'/dev/pve/root' [96.00 GB] inherit
>> =A0 ACTIVE =A0 =A0 =A0 =A0 =A0 =A0'/dev/pve/data' [354.26 GB] inheri=
t
>> =A0 inactive =A0 =A0 =A0 =A0 =A0'/dev/lvm-raid/RAID' [8.19 TB] inher=
it
>>
>> proxmox:/home/simon# vgchange -ay
>> =A0 3 logical volume(s) in volume group "pve" now active
>> =A0 1 logical volume(s) in volume group "lvm-raid" now active
>
> Heh. =A0Figures.
>
>> proxmox:/home/simon# fsck.ext4 -n /dev/mapper/lvm-raid-RAID
>
> Actually, I wanted you to try with a capital N. =A0Lower case 'n' is =
similar, but not quite the same.
>
>> e2fsck 1.41.3 (12-Oct-2008)
>> fsck.ext4: No such file or directory while trying to open /dev/mappe=
r/lvm-raid-RAID
>>
>> The superblock could not be read or does not describe a correct ext2
>> filesystem. =A0If the device is valid and it really contains an ext2
>> filesystem (and not swap or ufs or something else), then the superbl=
ock
>> is corrupt, and you might try running e2fsck with an alternate super=
block:
>> =A0 =A0 e2fsck -b 8193
>>
>> proxmox:/home/simon# fsck.ext4 -n /dev/mapper/
>> control =A0 =A0 =A0 =A0 lvm--raid-RAID =A0pve-data =A0 =A0 =A0 =A0pv=
e-root =A0 =A0 =A0 =A0pve-swap
>
> Strange. =A0I guess it does that to distinguish dashes in the VG name=
from dashes between VG and LV names.
>
>> proxmox:/home/simon# fsck.ext4 -n /dev/mapper/lvm--raid-RAID
>> e2fsck 1.41.3 (12-Oct-2008)
>> /dev/mapper/lvm--raid-RAID has unsupported feature(s): FEATURE_I31
>> e2fsck: Get a newer version of e2fsck!
>>
>> my version of e2fsck always worked before ?
>
> v1.41.14 was release 7 weeks ago. =A0But, I suspect there's corruptio=
n in the superblock. =A0Do you still have your disk images tucked away =
somewhere safe?
>
> If so, try:
>
> 1) The '-b' option to e2fsck. =A0We need to experiment with '-n -b of=
fset' to find the alternate superblock. =A0Trying 'offset' =3D to 8193,=
16384, and 32768, per the man-page.
>
> 2) A newer e2fsprogs.
>
> Finally,
> 3) mount -r /dev/lvm-raid/RAID /mnt/whatever
>
> Phil
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

--C7zPtVaVf+AK4Oqc
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Fri Feb 18, 2011 at 09:13:33AM +0000, Simon McNair wrote:

> Hi Roman,
> Sorry to be so persistent on something that is not Linux raid specific,=
=20
> but I still can't get blkid to run, the message is:
>=20
> proxmox:~# ls -la /usr/lib/debug/sbin/blkid
> -rwx------ 1 root root 19887 2008-10-13 04:54 /usr/lib/debug/sbin/blkid
>=20
> proxmox:~# /usr/lib/debug/sbin/blkid --help
> -bash: /usr/lib/debug/sbin/blkid: cannot execute binary file
>=20
> If I can't run it directly I'm pretty sure symlinking it will not make=20
> any difference. It's weird, normally when I apt-get things it modifies=
=20
> the path etc and does everything required to make it work, I would have=
=20
> thought if I installed a debug version of something it should add the=20
> debug path as well ?
>=20
I suspect that those are just the debug symbols - they're split off from
the binary so they can be automatically pulled in when needed, but don't
bloat the application itself. From the package naming, the actual blkid
binary should be in the e2fsprogs package - do you have that installed?

Cheers,
Robin
--=20
___ =20
( ' } | Robin Hill |
/ / ) | Little Jim says .... |
// !! | "He fallen in de water !!" |

--C7zPtVaVf+AK4Oqc
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.16 (GNU/Linux)

iEYEARECAAYFAk1ePfkACgkQShxCyD40xBKB0gCg4FWbYa1tLrwhvWwc6vCu nW1O
UvsAnj9K7vBAfri062hPqIKu7j1t2zAG
=Rwwt
-----END PGP SIGNATURE-----

--C7zPtVaVf+AK4Oqc--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

Hi Robin,

proxmox:~# apt-get install e2fsprogs
Reading package lists... Done
Building dependency tree
Reading state information... Done
e2fsprogs is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.

Yeah, It's installed :-)

Simon

On 18 February 2011 09:38, Robin Hill wrote:
> On Fri Feb 18, 2011 at 09:13:33AM +0000, Simon McNair wrote:
>
>> Hi Roman,
>> Sorry to be so persistent on something that is not Linux raid specif=
ic,
>> but I still can't get blkid to run, the message is:
>>
>> proxmox:~# ls -la /usr/lib/debug/sbin/blkid
>> -rwx------ 1 root root 19887 2008-10-13 04:54 /usr/lib/debug/sbin/bl=
kid
>>
>> proxmox:~# /usr/lib/debug/sbin/blkid --help
>> -bash: /usr/lib/debug/sbin/blkid: cannot execute binary file
>>
>> If I can't run it directly I'm pretty sure symlinking it will not ma=
ke
>> any difference. =A0It's weird, normally when I apt-get things it mod=
ifies
>> the path etc and does everything required to make it work, I would h=
ave
>> thought if I installed a debug version of something it should add th=
e
>> debug path as well ?
>>
> I suspect that those are just the debug symbols - they're split off f=
rom
> the binary so they can be automatically pulled in when needed, but do=
n't
> bloat the application itself. =A0From the package naming, the actual =
blkid
> binary should be in the e2fsprogs package - do you have that installe=
d?
>
> Cheers,
> =A0 =A0Robin
> --
> =A0 =A0 ___
> =A0 =A0( ' } =A0 =A0 | =A0 =A0 =A0 Robin Hill =A0 =A0 =A0 =A0 obinhill.me.uk> |
> =A0 / / ) =A0 =A0 =A0| Little Jim says .... =A0 =A0 =A0 =A0 =A0 =A0 =A0=
=A0 =A0 =A0 =A0 =A0 =A0 =A0|
> =A0// !! =A0 =A0 =A0 | =A0 =A0 =A0"He fallen in de water !!" =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 |
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

On 02/18/2011 04:31 AM, Simon Mcnair wrote:
> time passes. You are eaten by a Grue. Sheesh this is taking a long time.
>
> simon@proxmox:~$ top
>
> top - 09:26:19 up 20:36, 11 users, load average: 2.46, 2.49, 2.39
> Tasks: 334 total, 9 running, 325 sleeping, 0 stopped, 0 zombie
> Cpu(s): 52.8%us, 9.4%sy, 0.0%ni, 35.2%id, 2.5%wa, 0.0%hi, 0.2%si, 0.0%st
> Mem: 12299244k total, 12227584k used, 71660k free, 11042868k buffers
> Swap: 11534332k total, 0k used, 11534332k free, 208204k cached
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 18896 root 20 0 12288 1072 460 S 63 0.0 431:16.55 ntfs-3g
> 18909 root 20 0 12288 1064 460 R 62 0.0 419:17.11 ntfs-3g
> 18920 root 20 0 12288 1100 492 R 62 0.0 428:17.33 ntfs-3g
> 9210 root 20 0 4068 520 328 S 54 0.0 661:59.92 gzip
> 9138 root 20 0 4068 524 328 R 53 0.0 647:29.07 gzip
> 9247 root 20 0 4068 524 328 S 53 0.0 651:28.36 gzip
> 25678 root 20 0 4068 524 328 R 52 0.0 439:49.20 gzip
> 24957 root 20 0 4068 524 328 R 51 0.0 437:44.01 gzip
> 25792 root 20 0 4068 524 328 S 48 0.0 433:28.14 gzip
>
> This is hardly touching the CPU on the box (in my opinion), any advice
> on using renice ? I've never used it before, but now seems like a
> good time ?

Probably wouldn't help.

You have a total usage of 65%. I suspect that each of the processors running gzip are nearly pegged, and everything else is loafing along. Hit '1' in top to show the CPU usage per-cpu. Single-threaded gzip is holding you back.

If you had enough space for saving the partitions uncompressed, it would be much faster. The PCIe x4 interface on the SuperMicro has a theoretical performance of 1GByte/s, which would be ~ 125MB/s per drive. From what I've read, that card actually delivers ~ 75MB/s per drive when they're all busy.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

pci express x4 =3D 2.5*4gbps =3D 10gbps not?!

2011/2/18 Phil Turmel :
> On 02/18/2011 04:31 AM, Simon Mcnair wrote:
>> time passes. =A0You are eaten by a Grue. =A0Sheesh this is taking a =
long time.
>>
>> simon@proxmox:~$ top
>>
>> top - 09:26:19 up 20:36, 11 users, =A0load average: 2.46, 2.49, 2.39
>> Tasks: 334 total, =A0 9 running, 325 sleeping, =A0 0 stopped, =A0 0 =
zombie
>> Cpu(s): 52.8%us, =A09.4%sy, =A00.0%ni, 35.2%id, =A02.5%wa, =A00.0%hi=
, =A00.2%si, =A00.0%st
>> Mem: =A012299244k total, 12227584k used, =A0 =A071660k free, 1104286=
8k buffers
>> Swap: 11534332k total, =A0 =A0 =A0 =A00k used, 11534332k free, =A0 2=
08204k cached
>>
>> =A0 PID USER =A0 =A0 =A0PR =A0NI =A0VIRT =A0RES =A0SHR S %CPU %MEM =A0=
=A0TIME+ =A0COMMAND
>> 18896 root =A0 =A0 =A020 =A0 0 12288 1072 =A0460 S =A0 63 =A00.0 431=
:16.55 ntfs-3g
>> 18909 root =A0 =A0 =A020 =A0 0 12288 1064 =A0460 R =A0 62 =A00.0 419=
:17.11 ntfs-3g
>> 18920 root =A0 =A0 =A020 =A0 0 12288 1100 =A0492 R =A0 62 =A00.0 428=
:17.33 ntfs-3g
>> =A09210 root =A0 =A0 =A020 =A0 0 =A04068 =A0520 =A0328 S =A0 54 =A00=
0 661:59.92 gzip
>> =A09138 root =A0 =A0 =A020 =A0 0 =A04068 =A0524 =A0328 R =A0 53 =A00=
0 647:29.07 gzip
>> =A09247 root =A0 =A0 =A020 =A0 0 =A04068 =A0524 =A0328 S =A0 53 =A00=
0 651:28.36 gzip
>> 25678 root =A0 =A0 =A020 =A0 0 =A04068 =A0524 =A0328 R =A0 52 =A00.0=
439:49.20 gzip
>> 24957 root =A0 =A0 =A020 =A0 0 =A04068 =A0524 =A0328 R =A0 51 =A00.0=
437:44.01 gzip
>> 25792 root =A0 =A0 =A020 =A0 0 =A04068 =A0524 =A0328 S =A0 48 =A00.0=
433:28.14 gzip
>>
>> This is hardly touching the CPU on the box (in my opinion), any advi=
ce
>> on using renice ? =A0I've never used it before, but now seems like a
>> good time ?
>
> Probably wouldn't help.
>
> You have a total usage of 65%. =A0I suspect that each of the processo=
rs running gzip are nearly pegged, and everything else is loafing along=
=A0Hit '1' in top to show the CPU usage per-cpu. =A0Single-threaded=
gzip is holding you back.
>
> If you had enough space for saving the partitions uncompressed, it wo=
uld be much faster. =A0The PCIe x4 interface on the SuperMicro has a th=
eoretical performance of 1GByte/s, which would be ~ 125MB/s per drive. =
=A0From what I've read, that card actually delivers ~ 75MB/s per drive =
when they're all busy.
>
> Phil
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid"=
in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
>



--=20
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

On 02/18/2011 08:21 AM, Roberto Spadim wrote:
> pci express x4 = 2.5*4gbps = 10gbps not?!

Subtract transfer overhead (Framing, ECC, addressing, etc) and it takes >8 bits to transfer 8 bits. I haven't checked the spec, but it looks like just framing and ECC is +25%.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

top - 13:28:45 up 1 day, 39 min, 12 users, load average: 2.65, 2.49, 2=
37
Tasks: 340 total, 7 running, 333 sleeping, 0 stopped, 0 zombie
Cpu0 : 47.9%us, 15.0%sy, 0.0%ni, 13.7%id, 21.5%wa, 0.0%hi, 2.0%si, =
0.0%st
Cpu1 : 54.8%us, 8.7%sy, 0.0%ni, 33.3%id, 3.2%wa, 0.0%hi, 0.0%si, =
0.0%st
Cpu2 : 46.5%us, 13.7%sy, 0.0%ni, 37.9%id, 1.9%wa, 0.0%hi, 0.0%si, =
0.0%st
Cpu3 : 49.5%us, 9.5%sy, 0.0%ni, 37.6%id, 3.4%wa, 0.0%hi, 0.0%si, =
0.0%st
Cpu4 : 48.7%us, 8.2%sy, 0.0%ni, 17.3%id, 25.5%wa, 0.0%hi, 0.3%si, =
0.0%st
Cpu5 : 50.3%us, 7.2%sy, 0.0%ni, 41.8%id, 0.6%wa, 0.0%hi, 0.0%si, =
0.0%st
Cpu6 : 53.5%us, 6.6%sy, 0.0%ni, 38.4%id, 1.6%wa, 0.0%hi, 0.0%si, =
0.0%st
Cpu7 : 52.0%us, 6.2%sy, 0.0%ni, 41.2%id, 0.6%wa, 0.0%hi, 0.0%si, =
0.0%st
Mem: 12299244k total, 12227064k used, 72180k free, 10754600k buffer=
s
Swap: 11534332k total, 0k used, 11534332k free, 175916k cached


On 18 February 2011 13:16, Phil Turmel wrote:
> On 02/18/2011 04:31 AM, Simon Mcnair wrote:
>> time passes. =A0You are eaten by a Grue. =A0Sheesh this is taking a =
long time.
>>
>> simon@proxmox:~$ top
>>
>> top - 09:26:19 up 20:36, 11 users, =A0load average: 2.46, 2.49, 2.39
>> Tasks: 334 total, =A0 9 running, 325 sleeping, =A0 0 stopped, =A0 0 =
zombie
>> Cpu(s): 52.8%us, =A09.4%sy, =A00.0%ni, 35.2%id, =A02.5%wa, =A00.0%hi=
, =A00.2%si, =A00.0%st
>> Mem: =A012299244k total, 12227584k used, =A0 =A071660k free, 1104286=
8k buffers
>> Swap: 11534332k total, =A0 =A0 =A0 =A00k used, 11534332k free, =A0 2=
08204k cached
>>
>> =A0 PID USER =A0 =A0 =A0PR =A0NI =A0VIRT =A0RES =A0SHR S %CPU %MEM =A0=
=A0TIME+ =A0COMMAND
>> 18896 root =A0 =A0 =A020 =A0 0 12288 1072 =A0460 S =A0 63 =A00.0 431=
:16.55 ntfs-3g
>> 18909 root =A0 =A0 =A020 =A0 0 12288 1064 =A0460 R =A0 62 =A00.0 419=
:17.11 ntfs-3g
>> 18920 root =A0 =A0 =A020 =A0 0 12288 1100 =A0492 R =A0 62 =A00.0 428=
:17.33 ntfs-3g
>> =A09210 root =A0 =A0 =A020 =A0 0 =A04068 =A0520 =A0328 S =A0 54 =A00=
0 661:59.92 gzip
>> =A09138 root =A0 =A0 =A020 =A0 0 =A04068 =A0524 =A0328 R =A0 53 =A00=
0 647:29.07 gzip
>> =A09247 root =A0 =A0 =A020 =A0 0 =A04068 =A0524 =A0328 S =A0 53 =A00=
0 651:28.36 gzip
>> 25678 root =A0 =A0 =A020 =A0 0 =A04068 =A0524 =A0328 R =A0 52 =A00.0=
439:49.20 gzip
>> 24957 root =A0 =A0 =A020 =A0 0 =A04068 =A0524 =A0328 R =A0 51 =A00.0=
437:44.01 gzip
>> 25792 root =A0 =A0 =A020 =A0 0 =A04068 =A0524 =A0328 S =A0 48 =A00.0=
433:28.14 gzip
>>
>> This is hardly touching the CPU on the box (in my opinion), any advi=
ce
>> on using renice ? =A0I've never used it before, but now seems like a
>> good time ?
>
> Probably wouldn't help.
>
> You have a total usage of 65%. =A0I suspect that each of the processo=
rs running gzip are nearly pegged, and everything else is loafing along=
=A0Hit '1' in top to show the CPU usage per-cpu. =A0Single-threaded=
gzip is holding you back.
>
> If you had enough space for saving the partitions uncompressed, it wo=
uld be much faster. =A0The PCIe x4 interface on the SuperMicro has a th=
eoretical performance of 1GByte/s, which would be ~ 125MB/s per drive. =
=A0From what I've read, that card actually delivers ~ 75MB/s per drive =
when they're all busy.
>
> Phil
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

On 02/18/2011 08:29 AM, Simon Mcnair wrote:
> top - 13:28:45 up 1 day, 39 min, 12 users, load average: 2.65, 2.49, 2.37
> Tasks: 340 total, 7 running, 333 sleeping, 0 stopped, 0 zombie
> Cpu0 : 47.9%us, 15.0%sy, 0.0%ni, 13.7%id, 21.5%wa, 0.0%hi, 2.0%si, 0.0%st
> Cpu1 : 54.8%us, 8.7%sy, 0.0%ni, 33.3%id, 3.2%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu2 : 46.5%us, 13.7%sy, 0.0%ni, 37.9%id, 1.9%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu3 : 49.5%us, 9.5%sy, 0.0%ni, 37.6%id, 3.4%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu4 : 48.7%us, 8.2%sy, 0.0%ni, 17.3%id, 25.5%wa, 0.0%hi, 0.3%si, 0.0%st
> Cpu5 : 50.3%us, 7.2%sy, 0.0%ni, 41.8%id, 0.6%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu6 : 53.5%us, 6.6%sy, 0.0%ni, 38.4%id, 1.6%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu7 : 52.0%us, 6.2%sy, 0.0%ni, 41.2%id, 0.6%wa, 0.0%hi, 0.0%si, 0.0%st
> Mem: 12299244k total, 12227064k used, 72180k free, 10754600k buffers
> Swap: 11534332k total, 0k used, 11534332k free, 175916k cached

Hmmm. Not what I expected at all.

I'd be very interested in some performance tests of the Supermicro ports vs. motherboard ports on that system.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

Phil,
Tell me what you want me to do and I'm game for it. When I bought it
the reviews were not all that promising
(http://www.google.co.uk/products/catalog?hl=&q=AOC-SASLP-MV 8&cid=15596376504116122379&os=reviews)
but I figure getting up and running is more important than sata speed at
the moment.

cheers
Simon

On 18/02/2011 13:34, Phil Turmel wrote:
> On 02/18/2011 08:29 AM, Simon Mcnair wrote:
>> top - 13:28:45 up 1 day, 39 min, 12 users, load average: 2.65, 2.49, 2.37
>> Tasks: 340 total, 7 running, 333 sleeping, 0 stopped, 0 zombie
>> Cpu0 : 47.9%us, 15.0%sy, 0.0%ni, 13.7%id, 21.5%wa, 0.0%hi, 2.0%si, 0.0%st
>> Cpu1 : 54.8%us, 8.7%sy, 0.0%ni, 33.3%id, 3.2%wa, 0.0%hi, 0.0%si, 0.0%st
>> Cpu2 : 46.5%us, 13.7%sy, 0.0%ni, 37.9%id, 1.9%wa, 0.0%hi, 0.0%si, 0.0%st
>> Cpu3 : 49.5%us, 9.5%sy, 0.0%ni, 37.6%id, 3.4%wa, 0.0%hi, 0.0%si, 0.0%st
>> Cpu4 : 48.7%us, 8.2%sy, 0.0%ni, 17.3%id, 25.5%wa, 0.0%hi, 0.3%si, 0.0%st
>> Cpu5 : 50.3%us, 7.2%sy, 0.0%ni, 41.8%id, 0.6%wa, 0.0%hi, 0.0%si, 0.0%st
>> Cpu6 : 53.5%us, 6.6%sy, 0.0%ni, 38.4%id, 1.6%wa, 0.0%hi, 0.0%si, 0.0%st
>> Cpu7 : 52.0%us, 6.2%sy, 0.0%ni, 41.2%id, 0.6%wa, 0.0%hi, 0.0%si, 0.0%st
>> Mem: 12299244k total, 12227064k used, 72180k free, 10754600k buffers
>> Swap: 11534332k total, 0k used, 11534332k free, 175916k cached
> Hmmm. Not what I expected at all.
>
> I'd be very interested in some performance tests of the Supermicro ports vs. motherboard ports on that system.
>
> Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

On 02/18/2011 09:12 AM, Simon McNair wrote:
> Phil,
> Tell me what you want me to do and I'm game for it. When I bought it the reviews were not all that promising (http://www.google.co.uk/products/catalog?hl=&q=AOC-SASLP-MV 8&cid=15596376504116122379&os=reviews) but I figure getting up and running is more important than sata speed at the moment.

I perused the cards reviewed here:

http://blog.zorinaq.com/?e=10

I was aiming for the ~$100 mark, for personal use. I'd have chosen the one of LSI 1068E units, but the cheaper ones are reversed boards (parts on opposite side of normal). The Marvell 6480 was apparently the next best deal. I may have to rethink that.

In any case, some benchmarking options:

1) Spot check with "hdparm -t /dev/sd?". Not considered very accurate. It's read-only, though, so can run in parallel with other device activity (w/ corresponding reduced accuracy).

2) Run a packaged set of benchmarks:
http://www.coker.com.au/bonnie++/
http://www.iozone.org/
http://dbench.samba.org/

3) Script a series of basic tests of various tasks:
https://raid.wiki.kernel.org/index.php/Home_grown_testing_me thods

Most of these tests require a writable filesystem on the target device to work in.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

dd if=3D/dev/XXX of=3D/dev/null iflag=3Ddirect
on other terminal run
iostat -d 1 -k


2011/2/18 Phil Turmel :
> On 02/18/2011 09:12 AM, Simon McNair wrote:
>> Phil,
>> Tell me what you want me to do and I'm game for it. =A0When I bought=
it the reviews were not all that promising (http://www.google.co.uk/pr=
oducts/catalog?hl=3D&q=3DAOC-SASLP-MV8&cid=3D155963765041161 22379&os=3D=
reviews) but I figure getting up and running is more important than sat=
a speed at the moment.
>
> I perused the cards reviewed here:
>
> http://blog.zorinaq.com/?e=3D10
>
> I was aiming for the ~$100 mark, for personal use. =A0I'd have chosen=
the one of LSI 1068E units, but the cheaper ones are reversed boards (=
parts on opposite side of normal). =A0The Marvell 6480 was apparently t=
he next best deal. =A0I may have to rethink that.
>
> In any case, some benchmarking options:
>
> 1) Spot check with "hdparm -t /dev/sd?". =A0Not considered very accur=
ate. =A0It's read-only, though, so can run in parallel with other devic=
e activity (w/ corresponding reduced accuracy).
>
> 2) Run a packaged set of benchmarks:
> =A0 =A0 =A0 =A0http://www.coker.com.au/bonnie++/
> =A0 =A0 =A0 =A0http://www.iozone.org/
> =A0 =A0 =A0 =A0http://dbench.samba.org/
>
> 3) Script a series of basic tests of various tasks:
> =A0 =A0 =A0 =A0https://raid.wiki.kernel.org/index.php/Home_grown_test=
ing_methods
>
> Most of these tests require a writable filesystem on the target devic=
e to work in.
>
> Phil
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid"=
in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
>



--=20
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

Phil,
Sorry for the spamming, but I'm just keeping you informed :-).

proxmox:/home/simon# ./block2gz.sh sdd /media/ntfs3g/sdb
953869+1 records in
953869+1 records out
1000204886016 bytes (1.0 TB) copied, 95384 s, 10.5 MB/s

proxmox:/home/simon# ./block2gz.sh sde /media/ntfs3g/sdn
953869+1 records in
953869+1 records out
1000204886016 bytes (1.0 TB) copied, 106317 s, 9.4 MB/s

proxmox:/home/simon# ./block2gz.sh sdf /media/ntfs3g/sdc
953869+1 records in
953869+1 records out
1000204886016 bytes (1.0 TB) copied, 104711 s, 9.6 MB/s

proxmox:/home/simon# ./block2gz.sh sdg /media/ntfs3g/sdb
953869+1 records in
953869+1 records out
1000204886016 bytes (1.0 TB) copied, 104980 s, 9.5 MB/s

proxmox:/home/simon# ./block2gz.sh sdh /media/ntfs3g/sdn
953869+1 records in
953869+1 records out
1000204886016 bytes (1.0 TB) copied, 98105.1 s, 10.2 MB/s

proxmox:/home/simon# ./block2gz.sh sdi /media/ntfs3g/sdc
953869+1 records in
953869+1 records out
1000204886016 bytes (1.0 TB) copied, 98291 s, 10.2 MB/s
proxmox:/home/simon#

Have just started sdj,sdk,sdl,sdm. I was thinking of renaming the
gz's with the serial number as this would seem to me as more useful,
is this a good idea ?

These numbers are not at all reflective of the drive or controller
speed as I took the lazy route and was writing 2 sets of images at the
same time to the same drive and in addition gzip was also running
(although not really stressing the system from what I could tell).

I suspect also that the drives may be pretty fragmented as the space
was not allocated at the start of the write, so that may have had some
impact too.

Simon

On 17 February 2011 16:54, Phil Turmel wrote:
> On 02/17/2011 09:34 AM, Simon Mcnair wrote:
>> 19390 root =A0 =A0 =A0 17.49 M/s =A0 =A0 =A0 0 B/s =A00.00 % 21.36 %=
dd if /dev/sde bs 1M
>> 19333 root =A0 =A0 =A0 16.52 M/s =A0 =A0 =A0 0 B/s =A00.00 % 17.07 %=
dd if /dev/sdd bs 1M
>> 19503 root =A0 =A0 =A0 15.79 M/s =A0 =A0 =A0 0 B/s =A00.00 % 13.91 %=
dd if /dev/sdf bs 1M
>> 18896 root =A0 =A0 =A0 =A0 =A0 0 B/s =A0 16.10 M/s =A00.00 % =A00.00=
% ntfs-3g
>> /dev/sdb1 /media/ntfs3g/sdb
>> 18909 root =A0 =A0 =A0 =A0 =A0 0 B/s =A0 13.49 M/s =A00.00 % =A00.00=
% ntfs-3g
>> /dev/sdc1 /media/ntfs3g/sdc
>> 18920 root =A0 =A0 =A0 =A0 =A0 0 B/s =A0 15.73 M/s =A00.00 % =A00.00=
% ntfs-3g
>> /dev/sdn1 /media/ntfs3g/sdn
>>
>> this will certainly be quicker :-)
>
> But still a long time...
>
> Let me know.
>
> Phil
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

Hot plug isn't a problem, I have two supermicro CSE-M35T1 's.

It's wierd though that the drive order has changed so much because I
plugged the sata ports in to the board in the same order as the 3ware
card. i.e.

port
0->0
1->1
2->2
3->3
4->4
5->5
6->6
7->7
8->m/b 7
9->m/b 8

maybe it's the way linux/udev allocated them...

one last thing: Is echo 1 > /sys/block/sda/device/delete the correct
way to stop a AHCI SATA device ?

cheers
Simon

On 16 February 2011 18:14, Phil Turmel wrote:
> On 02/16/2011 12:49 PM, Simon McNair wrote:
>> Hi Phil,
>> =A0A couple of questions please.
>
> [trim /]
>
>>>> Simon
>>> I don't know why the serial numbers are formatted differently, but =
we can still tell them apart (the eight characters starting with "PAG")=

>>>
>>> So, our device order in your new setup is: [ihgfmlkjbc], where /dev=
/sdi corresponds to the original report's /dev/sdd, which matches the s=
ig grep in your other note.
>>>
>>> Another note: =A0The controller for sd[abc] is still showing ata_pi=
ix as its controller. =A0That means you cannot hot-plug those ports. =A0=
If you change your BIOS to AHCI mode instead of "Compatibility" or "Emu=
lation", the full-featured ahci driver will run those ports. =A0Not urg=
ent, but I highly recommend it.
>>>
>> Will do that now, before I forget
>
> Hot-pluggability with suitable trays is very handy! :)
>
> [trim /]
>
>>>> Error: The backup GPT table is not at the end of the disk, as it s=
hould be. =A0This might mean that another operating
>>>> system believes the disk is smaller. =A0Fix, by moving the backup =
to the end (and removing the old backup)?
>>>> Fix/Cancel? c
>>> The 3ware controller must have reserved some space at the end of ea=
ch drive for its own use. =A0Didn't know it'd do that. =A0You will have=
to fix that.
>>>
>>> [trim /]
>>>
>> Do you have any suggestions on how I can fix that ? =A0I don't have =
a clue
>
> Just do 'parted /dev/sd?' and on the ones it offers to fix, say yes. =
=A0Then request 'unit s' and 'print' to verify that it is correct.
>
> [trim /]
>
>> when I was trying to figure out the command for this using 'man part=
ed' I came across this:
>> " rescue start end
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Rescue =A0a =A0lost =A0pa=
rtition =A0that was located somewhere between start and end. =A0If a pa=
rtition is
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0found, parted will ask if=
you want to create an entry for it in the partition table."
>> Is it worth trying ?
>
> Nah. =A0That's for when you don't know exactly where the partition is=
=A0We know.
>
>> I originally created the partitions like so:
>> parted -s /dev/sdb rm 1
>> parted -s /dev/sdb mklabel gpt
>> parted -s --align optimal /dev/sdb mkpart primary ext4 .512 100%
>> parted -s /dev/sdb set 1 raid on
>> parted -s /dev/sdb align-check optimal 1
>>
>> so to recreate the above I would do:
>> parted -s /dev/sdb mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdc mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdf mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdg mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdh mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdi mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdj mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdk mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdl mkpart primary ext4 2048s 1953101823s
>> parted -s /dev/sdm mkpart primary ext4 2048s 1953101823s
>
> Only recreate the partition tables where you have to, i.e., the 'Fix'=
option above didn't work. =A0And don't specify a filesystem.
>
> Probably just /dev/sdh and /dev/sdi. =A0Like so, though:
>
> parted -s /dev/sdh mklabel gpt mkpart primary 2048s 1953101823s set 1=
raid on
> parted -s /dev/sdi mklabel gpt mkpart primary 2048s 1953101823s set 1=
raid on
>
>> I'm guessing the backups that I want to do can wait until any potent=
ial fsck ?
>
> Do an 'fsck -N' first, and if it passes, or has few errors, mount the=
filesystem readonly and grab your backup. =A0Then let fsck have at it =
for real. =A0If anything gets fixed, compare your backup from the read-=
only fs to the fixed fs.
>
> Given your flaky old controller, I expect there'll be *some* problems=

>
>> sorry if the questions are dumb but I'm not sure what I'm doing and =
I'd rather ask more questions than fewer and understand the implication=
s of what I'm doing.
>
> Oh, no. =A0You are right to be paranoid. =A0If anything looks funny, =
stop.
>
> Phil
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

On 18/02/11 11:38, Simon Mcnair wrote:
> Yeah, It's installed :-)

apt-get install --reinstall e2fsprogs
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

Jan,
yay. Thanks very much. That worked. I don't quite understand why
something should need a reinstall to work, but thanks v much.

regards
Simon

On 19/02/2011 11:46, Jan Ceuleers wrote:
> On 18/02/11 11:38, Simon Mcnair wrote:
>> Yeah, It's installed :-)
>
> apt-get install --reinstall e2fsprogs
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

On 02/19/2011 03:43 AM, Simon Mcnair wrote:
> Phil,
> Sorry for the spamming, but I'm just keeping you informed :-).
>
> proxmox:/home/simon# ./block2gz.sh sdd /media/ntfs3g/sdb
> 953869+1 records in
> 953869+1 records out
> 1000204886016 bytes (1.0 TB) copied, 95384 s, 10.5 MB/s

[trim /]

> Have just started sdj,sdk,sdl,sdm. I was thinking of renaming the
> .gz's with the serial number as this would seem to me as more useful,
> is this a good idea ?

Yes. I guess they'll finish some time tomorrow?

> These numbers are not at all reflective of the drive or controller
> speed as I took the lazy route and was writing 2 sets of images at the
> same time to the same drive and in addition gzip was also running
> (although not really stressing the system from what I could tell).

The CPU was busy, but not overloaded. The per-drive data rates were 1/5th to 1/10th what I would have expected. The Asus P6T SE motherboard has a lot of bandwidth. Something's odd.

Can you attach a copy of your dmesg?

> I suspect also that the drives may be pretty fragmented as the space
> was not allocated at the start of the write, so that may have had some
> impact too.

Ntfs-3g can push 70+ MB/s onto my heavily fragged Windows laptop partition with only 50% of a Core2 Duo. I doubt that's it.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

On 02/19/2011 10:36 AM, Simon Mcnair wrote:
> as attached (hopefully).
> regards
> Simon

Nothing obvious to me.

Are you running irqbalance?

cat /proc/interrupts ?
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

On 19/02/11 13:40, Simon McNair wrote:
> yay. Thanks very much. That worked. I don't quite understand why
> something should need a reinstall to work, but thanks v much.

If a package's binaries have been damaged or deleted, on purpose or
otherwise, then --reinstall helps. I have no idea what happened to your
copy of blkid, but --reinstall has brought it back for you.

Jan
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

Phil,
I don't know how to find out if I'm running irqbalance, it's whatever
was in the proxmox iso that I installed the OS from. I ran a ps -aux
| grep irq in case it shows anything of interest:

proxmox:/home/simon# ps -aux | grep irq
Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html
root 3 0.0 0.0 0 0 ? S Feb17 0:22 [ksoftirqd/0]
root 7 0.0 0.0 0 0 ? S Feb17 0:05 [ksoftirqd/1]
root 10 0.0 0.0 0 0 ? S Feb17 0:05 [ksoftirqd/2]
root 13 0.0 0.0 0 0 ? S Feb17 0:05 [ksoftirqd/3]
root 16 0.0 0.0 0 0 ? S Feb17 0:11 [ksoftirqd/4]
root 19 0.0 0.0 0 0 ? S Feb17 0:13 [ksoftirqd/5]
root 22 0.0 0.0 0 0 ? S Feb17 0:06 [ksoftirqd/6]
root 25 0.0 0.0 0 0 ? S Feb17 0:06 [ksoftirqd/7]
root 4024 0.0 0.0 0 0 ? S Feb17 0:00
[kvm-irqfd-clean]

proxmox:/home/simon# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4
CPU5 CPU6 CPU7
0: 66008972 0 0 0 0
0 0 0 IR-IO-APIC-edge timer
1: 265829 0 0 0 0
0 0 0 IR-IO-APIC-edge i8042
8: 1 0 0 0 0
0 0 0 IR-IO-APIC-edge rtc0
9: 0 0 0 0 0
0 0 0 IR-IO-APIC-fasteoi acpi
16: 4432639 0 0 0 0
0 0 0 IR-IO-APIC-fasteoi uhci_hcd:usb3, ahci
17: 124325 0 0 0 0
0 0 0 IR-IO-APIC-fasteoi pata_jmicron, eth1
18: 954710 0 0 0 0
0 0 0 IR-IO-APIC-fasteoi ehci_hcd:usb1,
uhci_hcd:usb8
19: 5994 0 0 0 0
0 0 0 IR-IO-APIC-fasteoi uhci_hcd:usb5,
uhci_hcd:usb7, firewire_ohci
21: 0 0 0 0 0
0 0 0 IR-IO-APIC-fasteoi uhci_hcd:usb4
23: 62 0 0 0 0
0 0 0 IR-IO-APIC-fasteoi ehci_hcd:usb2,
uhci_hcd:usb6
24: 273558 0 0 0 0
0 0 0 IR-IO-APIC-fasteoi nvidia
28: 30234710 0 0 0 0
0 0 0 IR-IO-APIC-fasteoi mvsas
64: 0 0 0 0 0
0 0 0 DMAR_MSI-edge dmar0
65: 0 0 0 0 0
0 0 0 DMAR_MSI-edge dmar1
73: 26838727 0 0 0 0
0 0 0 IR-PCI-MSI-edge eth0
74: 27057631 0 0 0 0
0 0 0 IR-PCI-MSI-edge ahci
75: 247 0 0 0 0
0 0 0 IR-PCI-MSI-edge hda_intel
NMI: 0 0 0 0 0
0 0 0 Non-maskable interrupts
LOC: 23087679 23088282 21638323 20580136 22974769
20856094 20144310 20084801 Local timer interrupts
SPU: 0 0 0 0 0
0 0 0 Spurious interrupts
PMI: 0 0 0 0 0
0 0 0 Performance monitoring interrupts
PND: 0 0 0 0 0
0 0 0 Performance pending work
RES: 15236622 9388860 8532928 7181879 13444866
6005599 4856933 3713707 Rescheduling interrupts
CAL: 3384 5773 5839 5859 4590
5748 5781 5691 Function call interrupts
TLB: 201989 195383 193545 196155 251338
249053 272032 319233 TLB shootdowns
TRM: 0 0 0 0 0
0 0 0 Thermal event interrupts
THR: 0 0 0 0 0
0 0 0 Threshold APIC interrupts
MCE: 0 0 0 0 0
0 0 0 Machine check exceptions
MCP: 830 830 830 830 830
830 830 830 Machine check polls
ERR: 7
MIS: 0

cheers
Simon

On 19 February 2011 16:19, Phil Turmel wrote:
> On 02/19/2011 10:36 AM, Simon Mcnair wrote:
>> as attached (hopefully).
>> regards
>> Simon
>
> Nothing obvious to me.
>
> Are you running irqbalance?
>
> cat /proc/interrupts ?
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

On 02/20/2011 12:03 PM, Simon Mcnair wrote:
> Hi Phil,
>
> Is this fsck (fsck.ext4 -n -b 32768 /dev/mapper/lvm--raid-RAID
>> fsck.txt) as bad as it looks ? :-(

It's bad. Either the original sdd has a lot more corruption than I expected, or the 3ware spread corruption over all the drives.

If the former, failing it out of the array might help. If the latter, your data is likely toast. Some identifiable data is being found, based on the used vs. free block/inode/directory counts in that report. That's good.

I suggest you do "mdadm /dev/md0 --fail /dev/sdi1" and repeat the "fsck -n" as above.

(It'll be noticably slower, as it'll be using parity to reconstruct 1 out every 9 chunks.)

If it the fsck results improve, or stay the same, proceed to "fsck -y", and we'll see.

Wouldn't hurt to run "iostat -xm 5" in another terminal during the fsck to see what kind of performance that array is getting.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

that's not good :-(

have done the --fail and running fsck at the moment. IOstat as below:

avg-cpu: %user %nice %system %iowait %steal %idle
10.96 0.00 4.01 10.98 0.00 74.05

Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s
avgrq-sz avgqu-sz await svctm %util
sda 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00
sda2 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00
sdd 4455.20 0.00 497.80 0.00 19.35 0.00
79.59 1.66 3.34 1.11 55.20
sdd1 4455.20 0.00 497.80 0.00 19.35 0.00
79.59 1.66 3.34 1.11 55.20
sde 4454.20 0.00 498.60 0.00 19.35 0.00
79.47 1.63 3.26 1.13 56.20
sde1 4454.20 0.00 498.60 0.00 19.35 0.00
79.47 1.63 3.26 1.13 56.20
sdf 4311.60 0.00 615.60 0.00 19.33 0.00
64.31 1.22 1.98 0.60 37.20
sdf1 4311.60 0.00 615.60 0.00 19.33 0.00
64.31 1.22 1.98 0.60 37.20
sdg 4262.60 0.00 659.60 0.00 19.35 0.00
60.08 1.83 2.77 0.79 52.20
sdg1 4262.60 0.00 659.60 0.00 19.35 0.00
60.08 1.83 2.77 0.79 52.20
sdh 4242.20 0.00 665.80 0.00 19.36 0.00
59.54 1.67 2.51 0.63 42.00
sdh1 4242.20 0.00 665.80 0.00 19.36 0.00
59.54 1.67 2.51 0.63 42.00
sdi 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00
sdi1 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00
sdj 4382.00 0.00 567.20 0.00 19.34 0.00
69.82 1.35 2.38 0.74 42.20
sdj1 4382.00 0.00 567.20 0.00 19.34 0.00
69.82 1.35 2.38 0.74 42.20
sdk 4341.40 0.00 605.60 0.00 19.34 0.00
65.42 1.71 2.82 0.89 53.80
sdk1 4341.40 0.00 605.60 0.00 19.34 0.00
65.42 1.71 2.82 0.89 53.80
sdl 4368.20 0.00 579.20 0.00 19.33 0.00
68.36 1.78 3.07 0.99 57.60
sdl1 4368.20 0.00 579.20 0.00 19.33 0.00
68.36 1.78 3.07 0.99 57.60
sdm 4351.00 0.00 591.40 0.00 19.32 0.00
66.92 2.17 3.68 1.09 64.60
sdm1 4351.00 0.00 591.40 0.00 19.32 0.00
66.92 2.17 3.68 1.09 64.60
dm-0 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00
dm-1 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00
dm-2 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00
md0 0.00 0.00 24808.00 0.00 96.91 0.00
8.00 0.00 0.00 0.00 0.00
dm-3 0.00 0.00 5735.60 0.00 22.40 0.00
8.00 26.08 4.55 0.17 99.20
sdr 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00
sdr1 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00
sds 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00
sds1 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00

Simon

On 20 February 2011 18:48, Phil Turmel wrote:
> On 02/20/2011 12:03 PM, Simon Mcnair wrote:
>> Hi Phil,
>>
>> Is this fsck (fsck.ext4 -n -b 32768 /dev/mapper/lvm--raid-RAID
>>> fsck.txt) as bad as it looks ? :-(
>
> It's bad. =A0Either the original sdd has a lot more corruption than I=
expected, or the 3ware spread corruption over all the drives.
>
> If the former, failing it out of the array might help. =A0If the latt=
er, your data is likely toast. =A0Some identifiable data is being found=
, based on the used vs. free block/inode/directory counts in that repor=
t. =A0That's good.
>
> I suggest you do "mdadm /dev/md0 --fail /dev/sdi1" and repeat the "fs=
ck -n" as above.
>
> (It'll be noticably slower, as it'll be using parity to reconstruct 1=
out every 9 chunks.)
>
> If it the fsck results improve, or stay the same, proceed to "fsck -y=
", and we'll see.
>
> Wouldn't hurt to run "iostat -xm 5" in another terminal during the fs=
ck to see what kind of performance that array is getting.
>
> Phil
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

On 02/20/2011 04:56 AM, Simon Mcnair wrote:
> Phil,
> I don't know how to find out if I'm running irqbalance, it's whatever
> was in the proxmox iso that I installed the OS from. I ran a ps -aux
> | grep irq in case it shows anything of interest:
>
> proxmox:/home/simon# ps -aux | grep irq
> Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html
> root 3 0.0 0.0 0 0 ? S Feb17 0:22 [ksoftirqd/0]
> root 7 0.0 0.0 0 0 ? S Feb17 0:05 [ksoftirqd/1]
> root 10 0.0 0.0 0 0 ? S Feb17 0:05 [ksoftirqd/2]
> root 13 0.0 0.0 0 0 ? S Feb17 0:05 [ksoftirqd/3]
> root 16 0.0 0.0 0 0 ? S Feb17 0:11 [ksoftirqd/4]
> root 19 0.0 0.0 0 0 ? S Feb17 0:13 [ksoftirqd/5]
> root 22 0.0 0.0 0 0 ? S Feb17 0:06 [ksoftirqd/6]
> root 25 0.0 0.0 0 0 ? S Feb17 0:06 [ksoftirqd/7]
> root 4024 0.0 0.0 0 0 ? S Feb17 0:00
> [kvm-irqfd-clean]

Not there. On ubuntu 10.10, the package is called "irqbalance", and the executable daemon is "irqbalance".

> proxmox:/home/simon# cat /proc/interrupts
> CPU0 CPU1 CPU2 CPU3 CPU4
> CPU5 CPU6 CPU7
> 0: 66008972 0 0 0 0
> 0 0 0 IR-IO-APIC-edge timer
> 1: 265829 0 0 0 0
> 0 0 0 IR-IO-APIC-edge i8042
> 8: 1 0 0 0 0
> 0 0 0 IR-IO-APIC-edge rtc0
> 9: 0 0 0 0 0
> 0 0 0 IR-IO-APIC-fasteoi acpi
> 16: 4432639 0 0 0 0
> 0 0 0 IR-IO-APIC-fasteoi uhci_hcd:usb3, ahci
> 17: 124325 0 0 0 0
> 0 0 0 IR-IO-APIC-fasteoi pata_jmicron, eth1
> 18: 954710 0 0 0 0
> 0 0 0 IR-IO-APIC-fasteoi ehci_hcd:usb1,
> uhci_hcd:usb8
> 19: 5994 0 0 0 0
> 0 0 0 IR-IO-APIC-fasteoi uhci_hcd:usb5,
> uhci_hcd:usb7, firewire_ohci
> 21: 0 0 0 0 0
> 0 0 0 IR-IO-APIC-fasteoi uhci_hcd:usb4
> 23: 62 0 0 0 0
> 0 0 0 IR-IO-APIC-fasteoi ehci_hcd:usb2,
> uhci_hcd:usb6
> 24: 273558 0 0 0 0
> 0 0 0 IR-IO-APIC-fasteoi nvidia
> 28: 30234710 0 0 0 0
> 0 0 0 IR-IO-APIC-fasteoi mvsas
> 64: 0 0 0 0 0
> 0 0 0 DMAR_MSI-edge dmar0
> 65: 0 0 0 0 0
> 0 0 0 DMAR_MSI-edge dmar1
> 73: 26838727 0 0 0 0
> 0 0 0 IR-PCI-MSI-edge eth0
> 74: 27057631 0 0 0 0
> 0 0 0 IR-PCI-MSI-edge ahci
> 75: 247 0 0 0 0
> 0 0 0 IR-PCI-MSI-edge hda_intel
> NMI: 0 0 0 0 0
> 0 0 0 Non-maskable interrupts
> LOC: 23087679 23088282 21638323 20580136 22974769
> 20856094 20144310 20084801 Local timer interrupts
> SPU: 0 0 0 0 0
> 0 0 0 Spurious interrupts
> PMI: 0 0 0 0 0
> 0 0 0 Performance monitoring interrupts
> PND: 0 0 0 0 0
> 0 0 0 Performance pending work
> RES: 15236622 9388860 8532928 7181879 13444866
> 6005599 4856933 3713707 Rescheduling interrupts
> CAL: 3384 5773 5839 5859 4590
> 5748 5781 5691 Function call interrupts
> TLB: 201989 195383 193545 196155 251338
> 249053 272032 319233 TLB shootdowns
> TRM: 0 0 0 0 0
> 0 0 0 Thermal event interrupts
> THR: 0 0 0 0 0
> 0 0 0 Threshold APIC interrupts
> MCE: 0 0 0 0 0
> 0 0 0 Machine check exceptions
> MCP: 830 830 830 830 830
> 830 830 830 Machine check polls
> ERR: 7
> MIS: 0

CPU0 is handling every single I/O interrupt. I really think you need irqbalance.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

UGhpbCwKWW91IGFyZSBhIGdvZCBhbW9uZ3N0IG1lbiA6LSkKCkkgc2VlbSB0 byBoYXZlIHRoZSB2
YXN0IG1ham9yaXR5IG9mIG15IGRhdGEuICBJIGNhbid0IHNlZSBhbnl0aGlu ZyBpbgpsb3N0K2Zv
dW5kIGFuZCB0aGUgZnNjayBsb2cgZG9lc24ndCBsaXN0IGFueSBmaWxlbmFt ZXMgZXRjIG9mIG1p
c3NpbmcKZGF0YS4gIElzIHRoZXJlIGFueSB3YXkgZm9yIG1lIHRvIGZpbmQg YW55dGhpbmcgdGhh
dCBJJ3ZlIGxvc3QgPwoKOi0pCjotKQo6LSkKClNpbW9uCgpPbiAyMCBGZWJy dWFyeSAyMDExIDE5
OjUwLCBQaGlsIFR1cm1lbCA8cGhpbGlwQHR1cm1lbC5vcmc+IHdyb3RlOgo+ IE9uIDAyLzIwLzIw
MTEgMDQ6NTYgQU0sIFNpbW9uIE1jbmFpciB3cm90ZToKPj4gUGhpbCwKPj4g SSBkb24ndCBrbm93
IGhvdyB0byBmaW5kIG91dCBpZiBJJ20gcnVubmluZyBpcnFiYWxhbmNlLCBp dCdzIHdoYXRldmVy
Cj4+IHdhcyBpbiB0aGUgcHJveG1veCBpc28gdGhhdCBJIGluc3RhbGxlZCB0 aGUgT1MgZnJvbS4g
oEkgcmFuIGEgcHMgLWF1eAo+PiB8IGdyZXAgaXJxIGluIGNhc2UgaXQgc2hv d3MgYW55dGhpbmcg
b2YgaW50ZXJlc3Q6Cj4+Cj4+IHByb3htb3g6L2hvbWUvc2ltb24jIHBzIC1h dXggfCBncmVwIGly
cQo+PiBXYXJuaW5nOiBiYWQgcHMgc3ludGF4LCBwZXJoYXBzIGEgYm9ndXMg Jy0nPyBTZWUgaHR0
cDovL3Byb2Nwcy5zZi5uZXQvZmFxLmh0bWwKPj4gcm9vdCCgIKAgoCCgIDMg oDAuMCCgMC4wIKAg
oCCgMCCgIKAgMCA/IKAgoCCgIKBTIKAgoEZlYjE3IKAgMDoyMiBba3NvZnRp cnFkLzBdCj4+IHJv
b3QgoCCgIKAgoCA3IKAwLjAgoDAuMCCgIKAgoDAgoCCgIDAgPyCgIKAgoCCg UyCgIKBGZWIxNyCg
IDA6MDUgW2tzb2Z0aXJxZC8xXQo+PiByb290IKAgoCCgIKAxMCCgMC4wIKAw LjAgoCCgIKAwIKAg
oCAwID8goCCgIKAgoFMgoCCgRmViMTcgoCAwOjA1IFtrc29mdGlycWQvMl0K Pj4gcm9vdCCgIKAg
oCCgMTMgoDAuMCCgMC4wIKAgoCCgMCCgIKAgMCA/IKAgoCCgIKBTIKAgoEZl YjE3IKAgMDowNSBb
a3NvZnRpcnFkLzNdCj4+IHJvb3QgoCCgIKAgoDE2IKAwLjAgoDAuMCCgIKAg oDAgoCCgIDAgPyCg
IKAgoCCgUyCgIKBGZWIxNyCgIDA6MTEgW2tzb2Z0aXJxZC80XQo+PiByb290 IKAgoCCgIKAxOSCg
MC4wIKAwLjAgoCCgIKAwIKAgoCAwID8goCCgIKAgoFMgoCCgRmViMTcgoCAw OjEzIFtrc29mdGly
cWQvNV0KPj4gcm9vdCCgIKAgoCCgMjIgoDAuMCCgMC4wIKAgoCCgMCCgIKAg MCA/IKAgoCCgIKBT
IKAgoEZlYjE3IKAgMDowNiBba3NvZnRpcnFkLzZdCj4+IHJvb3QgoCCgIKAg oDI1IKAwLjAgoDAu
MCCgIKAgoDAgoCCgIDAgPyCgIKAgoCCgUyCgIKBGZWIxNyCgIDA6MDYgW2tz b2Z0aXJxZC83XQo+
PiByb290IKAgoCCgNDAyNCCgMC4wIKAwLjAgoCCgIKAwIKAgoCAwID8goCCg IKAgoFMgoCCgRmVi
MTcgoCAwOjAwCj4+IFtrdm0taXJxZmQtY2xlYW5dCj4KPiBOb3QgdGhlcmUu IKBPbiB1YnVudHUg
MTAuMTAsIHRoZSBwYWNrYWdlIGlzIGNhbGxlZCAiaXJxYmFsYW5jZSIsIGFu ZCB0aGUgZXhlY3V0
YWJsZSBkYWVtb24gaXMgImlycWJhbGFuY2UiLgo+Cj4+IHByb3htb3g6L2hv bWUvc2ltb24jIGNh
dCAvcHJvYy9pbnRlcnJ1cHRzCj4+IKAgoCCgIKAgoCCgIENQVTAgoCCgIKAg Q1BVMSCgIKAgoCBD
UFUyIKAgoCCgIENQVTMgoCCgIKAgQ1BVNAo+PiBDUFU1IKAgoCCgIENQVTYg oCCgIKAgQ1BVNwo+
PiCgIKAwOiCgIDY2MDA4OTcyIKAgoCCgIKAgoDAgoCCgIKAgoCCgMCCgIKAg oCCgIKAwIKAgoCCg
IKAgoDAKPj4gMCCgIKAgoCCgIKAwIKAgoCCgIKAgoDAgoElSLUlPLUFQSUMt ZWRnZSCgIKAgoHRp
bWVyCj4+IKAgoDE6IKAgoCAyNjU4MjkgoCCgIKAgoCCgMCCgIKAgoCCgIKAw IKAgoCCgIKAgoDAg
oCCgIKAgoCCgMAo+PiAwIKAgoCCgIKAgoDAgoCCgIKAgoCCgMCCgSVItSU8t QVBJQy1lZGdlIKAg
oCCgaTgwNDIKPj4goCCgODogoCCgIKAgoCCgMSCgIKAgoCCgIKAwIKAgoCCg IKAgoDAgoCCgIKAg
oCCgMCCgIKAgoCCgIKAwCj4+IDAgoCCgIKAgoCCgMCCgIKAgoCCgIKAwIKBJ Ui1JTy1BUElDLWVk
Z2UgoCCgIKBydGMwCj4+IKAgoDk6IKAgoCCgIKAgoDAgoCCgIKAgoCCgMCCg IKAgoCCgIKAwIKAg
oCCgIKAgoDAgoCCgIKAgoCCgMAo+PiAwIKAgoCCgIKAgoDAgoCCgIKAgoCCg MCCgSVItSU8tQVBJ
Qy1mYXN0ZW9pIKAgYWNwaQo+PiCgIDE2OiCgIKA0NDMyNjM5IKAgoCCgIKAg oDAgoCCgIKAgoCCg
MCCgIKAgoCCgIKAwIKAgoCCgIKAgoDAKPj4gMCCgIKAgoCCgIKAwIKAgoCCg IKAgoDAgoElSLUlP
LUFQSUMtZmFzdGVvaSCgIHVoY2lfaGNkOnVzYjMsIGFoY2kKPj4goCAxNzog oCCgIDEyNDMyNSCg
IKAgoCCgIKAwIKAgoCCgIKAgoDAgoCCgIKAgoCCgMCCgIKAgoCCgIKAwCj4+ IDAgoCCgIKAgoCCg
MCCgIKAgoCCgIKAwIKBJUi1JTy1BUElDLWZhc3Rlb2kgoCBwYXRhX2ptaWNy b24sIGV0aDEKPj4g
oCAxODogoCCgIDk1NDcxMCCgIKAgoCCgIKAwIKAgoCCgIKAgoDAgoCCgIKAg oCCgMCCgIKAgoCCg
IKAwCj4+IDAgoCCgIKAgoCCgMCCgIKAgoCCgIKAwIKBJUi1JTy1BUElDLWZh c3Rlb2kgoCBlaGNp
X2hjZDp1c2IxLAo+PiB1aGNpX2hjZDp1c2I4Cj4+IKAgMTk6IKAgoCCgIDU5 OTQgoCCgIKAgoCCg
MCCgIKAgoCCgIKAwIKAgoCCgIKAgoDAgoCCgIKAgoCCgMAo+PiAwIKAgoCCg IKAgoDAgoCCgIKAg
oCCgMCCgSVItSU8tQVBJQy1mYXN0ZW9pIKAgdWhjaV9oY2Q6dXNiNSwKPj4g dWhjaV9oY2Q6dXNi
NywgZmlyZXdpcmVfb2hjaQo+PiCgIDIxOiCgIKAgoCCgIKAwIKAgoCCgIKAg oDAgoCCgIKAgoCCg
MCCgIKAgoCCgIKAwIKAgoCCgIKAgoDAKPj4gMCCgIKAgoCCgIKAwIKAgoCCg IKAgoDAgoElSLUlP
LUFQSUMtZmFzdGVvaSCgIHVoY2lfaGNkOnVzYjQKPj4goCAyMzogoCCgIKAg oCA2MiCgIKAgoCCg
IKAwIKAgoCCgIKAgoDAgoCCgIKAgoCCgMCCgIKAgoCCgIKAwCj4+IDAgoCCg IKAgoCCgMCCgIKAg
oCCgIKAwIKBJUi1JTy1BUElDLWZhc3Rlb2kgoCBlaGNpX2hjZDp1c2IyLAo+ PiB1aGNpX2hjZDp1
c2I2Cj4+IKAgMjQ6IKAgoCAyNzM1NTggoCCgIKAgoCCgMCCgIKAgoCCgIKAw IKAgoCCgIKAgoDAg
oCCgIKAgoCCgMAo+PiAwIKAgoCCgIKAgoDAgoCCgIKAgoCCgMCCgSVItSU8t QVBJQy1mYXN0ZW9p
IKAgbnZpZGlhCj4+IKAgMjg6IKAgMzAyMzQ3MTAgoCCgIKAgoCCgMCCgIKAg oCCgIKAwIKAgoCCg
IKAgoDAgoCCgIKAgoCCgMAo+PiAwIKAgoCCgIKAgoDAgoCCgIKAgoCCgMCCg SVItSU8tQVBJQy1m
YXN0ZW9pIKAgbXZzYXMKPj4goCA2NDogoCCgIKAgoCCgMCCgIKAgoCCgIKAw IKAgoCCgIKAgoDAg
oCCgIKAgoCCgMCCgIKAgoCCgIKAwCj4+IDAgoCCgIKAgoCCgMCCgIKAgoCCg IKAwIKBETUFSX01T
SS1lZGdlIKAgoCCgZG1hcjAKPj4goCA2NTogoCCgIKAgoCCgMCCgIKAgoCCg IKAwIKAgoCCgIKAg
oDAgoCCgIKAgoCCgMCCgIKAgoCCgIKAwCj4+IDAgoCCgIKAgoCCgMCCgIKAg oCCgIKAwIKBETUFS
X01TSS1lZGdlIKAgoCCgZG1hcjEKPj4goCA3MzogoCAyNjgzODcyNyCgIKAg oCCgIKAwIKAgoCCg
IKAgoDAgoCCgIKAgoCCgMCCgIKAgoCCgIKAwCj4+IDAgoCCgIKAgoCCgMCCg IKAgoCCgIKAwIKBJ
Ui1QQ0ktTVNJLWVkZ2UgoCCgIKBldGgwCj4+IKAgNzQ6IKAgMjcwNTc2MzEg oCCgIKAgoCCgMCCg
IKAgoCCgIKAwIKAgoCCgIKAgoDAgoCCgIKAgoCCgMAo+PiAwIKAgoCCgIKAg oDAgoCCgIKAgoCCg
MCCgSVItUENJLU1TSS1lZGdlIKAgoCCgYWhjaQo+PiCgIDc1OiCgIKAgoCCg MjQ3IKAgoCCgIKAg
oDAgoCCgIKAgoCCgMCCgIKAgoCCgIKAwIKAgoCCgIKAgoDAKPj4gMCCgIKAg oCCgIKAwIKAgoCCg
IKAgoDAgoElSLVBDSS1NU0ktZWRnZSCgIKAgoGhkYV9pbnRlbAo+PiCgTk1J OiCgIKAgoCCgIKAw
IKAgoCCgIKAgoDAgoCCgIKAgoCCgMCCgIKAgoCCgIKAwIKAgoCCgIKAgoDAK Pj4gMCCgIKAgoCCg
IKAwIKAgoCCgIKAgoDAgoCBOb24tbWFza2FibGUgaW50ZXJydXB0cwo+PiCg TE9DOiCgIDIzMDg3
Njc5IKAgMjMwODgyODIgoCAyMTYzODMyMyCgIDIwNTgwMTM2IKAgMjI5NzQ3 NjkKPj4gMjA4NTYw
OTQgoCAyMDE0NDMxMCCgIDIwMDg0ODAxIKAgTG9jYWwgdGltZXIgaW50ZXJy dXB0cwo+PiCgU1BV
OiCgIKAgoCCgIKAwIKAgoCCgIKAgoDAgoCCgIKAgoCCgMCCgIKAgoCCgIKAw IKAgoCCgIKAgoDAK
Pj4gMCCgIKAgoCCgIKAwIKAgoCCgIKAgoDAgoCBTcHVyaW91cyBpbnRlcnJ1 cHRzCj4+IKBQTUk6
IKAgoCCgIKAgoDAgoCCgIKAgoCCgMCCgIKAgoCCgIKAwIKAgoCCgIKAgoDAg oCCgIKAgoCCgMAo+
PiAwIKAgoCCgIKAgoDAgoCCgIKAgoCCgMCCgIFBlcmZvcm1hbmNlIG1vbml0 b3JpbmcgaW50ZXJy
dXB0cwo+PiCgUE5EOiCgIKAgoCCgIKAwIKAgoCCgIKAgoDAgoCCgIKAgoCCg MCCgIKAgoCCgIKAw
IKAgoCCgIKAgoDAKPj4gMCCgIKAgoCCgIKAwIKAgoCCgIKAgoDAgoCBQZXJm b3JtYW5jZSBwZW5k
aW5nIHdvcmsKPj4goFJFUzogoCAxNTIzNjYyMiCgIKA5Mzg4ODYwIKAgoDg1 MzI5MjggoCCgNzE4
MTg3OSCgIDEzNDQ0ODY2Cj4+IDYwMDU1OTkgoCCgNDg1NjkzMyCgIKAzNzEz NzA3IKAgUmVzY2hl
ZHVsaW5nIGludGVycnVwdHMKPj4goENBTDogoCCgIKAgMzM4NCCgIKAgoCA1 NzczIKAgoCCgIDU4
MzkgoCCgIKAgNTg1OSCgIKAgoCA0NTkwCj4+IDU3NDggoCCgIKAgNTc4MSCg IKAgoCA1NjkxIKAg
RnVuY3Rpb24gY2FsbCBpbnRlcnJ1cHRzCj4+IKBUTEI6IKAgoCAyMDE5ODkg oCCgIDE5NTM4MyCg
IKAgMTkzNTQ1IKAgoCAxOTYxNTUgoCCgIDI1MTMzOAo+PiAyNDkwNTMgoCCg IDI3MjAzMiCgIKAg
MzE5MjMzIKAgVExCIHNob290ZG93bnMKPj4goFRSTTogoCCgIKAgoCCgMCCg IKAgoCCgIKAwIKAg
oCCgIKAgoDAgoCCgIKAgoCCgMCCgIKAgoCCgIKAwCj4+IDAgoCCgIKAgoCCg MCCgIKAgoCCgIKAw
IKAgVGhlcm1hbCBldmVudCBpbnRlcnJ1cHRzCj4+IKBUSFI6IKAgoCCgIKAg oDAgoCCgIKAgoCCg
MCCgIKAgoCCgIKAwIKAgoCCgIKAgoDAgoCCgIKAgoCCgMAo+PiAwIKAgoCCg IKAgoDAgoCCgIKAg
oCCgMCCgIFRocmVzaG9sZCBBUElDIGludGVycnVwdHMKPj4goE1DRTogoCCg IKAgoCCgMCCgIKAg
oCCgIKAwIKAgoCCgIKAgoDAgoCCgIKAgoCCgMCCgIKAgoCCgIKAwCj4+IDAg oCCgIKAgoCCgMCCg
IKAgoCCgIKAwIKAgTWFjaGluZSBjaGVjayBleGNlcHRpb25zCj4+IKBNQ1A6 IKAgoCCgIKA4MzAg
oCCgIKAgoDgzMCCgIKAgoCCgODMwIKAgoCCgIKA4MzAgoCCgIKAgoDgzMAo+ PiA4MzAgoCCgIKAg
oDgzMCCgIKAgoCCgODMwIKAgTWFjaGluZSBjaGVjayBwb2xscwo+PiCgRVJS OiCgIKAgoCCgIKA3
Cj4+IKBNSVM6IKAgoCCgIKAgoDAKPgo+IENQVTAgaXMgaGFuZGxpbmcgZXZl cnkgc2luZ2xlIEkv
TyBpbnRlcnJ1cHQuIKBJIHJlYWxseSB0aGluayB5b3UgbmVlZCBpcnFiYWxh bmNlLgo+Cj4gUGhp
bAo+Cg==
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

On 02/20/2011 06:17 PM, Simon Mcnair wrote:
> Phil,
> You are a god amongst men :-)

Heh. I read that out loud to my wife....

> I seem to have the vast majority of my data. I can't see anything in
> lost+found and the fsck log doesn't list any filenames etc of missing
> data. Is there any way for me to find anything that I've lost ?

Not that I know of, short of a list of names from before the crash.

Glad to hear that the family treasures are safe.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

Phil,
I thought it was fixed, but every time I do a reboot it disappears
again. I did the:
pvcreate --restorefile 'simons-lvm-backup' --uuid
9fAJEz-HcaP-RQ51-fV8b-nxrN-Uqwb-PPnOLJ /dev/md0 and the vgcfgrestore

the only piece of your advice that I didn't follow was "zero the
superblock on the removed drive" as I thought re-adding it to the
array would do that anyway. Did this cause me problems ?

proxmox:~# mdadm --assemble --scan -v
mdadm: looking for devices for /dev/md/0
mdadm: cannot open device /dev/dm-2: Device or resource busy
mdadm: /dev/dm-2 has wrong uuid.
mdadm: cannot open device /dev/dm-1: Device or resource busy
mdadm: /dev/dm-1 has wrong uuid.
mdadm: cannot open device /dev/dm-0: Device or resource busy
mdadm: /dev/dm-0 has wrong uuid.
mdadm: /dev/sdo1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdo
mdadm: /dev/sdo has wrong uuid.
mdadm: /dev/sdn1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdn
mdadm: /dev/sdn has wrong uuid.
mdadm: /dev/sdm1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdm
mdadm: /dev/sdm has wrong uuid.
mdadm: /dev/sdl1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdl
mdadm: /dev/sdl has wrong uuid.
mdadm: /dev/sdk1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdk
mdadm: /dev/sdk has wrong uuid.
mdadm: /dev/sdj1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdj
mdadm: /dev/sdj has wrong uuid.
mdadm: /dev/sdi1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdi
mdadm: /dev/sdi has wrong uuid.
mdadm: /dev/sdh1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdh
mdadm: /dev/sdh has wrong uuid.
mdadm: /dev/sdg1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdg
mdadm: /dev/sdg has wrong uuid.
mdadm: /dev/sdf1 has wrong uuid.
mdadm: no RAID superblock on /dev/sdf
mdadm: /dev/sdf has wrong uuid.
mdadm: cannot open device /dev/sdc1: Device or resource busy
mdadm: /dev/sdc1 has wrong uuid.
mdadm: cannot open device /dev/sdc: Device or resource busy
mdadm: /dev/sdc has wrong uuid.
mdadm: cannot open device /dev/sdb1: Device or resource busy
mdadm: /dev/sdb1 has wrong uuid.
mdadm: cannot open device /dev/sdb: Device or resource busy
mdadm: /dev/sdb has wrong uuid.
mdadm: cannot open device /dev/sda2: Device or resource busy
mdadm: /dev/sda2 has wrong uuid.
mdadm: cannot open device /dev/sda1: Device or resource busy
mdadm: /dev/sda1 has wrong uuid.
mdadm: cannot open device /dev/sda: Device or resource busy
mdadm: /dev/sda has wrong uuid.

I'm getting frustrated now. It seems like it's fixzed, then next
reboot, BAM, it's gone again. I'm so out of my depth here.

Please can you help me fix this once and for all ?

cheers
Simon

On 20 February 2011 23:39, Phil Turmel wrote:
> On 02/20/2011 06:17 PM, Simon Mcnair wrote:
>> Phil,
>> You are a god amongst men :-)
>
> Heh. =A0I read that out loud to my wife....
>
>> I seem to have the vast majority of my data. =A0I can't see anything=
in
>> lost+found and the fsck log doesn't list any filenames etc of missin=
g
>> data. =A0Is there any way for me to find anything that I've lost ?
>
> Not that I know of, short of a list of names from before the crash.
>
> Glad to hear that the family treasures are safe.
>
> Phil
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

was this anything to do with the dist-upgrade that I performed ?

mdadm has upgraded again to
proxmox:~# mdadm -V
mdadm - v3.1.4 - 31st August 2010

if that's important.

Simon

On 22 February 2011 17:12, Simon Mcnair wrote:
> Phil,
> I thought it was fixed, but every time I do a reboot it disappears
> again. =A0I did the:
> pvcreate --restorefile 'simons-lvm-backup' --uuid
> 9fAJEz-HcaP-RQ51-fV8b-nxrN-Uqwb-PPnOLJ /dev/md0 and the vgcfgrestore
>
> the only piece of your advice that I didn't follow was "zero the
> superblock on the removed drive" as I thought re-adding it to the
> array would do that anyway. =A0Did this cause me problems ?
>
> proxmox:~# mdadm =A0--assemble --scan -v
> mdadm: looking for devices for /dev/md/0
> mdadm: cannot open device /dev/dm-2: Device or resource busy
> mdadm: /dev/dm-2 has wrong uuid.
> mdadm: cannot open device /dev/dm-1: Device or resource busy
> mdadm: /dev/dm-1 has wrong uuid.
> mdadm: cannot open device /dev/dm-0: Device or resource busy
> mdadm: /dev/dm-0 has wrong uuid.
> mdadm: /dev/sdo1 has wrong uuid.
> mdadm: no RAID superblock on /dev/sdo
> mdadm: /dev/sdo has wrong uuid.
> mdadm: /dev/sdn1 has wrong uuid.
> mdadm: no RAID superblock on /dev/sdn
> mdadm: /dev/sdn has wrong uuid.
> mdadm: /dev/sdm1 has wrong uuid.
> mdadm: no RAID superblock on /dev/sdm
> mdadm: /dev/sdm has wrong uuid.
> mdadm: /dev/sdl1 has wrong uuid.
> mdadm: no RAID superblock on /dev/sdl
> mdadm: /dev/sdl has wrong uuid.
> mdadm: /dev/sdk1 has wrong uuid.
> mdadm: no RAID superblock on /dev/sdk
> mdadm: /dev/sdk has wrong uuid.
> mdadm: /dev/sdj1 has wrong uuid.
> mdadm: no RAID superblock on /dev/sdj
> mdadm: /dev/sdj has wrong uuid.
> mdadm: /dev/sdi1 has wrong uuid.
> mdadm: no RAID superblock on /dev/sdi
> mdadm: /dev/sdi has wrong uuid.
> mdadm: /dev/sdh1 has wrong uuid.
> mdadm: no RAID superblock on /dev/sdh
> mdadm: /dev/sdh has wrong uuid.
> mdadm: /dev/sdg1 has wrong uuid.
> mdadm: no RAID superblock on /dev/sdg
> mdadm: /dev/sdg has wrong uuid.
> mdadm: /dev/sdf1 has wrong uuid.
> mdadm: no RAID superblock on /dev/sdf
> mdadm: /dev/sdf has wrong uuid.
> mdadm: cannot open device /dev/sdc1: Device or resource busy
> mdadm: /dev/sdc1 has wrong uuid.
> mdadm: cannot open device /dev/sdc: Device or resource busy
> mdadm: /dev/sdc has wrong uuid.
> mdadm: cannot open device /dev/sdb1: Device or resource busy
> mdadm: /dev/sdb1 has wrong uuid.
> mdadm: cannot open device /dev/sdb: Device or resource busy
> mdadm: /dev/sdb has wrong uuid.
> mdadm: cannot open device /dev/sda2: Device or resource busy
> mdadm: /dev/sda2 has wrong uuid.
> mdadm: cannot open device /dev/sda1: Device or resource busy
> mdadm: /dev/sda1 has wrong uuid.
> mdadm: cannot open device /dev/sda: Device or resource busy
> mdadm: /dev/sda has wrong uuid.
>
> I'm getting frustrated now. =A0It seems like it's fixzed, then next
> reboot, BAM, it's gone again. =A0I'm so out of my depth here.
>
> Please can you help me fix this once and for all ?
>
> cheers
> Simon
>
> On 20 February 2011 23:39, Phil Turmel wrote:
>> On 02/20/2011 06:17 PM, Simon Mcnair wrote:
>>> Phil,
>>> You are a god amongst men :-)
>>
>> Heh. =A0I read that out loud to my wife....
>>
>>> I seem to have the vast majority of my data. =A0I can't see anythin=
g in
>>> lost+found and the fsck log doesn't list any filenames etc of missi=
ng
>>> data. =A0Is there any way for me to find anything that I've lost ?
>>
>> Not that I know of, short of a list of names from before the crash.
>>
>> Glad to hear that the family treasures are safe.
>>
>> Phil
>>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

On 02/22/2011 12:14 PM, Simon Mcnair wrote:
> was this anything to do with the dist-upgrade that I performed ?
>
> mdadm has upgraded again to
> proxmox:~# mdadm -V
> mdadm - v3.1.4 - 31st August 2010

No, but you probably can't do any more "--create --assume-clean" operations with that array.

As for the assembly errors, its probably an out-of-date mdadm.conf.

My server's looks like this:

> DEVICE /dev/sd[a-z][1-9]
>
> ARRAY /dev/md23 UUID=c3cbe096:fc43d939:8aa66230:708c5670
> ARRAY /dev/md22 UUID=1438a239:aa03c3f9:68e051d7:b59a6219
> ARRAY /dev/md21 UUID=4b47d0f6:16f0e352:c67a2185:feb0b573
> ARRAY /dev/md20 UUID=7676c77e:6ae70f65:30a170a2:b1a1b242

Yours probably needs to be updated with the new md0 uuid. Its not your boot drive, so your initramfs shouldn't matter.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

Phil,
phew I didn't know that:
"mdadm: /dev/sdo has wrong uuid.
mdadm: /dev/sdn1 has wrong uuid."

was just that the array UUID didn't match mdadm.conf, it would be nice=20
if it said:

"mdadm: /dev/sdo uuid does not match mdadm.conf.
mdadm: /dev/sdn1 uuid does not match mdadm.conf"
my mdadm.conf now reads

DEVICE partitions
CREATE owner=3Droot group=3Ddisk mode=3D0660 auto=3Dyes
HOMEHOST
MAILADDR root
# ARRAY /dev/md/0 level=3Draid5 metadata=3D1.1 num-devices=3D10=20
UUID=3D0a72e40f:aec6f80f:a7004457:1a84a7a8 name=3Dproï¿=BDlox:0
ARRAY /dev/md/0 metadata=3D1.1 UUID=3D12c2af00:10681e10:fb17e449:140473=
9c=20
name=3Dproxmox:0

I'm afraid to do another reboot incase something else goes wrong ;-). =20
Just for my info, when I only have a single array there isn't much=20
chance of it being assigned anything other than md0, so could I not jus=
t=20
leave mdadm.conf empty ?
cheers

Simon

On 22/02/2011 18:23, Phil Turmel wrote:
> On 02/22/2011 12:14 PM, Simon Mcnair wrote:
>> was this anything to do with the dist-upgrade that I performed ?
>>
>> mdadm has upgraded again to
>> proxmox:~# mdadm -V
>> mdadm - v3.1.4 - 31st August 2010
> No, but you probably can't do any more "--create --assume-clean" oper=
ations with that array.
>
> As for the assembly errors, its probably an out-of-date mdadm.conf.
>
> My server's looks like this:
>
>> DEVICE /dev/sd[a-z][1-9]
>>
>> ARRAY /dev/md23 UUID=3Dc3cbe096:fc43d939:8aa66230:708c5670
>> ARRAY /dev/md22 UUID=3D1438a239:aa03c3f9:68e051d7:b59a6219
>> ARRAY /dev/md21 UUID=3D4b47d0f6:16f0e352:c67a2185:feb0b573
>> ARRAY /dev/md20 UUID=3D7676c77e:6ae70f65:30a170a2:b1a1b242
> Yours probably needs to be updated with the new md0 uuid. Its not yo=
ur boot drive, so your initramfs shouldn't matter.
>
> Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Linux software RAID assistance

On 02/22/2011 01:36 PM, Simon McNair wrote:
> Phil,
> phew I didn't know that:
> "mdadm: /dev/sdo has wrong uuid.
> mdadm: /dev/sdn1 has wrong uuid."
>=20
> was just that the array UUID didn't match mdadm.conf, it would be nic=
e if it said:
>=20
> "mdadm: /dev/sdo uuid does not match mdadm.conf.
> mdadm: /dev/sdn1 uuid does not match mdadm.conf"
> my mdadm.conf now reads
>=20
> DEVICE partitions
> CREATE owner=3Droot group=3Ddisk mode=3D0660 auto=3Dyes
> HOMEHOST
> MAILADDR root
> # ARRAY /dev/md/0 level=3Draid5 metadata=3D1.1 num-devices=3D10 UUID=3D=
0a72e40f:aec6f80f:a7004457:1a84a7a8 name=3Dproï¿=BDlox:0
> ARRAY /dev/md/0 metadata=3D1.1 UUID=3D12c2af00:10681e10:fb17e449:1404=
739c name=3Dproxmox:0

You can strip it down to be effectively "auto", but I wouldn't remove i=
t. Since you are using partitioned devices throughout your system, som=
ething like this:

DEVICE /dev/sd*[0-9]
AUTO all

Or slightly more restrictive:

DEVICE /dev/sd*[0-9]
ARRAY /dev/md/0 UUID=3D12c2af00:10681e10:fb17e449:1404739c

> I'm afraid to do another reboot incase something else goes wrong ;-).=
Just for my info, when I only have a single array there isn't much ch=
ance of it being assigned anything other than md0, so could I not just =
leave mdadm.conf empty ?

LVM doesn't care what device name the physical volumes end up with. Pl=
ease reboot when the system is quiet again to make sure your new mdadm.=
conf works. Or stop your array, and then do a "mdadm --assemble --scan=
", which is effectively the same.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html