Antw: Problems creating MD-RAID1: "device .. not suitable forany style of raid array" / &q

Hi!

I could reduce the problem to "--bitmap internal": It works if I leave it out. Unfortunately the RAID will fully resync every assembly when there was an unclean shutdown.
The thing that kept one device busy was MD-RAID itself: "mdadm -- stop" released the device.

So to summarize:
1) mdadm fails to set up an internal bitmap
2) Even when mdadm fails to do 1), it starts the array

Regards,
Ulrich


>>> Ulrich Windl schrieb am 06.05.2011 um 14:15 in Nachricht <4DC3E67F.8BA : 161 :
60728>:
> Hello!
>
> I'm having strange trouble with SLES11 SP1 amd MD RAID1 (mdadm - v3.0.3
> (mdadm-3.0.3-0.22.4), 2.6.32.36-0.5-xen):
>
> I was able to create one RAID1 array, but not a second one. I have no idea
> what's wrong, but my guesses are:
>
> 1) An error when using "--bitmap internal" for a 30GB disk
> 2) Unsure whether the disks needs an msdos signature or a RAID partition
> 3) It seems a failed attempt to create the array keeps one device busy (a
> reboot(!) resolves that problem for one attempt)
>
> Some output:
> # mdadm -C -l1 -n2 --bitmap internal /dev/md0 /dev/xvdd /dev/xvde
> mdadm: device /dev/xvdd not suitable for any style of array
> (Reboot)
> # mdadm -C -l1 -n2 /dev/md0 /dev/xvdd /dev/xvde
> mdadm: /dev/xvdd appears to be part of a raid array:
> level=raid1 devices=2 ctime=Wed May 4 12:00:12 2011
> Continue creating array? y
> mdadm: array /dev/md0 started.
>
> # mdadm --grow --bitmap internal /dev/md0
> mdadm: failed to set internal bitmap.
> # mdadm --stop /dev/md0
> mdadm: stopped /dev/md0
> # mdadm -C -l1 -n2 --bitmap internal /dev/md0 /dev/xvdd /dev/xvde
> mdadm: /dev/xvdd appears to be part of a raid array:
> level=raid1 devices=2 ctime=Wed May 4 12:38:20 2011
> mdadm: /dev/xvde appears to be part of a raid array:
> level=raid1 devices=2 ctime=Wed May 4 12:38:20 2011
> Continue creating array? y
> mdadm: failed to write superblock to /dev/xvdd
> mdadm: ADD_NEW_DISK for /dev/xvde failed: Device or resource busy
> # mdadm -C -l1 -n2 --bitmap internal /dev/md0 /dev/xvdd /dev/xvde
> mdadm: device /dev/xvdd not suitable for any style of array
> # xm reboot rksapv01
> # mdadm -C -l1 -n2 --bitmap internal /dev/md0 /dev/xvdd /dev/xvde
> mdadm: /dev/xvdd appears to be part of a raid array:
> level=raid1 devices=2 ctime=Wed May 4 13:26:55 2011
> mdadm: /dev/xvde appears to be part of a raid array:
> level=raid1 devices=2 ctime=Wed May 4 12:38:20 2011
> Continue creating array? y
> mdadm: failed to write superblock to /dev/xvdd
> mdadm: ADD_NEW_DISK for /dev/xvde failed: Device or resource busy
>
> # mdadm -C -l1 -n2 --bitmap internal /dev/md0 /dev/xvdd /dev/xvde
> mdadm: /dev/xvdd appears to be part of a raid array:
> level=raid1 devices=2 ctime=Wed May 4 13:37:28 2011
> mdadm: /dev/xvde appears to be part of a raid array:
> level=raid1 devices=2 ctime=Wed May 4 12:38:20 2011
> Continue creating array? y
> mdadm: failed to write superblock to /dev/xvdd
> mdadm: ADD_NEW_DISK for /dev/xvde failed: Device or resource busy
> # mdadm -C -l1 -n2 --bitmap internal /dev/md0 /dev/xvdd /dev/xvde
> mdadm: device /dev/xvdd not suitable for any style of array
>
> (At this point I filed a service request for SLES with no result until now)
>
> Trying some other disks (of varying size):
> # mdadm -C -l1 -n2 --bitmap internal /dev/md0 /dev/xvdd /dev/xvde
> mdadm: /dev/xvdd appears to be part of a raid array:
> level=raid1 devices=2 ctime=Wed May 4 13:37:28 2011
> mdadm: /dev/xvde appears to be part of a raid array:
> level=raid1 devices=2 ctime=Wed May 4 12:38:20 2011
> Continue creating array? y
> mdadm: failed to write superblock to /dev/xvdd
> mdadm: ADD_NEW_DISK for /dev/xvde failed: Device or resource busy
> # mdadm -C -l1 -n2 --bitmap internal /dev/md0 /dev/xvdd /dev/xvde
> mdadm: device /dev/xvdd not suitable for any style of array
>
> (another Reboot)
> # mdadm -C -l1 -n2 --bitmap internal /dev/md1 /dev/xvdf /dev/xvdg
> mdadm: /dev/xvdf appears to be part of a raid array:
> level=raid1 devices=2 ctime=Wed May 4 16:36:59 2011
> Continue creating array? y
> mdadm: failed to write superblock to /dev/xvdf
> mdadm: ADD_NEW_DISK for /dev/xvdg failed: Device or resource busy
> # mdadm -C -l1 -n2 --bitmap internal /dev/md2 /dev/xvdh /dev/xvdi
> mdadm: /dev/xvdh appears to be part of a raid array:
> level=raid1 devices=2 ctime=Wed May 4 16:37:26 2011
> Continue creating array? y
> mdadm: failed to write superblock to /dev/xvdh
> mdadm: ADD_NEW_DISK for /dev/xvdi failed: Device or resource busy
> # mdadm -C -l1 -n2 --bitmap internal /dev/md3 /dev/xvdj /dev/xvdk
> mdadm: /dev/xvdj appears to be part of a raid array:
> level=raid1 devices=2 ctime=Wed May 4 16:37:38 2011
> Continue creating array? y
> mdadm: failed to write superblock to /dev/xvdj
> mdadm: ADD_NEW_DISK for /dev/xvdk failed: Device or resource busy
>
> Corresponding Syslog messages:
> May 4 17:18:54 rksapv01 kernel: [ 231.942241] md: bind
> May 4 17:18:54 rksapv01 kernel: [ 231.942265] md: could not bd_claim xvdg.
> May 4 17:18:54 rksapv01 kernel: [ 231.942269] md: md_import_device
> returned
> -16
> May 4 17:19:13 rksapv01 kernel: [ 250.118561] md: bind
> May 4 17:19:13 rksapv01 kernel: [ 250.118586] md: could not bd_claim xvdi.
> May 4 17:19:13 rksapv01 kernel: [ 250.118590] md: md_import_device
> returned
> -16
> May 4 17:19:27 rksapv01 kernel: [ 264.505337] md: bind
> May 4 17:19:27 rksapv01 kernel: [ 264.505365] md: could not bd_claim xvdk.
> May 4 17:19:27 rksapv01 kernel: [ 264.505368] md: md_import_device
> returned
> -16
>
> Do you understand that I'm quite frustrated? Maybe I should mention that the
> disks are from a FC-SAN with a 4-way multipath
> (multipath-tools-0.4.8-40.25.1). Also "lsof" finds no process that has the
> device open.
>
> Finally: I had a similar problem with SLES10 SP3 which made me quit using
> MD-RAID about two years ago...
>
> Regards,
> Ulrich
> P.S: I'm not subscribed to the list, so please CC: -- Thank you
>
>



--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Antw: Problems creating MD-RAID1: "device .. not suitable forany style of raid array"

On 11/05/2011 15:38, Ulrich Windl wrote:
> Hi!
>
> I could reduce the problem to "--bitmap internal": It works if I leave it out. Unfortunately the RAID will fully resync every assembly when there was an unclean shutdown.
> The thing that kept one device busy was MD-RAID itself: "mdadm -- stop" released the device.
>
> So to summarize:
> 1) mdadm fails to set up an internal bitmap
> 2) Even when mdadm fails to do 1), it starts the array

OK. First up, the syntax is "--bitmap=internal" - perhaps you have a
mdadm which is partially parsing what you've said but not got it 100%
right. If using the correct syntax doesn't work, instead try creating
the array without the bitmap, then adding one with
mdadm --grow /dev/mdX --bitmap=internal

Cheers,

John.

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Antw: Problems creating MD-RAID1: "device .. not suitablefor any style of raid array"

>>> John Robinson schrieb am 11.05.2011 um 19:00
in Nachricht <4DCAC0B3.8010509@anonymous.org.uk>:
> On 11/05/2011 15:38, Ulrich Windl wrote:
> > Hi!
> >
> > I could reduce the problem to "--bitmap internal": It works if I leave it
> out. Unfortunately the RAID will fully resync every assembly when there was
> an unclean shutdown.
> > The thing that kept one device busy was MD-RAID itself: "mdadm -- stop"
> released the device.
> >
> > So to summarize:
> > 1) mdadm fails to set up an internal bitmap
> > 2) Even when mdadm fails to do 1), it starts the array
>
> OK. First up, the syntax is "--bitmap=internal" - perhaps you have a
> mdadm which is partially parsing what you've said but not got it 100%
> right. If using the correct syntax doesn't work, instead try creating
> the array without the bitmap, then adding one with
> mdadm --grow /dev/mdX --bitmap=internal

Hi John!

Thanks for that. Of course (using my syntax) the "--grow" variant also did not work. I would also expect that mdadm can either handle "--bitmap internal" as a variant of "--bitmap=internal" or emit a syntax error. Unfortunately I don't have the time right now to inspect the sources.

Regards,
Ulrich
P.S. Amazing things happen now and then:
# cat /proc/mdstat
Personalities : [linear] [raid1]
md2 : active raid1 dm-17[1] dm-10[0]
209715136 blocks [2/2] [UU]

md1 : active raid1 dm-12[0] dm-14[1]
20971456 blocks [2/2] [UU]

md3 : active raid1 dm-16[0] dm-9[1]
31457216 blocks [2/2] [UU]

md0 : active raid1 dm-11[0] dm-8[1]
525336512 blocks [2/2] [UU]

unused devices:
# mdadm --grow --bitmap=internal /dev/md0
# mdadm --grow --bitmap internal /dev/md1
# mdadm --grow --bitmap internal /dev/md2
# mdadm --grow --bitmap=internal /dev/md3
mdadm: failed to set internal bitmap.
# cat /proc/mdstat
Personalities : [linear] [raid1]
md2 : active raid1 dm-17[1] dm-10[0]
209715136 blocks [2/2] [UU]
bitmap: 0/200 pages [0KB], 512KB chunk

md1 : active raid1 dm-12[0] dm-14[1]
20971456 blocks [2/2] [UU]
bitmap: 0/160 pages [0KB], 64KB chunk

md3 : active raid1 dm-16[0] dm-9[1]
31457216 blocks [2/2] [UU]

md0 : active raid1 dm-11[0] dm-8[1]
525336512 blocks [2/2] [UU]
bitmap: 0/126 pages [0KB], 2048KB chunk

unused devices:


--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Antw: Problems creating MD-RAID1: "device .. not suitablefor any style of raid array"

>>> Ulrich Windl schrieb am 12.05.2011 um 08:07 in Nachricht <4DCB793A.C19 : 161 :
60728>:

The problem seems to be a mis-calculated bitmap size or position, indeed:

May 12 08:06:00 hostname kernel: [55956.959794] md3: bitmap file is out of date
(0 < 70) -- forcing full recovery
May 12 08:06:00 hostname kernel: [55956.959798] md3: bitmap file is out of
date, doing full recovery
May 12 08:06:00 hostname kernel: [55957.054583] attempt to access beyond end of
device
May 12 08:06:00 hostname kernel: [55957.054585] dm-16: rw=192, want=62914561,
limit=62914560
May 12 08:06:00 hostname kernel: [55957.054587] attempt to access beyond end of
device
May 12 08:06:00 hostname kernel: [55957.054588] dm-9: rw=192, want=62914561,
limit=62914560
May 12 08:06:00 hostname kernel: [55957.054589] md3: bitmap initialisation
failed: -5
May 12 08:06:00 hostname kernel: [55957.055058] md: couldn't update array info.
-5

[...]
> P.S. Amazing things happen now and then:
> # cat /proc/mdstat
> Personalities : [linear] [raid1]
> md2 : active raid1 dm-17[1] dm-10[0]
> 209715136 blocks [2/2] [UU]
>
> md1 : active raid1 dm-12[0] dm-14[1]
> 20971456 blocks [2/2] [UU]
>
> md3 : active raid1 dm-16[0] dm-9[1]
> 31457216 blocks [2/2] [UU]
>
> md0 : active raid1 dm-11[0] dm-8[1]
> 525336512 blocks [2/2] [UU]
>
> unused devices:
> # mdadm --grow --bitmap=internal /dev/md0
> # mdadm --grow --bitmap internal /dev/md1
> # mdadm --grow --bitmap internal /dev/md2
> # mdadm --grow --bitmap=internal /dev/md3
> mdadm: failed to set internal bitmap.
> # cat /proc/mdstat
> Personalities : [linear] [raid1]
> md2 : active raid1 dm-17[1] dm-10[0]
> 209715136 blocks [2/2] [UU]
> bitmap: 0/200 pages [0KB], 512KB chunk
>
> md1 : active raid1 dm-12[0] dm-14[1]
> 20971456 blocks [2/2] [UU]
> bitmap: 0/160 pages [0KB], 64KB chunk
>
> md3 : active raid1 dm-16[0] dm-9[1]
> 31457216 blocks [2/2] [UU]
>
> md0 : active raid1 dm-11[0] dm-8[1]
> 525336512 blocks [2/2] [UU]
> bitmap: 0/126 pages [0KB], 2048KB chunk
>
> unused devices:
>
>




--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html