RAID5 Shrinking array-size nearly killed the system

Using Ubuntu 10.10 mdadm v 3.2. ext4 filesystem. I wanted to shrin=
k
from 6 disks to 4. I have about 2TB of files on the disks. So, I ran
$ sudo mdadm -G -n 4 /dev/md0
which gave the message:

mdadm: this change will reduce the size of the array.
       use --grow --array-size first to truncate ar=
ray.
       e.g. mdadm --grow /dev/md0 --array-size 5857=
612608

then ran
$ sudo mdadm --grow /dev/md0 --array-size 5857612608
and started testing the filesystem prior to reducing the array. I
quickly found out that the filesystem was broken. It was broken enough
that I couldn't even get access to commands, including sudo, mdadm,
reboot, and ls. I had to power down the system and restart it. There
were a number of disk errors, but managed to restart the system and it
looks like there wasn't much damage. fstab is listed after my
questions.
Questions:
1. Is there a safer way to shrink the file system prior to reducing
the number of disks in the array?
2. Is there a way of rearranging the files and directories to make the
file system shrink safer?
3. Is there something I did that caused the crash?
Thanks-Rory


# /etc/fstab: static file system information.
#
# Use 'blkid -o value -s UUID' to print the universally unique identifi=
er
# for a device; this may be used with UUID=3D as a more robust way to n=
ame
# devices that works even if disks are added and removed. See fstab(5).
#
#        =
   
proc            /proc      =
    proc    nodev,noexec,nosuid 0     =C2=
=A0 0
# / was on /dev/md0 during installation
UUID=3D9a978b70-e034-4d79-9e1d-237a67b553d5 /        =
      ext4
commit=3D60,errors=3Dremount-ro,usrjquota=3Daquota.user,grpj quota=3Daqu=
ota.group,jqfmt=3Dvfsv1,barrier=3D0
 0       1
# /boot was on /dev/sdb1 during installation
UUID=3D5beb5144-6d2f-4a73-b9e9-442355d8f529 /boot       =
    ext2
defaults        0       2
# swap was on /dev/sda1 during installation
UUID=3D31660a21-3f99-4ffb-81cc-501dc6ce5de7 none       =C2=
=A0    swap    sw
           0       0
# swap was on /dev/sdc1 during installation
UUID=3Df168588d-8b9e-45c3-b9ae-f90f66906616 none       =C2=
=A0    swap    sw
           0       0
# swap was on /dev/sdd1 during installation
UUID=3D05cadc48-59df-479e-b5ed-b9e9322cb905 none       =C2=
=A0    swap    sw
           0       0
# swap was on /dev/sde1 during installation
UUID=3D61fba94d-e6c5-4a58-b0cd-9d878b55b65c none       =C2=
=A0    swap    sw
           0       0
# swap was on /dev/sdf1 during installation
UUID=3D47737641-7555-4cbc-9bf6-508c9f2035bc none       =C2=
=A0    swap    sw
           0       0
# swap was on /dev/sdg1 during installation
UUID=3Dad06f3d6-a6ec-445a-bcfb-427fec72725b none       =C2=
=A0    swap    sw
           0       0
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: RAID5 Shrinking array-size nearly killed the system

On Fri, 11 Mar 2011, Rory Jaffe wrote:

> 3. Is there something I did that caused the crash?

Mdadm has no concept of filesystem, it works on the block device.

From your text you give no indication that you shrunk the filesystem
before shrinking the block device. Did you?

--
Mikael Abrahamsson email: swmike@swm.pp.se
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: RAID5 Shrinking array-size nearly killed the system

On 03/12/2011 12:56 AM, Mikael Abrahamsson wrote:
> On Fri, 11 Mar 2011, Rory Jaffe wrote:
>
>> 3. Is there something I did that caused the crash?
>
> Mdadm has no concept of filesystem, it works on the block device.
>
> From your text you give no indication that you shrunk the filesystem before shrinking the block device. Did you?
>

Also, you didn't say what kind of filesystem it is. Some don't support shrinking at all (XFS is one), or don't support shrinking while running (most of them).

You will very likely need to do this from a LiveCD of one form or another. If your filesystem cannot be shrunk, you'll need to make a fresh backup, reformat, and restore.

(You do have a backup, don't you?)

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[PATCH] Add more warnings to --grow documentation (was: RAID5 Shrinkingarray-size nearly killed the

[CC restored. Always use reply-to-all on kernel.org lists.]

On 03/12/2011 11:07 AM, Rory Jaffe wrote:
> It's ext4. I've backed up the critical data, so a complete loss of the
> files won't be a catastrophe, just a major pain. I was planning to
> unmount the filesystem, shrink it, remount it, then proceed with
> shrinking the block device.
>
> Mikael also asked about what I would change in the documentation. I'd
> change the following two paragraphs (my changes are in all caps).
>
> Note that when an array changes size, any filesystem that may be stored
> in the array will not automatically grow OR SHRINK to use
> the space. The
> filesystem will need to be explicitly told to use the extra space
> AFTER GROWING THE ARRAY, OR TO REDUCE ITS SIZE PRIOR TO SHRINKING THE
> ARRAY.
>
> When decreasing the number of devices, the size of the array will also
> decrease. If there was data in the array, it could get destroyed and
> this is not reversible. FIRST, SHRINK THE FILESYSTEMS ON THE
> ARRAY TO ACCOMODATE THE NEW SIZE. To help prevent accidents, mdadm
> requires that
> the size of the array be decreased first with mdadm --grow --array-
> size. This is a reversible change which simply makes the end of the
> array inaccessible. The integrity of any data can then be checked
> before the non-reversible reduction in the number of devices is
> request.

I've incorporated some of the above suggestions in the following patch,
paraphrased somewhat, with additional text in the summary section.

8<======================
From 027600b0db6bb9043fc5141a5ad6db32f5ba5ab5 Mon Sep 17 00:00:00 2001
From: Philip J. Turmel
Date: Sat, 12 Mar 2011 12:30:23 -0500
Subject: [PATCH] Grow: Improve the documentation of shrinking

Users often report difficulties caused by shrinking an array
without having shrunk the contained filesystem. Add a note
warning about this in the summary description for --grow, and
elaborate in the SIZE CHANGES subsection of the GROW mode
detailed description.

Inspired-by: Rory Jaffe
Signed-off-by: Philip J. Turmel
---
mdadm.8.in | 15 +++++++++++++--
1 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/mdadm.8.in b/mdadm.8.in
index 08e4255..43c2022 100644
--- a/mdadm.8.in
+++ b/mdadm.8.in
@@ -126,6 +126,13 @@ of component devices and changing the number of active devices in RAID
levels 1/4/5/6, changing the RAID level between 1, 5, and 6, changing
the chunk size and layout for RAID5 and RAID5, as well as adding or
removing a write-intent bitmap.
+.B "Note:"
+The contained filesystem, if any, is
+.B NOT
+adjusted automatically.
+Shrinking without prior filesystem adjustment
+.B WILL
+do irreversible damage.

.TP
.B "Incremental Assembly"
@@ -2158,8 +2165,12 @@ space to start being used. If the size is increased in this way, a
are synchronised.

Note that when an array changes size, any filesystem that may be
-stored in the array will not automatically grow to use the space. The
-filesystem will need to be explicitly told to use the extra space.
+stored in the array will not automatically grow to use the space.
+Nor will it automatically shrink to fit a smaller size. The
+filesystem will need to be explicitly told to use the new size.
+.B "Note:"
+Some filesystems can not be shrunk at all. Most filesystems can not
+be shrunk while mounted.

Also the size of an array cannot be changed while it has an active
bitmap. If an array has a bitmap, it must be removed before the size
--
1.7.4.1


--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: RAID5 Shrinking array-size nearly killed the system

[CC restored]

On 03/12/2011 12:37 PM, Rory Jaffe wrote:
> This is my plan now--did I get this right? -- thanks --
>
> shutdown -r now # go to live cd
> umount /dev/md0 #just to make sure
> e2fsck /dev/md0
> resize2fs /dev/md0 3800G #3.2T currently in use
> shutdown -r now # go back to main system
> mdadm --grow /dev/md0 --array-size 4000000000
> mdadm -G -n 4 -x 2 --backup-file=/path/to/file.bak /dev/md0
> resize2fs /dev/md0

I would do everything in the LiveCD environment, and I would add an fsck after the resize, and again at the end.

In the LiveCD, there's a good chance the array will be assembled for you, but as a different number. That shouldn't cause any problems, but it affects the commands you'll type. "cat /proc/mdstat" will give you a quick summary of where you stand.

I can't comment on the size figures you've chosen, as you haven't shared the output of "mdadm -D /dev/md0" and "mdadm -E" for each of the component devices.

Also note that the backup file needed by mdadm cannot be *inside* the array you are resizing. You *must* have another storage device for it. I use a thumb drive with my LiveCD for this kind of task.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: RAID5 Shrinking array-size nearly killed the system

On Sat, Mar 12, 2011 at 5:58 PM, Phil Turmel wrote:
>
> [CC restored]
>
> On 03/12/2011 12:37 PM, Rory Jaffe wrote:
> > This is my plan now--did I get this right? -- thanks --
> >
> > shutdown -r now # go to live cd
> > umount /dev/md0 #just to make sure
> > e2fsck /dev/md0
> > resize2fs /dev/md0 3800G #3.2T currently in use
> > shutdown -r now # go back to main system
> > mdadm --grow /dev/md0 --array-size 4000000000
> > mdadm -G -n 4 -x 2 --backup-file=3D/path/to/file.bak /dev/md0
> > resize2fs /dev/md0
>
> I would do everything in the LiveCD environment, and I would add an f=
sck after the resize, and again at the end.
>
> In the LiveCD, there's a good chance the array will be assembled for =
you, but as a different number.  That shouldn't cause any problems=
, but it affects the commands you'll type.  "cat /proc/mdstat" wil=
l give you a quick summary of where you stand.
>
> I can't comment on the size figures you've chosen, as you haven't sha=
red the output of "mdadm -D /dev/md0" and "mdadm -E" for each of the co=
mponent devices.
>
> Also note that the backup file needed by mdadm cannot be *inside* the=
array you are resizing.  You *must* have another storage device f=
or it.  I use a thumb drive with my LiveCD for this kind of task.
>
> Phil
Here's the data on array sizes
sudo mdadm -D /dev/md/0_0
/dev/md/0_0:
Version : 0.90
Creation Time : Thu Jan 6 06:13:08 2011
Raid Level : raid5
Array Size : 9762687680 (9310.42 GiB 9996.99 GB)
Used Dev Size : 1952537536 (1862.08 GiB 1999.40 GB)
Raid Devices : 6
Total Devices : 6
Preferred Minor : 127
Persistence : Superblock is persistent

Update Time : Sat Mar 12 17:56:34 2011
State : clean
Active Devices : 6
Working Devices : 6
Failed Devices : 0
Spare Devices : 0

Layout : left-symmetric
Chunk Size : 64K

UUID : 7e946e9d:b6a3395c:b57e8a13:68af0467
Events : 0.72

Number Major Minor RaidDevice State
0 8 2 0 active sync /dev/sda2
1 8 50 1 active sync /dev/sdd2
2 8 66 2 active sync /dev/sde2
3 8 82 3 active sync /dev/sdf2
4 8 98 4 active sync /dev/sdg2
5 8 114 5 active sync /dev/sdh2
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: RAID5 Shrinking array-size nearly killed the system

On 03/12/2011 01:28 PM, Rory Jaffe wrote:
> On Sat, Mar 12, 2011 at 5:58 PM, Phil Turmel wrote:
>
>> [CC restored]
>>
>> On 03/12/2011 12:37 PM, Rory Jaffe wrote:
>>> This is my plan now--did I get this right? -- thanks --
>>>
>>> shutdown -r now # go to live cd
>>> umount /dev/md0 #just to make sure
>>> e2fsck /dev/md0
>>> resize2fs /dev/md0 3800G #3.2T currently in use
>>> shutdown -r now # go back to main system
>>> mdadm --grow /dev/md0 --array-size 4000000000
>>> mdadm -G -n 4 -x 2 --backup-file=/path/to/file.bak /dev/md0
>>> resize2fs /dev/md0
>>
>> I would do everything in the LiveCD environment, and I would add an fsck
>> after the resize, and again at the end.
>>
>> In the LiveCD, there's a good chance the array will be assembled for you,
>> but as a different number. That shouldn't cause any problems, but it
>> affects the commands you'll type. "cat /proc/mdstat" will give you a quick
>> summary of where you stand.
>>
>> I can't comment on the size figures you've chosen, as you haven't shared
>> the output of "mdadm -D /dev/md0" and "mdadm -E" for each of the component
>> devices.
>>
>> Also note that the backup file needed by mdadm cannot be *inside* the array
>> you are resizing. You *must* have another storage device for it. I use a
>> thumb drive with my LiveCD for this kind of task.
>>
>> Phil
>>
> Here's the data on array sizes
> sudo mdadm -D /dev/md/0_0
> /dev/md/0_0:
> Version : 0.90
> Creation Time : Thu Jan 6 06:13:08 2011
> Raid Level : raid5
> Array Size : 9762687680 (9310.42 GiB 9996.99 GB)
> Used Dev Size : 1952537536 (1862.08 GiB 1999.40 GB)
> Raid Devices : 6
> Total Devices : 6
> Preferred Minor : 127
> Persistence : Superblock is persistent
>
> Update Time : Sat Mar 12 17:56:34 2011
> State : clean
> Active Devices : 6
> Working Devices : 6
> Failed Devices : 0
> Spare Devices : 0
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> UUID : 7e946e9d:b6a3395c:b57e8a13:68af0467
> Events : 0.72
>
> Number Major Minor RaidDevice State
> 0 8 2 0 active sync /dev/sda2
> 1 8 50 1 active sync /dev/sdd2
> 2 8 66 2 active sync /dev/sde2
> 3 8 82 3 active sync /dev/sdf2
> 4 8 98 4 active sync /dev/sdg2
> 5 8 114 5 active sync /dev/sdh2
>

OK, so your new array size will be 5857612608 (1952537536 * 3) == 5586GB

You can use an initial resize2fs to 5.4T to speed things up, as you don't really need to move items that are currently located between the 3.8T and 5.4T mark. Then use the exact "mdadm --grow /dev/md0 --array-size=5857612608" afterwards before you fsck it.

If that passes, do the rest. The final resize2fs should be very quick.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: RAID5 Shrinking array-size nearly killed the system

> OK, so your new array size will be 5857612608 (1952537536 * 3) ===
5586GB
>
> You can use an initial resize2fs to 5.4T to speed things up, as you d=
on't really need to move items that are currently located between the 3=
8T and 5.4T mark.  Then use the exact "mdadm --grow /dev/md0 --=
array-size=3D5857612608" afterwards before you fsck it.
>
> If that passes, do the rest.  The final resize2fs should be very=
quick.
>
> Phil
>

One more glitch? I ran the following command, trying several different
locations for the backup file, all of which have plenty of space and
are not on the array.

sudo mdadm -G /dev/md/0_0 -n 4 --backup-file=3D/tmp/backmd

mdadm gives the message "mdadm: Need to backup 960K of critical
section.." and it immediately returns to the command prompt without
shrinking the array.

=46ollowing is the current array info

ubuntu@ubuntu:~/mdadm-3.2$ sudo mdadm -D /dev/md/0_0
/dev/md/0_0:
Version : 0.90
Creation Time : Thu Jan 6 06:13:08 2011
Raid Level : raid5
Array Size : 5857612608 (5586.25 GiB 5998.20 GB)
Used Dev Size : 1952537536 (1862.08 GiB 1999.40 GB)
Raid Devices : 6
Total Devices : 6
Preferred Minor : 127
Persistence : Superblock is persistent

Update Time : Sun Mar 13 06:46:37 2011
State : clean
Active Devices : 6
Working Devices : 6
Failed Devices : 0
Spare Devices : 0

Layout : left-symmetric
Chunk Size : 64K

UUID : 7e946e9d:b6a3395c:b57e8a13:68af0467
Events : 0.73

Number Major Minor RaidDevice State
0 8 2 0 active sync /dev/sda2
1 8 50 1 active sync /dev/sdd2
2 8 66 2 active sync /dev/sde2
3 8 82 3 active sync /dev/sdf2
4 8 98 4 active sync /dev/sdg2
5 8 114 5 active sync /dev/sdh2
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: RAID5 Shrinking array-size nearly killed the system

Hi Rory,

On 03/13/2011 01:56 AM, Rory Jaffe wrote:
>> OK, so your new array size will be 5857612608 (1952537536 * 3) == 5586GB
>>
>> You can use an initial resize2fs to 5.4T to speed things up, as you don't really need to move items that are currently located between the 3.8T and 5.4T mark. Then use the exact "mdadm --grow /dev/md0 --array-size=5857612608" afterwards before you fsck it.
>>
>> If that passes, do the rest. The final resize2fs should be very quick.
>>
>> Phil
>>
>
> One more glitch? I ran the following command, trying several different
> locations for the backup file, all of which have plenty of space and
> are not on the array.
>
> sudo mdadm -G /dev/md/0_0 -n 4 --backup-file=/tmp/backmd
>
> mdadm gives the message "mdadm: Need to backup 960K of critical
> section.." and it immediately returns to the command prompt without
> shrinking the array.

Are you sure its not doing the reshape? "cat /proc/mdstat" will show whats happening in the background.

Also, check your dmesg to see if there are any explanatory messages.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: RAID5 Shrinking array-size nearly killed the system

>> One more glitch? I ran the following command, trying several differe=
nt
>> locations for the backup file, all of which have plenty of space and
>> are not on the array.
>>
>> sudo mdadm -G /dev/md/0_0 -n 4 --backup-file=3D/tmp/backmd
>>
>> mdadm gives the message "mdadm: Need to backup 960K of critical
>> section.." and it immediately returns to the command prompt without
>> shrinking the array.
>
> Are you sure its not doing the reshape?  "cat /proc/mdstat" will=
show whats happening in the background.
>
> Also, check your dmesg to see if there are any explanatory messages.
>
> Phil
>
I tried again, with the same results. Details follow:

To assemble the array, I used
ubuntu@ubuntu:~/mdadm-3.2$ sudo mdadm --assemble --scan
then
I resynced the array.
then
ubuntu@ubuntu:~/mdadm-3.2$ sudo mdadm --grow /dev/md127 --array-size 58=
57612608
then
ubuntu@ubuntu:~/mdadm-3.2$ sudo mdadm -G -n 4 --backup-file=3Dmdbak /de=
v/md127
and again received the messages:
ubuntu@ubuntu:~/mdadm-3.2$ sudo mdadm -G -n 4 --backup-file=3Dmdback /d=
ev/md127
mdadm: Need to backup 960K of critical section..
ubuntu@ubuntu:~/mdadm-3.2$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md127 : active raid5 sda2[0] sdh2[5] sdg2[4] sdf2[3] sde2[2] sdd2[1]
5857612608 blocks level 5, 64k chunk, algorithm 2 [6/6] [UUUUUU]

unused devices:
ubuntu@ubuntu:~/mdadm-3.2$ mdadm -V
mdadm - v3.2 DEVELOPER_ONLY - 1st February 2011 (USE WITH CARE)


The following appear to be the relevant parts of dmesg--

[ 758.516860] md: md127 stopped.
[ 758.522499] md: bind
[ 758.523731] md: bind
[ 758.525170] md: bind
[ 758.525588] md: bind
[ 758.526003] md: bind
[ 758.526748] md: bind
[ 758.567380] async_tx: api initialized (async)
[ 758.740173] raid6: int64x1 335 MB/s
[ 758.910051] raid6: int64x2 559 MB/s
[ 759.080062] raid6: int64x4 593 MB/s
[ 759.250058] raid6: int64x8 717 MB/s
[ 759.420148] raid6: sse2x1 437 MB/s
[ 759.590013] raid6: sse2x2 599 MB/s
[ 759.760037] raid6: sse2x4 634 MB/s
[ 759.760044] raid6: using algorithm sse2x4 (634 MB/s)
[ 759.793413] md: raid6 personality registered for level 6
[ 759.793423] md: raid5 personality registered for level 5
[ 759.793429] md: raid4 personality registered for level 4
[ 759.798708] md/raid:md127: device sda2 operational as raid disk 0
[ 759.798720] md/raid:md127: device sdh2 operational as raid disk 5
[ 759.798729] md/raid:md127: device sdg2 operational as raid disk 4
[ 759.798739] md/raid:md127: device sdf2 operational as raid disk 3
[ 759.798747] md/raid:md127: device sde2 operational as raid disk 2
[ 759.798756] md/raid:md127: device sdd2 operational as raid disk 1
[ 759.800722] md/raid:md127: allocated 6386kB
[ 759.810239] md/raid:md127: raid level 5 active with 6 out of 6
devices, algorithm 2
[ 759.810249] RAID conf printout:
[ 759.810255] --- level:5 rd:6 wd:6
[ 759.810263] disk 0, o:1, dev:sda2
[ 759.810271] disk 1, o:1, dev:sdd2
[ 759.810278] disk 2, o:1, dev:sde2
[ 759.810285] disk 3, o:1, dev:sdf2
[ 759.810293] disk 4, o:1, dev:sdg2
[ 759.810300] disk 5, o:1, dev:sdh2
[ 759.810416] md127: detected capacity change from 0 to 9996992184320
[ 759.825149] md127: unknown partition table
[ 810.381494] md127: detected capacity change from 9996992184320 to
5998195310592
[ 810.384868] md127: unknown partition table

and here is the information about the array.
sudo mdadm -D /dev/md127
/dev/md127:
Version : 0.90
Creation Time : Thu Jan 6 06:13:08 2011
Raid Level : raid5
Array Size : 5857612608 (5586.25 GiB 5998.20 GB)
Used Dev Size : 1952537536 (1862.08 GiB 1999.40 GB)
Raid Devices : 6
Total Devices : 6
Preferred Minor : 127
Persistence : Superblock is persistent

Update Time : Tue Mar 15 00:45:28 2011
State : clean
Active Devices : 6
Working Devices : 6
Failed Devices : 0
Spare Devices : 0

Layout : left-symmetric
Chunk Size : 64K

UUID : 7e946e9d:b6a3395c:b57e8a13:68af0467
Events : 0.76

Number Major Minor RaidDevice State
0 8 2 0 active sync /dev/sda2
1 8 50 1 active sync /dev/sdd2
2 8 66 2 active sync /dev/sde2
3 8 82 3 active sync /dev/sdf2
4 8 98 4 active sync /dev/sdg2
5 8 114 5 active sync /dev/sdh2
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: RAID5 Shrinking array-size nearly killed the system

On Tue, 15 Mar 2011 05:26:44 +0000 Rory Jaffe wrote=
:

> >> One more glitch? I ran the following command, trying several diffe=
rent
> >> locations for the backup file, all of which have plenty of space a=
nd
> >> are not on the array.
> >>
> >> sudo mdadm -G /dev/md/0_0 -n 4 --backup-file=3D/tmp/backmd
> >>
> >> mdadm gives the message "mdadm: Need to backup 960K of critical
> >> section.." and it immediately returns to the command prompt withou=
t
> >> shrinking the array.
> >
> > Are you sure its not doing the reshape? =A0"cat /proc/mdstat" will =
show whats happening in the background.
> >
> > Also, check your dmesg to see if there are any explanatory messages=

> >
> > Phil
> >
> I tried again, with the same results. Details follow:
>=20
> To assemble the array, I used
> ubuntu@ubuntu:~/mdadm-3.2$ sudo mdadm --assemble --scan
> then
> I resynced the array.
> then
> ubuntu@ubuntu:~/mdadm-3.2$ sudo mdadm --grow /dev/md127 --array-size =
5857612608
> then
> ubuntu@ubuntu:~/mdadm-3.2$ sudo mdadm -G -n 4 --backup-file=3Dmdbak /=
dev/md127
> and again received the messages:
> ubuntu@ubuntu:~/mdadm-3.2$ sudo mdadm -G -n 4 --backup-file=3Dmdback =
/dev/md127
> mdadm: Need to backup 960K of critical section..
> ubuntu@ubuntu:~/mdadm-3.2$ cat /proc/mdstat
> Personalities : [raid6] [raid5] [raid4]
> md127 : active raid5 sda2[0] sdh2[5] sdg2[4] sdf2[3] sde2[2] sdd2[1]
> 5857612608 blocks level 5, 64k chunk, algorithm 2 [6/6] [UUUUUU=
]
>=20
> unused devices:
> ubuntu@ubuntu:~/mdadm-3.2$ mdadm -V
> mdadm - v3.2 DEVELOPER_ONLY - 1st February 2011 (USE WITH CARE)
^^^^^^^^^^^^^^ ^^^^^^^^^^^^^

I guess you must be a developer, so probably don't need any help....

But may I suggest trying mdadm-3.1.4 instead??

NeilBrown




>=20
>=20
> The following appear to be the relevant parts of dmesg--
>=20
> [ 758.516860] md: md127 stopped.
> [ 758.522499] md: bind
> [ 758.523731] md: bind
> [ 758.525170] md: bind
> [ 758.525588] md: bind
> [ 758.526003] md: bind
> [ 758.526748] md: bind
> [ 758.567380] async_tx: api initialized (async)
> [ 758.740173] raid6: int64x1 335 MB/s
> [ 758.910051] raid6: int64x2 559 MB/s
> [ 759.080062] raid6: int64x4 593 MB/s
> [ 759.250058] raid6: int64x8 717 MB/s
> [ 759.420148] raid6: sse2x1 437 MB/s
> [ 759.590013] raid6: sse2x2 599 MB/s
> [ 759.760037] raid6: sse2x4 634 MB/s
> [ 759.760044] raid6: using algorithm sse2x4 (634 MB/s)
> [ 759.793413] md: raid6 personality registered for level 6
> [ 759.793423] md: raid5 personality registered for level 5
> [ 759.793429] md: raid4 personality registered for level 4
> [ 759.798708] md/raid:md127: device sda2 operational as raid disk 0
> [ 759.798720] md/raid:md127: device sdh2 operational as raid disk 5
> [ 759.798729] md/raid:md127: device sdg2 operational as raid disk 4
> [ 759.798739] md/raid:md127: device sdf2 operational as raid disk 3
> [ 759.798747] md/raid:md127: device sde2 operational as raid disk 2
> [ 759.798756] md/raid:md127: device sdd2 operational as raid disk 1
> [ 759.800722] md/raid:md127: allocated 6386kB
> [ 759.810239] md/raid:md127: raid level 5 active with 6 out of 6
> devices, algorithm 2
> [ 759.810249] RAID conf printout:
> [ 759.810255] --- level:5 rd:6 wd:6
> [ 759.810263] disk 0, o:1, dev:sda2
> [ 759.810271] disk 1, o:1, dev:sdd2
> [ 759.810278] disk 2, o:1, dev:sde2
> [ 759.810285] disk 3, o:1, dev:sdf2
> [ 759.810293] disk 4, o:1, dev:sdg2
> [ 759.810300] disk 5, o:1, dev:sdh2
> [ 759.810416] md127: detected capacity change from 0 to 999699218432=
0
> [ 759.825149] md127: unknown partition table
> [ 810.381494] md127: detected capacity change from 9996992184320 to
> 5998195310592
> [ 810.384868] md127: unknown partition table
>=20
> and here is the information about the array.
> sudo mdadm -D /dev/md127
> /dev/md127:
> Version : 0.90
> Creation Time : Thu Jan 6 06:13:08 2011
> Raid Level : raid5
> Array Size : 5857612608 (5586.25 GiB 5998.20 GB)
> Used Dev Size : 1952537536 (1862.08 GiB 1999.40 GB)
> Raid Devices : 6
> Total Devices : 6
> Preferred Minor : 127
> Persistence : Superblock is persistent
>=20
> Update Time : Tue Mar 15 00:45:28 2011
> State : clean
> Active Devices : 6
> Working Devices : 6
> Failed Devices : 0
> Spare Devices : 0
>=20
> Layout : left-symmetric
> Chunk Size : 64K
>=20
> UUID : 7e946e9d:b6a3395c:b57e8a13:68af0467
> Events : 0.76
>=20
> Number Major Minor RaidDevice State
> 0 8 2 0 active sync /dev/sda2
> 1 8 50 1 active sync /dev/sdd2
> 2 8 66 2 active sync /dev/sde2
> 3 8 82 3 active sync /dev/sdf2
> 4 8 98 4 active sync /dev/sdg2
> 5 8 114 5 active sync /dev/sdh2
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid"=
in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: RAID5 Shrinking array-size nearly killed the system

> ubuntu@ubuntu:~/mdadm-3.2$ mdadm -V
>> mdadm - v3.2 DEVELOPER_ONLY - 1st February 2011 (USE WITH CARE)
>               ^^^^^^^^^^^^^^  =
                   ^=
^^^^^^^^^^^^
>
> I guess you must be a developer, so probably don't need any help....
>
> But may I suggest trying mdadm-3.1.4 instead??
>
> NeilBrown
Well, I was developing a headache. Does that qualify me as a developer?

Seriously, point well-taken. I've now installed 3.1.4. 3.1.4 seems to
be doing the job correctly.

It's now reshaping.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html