Incompatibility of internal bitmap with ext4 barriers?

Incompatibility of internal bitmap with ext4 barriers?

am 17.05.2011 14:10:07 von Jason Tinker

--e0cb4e43cd551fb9f304a377a7b6
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hello

I have encountered a weird bug when trying to use ext4 partition on
top of mdadm RAID5 array. Mdadm array has internal bitmap, and ext4 is
mounted with default options - which means that barriers are enabled.
When trying to write large enough amount of files system just locks up
indefinitely, only hard reset helps. This seems to happen at random
times, yet consistently after several minutes of usage.
I tried different configuration options and it seems to happen only
when both barriers on ext4 and mdadm's internal bitmaps are enabled.
After disabling ext4 barriers for good (it seemed like a lesser evil)
no lock ups happened for 3 months.
Mdadm is version 3.1.3, kernel is 2.6.32 (rhel6)

Initialy it happened on this configuration:
1 RAID0 of 2x1TB drives + 1 RAID0 of 2x1TB drives +1x2TB drive
Each array had 1 hdd on PCI SATA (SiliconImage) controller and 1 on
internal ICH7 (Intel G41 chipset), 2Tb drive was on PCI SATA
controller.

Later I successfully reproduced the same bug in a test setup with all
partitions on a single drive.
The following steps where taken:
1. Created 6 identical blank partitions (I made 6*1GB partitions on a
single hdd).
2. Created 3 RAID0 arrays from these partitions: 0&1, 2&3, 4&5.
3. Created MBR and blank primary partition on each of these arrays
using fdisk (this step is
probably optional).
4. Created 1 RAID5 array from these 3 partitions with --bitmap=3Dinternal,
everything else default.
5. Created ext4 filesystem on RAID5 with default options, mount with defaul=
t
options (barriers are enabled by default).
6. Tried to rsync several hundred of ~1-20 MB files to mounted directory.

/var/log/messages at the moment of lock up:

kernel: INFO: task md90_raid5:13736 blocked for more than 120 seconds.
kernel: "echo 0 > /proc/sys/kernel/hung_task_
timeout_secs" disables this message.
kernel: md90_raid5  =A0 D 0000000000000000     0 13736  =A0=
   2 0x00000080
kernel: ffff88003e0b7ab0 0000000000000046 ffff88006d273e60 ffff8800716b8240
kernel: ffff88003e0b7a50 ffffffff8123a553 ffff880072ea4800 0000000000000810
kernel: ffff88006f50ba98 ffff88003e0b7fd8 0000000000010518 ffff88006f50ba98
kernel: Call Trace:
kernel: [] ? elv_insert+0x133/0x1f0
kernel: [] ? prepare_to_wait+0x4e/0x80
kernel: [] md_make_request+0x85/0x230
kernel: [] ? autoremove_wake_function+0x0/0x40
kernel: [] ? generic_make_request+0x1b2/0x4f0
kernel: [] generic_make_request+0x1b2/0x4f0
kernel: [] ? find_busiest_group+0x96a/0xb40
kernel: [] ops_run_io+0x22d/0x330 [raid456]
kernel: [] ? md_super_write+0xd6/0xe0
kernel: [] handle_stripe+0x4d5/0x22e0 [raid456]
kernel: [] ? finish_task_switch+0x42/0xd0
kernel: [] raid5d+0x49f/0x690 [raid456]
kernel: [] md_thread+0x5c/0x130
kernel: [] ? autoremove_wake_function+0x0/0x40
kernel: [] ? md_thread+0x0/0x130
kernel: [] kthread+0x96/0xa0
kernel: [] child_rip+0xa/0x20
kernel: [] ? kthread+0x0/0xa0
kernel: [] ? child_rip+0x0/0x20

I have included all additional info and logs about test setup in
separate attachments.
According to logs it seems that the bug is in mdadm, but I'm not sure
since I haven't found any similar reports anywhere.
It would be great if someone tried to reproduce it on their machine,
this shouldn't take long, maybe 20 minutes or so...

--e0cb4e43cd551fb9f304a377a7b6
Content-Type: application/octet-stream; name=fdisk-info
Content-Disposition: attachment; filename=fdisk-info
Content-Transfer-Encoding: base64
X-Attachment-Id: f_gnssr6es0

RGlzayAvZGV2L3NkYjogMTAwMC4yIEdCLCAxMDAwMjAzODA0MTYwIGJ5dGVz CjI1NSBoZWFkcywg
NjMgc2VjdG9ycy90cmFjaywgMTIxNjAxIGN5bGluZGVycywgdG90YWwgMTk1 MzUyMzA1NSBzZWN0
b3JzClVuaXRzID0gc2VjdG9ycyBvZiAxICogNTEyID0gNTEyIGJ5dGVzClNl Y3RvciBzaXplIChs
b2dpY2FsL3BoeXNpY2FsKTogNTEyIGJ5dGVzIC8gNTEyIGJ5dGVzCkkvTyBz aXplIChtaW5pbXVt
L29wdGltYWwpOiA1MTIgYnl0ZXMgLyA1MTIgYnl0ZXMKRGlzayBpZGVudGlm aWVyOiAweDAwMDBj
MWIzCgogICBEZXZpY2UgQm9vdCAgICAgIFN0YXJ0ICAgICAgICAgRW5kICAg ICAgQmxvY2tzICAg
SWQgIFN5c3RlbQovZGV2L3NkYjEgICAqICAgICAgICAyMDQ4ICAgICAxMDUw NjIzICAgICAgNTI0
Mjg4ICAgODMgIExpbnV4Ci9kZXYvc2RiMiAgICAgICAgIDEwNTA2MjQgICAg NjM5NjUxODMgICAg
MzE0NTcyODAgICA4MyAgTGludXgKL2Rldi9zZGIzICAgICAgICA2Mzk2NTE4 NCAgIDEyNjg3OTc0
MyAgICAzMTQ1NzI4MCAgIDgzICBMaW51eAovZGV2L3NkYjQgICAgICAgMTI2 ODc5NzQ0ICAxOTUz
NTIxNjYzICAgOTEzMzIwOTYwICAgIDUgIEV4dGVuZGVkCi9kZXYvc2RiNSAg ICAgICAxMjY4ODM4
NDAgICAxODk3OTgzOTkgICAgMzE0NTcyODAgICA4MyAgTGludXgKL2Rldi9z ZGI2ICAgICAgIDE4
OTgwMDQ0OCAgIDE5ODE4OTA1NSAgICAgNDE5NDMwNCAgIDgyICBMaW51eCBz d2FwIC8gU29sYXJp
cwovZGV2L3NkYjcgICAgICAgMTk4MTkzOTY4ICAgMjAwMTM3NzY5ICAgICAg OTcxOTAxICAgODMg
IExpbnV4Ci9kZXYvc2RiOCAgICAgICAyMDAxMzc4MzMgICAyMDIwOTc2OTkg ICAgICA5Nzk5MzMr
ICA4MyAgTGludXgKL2Rldi9zZGI5ICAgICAgIDIwMjA5Nzc2MyAgIDIwNDA1 NzYyOSAgICAgIDk3
OTkzMysgIDgzICBMaW51eAovZGV2L3NkYjEwICAgICAgMjA0MDU3NjkzICAg MjA2MDE3NTU5ICAg
ICAgOTc5OTMzKyAgODMgIExpbnV4Ci9kZXYvc2RiMTEgICAgICAyMDYwMTc2 MjMgICAyMDc5Nzc0
ODkgICAgICA5Nzk5MzMrICA4MyAgTGludXgKL2Rldi9zZGIxMiAgICAgIDIw Nzk3NzU1MyAgIDIw
OTkzNzQxOSAgICAgIDk3OTkzMysgIDgzICBMaW51eAoK
--e0cb4e43cd551fb9f304a377a7b6
Content-Type: application/octet-stream; name=messages
Content-Disposition: attachment; filename=messages
Content-Transfer-Encoding: base64
X-Attachment-Id: f_gnssr6eu1

RmViICA4IDE5OjA0OjI0IHN0b3JlLWVsNiBrZXJuZWw6IG1kOiBiaW5kPG1k OTEwcDE+CkZlYiAg
OCAxOTowNDoyNCBzdG9yZS1lbDYga2VybmVsOiBtZDogYmluZDxtZDkxMXAx PgpGZWIgIDggMTk6
MDQ6MjQgc3RvcmUtZWw2IGtlcm5lbDogbWQ6IGJpbmQ8bWQ5MTJwMT4KRmVi ICA4IDE5OjA0OjI0
IHN0b3JlLWVsNiBrZXJuZWw6IHJhaWQ1OiBkZXZpY2UgbWQ5MTFwMSBvcGVy YXRpb25hbCBhcyBy
YWlkIGRpc2sgMQpGZWIgIDggMTk6MDQ6MjQgc3RvcmUtZWw2IGtlcm5lbDog cmFpZDU6IGRldmlj
ZSBtZDkxMHAxIG9wZXJhdGlvbmFsIGFzIHJhaWQgZGlzayAwCkZlYiAgOCAx OTowNDoyNCBzdG9y
ZS1lbDYga2VybmVsOiByYWlkNTogYWxsb2NhdGVkIDMyMzBrQiBmb3IgbWQ5 MApGZWIgIDggMTk6
MDQ6MjQgc3RvcmUtZWw2IGtlcm5lbDogMTogdz0xIHBhPTAgcHI9MyBtPTEg YT0yIHI9MyBvcDE9
MCBvcDI9MApGZWIgIDggMTk6MDQ6MjQgc3RvcmUtZWw2IGtlcm5lbDogMDog dz0yIHBhPTAgcHI9
MyBtPTEgYT0yIHI9MyBvcDE9MCBvcDI9MApGZWIgIDggMTk6MDQ6MjQgc3Rv cmUtZWw2IGtlcm5l
bDogcmFpZDU6IHJhaWQgbGV2ZWwgNSBzZXQgbWQ5MCBhY3RpdmUgd2l0aCAy IG91dCBvZiAzIGRl
dmljZXMsIGFsZ29yaXRobSAyCkZlYiAgOCAxOTowNDoyNCBzdG9yZS1lbDYg a2VybmVsOiBSQUlE
NSBjb25mIHByaW50b3V0OgpGZWIgIDggMTk6MDQ6MjQgc3RvcmUtZWw2IGtl cm5lbDogLS0tIHJk
OjMgd2Q6MgpGZWIgIDggMTk6MDQ6MjQgc3RvcmUtZWw2IGtlcm5lbDogZGlz ayAwLCBvOjEsIGRl
djptZDkxMHAxCkZlYiAgOCAxOTowNDoyNCBzdG9yZS1lbDYga2VybmVsOiBk aXNrIDEsIG86MSwg
ZGV2Om1kOTExcDEKRmViICA4IDE5OjA0OjI0IHN0b3JlLWVsNiBrZXJuZWw6 IG1kOTA6IGJpdG1h
cCBpbml0aWFsaXplZCBmcm9tIGRpc2s6IHJlYWQgMS8xIHBhZ2VzLCBzZXQg MzAgYml0cwpGZWIg
IDggMTk6MDQ6MjQgc3RvcmUtZWw2IGtlcm5lbDogY3JlYXRlZCBiaXRtYXAg KDEgcGFnZXMpIGZv
ciBkZXZpY2UgbWQ5MApGZWIgIDggMTk6MDQ6MjQgc3RvcmUtZWw2IGtlcm5l bDogbWQ5MDogZGV0
ZWN0ZWQgY2FwYWNpdHkgY2hhbmdlIGZyb20gMCB0byAzOTg2Njg1OTUyCkZl YiAgOCAxOTowNDoy
NCBzdG9yZS1lbDYga2VybmVsOiBSQUlENSBjb25mIHByaW50b3V0OgpGZWIg IDggMTk6MDQ6MjQg
c3RvcmUtZWw2IGtlcm5lbDogLS0tIHJkOjMgd2Q6MgpGZWIgIDggMTk6MDQ6 MjQgc3RvcmUtZWw2
IGtlcm5lbDogZGlzayAwLCBvOjEsIGRldjptZDkxMHAxCkZlYiAgOCAxOTow NDoyNCBzdG9yZS1l
bDYga2VybmVsOiBkaXNrIDEsIG86MSwgZGV2Om1kOTExcDEKRmViICA4IDE5 OjA0OjI0IHN0b3Jl
LWVsNiBrZXJuZWw6IGRpc2sgMiwgbzoxLCBkZXY6bWQ5MTJwMQpGZWIgIDgg MTk6MDQ6MjQgc3Rv
cmUtZWw2IGtlcm5lbDogbWQ6IHJlY292ZXJ5IG9mIFJBSUQgYXJyYXkgbWQ5 MApGZWIgIDggMTk6
MDQ6MjQgc3RvcmUtZWw2IGtlcm5lbDogbWQ6IG1pbmltdW0gX2d1YXJhbnRl ZWRfICBzcGVlZDog
MTAwMCBLQi9zZWMvZGlzay4KRmViICA4IDE5OjA0OjI0IHN0b3JlLWVsNiBr ZXJuZWw6IG1kOiB1
c2luZyBtYXhpbXVtIGF2YWlsYWJsZSBpZGxlIElPIGJhbmR3aWR0aCAoYnV0 IG5vdCBtb3JlIHRo
YW4gMjAwMDAwIEtCL3NlYykgZm9yIHJlY292ZXJ5LgpGZWIgIDggMTk6MDQ6 MjQgc3RvcmUtZWw2
IGtlcm5lbDogbWQ6IHVzaW5nIDEyOGsgd2luZG93LCBvdmVyIGEgdG90YWwg b2YgMTk0NjYyNCBi
bG9ja3MuCkZlYiAgOCAxOTowNDoyNSBzdG9yZS1lbDYga2VybmVsOiBtZDkw OiB1bmtub3duIHBh
cnRpdGlvbiB0YWJsZQpGZWIgIDggMTk6MDU6NTAgc3RvcmUtZWw2IGtlcm5l bDogRVhUNC1mcyAo
bWQ5MCk6IG1vdW50ZWQgZmlsZXN5c3RlbSB3aXRoIG9yZGVyZWQgZGF0YSBt b2RlCkZlYiAgOCAx
OTowNjozMSBzdG9yZS1lbDYga2VybmVsOiBtZDogbWQ5MDogcmVjb3Zlcnkg ZG9uZS4KRmViICA4
IDE5OjA2OjMyIHN0b3JlLWVsNiBrZXJuZWw6IFJBSUQ1IGNvbmYgcHJpbnRv dXQ6CkZlYiAgOCAx
OTowNjozMiBzdG9yZS1lbDYga2VybmVsOiAtLS0gcmQ6MyB3ZDozCkZlYiAg OCAxOTowNjozMiBz
dG9yZS1lbDYga2VybmVsOiBkaXNrIDAsIG86MSwgZGV2Om1kOTEwcDEKRmVi ICA4IDE5OjA2OjMy
IHN0b3JlLWVsNiBrZXJuZWw6IGRpc2sgMSwgbzoxLCBkZXY6bWQ5MTFwMQpG ZWIgIDggMTk6MDY6
MzIgc3RvcmUtZWw2IGtlcm5lbDogZGlzayAyLCBvOjEsIGRldjptZDkxMnAx CkZlYiAgOCAxOTow
NjozNSBzdG9yZS1lbDYga2VybmVsOiBFWFQ0LWZzIChtZDkwKTogbW91bnRl ZCBmaWxlc3lzdGVt
IHdpdGggb3JkZXJlZCBkYXRhIG1vZGUKCiMKIyBoZXJlIGkgc3RhcnRlZCAi cnN5bmMgLWF2IC9z
b21ld2hlcmUgL21udC90ZXN0LXJhaWQiIGluIGdub21lLXRlcm1pbmFsCiMK CkZlYiAgOCAxOTow
OTozOSBzdG9yZS1lbDYga2VybmVsOiBJTkZPOiB0YXNrIGdub21lLXRlcm1p bmFsOjU0NjMgYmxv
Y2tlZCBmb3IgbW9yZSB0aGFuIDEyMCBzZWNvbmRzLgpGZWIgIDggMTk6MDk6 Mzkgc3RvcmUtZWw2
IGtlcm5lbDogImVjaG8gMCA+IC9wcm9jL3N5cy9rZXJuZWwvaHVuZ190YXNr X3RpbWVvdXRfc2Vj
cyIgZGlzYWJsZXMgdGhpcyBtZXNzYWdlLgpGZWIgIDggMTk6MDk6Mzkgc3Rv cmUtZWw2IGtlcm5l
bDogZ25vbWUtdGVybWluYSBEIGZmZmY4ODAwNDEzZWQ5OTggICAgIDAgIDU0 NjMgICA1NDU5IDB4
MDAwMDAwODAKRmViICA4IDE5OjA5OjM5IHN0b3JlLWVsNiBrZXJuZWw6IGZm ZmY4ODAwNDEzZWQ4
YjggMDAwMDAwMDAwMDAwMDA4NiBmZmZmODgwMDRiZDJlMTM4IGZmZmY4ODAw NGJkMmUxMzgKRmVi
ICA4IDE5OjA5OjM5IHN0b3JlLWVsNiBrZXJuZWw6IGZmZmY4ODAwMDFlOTY5 ODAgZmZmZjg4MDAw
MWU5Njk4MCBmZmZmODgwMDY2ZTQ4MDgwIGZmZmY4ODAwMDFlOTY5ODAKRmVi ICA4IDE5OjA5OjM5
IHN0b3JlLWVsNiBrZXJuZWw6IGZmZmY4ODAwNjZlNDg2MzggZmZmZjg4MDA0 MTNlZGZkOCAwMDAw
MDAwMDAwMDEwNTE4IGZmZmY4ODAwNjZlNDg2MzgKRmViICA4IDE5OjA5OjM5 IHN0b3JlLWVsNiBr
ZXJuZWw6IENhbGwgVHJhY2U6CkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYg a2VybmVsOiBbPGZm
ZmZmZmZmODE0YzlhZDU+XSBzY2hlZHVsZV90aW1lb3V0KzB4MjI1LzB4MmYw CkZlYiAgOCAxOTow
OTozOSBzdG9yZS1lbDYga2VybmVsOiBbPGZmZmZmZmZmODEwMWFlNDU+XSA/ IG5hdGl2ZV9zY2hl
ZF9jbG9jaysweDE1LzB4NzAKRmViICA4IDE5OjA5OjM5IHN0b3JlLWVsNiBr ZXJuZWw6IFs8ZmZm
ZmZmZmY4MTAxYTRjOT5dID8gc2NoZWRfY2xvY2srMHg5LzB4MTAKRmViICA4 IDE5OjA5OjM5IHN0
b3JlLWVsNiBrZXJuZWw6IFs8ZmZmZmZmZmY4MTA5N2YyNT5dID8gc2NoZWRf Y2xvY2tfbG9jYWwr
MHgyNS8weDkwCkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBb PGZmZmZmZmZmODE0
Yzk3NDM+XSB3YWl0X2Zvcl9jb21tb24rMHgxMjMvMHgxODAKRmViICA4IDE5 OjA5OjM5IHN0b3Jl
LWVsNiBrZXJuZWw6IFs8ZmZmZmZmZmY4MTA1YzUzMD5dID8gZGVmYXVsdF93 YWtlX2Z1bmN0aW9u
KzB4MC8weDIwCkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBb PGZmZmZmZmZmODE0
Yzk4NWQ+XSB3YWl0X2Zvcl9jb21wbGV0aW9uKzB4MWQvMHgyMApGZWIgIDgg MTk6MDk6Mzkgc3Rv
cmUtZWw2IGtlcm5lbDogWzxmZmZmZmZmZjgxMDhkMTA3Pl0gZmx1c2hfd29y aysweDc3LzB4YzAK
RmViICA4IDE5OjA5OjM5IHN0b3JlLWVsNiBrZXJuZWw6IFs8ZmZmZmZmZmY4 MTA4Y2I5MD5dID8g
d3FfYmFycmllcl9mdW5jKzB4MC8weDIwCkZlYiAgOCAxOTowOTozOSBzdG9y ZS1lbDYga2VybmVs
OiBbPGZmZmZmZmZmODEwOGQzMjQ+XSBmbHVzaF9kZWxheWVkX3dvcmsrMHg1 NC8weDcwCkZlYiAg
OCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBbPGZmZmZmZmZmODEyZmQ1 ZDU+XSB0dHlfZmx1
c2hfdG9fbGRpc2MrMHgxNS8weDIwCkZlYiAgOCAxOTowOTozOSBzdG9yZS1l bDYga2VybmVsOiBb
PGZmZmZmZmZmODEyZjgxMTc+XSBuX3R0eV9wb2xsKzB4NjcvMHgxZDAKRmVi ICA4IDE5OjA5OjM5
IHN0b3JlLWVsNiBrZXJuZWw6IFs8ZmZmZmZmZmY4MTJmM2JhYT5dIHR0eV9w b2xsKzB4OGEvMHhh
MApGZWIgIDggMTk6MDk6Mzkgc3RvcmUtZWw2IGtlcm5lbDogWzxmZmZmZmZm ZjgxMTgyNGFiPl0g
ZG9fc3lzX3BvbGwrMHgyOWIvMHg1MjAKRmViICA4IDE5OjA5OjM5IHN0b3Jl LWVsNiBrZXJuZWw6
IFs8ZmZmZmZmZmY4MTE4MjBjMD5dID8gX19wb2xsd2FpdCsweDAvMHhmMApG ZWIgIDggMTk6MDk6
Mzkgc3RvcmUtZWw2IGtlcm5lbDogWzxmZmZmZmZmZjgxMTgyMWIwPl0gPyBw b2xsd2FrZSsweDAv
MHg2MApGZWIgIDggMTk6MDk6Mzkgc3RvcmUtZWw2IGtlcm5lbDogWzxmZmZm ZmZmZjgxMTgyMWIw
Pl0gPyBwb2xsd2FrZSsweDAvMHg2MApGZWIgIDggMTk6MDk6Mzkgc3RvcmUt ZWw2IGtlcm5lbDog
WzxmZmZmZmZmZjgxMTgyMWIwPl0gPyBwb2xsd2FrZSsweDAvMHg2MApGZWIg IDggMTk6MDk6Mzkg
c3RvcmUtZWw2IGtlcm5lbDogWzxmZmZmZmZmZjgxMTgyMWIwPl0gPyBwb2xs d2FrZSsweDAvMHg2
MApGZWIgIDggMTk6MDk6Mzkgc3RvcmUtZWw2IGtlcm5lbDogWzxmZmZmZmZm ZjgxMTgyMWIwPl0g
PyBwb2xsd2FrZSsweDAvMHg2MApGZWIgIDggMTk6MDk6Mzkgc3RvcmUtZWw2 IGtlcm5lbDogWzxm
ZmZmZmZmZjgxMTgyMWIwPl0gPyBwb2xsd2FrZSsweDAvMHg2MApGZWIgIDgg MTk6MDk6Mzkgc3Rv
cmUtZWw2IGtlcm5lbDogWzxmZmZmZmZmZjgxMTgyMWIwPl0gPyBwb2xsd2Fr ZSsweDAvMHg2MApG
ZWIgIDggMTk6MDk6Mzkgc3RvcmUtZWw2IGtlcm5lbDogWzxmZmZmZmZmZjgx MTgyMWIwPl0gPyBw
b2xsd2FrZSsweDAvMHg2MApGZWIgIDggMTk6MDk6Mzkgc3RvcmUtZWw2IGtl cm5lbDogWzxmZmZm
ZmZmZjgxMTgyMWIwPl0gPyBwb2xsd2FrZSsweDAvMHg2MApGZWIgIDggMTk6 MDk6Mzkgc3RvcmUt
ZWw2IGtlcm5lbDogWzxmZmZmZmZmZjgxMjBjNmNmPl0gPyBzZWxpbnV4X2Zp bGVfcGVybWlzc2lv
bisweGJmLzB4MTUwCkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVs OiBbPGZmZmZmZmZm
ODExZmZiNzY+XSA/IHNlY3VyaXR5X2ZpbGVfcGVybWlzc2lvbisweDE2LzB4 MjAKRmViICA4IDE5
OjA5OjM5IHN0b3JlLWVsNiBrZXJuZWw6IFs8ZmZmZmZmZmY4MTE2ZDlkMT5d ID8gdmZzX3JlYWQr
MHgxODEvMHgxYTAKRmViICA4IDE5OjA5OjM5IHN0b3JlLWVsNiBrZXJuZWw6 IFs8ZmZmZmZmZmY4
MTE4MjkyYz5dIHN5c19wb2xsKzB4N2MvMHgxMTAKRmViICA4IDE5OjA5OjM5 IHN0b3JlLWVsNiBr
ZXJuZWw6IFs8ZmZmZmZmZmY4MTAxMzE3Mj5dIHN5c3RlbV9jYWxsX2Zhc3Rw YXRoKzB4MTYvMHgx
YgpGZWIgIDggMTk6MDk6Mzkgc3RvcmUtZWw2IGtlcm5lbDogSU5GTzogdGFz ayBtZDkwX3JhaWQ1
OjEzNzM2IGJsb2NrZWQgZm9yIG1vcmUgdGhhbiAxMjAgc2Vjb25kcy4KRmVi ICA4IDE5OjA5OjM5
IHN0b3JlLWVsNiBrZXJuZWw6ICJlY2hvIDAgPiAvcHJvYy9zeXMva2VybmVs L2h1bmdfdGFza190
aW1lb3V0X3NlY3MiIGRpc2FibGVzIHRoaXMgbWVzc2FnZS4KRmViICA4IDE5 OjA5OjM5IHN0b3Jl
LWVsNiBrZXJuZWw6IG1kOTBfcmFpZDUgICAgRCAwMDAwMDAwMDAwMDAwMDAw ICAgICAwIDEzNzM2
ICAgICAgMiAweDAwMDAwMDgwCkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYg a2VybmVsOiBmZmZm
ODgwMDNlMGI3YWIwIDAwMDAwMDAwMDAwMDAwNDYgZmZmZjg4MDA2ZDI3M2U2 MCBmZmZmODgwMDcx
NmI4MjQwCkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBmZmZm ODgwMDNlMGI3YTUw
IGZmZmZmZmZmODEyM2E1NTMgZmZmZjg4MDA3MmVhNDgwMCAwMDAwMDAwMDAw MDAwODEwCkZlYiAg
OCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBmZmZmODgwMDZmNTBiYTk4 IGZmZmY4ODAwM2Uw
YjdmZDggMDAwMDAwMDAwMDAxMDUxOCBmZmZmODgwMDZmNTBiYTk4CkZlYiAg OCAxOTowOTozOSBz
dG9yZS1lbDYga2VybmVsOiBDYWxsIFRyYWNlOgpGZWIgIDggMTk6MDk6Mzkg c3RvcmUtZWw2IGtl
cm5lbDogWzxmZmZmZmZmZjgxMjNhNTUzPl0gPyBlbHZfaW5zZXJ0KzB4MTMz LzB4MWYwCkZlYiAg
OCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBbPGZmZmZmZmZmODEwOTIw Y2U+XSA/IHByZXBh
cmVfdG9fd2FpdCsweDRlLzB4ODAKRmViICA4IDE5OjA5OjM5IHN0b3JlLWVs NiBrZXJuZWw6IFs8
ZmZmZmZmZmY4MTNkMDUzNT5dIG1kX21ha2VfcmVxdWVzdCsweDg1LzB4MjMw CkZlYiAgOCAxOTow
OTozOSBzdG9yZS1lbDYga2VybmVsOiBbPGZmZmZmZmZmODEwOTFkZTA+XSA/ IGF1dG9yZW1vdmVf
d2FrZV9mdW5jdGlvbisweDAvMHg0MApGZWIgIDggMTk6MDk6Mzkgc3RvcmUt ZWw2IGtlcm5lbDog
WzxmZmZmZmZmZjgxMjQxNjUyPl0gPyBnZW5lcmljX21ha2VfcmVxdWVzdCsw eDFiMi8weDRmMApG
ZWIgIDggMTk6MDk6Mzkgc3RvcmUtZWw2IGtlcm5lbDogWzxmZmZmZmZmZjgx MjQxNjUyPl0gZ2Vu
ZXJpY19tYWtlX3JlcXVlc3QrMHgxYjIvMHg0ZjAKRmViICA4IDE5OjA5OjM5 IHN0b3JlLWVsNiBr
ZXJuZWw6IFs8ZmZmZmZmZmY4MTA2MzMzYT5dID8gZmluZF9idXNpZXN0X2dy b3VwKzB4OTZhLzB4
YjQwCkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBbPGZmZmZm ZmZmYTAzZDhkOWQ+
XSBvcHNfcnVuX2lvKzB4MjJkLzB4MzMwIFtyYWlkNDU2XQpGZWIgIDggMTk6 MDk6Mzkgc3RvcmUt
ZWw2IGtlcm5lbDogWzxmZmZmZmZmZjgxM2QxZWY2Pl0gPyBtZF9zdXBlcl93 cml0ZSsweGQ2LzB4
ZTAKRmViICA4IDE5OjA5OjM5IHN0b3JlLWVsNiBrZXJuZWw6IFs8ZmZmZmZm ZmZhMDNkYjlmNT5d
IGhhbmRsZV9zdHJpcGUrMHg0ZDUvMHgyMmUwIFtyYWlkNDU2XQpGZWIgIDgg MTk6MDk6Mzkgc3Rv
cmUtZWw2IGtlcm5lbDogWzxmZmZmZmZmZjgxMDU5ZGIyPl0gPyBmaW5pc2hf dGFza19zd2l0Y2gr
MHg0Mi8weGQwCkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBb PGZmZmZmZmZmYTAz
ZGRjOWY+XSByYWlkNWQrMHg0OWYvMHg2OTAgW3JhaWQ0NTZdCkZlYiAgOCAx OTowOTozOSBzdG9y
ZS1lbDYga2VybmVsOiBbPGZmZmZmZmZmODEzZDE4MmM+XSBtZF90aHJlYWQr MHg1Yy8weDEzMApG
ZWIgIDggMTk6MDk6Mzkgc3RvcmUtZWw2IGtlcm5lbDogWzxmZmZmZmZmZjgx MDkxZGUwPl0gPyBh
dXRvcmVtb3ZlX3dha2VfZnVuY3Rpb24rMHgwLzB4NDAKRmViICA4IDE5OjA5 OjM5IHN0b3JlLWVs
NiBrZXJuZWw6IFs8ZmZmZmZmZmY4MTNkMTdkMD5dID8gbWRfdGhyZWFkKzB4 MC8weDEzMApGZWIg
IDggMTk6MDk6Mzkgc3RvcmUtZWw2IGtlcm5lbDogWzxmZmZmZmZmZjgxMDkx YTc2Pl0ga3RocmVh
ZCsweDk2LzB4YTAKRmViICA4IDE5OjA5OjM5IHN0b3JlLWVsNiBrZXJuZWw6 IFs8ZmZmZmZmZmY4
MTAxNDFjYT5dIGNoaWxkX3JpcCsweGEvMHgyMApGZWIgIDggMTk6MDk6Mzkg c3RvcmUtZWw2IGtl
cm5lbDogWzxmZmZmZmZmZjgxMDkxOWUwPl0gPyBrdGhyZWFkKzB4MC8weGEw CkZlYiAgOCAxOTow
OTozOSBzdG9yZS1lbDYga2VybmVsOiBbPGZmZmZmZmZmODEwMTQxYzA+XSA/ IGNoaWxkX3JpcCsw
eDAvMHgyMApGZWIgIDggMTk6MDk6Mzkgc3RvcmUtZWw2IGtlcm5lbDogSU5G TzogdGFzayBmbHVz
aC05OjkwOjEzODgzIGJsb2NrZWQgZm9yIG1vcmUgdGhhbiAxMjAgc2Vjb25k cy4KRmViICA4IDE5
OjA5OjM5IHN0b3JlLWVsNiBrZXJuZWw6ICJlY2hvIDAgPiAvcHJvYy9zeXMv a2VybmVsL2h1bmdf
dGFza190aW1lb3V0X3NlY3MiIGRpc2FibGVzIHRoaXMgbWVzc2FnZS4KRmVi ICA4IDE5OjA5OjM5
IHN0b3JlLWVsNiBrZXJuZWw6IGZsdXNoLTk6OTAgICAgRCAwMDAwMDAwMDAw MDAwMDAxICAgICAw
IDEzODgzICAgICAgMiAweDAwMDAwMDgwCkZlYiAgOCAxOTowOTozOSBzdG9y ZS1lbDYga2VybmVs
OiBmZmZmODgwMDNlMTZiNmQwIDAwMDAwMDAwMDAwMDAwNDYgZmZmZjg4MDAz ZTE2YjZhMCBmZmZm
ODgwMDc1Mzg0ZGQ4CkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVs OiBmZmZmODgwMDcw
ZWZhNjE4IGZmZmZmZmZmYTAxZTVhMDAgZmZmZjg4MDA3Mzc0NDYxMCAwMDAw MDAwMDAwMDAwMDAw
CkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBmZmZmODgwMDUw MjNkMGU4IGZmZmY4
ODAwM2UxNmJmZDggMDAwMDAwMDAwMDAxMDUxOCBmZmZmODgwMDUwMjNkMGU4 CkZlYiAgOCAxOTow
OTozOSBzdG9yZS1lbDYga2VybmVsOiBDYWxsIFRyYWNlOgpGZWIgIDggMTk6 MDk6Mzkgc3RvcmUt
ZWw2IGtlcm5lbDogWzxmZmZmZmZmZjgxMDkyMGNlPl0gPyBwcmVwYXJlX3Rv X3dhaXQrMHg0ZS8w
eDgwCkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBbPGZmZmZm ZmZmODEzZDA1MzU+
XSBtZF9tYWtlX3JlcXVlc3QrMHg4NS8weDIzMApGZWIgIDggMTk6MDk6Mzkg c3RvcmUtZWw2IGtl
cm5lbDogWzxmZmZmZmZmZjgxMDkxZGUwPl0gPyBhdXRvcmVtb3ZlX3dha2Vf ZnVuY3Rpb24rMHgw
LzB4NDAKRmViICA4IDE5OjA5OjM5IHN0b3JlLWVsNiBrZXJuZWw6IFs8ZmZm ZmZmZmY4MTI0MTY1
Mj5dIGdlbmVyaWNfbWFrZV9yZXF1ZXN0KzB4MWIyLzB4NGYwCkZlYiAgOCAx OTowOTozOSBzdG9y
ZS1lbDYga2VybmVsOiBbPGZmZmZmZmZmODExMGU4NjU+XSA/IG1lbXBvb2xf YWxsb2Nfc2xhYisw
eDE1LzB4MjAKRmViICA4IDE5OjA5OjM5IHN0b3JlLWVsNiBrZXJuZWw6IFs8 ZmZmZmZmZmY4MTA5
MWRjZj5dID8gd2FrZV91cF9iaXQrMHgyZi8weDQwCkZlYiAgOCAxOTowOToz OSBzdG9yZS1lbDYg
a2VybmVsOiBbPGZmZmZmZmZmODEyNDFhMWY+XSBzdWJtaXRfYmlvKzB4OGYv MHgxMjAKRmViICA4
IDE5OjA5OjM5IHN0b3JlLWVsNiBrZXJuZWw6IFs8ZmZmZmZmZmY4MTE5ZDMy ND5dIHN1Ym1pdF9i
aCsweGY0LzB4MTQwCkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVs OiBbPGZmZmZmZmZm
ODExOWYzMjA+XSBfX2Jsb2NrX3dyaXRlX2Z1bGxfcGFnZSsweDFlMC8weDNi MApGZWIgIDggMTk6
MDk6Mzkgc3RvcmUtZWw2IGtlcm5lbDogWzxmZmZmZmZmZjgxMTllOWIwPl0g PyBlbmRfYnVmZmVy
X2FzeW5jX3dyaXRlKzB4MC8weDE5MApGZWIgIDggMTk6MDk6Mzkgc3RvcmUt ZWw2IGtlcm5lbDog
WzxmZmZmZmZmZmEwMWI0YzAwPl0gPyBub2FsbG9jX2dldF9ibG9ja193cml0 ZSsweDAvMHg2MCBb
ZXh0NF0KRmViICA4IDE5OjA5OjM5IHN0b3JlLWVsNiBrZXJuZWw6IFs8ZmZm ZmZmZmZhMDFiNGMw
MD5dID8gbm9hbGxvY19nZXRfYmxvY2tfd3JpdGUrMHgwLzB4NjAgW2V4dDRd CkZlYiAgOCAxOTow
OTozOSBzdG9yZS1lbDYga2VybmVsOiBbPGZmZmZmZmZmODExOWZjMjA+XSBi bG9ja193cml0ZV9m
dWxsX3BhZ2VfZW5kaW8rMHhlMC8weDEyMApGZWIgIDggMTk6MDk6Mzkgc3Rv cmUtZWw2IGtlcm5l
bDogWzxmZmZmZmZmZmEwMWFmOTYwPl0gPyBleHQ0X2JoX2RlbGF5X29yX3Vu d3JpdHRlbisweDAv
MHgzMCBbZXh0NF0KRmViICA4IDE5OjA5OjM5IHN0b3JlLWVsNiBrZXJuZWw6 IFs8ZmZmZmZmZmY4
MTE5ZmM3NT5dIGJsb2NrX3dyaXRlX2Z1bGxfcGFnZSsweDE1LzB4MjAKRmVi ICA4IDE5OjA5OjM5
IHN0b3JlLWVsNiBrZXJuZWw6IFs8ZmZmZmZmZmZhMDFiMGVhNj5dIGV4dDRf d3JpdGVwYWdlKzB4
ZDYvMHgzYTAgW2V4dDRdCkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYga2Vy bmVsOiBbPGZmZmZm
ZmZmYTAxYjBkMmM+XSBtcGFnZV9kYV9zdWJtaXRfaW8rMHgxNGMvMHgxZDAg W2V4dDRdCkZlYiAg
OCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBbPGZmZmZmZmZmYTAxYjYx NDk+XSBleHQ0X2Rh
X3dyaXRlcGFnZXMrMHgzZDkvMHg2MDAgW2V4dDRdCkZlYiAgOCAxOTowOToz OSBzdG9yZS1lbDYg
a2VybmVsOiBbPGZmZmZmZmZmODExMjBiYzE+XSBkb193cml0ZXBhZ2VzKzB4 MjEvMHg0MApGZWIg
IDggMTk6MDk6Mzkgc3RvcmUtZWw2IGtlcm5lbDogWzxmZmZmZmZmZjgxMTk2 NTZkPl0gd3JpdGVi
YWNrX3NpbmdsZV9pbm9kZSsweGRkLzB4MmMwCkZlYiAgOCAxOTowOTozOSBz dG9yZS1lbDYga2Vy
bmVsOiBbPGZmZmZmZmZmODExOTY5NmU+XSB3cml0ZWJhY2tfc2JfaW5vZGVz KzB4Y2UvMHgxODAK
RmViICA4IDE5OjA5OjM5IHN0b3JlLWVsNiBrZXJuZWw6IFs8ZmZmZmZmZmY4 MTE5NmFjMz5dIHdy
aXRlYmFja19pbm9kZXNfd2IrMHhhMy8weDFhMApGZWIgIDggMTk6MDk6Mzkg c3RvcmUtZWw2IGtl
cm5lbDogWzxmZmZmZmZmZjgxMTk2ZTViPl0gd2Jfd3JpdGViYWNrKzB4Mjli LzB4M2YwCkZlYiAg
OCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBbPGZmZmZmZmZmODE0Yzhk OTY+XSA/IHRocmVh
ZF9yZXR1cm4rMHg0ZS8weDc3OApGZWIgIDggMTk6MDk6Mzkgc3RvcmUtZWw2 IGtlcm5lbDogWzxm
ZmZmZmZmZjgxMTk3MTQ5Pl0gd2JfZG9fd3JpdGViYWNrKzB4MTk5LzB4MjQw CkZlYiAgOCAxOTow
OTozOSBzdG9yZS1lbDYga2VybmVsOiBbPGZmZmZmZmZmODExOTcyNTM+XSBi ZGlfd3JpdGViYWNr
X3Rhc2srMHg2My8weDFiMApGZWIgIDggMTk6MDk6Mzkgc3RvcmUtZWw2IGtl cm5lbDogWzxmZmZm
ZmZmZjgxMDkxY2E3Pl0gPyBiaXRfd2FpdHF1ZXVlKzB4MTcvMHhkMApGZWIg IDggMTk6MDk6Mzkg
c3RvcmUtZWw2IGtlcm5lbDogWzxmZmZmZmZmZjgxMTJlZjcwPl0gPyBiZGlf c3RhcnRfZm4rMHgw
LzB4MTAwCkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBbPGZm ZmZmZmZmODExMmVm
ZjY+XSBiZGlfc3RhcnRfZm4rMHg4Ni8weDEwMApGZWIgIDggMTk6MDk6Mzkg c3RvcmUtZWw2IGtl
cm5lbDogWzxmZmZmZmZmZjgxMTJlZjcwPl0gPyBiZGlfc3RhcnRfZm4rMHgw LzB4MTAwCkZlYiAg
OCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBbPGZmZmZmZmZmODEwOTFh NzY+XSBrdGhyZWFk
KzB4OTYvMHhhMApGZWIgIDggMTk6MDk6Mzkgc3RvcmUtZWw2IGtlcm5lbDog WzxmZmZmZmZmZjgx
MDE0MWNhPl0gY2hpbGRfcmlwKzB4YS8weDIwCkZlYiAgOCAxOTowOTozOSBz dG9yZS1lbDYga2Vy
bmVsOiBbPGZmZmZmZmZmODEwOTE5ZTA+XSA/IGt0aHJlYWQrMHgwLzB4YTAK RmViICA4IDE5OjA5
OjM5IHN0b3JlLWVsNiBrZXJuZWw6IFs8ZmZmZmZmZmY4MTAxNDFjMD5dID8g Y2hpbGRfcmlwKzB4
MC8weDIwCkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBJTkZP OiB0YXNrIGpiZDIv
bWQ5MC04OjE3ODAzIGJsb2NrZWQgZm9yIG1vcmUgdGhhbiAxMjAgc2Vjb25k cy4KRmViICA4IDE5
OjA5OjM5IHN0b3JlLWVsNiBrZXJuZWw6ICJlY2hvIDAgPiAvcHJvYy9zeXMv a2VybmVsL2h1bmdf
dGFza190aW1lb3V0X3NlY3MiIGRpc2FibGVzIHRoaXMgbWVzc2FnZS4KRmVi ICA4IDE5OjA5OjM5
IHN0b3JlLWVsNiBrZXJuZWw6IGpiZDIvbWQ5MC04ICAgRCAwMDAwMDAwMDAw MDAwMDAyICAgICAw
IDE3ODAzICAgICAgMiAweDAwMDAwMDgwCkZlYiAgOCAxOTowOTozOSBzdG9y ZS1lbDYga2VybmVs
OiBmZmZmODgwMDc0MWQ1YzEwIDAwMDAwMDAwMDAwMDAwNDYgZmZmZjg4MDA3 NDFkNWJiMCBmZmZm
ZmZmZmEwM2Q5MmU4CkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVs OiBmZmZmODgwMDQ3
YmM5YzAwIGZmZmY4ODAwNDdiYzlkYTAgZmZmZjg4MDA0N2JjOWMwMCBmZmZm ODgwMDcyMDQ0MjAw
CkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBmZmZmODgwMDNl MTgzYTk4IGZmZmY4
ODAwNzQxZDVmZDggMDAwMDAwMDAwMDAxMDUxOCBmZmZmODgwMDNlMTgzYTk4 CkZlYiAgOCAxOTow
OTozOSBzdG9yZS1lbDYga2VybmVsOiBDYWxsIFRyYWNlOgpGZWIgIDggMTk6 MDk6Mzkgc3RvcmUt
ZWw2IGtlcm5lbDogWzxmZmZmZmZmZmEwM2Q5MmU4Pl0gPyB1bnBsdWdfc2xh dmVzKzB4OTgvMHhl
MCBbcmFpZDQ1Nl0KRmViICA4IDE5OjA5OjM5IHN0b3JlLWVsNiBrZXJuZWw6 IFs8ZmZmZmZmZmY4
MTA5YmFlOT5dID8ga3RpbWVfZ2V0X3RzKzB4YTkvMHhlMApGZWIgIDggMTk6 MDk6Mzkgc3RvcmUt
ZWw2IGtlcm5lbDogWzxmZmZmZmZmZjgxMTllNTgwPl0gPyBzeW5jX2J1ZmZl cisweDAvMHg1MApG
ZWIgIDggMTk6MDk6Mzkgc3RvcmUtZWw2IGtlcm5lbDogWzxmZmZmZmZmZjgx NGM5NTMzPl0gaW9f
c2NoZWR1bGUrMHg3My8weGMwCkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYg a2VybmVsOiBbPGZm
ZmZmZmZmODExOWU1YzA+XSBzeW5jX2J1ZmZlcisweDQwLzB4NTAKRmViICA4 IDE5OjA5OjM5IHN0
b3JlLWVsNiBrZXJuZWw6IFs8ZmZmZmZmZmY4MTRjOWRhZj5dIF9fd2FpdF9v bl9iaXQrMHg1Zi8w
eDkwCkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBbPGZmZmZm ZmZmODExOWU1ODA+
XSA/IHN5bmNfYnVmZmVyKzB4MC8weDUwCkZlYiAgOCAxOTowOTozOSBzdG9y ZS1lbDYga2VybmVs
OiBbPGZmZmZmZmZmODE0YzllNTg+XSBvdXRfb2ZfbGluZV93YWl0X29uX2Jp dCsweDc4LzB4OTAK
RmViICA4IDE5OjA5OjM5IHN0b3JlLWVsNiBrZXJuZWw6IFs8ZmZmZmZmZmY4 MTA5MWUyMD5dID8g
d2FrZV9iaXRfZnVuY3Rpb24rMHgwLzB4NTAKRmViICA4IDE5OjA5OjM5IHN0 b3JlLWVsNiBrZXJu
ZWw6IFs8ZmZmZmZmZmY4MTE5ZTU3Nj5dIF9fd2FpdF9vbl9idWZmZXIrMHgy Ni8weDMwCkZlYiAg
OCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBbPGZmZmZmZmZmYTAxODQ4 MDE+XSBqYmQyX2pv
dXJuYWxfY29tbWl0X3RyYW5zYWN0aW9uKzB4MTBkMS8weDE0ZTAgW2piZDJd CkZlYiAgOCAxOTow
OTozOSBzdG9yZS1lbDYga2VybmVsOiBbPGZmZmZmZmZmODEwOTFkZTA+XSA/ IGF1dG9yZW1vdmVf
d2FrZV9mdW5jdGlvbisweDAvMHg0MApGZWIgIDggMTk6MDk6Mzkgc3RvcmUt ZWw2IGtlcm5lbDog
WzxmZmZmZmZmZmEwMThhMGI4Pl0ga2pvdXJuYWxkMisweGI4LzB4MjIwIFtq YmQyXQpGZWIgIDgg
MTk6MDk6Mzkgc3RvcmUtZWw2IGtlcm5lbDogWzxmZmZmZmZmZjgxMDkxZGUw Pl0gPyBhdXRvcmVt
b3ZlX3dha2VfZnVuY3Rpb24rMHgwLzB4NDAKRmViICA4IDE5OjA5OjM5IHN0 b3JlLWVsNiBrZXJu
ZWw6IFs8ZmZmZmZmZmZhMDE4YTAwMD5dID8ga2pvdXJuYWxkMisweDAvMHgy MjAgW2piZDJdCkZl
YiAgOCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBbPGZmZmZmZmZmODEw OTFhNzY+XSBrdGhy
ZWFkKzB4OTYvMHhhMApGZWIgIDggMTk6MDk6Mzkgc3RvcmUtZWw2IGtlcm5l bDogWzxmZmZmZmZm
ZjgxMDE0MWNhPl0gY2hpbGRfcmlwKzB4YS8weDIwCkZlYiAgOCAxOTowOToz OSBzdG9yZS1lbDYg
a2VybmVsOiBbPGZmZmZmZmZmODEwOTE5ZTA+XSA/IGt0aHJlYWQrMHgwLzB4 YTAKRmViICA4IDE5
OjA5OjM5IHN0b3JlLWVsNiBrZXJuZWw6IFs8ZmZmZmZmZmY4MTAxNDFjMD5d ID8gY2hpbGRfcmlw
KzB4MC8weDIwCkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBJ TkZPOiB0YXNrIHJz
eW5jOjE3ODgwIGJsb2NrZWQgZm9yIG1vcmUgdGhhbiAxMjAgc2Vjb25kcy4K RmViICA4IDE5OjA5
OjM5IHN0b3JlLWVsNiBrZXJuZWw6ICJlY2hvIDAgPiAvcHJvYy9zeXMva2Vy bmVsL2h1bmdfdGFz
a190aW1lb3V0X3NlY3MiIGRpc2FibGVzIHRoaXMgbWVzc2FnZS4KRmViICA4 IDE5OjA5OjM5IHN0
b3JlLWVsNiBrZXJuZWw6IHJzeW5jICAgICAgICAgRCAwMDAwMDAwMDAwMDAw MTAwICAgICAwIDE3
ODgwICAxNzg3OSAweDAwMDAwMDgwCkZlYiAgOCAxOTowOTozOSBzdG9yZS1l bDYga2VybmVsOiBm
ZmZmODgwMDEyNjI1OTE4IDAwMDAwMDAwMDAwMDAwODYgZmZmZjg4MDAxMjYy NTk2OCBmZmZmZmZm
ZjgxMTFlNDExCkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBm ZmZmODgwMDAwMDEy
NmMwIGZmZmZlYTAwMDA2OTcxODggZmZmZjg4MDA3NzJmNTExOCAwMDAwMDAw MDAwMDAxMDAwCkZl
YiAgOCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBmZmZmODgwMDM3ZGI1 YTU4IGZmZmY4ODAw
MTI2MjVmZDggMDAwMDAwMDAwMDAxMDUxOCBmZmZmODgwMDM3ZGI1YTU4CkZl YiAgOCAxOTowOToz
OSBzdG9yZS1lbDYga2VybmVsOiBDYWxsIFRyYWNlOgpGZWIgIDggMTk6MDk6 Mzkgc3RvcmUtZWw2
IGtlcm5lbDogWzxmZmZmZmZmZjgxMTFlNDExPl0gPyBfX2FsbG9jX3BhZ2Vz X25vZGVtYXNrKzB4
MTExLzB4ODUwCkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBb PGZmZmZmZmZmODEw
OTIwY2U+XSA/IHByZXBhcmVfdG9fd2FpdCsweDRlLzB4ODAKRmViICA4IDE5 OjA5OjM5IHN0b3Jl
LWVsNiBrZXJuZWw6IFs8ZmZmZmZmZmY4MTNkMDUzNT5dIG1kX21ha2VfcmVx dWVzdCsweDg1LzB4
MjMwCkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBbPGZmZmZm ZmZmODEwOTFkZTA+
XSA/IGF1dG9yZW1vdmVfd2FrZV9mdW5jdGlvbisweDAvMHg0MApGZWIgIDgg MTk6MDk6Mzkgc3Rv
cmUtZWw2IGtlcm5lbDogWzxmZmZmZmZmZjgxMTlkNWVmPl0gPyBfX2ZpbmRf Z2V0X2Jsb2NrX3Ns
b3crMHhhZi8weDEzMApGZWIgIDggMTk6MDk6Mzkgc3RvcmUtZWw2IGtlcm5l bDogWzxmZmZmZmZm
ZjgxMjQxNjUyPl0gZ2VuZXJpY19tYWtlX3JlcXVlc3QrMHgxYjIvMHg0ZjAK RmViICA4IDE5OjA5
OjM5IHN0b3JlLWVsNiBrZXJuZWw6IFs8ZmZmZmZmZmY4MTEwZTg2NT5dID8g bWVtcG9vbF9hbGxv
Y19zbGFiKzB4MTUvMHgyMApGZWIgIDggMTk6MDk6Mzkgc3RvcmUtZWw2IGtl cm5lbDogWzxmZmZm
ZmZmZjgxMTlkY2ZjPl0gPyBfX2dldGJsaysweDljLzB4MmUwCkZlYiAgOCAx OTowOTozOSBzdG9y
ZS1lbDYga2VybmVsOiBbPGZmZmZmZmZmODEyNDFhMWY+XSBzdWJtaXRfYmlv KzB4OGYvMHgxMjAK
RmViICA4IDE5OjA5OjM5IHN0b3JlLWVsNiBrZXJuZWw6IFs8ZmZmZmZmZmY4 MTE5ZDMyND5dIHN1
Ym1pdF9iaCsweGY0LzB4MTQwCkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYg a2VybmVsOiBbPGZm
ZmZmZmZmODExOWVjODM+XSBsbF9yd19ibG9jaysweDE0My8weDE1MApGZWIg IDggMTk6MDk6Mzkg
c3RvcmUtZWw2IGtlcm5lbDogWzxmZmZmZmZmZmEwMWJhYTU5Pl0gZXh0NF9m aW5kX2VudHJ5KzB4
MTc5LzB4NGMwIFtleHQ0XQpGZWIgIDggMTk6MDk6Mzkgc3RvcmUtZWw2IGtl cm5lbDogWzxmZmZm
ZmZmZmEwMWJhZGVkPl0gZXh0NF9sb29rdXArMHg0ZC8weDE0MCBbZXh0NF0K RmViICA4IDE5OjA5
OjM5IHN0b3JlLWVsNiBrZXJuZWw6IFs8ZmZmZmZmZmY4MTE3YWUwYj5dIGRv X2xvb2t1cCsweDE4
Yi8weDIyMApGZWIgIDggMTk6MDk6Mzkgc3RvcmUtZWw2IGtlcm5lbDogWzxm ZmZmZmZmZjgxMTdi
OWM1Pl0gX19saW5rX3BhdGhfd2FsaysweDZmNS8weDEwNDAKRmViICA4IDE5 OjA5OjM5IHN0b3Jl
LWVsNiBrZXJuZWw6IFs8ZmZmZmZmZmY4MTE3YzU5YT5dIHBhdGhfd2Fsaysw eDZhLzB4ZTAKRmVi
ICA4IDE5OjA5OjM5IHN0b3JlLWVsNiBrZXJuZWw6IFs8ZmZmZmZmZmY4MTE3 Yzc2Yj5dIGRvX3Bh
dGhfbG9va3VwKzB4NWIvMHhhMApGZWIgIDggMTk6MDk6Mzkgc3RvcmUtZWw2 IGtlcm5lbDogWzxm
ZmZmZmZmZjgxMTZlNmExPl0gPyBnZXRfZW1wdHlfZmlscCsweGExLzB4MTcw CkZlYiAgOCAxOTow
OTozOSBzdG9yZS1lbDYga2VybmVsOiBbPGZmZmZmZmZmODExN2Q2YTY+XSBk b19maWxwX29wZW4r
MHgxMDYvMHhkNTAKRmViICA4IDE5OjA5OjM5IHN0b3JlLWVsNiBrZXJuZWw6 IFs8ZmZmZmZmZmY4
MTIwYzZjZj5dID8gc2VsaW51eF9maWxlX3Blcm1pc3Npb24rMHhiZi8weDE1 MApGZWIgIDggMTk6
MDk6Mzkgc3RvcmUtZWw2IGtlcm5lbDogWzxmZmZmZmZmZjgxMTg5YjgyPl0g PyBhbGxvY19mZCsw
eDkyLzB4MTYwCkZlYiAgOCAxOTowOTozOSBzdG9yZS1lbDYga2VybmVsOiBb PGZmZmZmZmZmODEx
NmExYTk+XSBkb19zeXNfb3BlbisweDY5LzB4MTQwCkZlYiAgOCAxOTowOToz OSBzdG9yZS1lbDYg
a2VybmVsOiBbPGZmZmZmZmZmODExNmEyYzA+XSBzeXNfb3BlbisweDIwLzB4 MzAKRmViICA4IDE5
OjA5OjM5IHN0b3JlLWVsNiBrZXJuZWw6IFs8ZmZmZmZmZmY4MTAxMzE3Mj5d IHN5c3RlbV9jYWxs
X2Zhc3RwYXRoKzB4MTYvMHgxYgpGZWIgIDggMTk6MDk6NTAgc3RvcmUtZWw2 IG50cGRbMjAyNF06
IHN5bmNocm9uaXplZCB0byA5Mi40My4xODQuNDQsIHN0cmF0dW0gMgoKIwoj IGhlcmUgaSB3YXMg
Zm9yY2VkIHRvIG1ha2UgYSBoYXJkIHJlc2V0LCBuZWl0aGVyIHJlbW90ZSB4 IHNlc3Npb24gd2Fz
IG5vdCByZXNwb25naW5nIG5vciBsb2NhbCB0dHlzCiMKCkZlYiAgOCAxOTox MTozNSBzdG9yZS1l
bDYga2VybmVsOiBpbWtsb2cgNC42LjIsIGxvZyBzb3VyY2UgPSAvcHJvYy9r bXNnIHN0YXJ0ZWQu
CkZlYiAgOCAxOToxMTozNSBzdG9yZS1lbDYgcnN5c2xvZ2Q6IFtvcmlnaW4g c29mdHdhcmU9InJz
eXNsb2dkIiBzd1ZlcnNpb249IjQuNi4yIiB4LXBpZD0iMTY3MCIgeC1pbmZv PSJodHRwOi8vd3d3
LnJzeXNsb2cuY29tIl0gKHJlKXN0YXJ0CkZlYiAgOCAxOToxMTozNSBzdG9y ZS1lbDYga2VybmVs
OiBJbml0aWFsaXppbmcgY2dyb3VwIHN1YnN5cyBjcHVzZXQKRmViICA4IDE5 OjExOjM1IHN0b3Jl
LWVsNiBrZXJuZWw6IEluaXRpYWxpemluZyBjZ3JvdXAgc3Vic3lzIGNwdQpG ZWIgIDggMTk6MTE6
MzUgc3RvcmUtZWw2IGtlcm5lbDogTGludXggdmVyc2lvbiAyLjYuMzItNzEu MTQuMS5lbDYueDg2
XzY0IChtb2NrYnVpbGRAbHMyMC1iYzItMTQuYnVpbGQucmVkaGF0LmNvbSkg KGdjYyB2ZXJzaW9u
IDQuNC40IDIwMTAwNzI2IChSZWQgSGF0IDQuNC40LTEzKSAoR0NDKSApICMx IFNNUCBXZWQgSmFu
IDUgMTc6MDE6MDEgRVNUIDIwMTEKRmViICA4IDE5OjExOjM1IHN0b3JlLWVs NiBrZXJuZWw6IENv
bW1hbmQgbGluZTogcm8gcm9vdD1VVUlEPWE1OWVhMjJiLTZhZGYtNDk3Yi05 NDA5LTRjZjkwNjJj
MGI5OSByZF9OT19MVUtTIHJkX05PX0xWTSByZF9OT19NRCByZF9OT19ETSBM QU5HPWVuX1VTLlVU
Ri04IFNZU0ZPTlQ9bGF0YXJjeXJoZWItc3VuMTYgS0VZQk9BUkRUWVBFPXBj IEtFWVRBQkxFPXVz
IGNyYXNoa2VybmVsPWF1dG8gcmhnYiBxdWlldAoK
--e0cb4e43cd551fb9f304a377a7b6
Content-Type: application/octet-stream; name=raid-conf
Content-Disposition: attachment; filename=raid-conf
Content-Transfer-Encoding: base64
X-Attachment-Id: f_gnssr6ev2

REVWSUNFIHBhcnRpdGlvbnMKTUFJTEFERFIgcm9vdAoKQVJSQVkgL2Rldi9t ZDkxMCBsZXZlbD1y
YWlkMCBtZXRhZGF0YT0xLjIgbnVtLWRldmljZXM9MiAKCVVVSUQ9NWViZGRk ODE6OWEyZDBhZWI6
ZDU1ZjNjMTM6ZTFjNTMyY2YgbmFtZT1zdG9yZS1lbDY6dGVzdDEKCWRldmlj ZXM9L2Rldi9zZGI4
LC9kZXYvc2RiNwpBUlJBWSAvZGV2L21kOTExIGxldmVsPXJhaWQwIG1ldGFk YXRhPTEuMiBudW0t
ZGV2aWNlcz0yIAoJVVVJRD02YTA2YjM1YTpiMzllOTdlNDo3N2IyNmFkYTpm MzI0ZThmYSBuYW1l
PXN0b3JlLWVsNjp0ZXN0MgoJZGV2aWNlcz0vZGV2L3NkYjEwLC9kZXYvc2Ri OQpBUlJBWSAvZGV2
L21kOTEyIGxldmVsPXJhaWQwIG1ldGFkYXRhPTEuMiBudW0tZGV2aWNlcz0y IAoJVVVJRD02NWM5
ZTVmMzoxNTk0ZTU5ZTpiNDFkMWQ4ZTo2ZjI2NzFkZSBuYW1lPXN0b3JlLWVs Njp0ZXN0MwoJZGV2
aWNlcz0vZGV2L3NkYjEyLC9kZXYvc2RiMTEKQVJSQVkgL2Rldi9tZDkwIGxl dmVsPXJhaWQ1IG1l
dGFkYXRhPTEuMiBudW0tZGV2aWNlcz0zIAoJVVVJRD1hZThiYTc1ZDpkMDQx NWYwNzo1ZDYwZWVk
ZDozOTRmMjhlYiBuYW1lPXN0b3JlLWVsNjp0ZXN0CglkZXZpY2VzPS9kZXYv bWQ5MTBwMSwvZGV2
L21kOTExcDEsL2Rldi9tZDkxMnAxCgoK
--e0cb4e43cd551fb9f304a377a7b6
Content-Type: application/octet-stream; name=raid-details
Content-Disposition: attachment; filename=raid-details
Content-Transfer-Encoding: base64
X-Attachment-Id: f_gnssr6ev3

L2Rldi9tZDkwOgogICAgICAgIFZlcnNpb24gOiAxLjIKICBDcmVhdGlvbiBU aW1lIDogVHVlIEZl
YiAgOCAxOTowNDoyNCAyMDExCiAgICAgUmFpZCBMZXZlbCA6IHJhaWQ1CiAg ICAgQXJyYXkgU2l6
ZSA6IDM4OTMyNDggKDMuNzEgR2lCIDMuOTkgR0IpCiAgVXNlZCBEZXYgU2l6 ZSA6IDE5NDY2MjQg
KDE5MDEuMzIgTWlCIDE5OTMuMzQgTUIpCiAgIFJhaWQgRGV2aWNlcyA6IDMK ICBUb3RhbCBEZXZp
Y2VzIDogMwogICAgUGVyc2lzdGVuY2UgOiBTdXBlcmJsb2NrIGlzIHBlcnNp c3RlbnQKCiAgSW50
ZW50IEJpdG1hcCA6IEludGVybmFsCgogICAgVXBkYXRlIFRpbWUgOiBUdWUg RmViICA4IDE5OjMy
OjI4IDIwMTEKICAgICAgICAgIFN0YXRlIDogYWN0aXZlCiBBY3RpdmUgRGV2 aWNlcyA6IDMKV29y
a2luZyBEZXZpY2VzIDogMwogRmFpbGVkIERldmljZXMgOiAwCiAgU3BhcmUg RGV2aWNlcyA6IDAK
CiAgICAgICAgIExheW91dCA6IGxlZnQtc3ltbWV0cmljCiAgICAgQ2h1bmsg U2l6ZSA6IDUxMksK
CiAgICAgICAgICAgTmFtZSA6IHN0b3JlLWVsNjp0ZXN0ICAobG9jYWwgdG8g aG9zdCBzdG9yZS1l
bDYpCiAgICAgICAgICAgVVVJRCA6IDg2OTE0NDg2OjgyNzU2MTcyOmNmMzc3 ZjUzOjk1Yzc3ZjBh
CiAgICAgICAgIEV2ZW50cyA6IDQ2CgogICAgTnVtYmVyICAgTWFqb3IgICBN aW5vciAgIFJhaWRE
ZXZpY2UgU3RhdGUKICAgICAgIDAgICAgIDI1OSAgICAgICAgMiAgICAgICAg MCAgICAgIGFjdGl2
ZSBzeW5jICAgL2Rldi9tZDkxMHAxCiAgICAgICAxICAgICAyNTkgICAgICAg IDMgICAgICAgIDEg
ICAgICBhY3RpdmUgc3luYyAgIC9kZXYvbWQ5MTFwMQogICAgICAgMyAgICAg MjU5ICAgICAgICA0
ICAgICAgICAyICAgICAgYWN0aXZlIHN5bmMgICAvZGV2L21kOTEycDEKCg==
--e0cb4e43cd551fb9f304a377a7b6
Content-Type: application/octet-stream; name=raid-mdstat
Content-Disposition: attachment; filename=raid-mdstat
Content-Transfer-Encoding: base64
X-Attachment-Id: f_gnssr6ew4

UGVyc29uYWxpdGllcyA6IFtyYWlkMF0gW3JhaWQ2XSBbcmFpZDVdIFtyYWlk NF0gCm1kOTAgOiBh
Y3RpdmUgcmFpZDUgbWQ5MTBwMVswXSBtZDkxMnAxWzNdIG1kOTExcDFbMV0K ICAgICAgMzg5MzI0
OCBibG9ja3Mgc3VwZXIgMS4yIGxldmVsIDUsIDUxMmsgY2h1bmssIGFsZ29y aXRobSAyIFszLzNd
IFtVVVVdCiAgICAgIGJpdG1hcDogMC8xIHBhZ2VzIFswS0JdLCA2NTUzNktC IGNodW5rCgptZDkx
MiA6IGFjdGl2ZSByYWlkMCBzZGIxMlsxXSBzZGIxMVswXQogICAgICAxOTU2 ODY0IGJsb2NrcyBz
dXBlciAxLjIgNTEyayBjaHVua3MKICAgICAgCm1kOTEwIDogYWN0aXZlIHJh aWQwIHNkYjdbMF0g
c2RiOFsxXQogICAgICAxOTQ5MTg0IGJsb2NrcyBzdXBlciAxLjIgNTEyayBj aHVua3MKICAgICAg
Cm1kOTExIDogYWN0aXZlIHJhaWQwIHNkYjlbMF0gc2RiMTBbMV0KICAgICAg MTk1Njg2NCBibG9j
a3Mgc3VwZXIgMS4yIDUxMmsgY2h1bmtzCgo=
--e0cb4e43cd551fb9f304a377a7b6--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Incompatibility of internal bitmap with ext4 barriers?

am 17.05.2011 15:17:46 von Roman Mamedov

--Sig_/j1n8U28U/d.OmUcMHuE8LS0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

On Tue, 17 May 2011 16:10:07 +0400
Jason Tinker wrote:

> I have encountered a weird bug when trying to use ext4 partition on
> top of mdadm RAID5 array. Mdadm array has internal bitmap, and ext4 is
> mounted with default options - which means that barriers are enabled.

The same combination works perfectly for me on 2.6.38.

> When trying to write large enough amount of files system just locks up
> indefinitely, only hard reset helps. This seems to happen at random
> times, yet consistently after several minutes of usage.
> I tried different configuration options and it seems to happen only
> when both barriers on ext4 and mdadm's internal bitmaps are enabled.
> After disabling ext4 barriers for good (it seemed like a lesser evil)
> no lock ups happened for 3 months.
> Mdadm is version 3.1.3, kernel is 2.6.32 (rhel6)

How about a newer kernel, maybe?

--=20
With respect,
Roman

--Sig_/j1n8U28U/d.OmUcMHuE8LS0
Content-Type: application/pgp-signature; name=signature.asc
Content-Disposition: attachment; filename=signature.asc

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAk3SdXoACgkQTLKSvz+PZwgdKwCfWYCsrDZtxPK0PN3+BNEv JNi5
oi0AnjFs1MMPrUo9T8pStCvXlSFa+3oL
=wXhE
-----END PGP SIGNATURE-----

--Sig_/j1n8U28U/d.OmUcMHuE8LS0--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Incompatibility of internal bitmap with ext4 barriers?

am 17.05.2011 20:43:48 von Jason Tinker

2011/5/17 Roman Mamedov :
> On Tue, 17 May 2011 16:10:07 +0400
> Jason Tinker wrote:
>
> The same combination works perfectly for me on 2.6.38.

Do you have RAID5 over RAID0's or an ordinary RAID5? On my box
ordinary RAID5 works perfectly too.

>
> How about a newer kernel, maybe?

This is out of the question, unfortunately. Compiling my own kernel on
top of RHEL would kind of defeat the purpose of having safe low
maintenance distro. And both stable Debian and Ubuntu LTS have the
same kernel.

If your configuration is also RAID5 over RAID0 and it works fine on
2.6.38 then I'll just wait for next ubuntu lts...
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Incompatibility of internal bitmap with ext4 barriers?

am 17.05.2011 21:00:20 von Roman Mamedov

--Sig_/PKoTv=co9lF5vAs40NIAwag
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

On Tue, 17 May 2011 22:43:48 +0400
Jason Tinker wrote:

> Do you have RAID5 over RAID0's or an ordinary RAID5? On my box
> ordinary RAID5 works perfectly too.

Yes, I ran RAID5 of 4x2TB+(1+1TB RAID0)+(1.5+0.5TB RAID0), recently changed=
to
RAID6 though, also replaced one of the RAID0s with another 2TB drive.

> If your configuration is also RAID5 over RAID0 and it works fine on
> 2.6.38 then I'll just wait for next ubuntu lts...

I am not saying you should just give up. The logs you posted seem like
they can be very helpful in tracking this down. But if it is indeed a kernel
issue, how do you fix or even debug it it without replacing/compiling a new
kernel, which is exactly what you do not want to do. And if you're already
replacing a kernel, why not try a 2.6.38 right away, to check if your issue=
is
already solved in there (after all 2.6.32 vs 2.6.38 are eons apart BOTH
mdadm-wise and ext4-wise).

--=20
With respect,
Roman

--Sig_/PKoTv=co9lF5vAs40NIAwag
Content-Type: application/pgp-signature; name=signature.asc
Content-Disposition: attachment; filename=signature.asc

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAk3SxcQACgkQTLKSvz+PZwgwfwCaA5t3jd8QjLw4MGwNkV5S ICrA
fU0AoJi2LOCl27EzZ9+356IA02abSF6X
=DZ/6
-----END PGP SIGNATURE-----

--Sig_/PKoTv=co9lF5vAs40NIAwag--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Incompatibility of internal bitmap with ext4 barriers?

am 17.05.2011 22:20:43 von John Robinson

On 17/05/2011 20:00, Roman Mamedov wrote:
> On Tue, 17 May 2011 22:43:48 +0400
> Jason Tinker wrote:
>
>> Do you have RAID5 over RAID0's or an ordinary RAID5? On my box
>> ordinary RAID5 works perfectly too.
>
> Yes, I ran RAID5 of 4x2TB+(1+1TB RAID0)+(1.5+0.5TB RAID0), recently changed to
> RAID6 though, also replaced one of the RAID0s with another 2TB drive.
>
>> If your configuration is also RAID5 over RAID0 and it works fine on
>> 2.6.38 then I'll just wait for next ubuntu lts...
>
> I am not saying you should just give up. The logs you posted seem like
> they can be very helpful in tracking this down. But if it is indeed a kernel
> issue, how do you fix or even debug it it without replacing/compiling a new
> kernel, which is exactly what you do not want to do. And if you're already
> replacing a kernel, why not try a 2.6.38 right away, to check if your issue is
> already solved in there (after all 2.6.32 vs 2.6.38 are eons apart BOTH
> mdadm-wise and ext4-wise).

I've said it before and no doubt I'll say it again, but the RHEL kernels
are often a lot closer to the most recent than their version numbers
suggest. It's a little more difficult to tell with EL6, because Red Hat
don't ship vanilla+patches sources any more, but their .32 series
includes lots of backported fixes from .38 etc.

Jason, it's probably worth testing with vanilla .38 to see if that does
fix your problem, then post to Red Hat bugzilla saying so, especially as
it looks to me like the configuration you're trying to use ought to be a
supported one - then a future RHEL .32 might well include the fix.

Cheers,

John.
(Not a spokesman for Red Hat, nor anyone else other than myself, and
sometimes not even that ;-)

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html