systemd kills mdmon if it was started manually by user

If user starts array manually (mdadm -A -s as example) from within
user session and array needs mdmon, mdmon becomes part of user session
control group:

â”=9C user
â”=82 â”=94 root
â”=82 â”=94 1
â”=82 â”=9C 1916 login -- root
â”=82 â”=9C 1930 -bash
â”=82 â”=9C 1964 gpg-agent --keep-display --daemon --write-=
env-file /root/.gnup...
â”=82 â”=94 2062 mdmon md127


It is then killed by systemd during shutdown as part of user session.
It results in dirty array on next boot.

Is there any magic that allows daemon to be exempted from killing?

TIA

-andrey
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: systemd kills mdmon if it was started manually

T24gU2F0dXJkYXksIERlY2VtYmVyIDA0LCAyMDEwIDA5OjQxOjI2IGFtIEFu ZHJleSBCb3J6ZW5r
b3Ygd3JvdGU6Cj4gSWYgdXNlciBzdGFydHMgYXJyYXkgbWFudWFsbHkgKG1k YWRtIC1BIC1zIGFz
IGV4YW1wbGUpIGZyb20gd2l0aGluCj4gdXNlciBzZXNzaW9uIGFuZCBhcnJh eSBuZWVkcyBtZG1v
biwgbWRtb24gYmVjb21lcyBwYXJ0IG9mIHVzZXIgc2Vzc2lvbgo+IGNvbnRy b2wgZ3JvdXA6Cj4g
Cj4g4pScIHVzZXIKPiDilIIg4pSUIHJvb3QKPiDilIIgICDilJQgMQo+IOKU giAgICAg4pScIDE5
MTYgbG9naW4gLS0gcm9vdAo+IOKUgiAgICAg4pScIDE5MzAgLWJhc2gKPiDi lIIgICAgIOKUnCAx
OTY0IGdwZy1hZ2VudCAtLWtlZXAtZGlzcGxheSAtLWRhZW1vbiAtLXdyaXRl LWVudi1maWxlCj4g
L3Jvb3QvLmdudXAuLi4g4pSCICAgICDilJQgMjA2MiBtZG1vbiBtZDEyNwo+ IAo+IAo+IEl0IGlz
IHRoZW4ga2lsbGVkIGJ5IHN5c3RlbWQgZHVyaW5nIHNodXRkb3duIGFzIHBh cnQgb2YgdXNlciBz
ZXNzaW9uLgo+IEl0IHJlc3VsdHMgaW4gZGlydHkgYXJyYXkgb24gbmV4dCBi b290Lgo+IAo+IElz
IHRoZXJlIGFueSBtYWdpYyB0aGF0IGFsbG93cyBkYWVtb24gdG8gYmUgZXhl bXB0ZWQgZnJvbSBr
aWxsaW5nPwoKV2hpbGUgeW91ciByYWlkIHNob3VsZCBhYnNvbHV0ZWx5IG5v dCBiZSBjb3JydXB0
ZWQgb24gbmV4dCByZWJvb3QgCndoZW4gbWRtb24gcmVjZWl2ZXMgYSBTSUdU RVJNLCBJIHN1c3Bl
Y3QgeW91J3JlIHVzaW5nIHBhbV9zeXN0ZW1kLnNvIAooL2V0Yy9wYW0uZC9z eXN0ZW0tYXV0aCks
IHdoaWNoIGF1dG9tYXRpY2FsbHkgY3JlYXRlcyBjZ3JvdXBzIGJ5IGxvZ2lu IApzZXNzaW9uLCB3
aGljaCBpbiB0dXJuIGdldHMga2lsbGVkIHdoZW4gdGhlIHVzZXIgaGFzICJj b21wbGV0ZWx5IGxv
Z2dlZCBvdXQiLgpUaGF0IGlzIHdoeSB5b3VyIG1kYWRtIGdldHMgdGVybWlu YXRlZCwgdG9vLgpZ
b3UgY2FuIGF2b2lkIHRoYXQgYnkgYWRkaW5nIGNyZWF0ZS1zZXNzaW9uPTAg dG8gaXQsIGxpa2U6
CgojIGdyZXAgcGFtX3N5c3RlbWQgL2V0Yy9wYW0uZC9zeXN0ZW1kLWF1dGgK c2Vzc2lvbiAgICAg
b3B0aW9uYWwgICAgcGFtX3N5c3RlbWQuc28gY3JlYXRlLXNlc3Npb249MAoK V2hpY2ggaXMgdGhl
IHJlY29tbWVudGVkIHdheSBpZiB5b3Ugd2FudCBwcm9jZXNzZXMgKGNyZWF0 ZWQgYnkgdGhlICJ1
c2VyIikgdG8gCmxpdmUgb24gZXZlbiB3aGVuIHRoaXMgdXNlciBoYXMgZnVs bHkgbG9nZ2VkIG91
dC4KClJlZ2FyZHMsCkNocmlzdGlhbiBQYXJwYXJ0LgoKcC5zLjogc2VlIHBh bV9zeXN0ZW1kKDgp
Cl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fCnN5c3RlbWQt
ZGV2ZWwgbWFpbGluZyBsaXN0CnN5c3RlbWQtZGV2ZWxAbGlzdHMuZnJlZWRl c2t0b3Aub3JnCmh0
dHA6Ly9saXN0cy5mcmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5mby9z eXN0ZW1kLWRldmVs
Cg==

Re: systemd kills mdmon if it was started manually

T24gU2F0LCBEZWMgNCwgMjAxMCBhdCAxMjoxMiBQTSwgQ2hyaXN0aWFuIFBh cnBhcnQgPHRyYXBu
aUBnZW50b28ub3JnPiB3cm90ZToKPiBPbiBTYXR1cmRheSwgRGVjZW1iZXIg MDQsIDIwMTAgMDk6
NDE6MjYgYW0gQW5kcmV5IEJvcnplbmtvdiB3cm90ZToKPj4gSWYgdXNlciBz dGFydHMgYXJyYXkg
bWFudWFsbHkgKG1kYWRtIC1BIC1zIGFzIGV4YW1wbGUpIGZyb20gd2l0aGlu Cj4+IHVzZXIgc2Vz
c2lvbiBhbmQgYXJyYXkgbmVlZHMgbWRtb24sIG1kbW9uIGJlY29tZXMgcGFy dCBvZiB1c2VyIHNl
c3Npb24KPj4gY29udHJvbCBncm91cDoKPj4KPj4g4pScIHVzZXIKPj4g4pSC IOKUlCByb290Cj4+
IOKUgiDCoCDilJQgMQo+PiDilIIgwqAgwqAg4pScIDE5MTYgbG9naW4gLS0g cm9vdAo+PiDilIIg
wqAgwqAg4pScIDE5MzAgLWJhc2gKPj4g4pSCIMKgIMKgIOKUnCAxOTY0IGdw Zy1hZ2VudCAtLWtl
ZXAtZGlzcGxheSAtLWRhZW1vbiAtLXdyaXRlLWVudi1maWxlCj4+IC9yb290 Ly5nbnVwLi4uIOKU
giDCoCDCoCDilJQgMjA2MiBtZG1vbiBtZDEyNwo+Pgo+Pgo+PiBJdCBpcyB0 aGVuIGtpbGxlZCBi
eSBzeXN0ZW1kIGR1cmluZyBzaHV0ZG93biBhcyBwYXJ0IG9mIHVzZXIgc2Vz c2lvbi4KPj4gSXQg
cmVzdWx0cyBpbiBkaXJ0eSBhcnJheSBvbiBuZXh0IGJvb3QuCj4+Cj4+IElz IHRoZXJlIGFueSBt
YWdpYyB0aGF0IGFsbG93cyBkYWVtb24gdG8gYmUgZXhlbXB0ZWQgZnJvbSBr aWxsaW5nPwo+Cj4g
V2hpbGUgeW91ciByYWlkIHNob3VsZCBhYnNvbHV0ZWx5IG5vdCBiZSBjb3Jy dXB0ZWQgb24gbmV4
dCByZWJvb3QKPiB3aGVuIG1kbW9uIHJlY2VpdmVzIGEgU0lHVEVSTSwKClRo aXMgd29uJ3QgYmUg
Y29ycnVwdGVkIGJ1dCBpdCB3aWxsIGluaXRpYXRlIHJlYnVpbHQuIEkgaGF2 ZSByZXBvcnRzCnRo
YXQgc3VjaCByZWJ1aWxkIG1heSB0YWtlIGhvdXJzLCBjb3N0aW5nIHBlcmZv cm1hbmNlIGFuZCBs
b3NzIG9mCnJlZHVuZGFuY3kuCgo+ICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIEkgc3VzcGVjdCB5b3UncmUgdXNpbmcgcGFt X3N5c3RlbWQuc28K
ClllcwoKPiAoL2V0Yy9wYW0uZC9zeXN0ZW0tYXV0aCksIHdoaWNoIGF1dG9t YXRpY2FsbHkgY3Jl
YXRlcyBjZ3JvdXBzIGJ5IGxvZ2luCj4gc2Vzc2lvbiwgd2hpY2ggaW4gdHVy biBnZXRzIGtpbGxl
ZCB3aGVuIHRoZSB1c2VyIGhhcyAiY29tcGxldGVseSBsb2dnZWQgb3V0Ii4K PiBUaGF0IGlzIHdo
eSB5b3VyIG1kYWRtIGdldHMgdGVybWluYXRlZCwgdG9vLgoKU3VyZS4KCj4g WW91IGNhbiBhdm9p
ZCB0aGF0IGJ5IGFkZGluZyBjcmVhdGUtc2Vzc2lvbj0wIHRvIGl0LCBsaWtl Ogo+Cj4gIyBncmVw
IHBhbV9zeXN0ZW1kIC9ldGMvcGFtLmQvc3lzdGVtZC1hdXRoCj4gc2Vzc2lv biDCoCDCoCBvcHRp
b25hbCDCoCDCoHBhbV9zeXN0ZW1kLnNvIGNyZWF0ZS1zZXNzaW9uPTAKPgoK QnV0IEkgZG8gd2Fu
dCB1c2VyIHNlc3Npb24gdG8gYmUgY3JlYXRlZDsgYW5kIHN5c3RlbWQgd2Fz IHNwZWNpZmljYWxs
eQpleHRlbmRlZCB0byBwcm9wZXJseSB0ZXJtaW5hdGUgdXNlciBzZXNzaW9u cyBvbiBzaHV0ZG93
bi4gSXQgaXMganVzdAp0aGF0IG1kbW9uIGRvZXMgbm90IGJlbG9uZyB0byB1 c2VyIHNlc3Npb24g
YXQgYWxsLgoKPiBXaGljaCBpcyB0aGUgcmVjb21tZW50ZWQgd2F5IGlmIHlv dSB3YW50IHByb2Nl
c3NlcyAoY3JlYXRlZCBieSB0aGUgInVzZXIiKSB0bwo+IGxpdmUgb24gZXZl biB3aGVuIHRoaXMg
dXNlciBoYXMgZnVsbHkgbG9nZ2VkIG91dC4KPgoKbWRtb24gZG9lcyBub3Qg YmVsb25nIHRvIHVz
ZXIuIFVzZXIgaXMgbm90IGV2ZW4gYXdhcmUgdGhhdCBpdCBpcwpzdGFydGVk LiBBbmQgaXQgaXMg
bGlrZWx5IG5vdCB0aGUgbGFzdCBjYXNlLiBTbyBzeXN0ZW1kIGRvZXMgbmVl ZCBzb21lCmZyYW1l
d29yayB3aGljaCBjYW4gbW92ZSBzdWNoIHByb2Nlc3NlcyBvdXQgb2YgdXNl ciBzZXNzaW9uLiBJ
dApwcm9iYWJseSBuZWVkcyBzb21lIHNkX2RhZW1vbiBBUEkgdG8gbm90aWZ5 IHN5c3RlbWQgdGhh
dCBpdCBpcyBzeXN0ZW0KbGV2ZWwgdGFzayBldmVuIGlmIGl0IHdhcyBzdGFy dGVkIGFzIHJlc3Vs
dCBvZiB1c2VyIGludGVyYWN0aW9uLgpfX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19f
X19fX19fX19fX19fXwpzeXN0ZW1kLWRldmVsIG1haWxpbmcgbGlzdApzeXN0 ZW1kLWRldmVsQGxp
c3RzLmZyZWVkZXNrdG9wLm9yZwpodHRwOi8vbGlzdHMuZnJlZWRlc2t0b3Au b3JnL21haWxtYW4v
bGlzdGluZm8vc3lzdGVtZC1kZXZlbAo=

Re: [systemd-devel] systemd kills mdmon if it was started manually

On Sat, Dec 04, 2010 at 03:08:05PM +0300, Andrey Borzenkov wrote:
>mdmon does not belong to user. User is not even aware that it is
>started. And it is likely not the last case. So systemd does need some
>framework which can move such processes out of user session. It
>probably needs some sd_daemon API to notify systemd that it is system
>level task even if it was started as result of user interaction.

what about running mdmon --all --takeover outside of user context at
shutdown, it should replace all mdmon processes with new ones that won't
be killed when user sessions are being closed?

Regards,
L.

--
Luca Berra -- bluca@comedia.it
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: systemd kills mdmon if it was started manuallyby user

T24gU2F0LCAwNC4xMi4xMCAxMTo0MSwgQW5kcmV5IEJvcnplbmtvdiAoYXJ2 aWRqYWFyQGdtYWls
LmNvbSkgd3JvdGU6Cgo+IElmIHVzZXIgc3RhcnRzIGFycmF5IG1hbnVhbGx5 IChtZGFkbSAtQSAt
cyBhcyBleGFtcGxlKSBmcm9tIHdpdGhpbgo+IHVzZXIgc2Vzc2lvbiBhbmQg YXJyYXkgbmVlZHMg
bWRtb24sIG1kbW9uIGJlY29tZXMgcGFydCBvZiB1c2VyIHNlc3Npb24KPiBj b250cm9sIGdyb3Vw
OgoKQXJlIHlvdSBzdWdnZXN0aW5nIHRoYXQgbWRhZG0gZm9ya3Mgb2ZmIG1k bW9uIGZyb20gd2l0
aGluIHRoZSB1c2VyCnNlc3Npb24/IFRoaXMgaXMgaG9ycmlibHkgdWdseSBh bmQgYnJva2VuIGFu
ZCB0aGV5IHNob3VsZG4ndCBkbyB0aGF0LgoKPiAKPiDilJwgdXNlcgo+IOKU giDilJQgcm9vdAo+
IOKUgiAgIOKUlCAxCj4g4pSCICAgICDilJwgMTkxNiBsb2dpbiAtLSByb290 Cj4g4pSCICAgICDi
lJwgMTkzMCAtYmFzaAo+IOKUgiAgICAg4pScIDE5NjQgZ3BnLWFnZW50IC0t a2VlcC1kaXNwbGF5
IC0tZGFlbW9uIC0td3JpdGUtZW52LWZpbGUgL3Jvb3QvLmdudXAuLi4KPiDi lIIgICAgIOKUlCAy
MDYyIG1kbW9uIG1kMTI3Cj4gCj4gCj4gSXQgaXMgdGhlbiBraWxsZWQgYnkg c3lzdGVtZCBkdXJp
bmcgc2h1dGRvd24gYXMgcGFydCBvZiB1c2VyIHNlc3Npb24uCgpXZWxsLCBv bmx5IGlmIHlvdSBl
bmFibGUgdGhhdCB0aGUgdXNlciBzZXNzaW9uIGlzIGNvbXBsZXRlbHkga2ls bGVkIG9uCmxvZ291
dCwgd2hpY2ggd2UgY3VycmVudGx5IGRvbid0IGRvIGJ5IGRlZmF1bHQuCgpJ IHdvbmRlciBpZiBp
dCB3b3VsZCBtYWtlIHNlbnNlIHRvIGFkZCBhbiBvcHRpb24gd2hpY2gga2ls bHMgdXNlcgpzZXNz
aW9ucyBvbiBsb2cgb3V0IG9ubHkgZm9yIHVpZCAhPSAwLiBUaGlzIG1pZ2h0 IGhlbHAgaGVyZSwg
YnV0IG9ubHkKaGFsZi13YXksIHNpbmNlIHN1ZG8gd291bGQgc3RpbGwgYnJl YWsuIEJ1dCBhbnl3
YXksIEknbGwgYWRkIHRoaXMgdG8gdGhlCnRvZG8gbGlzdC4KCj4gSXQgcmVz dWx0cyBpbiBkaXJ0
eSBhcnJheSBvbiBuZXh0IGJvb3QuCgpIbW0sIHRoYXQgc2hvdWxkbid0IGhh cHBlbi4KCj4gSXMg
dGhlcmUgYW55IG1hZ2ljIHRoYXQgYWxsb3dzIGRhZW1vbiB0byBiZSBleGVt cHRlZCBmcm9tIGtp
bGxpbmc/CgpXZWxsLCBJIGhhdmUgYmVlbiBkaXNjdXNzaW5nIHRoaXMgd2l0 aCBLYXkgYW5kIHdl
J2xsIG1vc3QgbGlrZWx5IGFkZApzb21ldGhpbmcgbGlrZSBEb250S2lsbE9u U2h1dGRvd249eWVz
IG9yIHNvLCB3aGljaCBpZiBhZGRlZCB0byBhIHVuaXQKZmlsZSB3aWxsIGV4 ZW1wdCBpdCBmcm9t
IGtpbGxpbmcgZHVyaW5nIHRoZSBub3JtYWwgc2VydmljZSBzaHV0ZG93bgpw aGFzZSwgYW5kIHRo
ZSBmaXJzdCBraWxsaW5nIHNwcmVlIChidXQgbm90IHRoZSBzZWNvbmQsIHBv c3QtdW1vdW50Cmtp
bGxpbmcgc3ByZWUpLiBCdXQgdGhhdCBvZiBjb3Vyc2Ugd291bGQgcmVxdWly ZSBtZG1vbiB0byBi
ZSBzdGFydGVkCmxpa2UgYW55IG90aGVyIGRhZW1vbiwgYW5kIG5vdCBmb3Jr ZWQgb2ZmIG1kYWRt
LgoKVGhhdCBzaG91bGQgbW9zdGx5IGZpeCB0aGUgcHJvYmxlbSwgYnV0IHRo ZW4gYWdhaW4gSSBk
byBiZWxpZXZlIHRoYXQgdGhlCndob2xlIGlkZWEgb2YgbWRtb24gaXMganVz dCBib3JrZWQsIHNp
bmNlIGl0IHdpbGwgbmVjZXNzYXJpbHkgcGluIHBhZ2UKZnJvbSB0aGUgcm9v dCBmcyBpbnRvIG1l
bW9yeSB3aGljaCB3aWxsIGNyZWF0ZSBhbGwga2luZHMgb2YgcHJvYmxlbXMs CmZvciBleGFtcGxl
IGFmdGVyIHVwZ3JhZGVzIChpLmUuIG1kbW9uIG1hcHMgbGliYyBpbnRvIG1l bW9yeSwgbGliYyBn
ZXRzCnVwZGF0ZWQsIHRoZSBvbGQgbGliYyBkZWxldGVkLCB3aGljaCBjYW5u b3QgYmUgd3JpdHRl
biB0byBkaXNrIGFzIGxvbmcKYXMgbWRtb24gc3RheXMgcnVubmluZyBwaW5u aW5nIGl0LCB3aGlj
aCB3aWxsIGRpc2FsbG93IHRoZSB1bHRpbWF0ZQp1bm1vdW50aW5nL3JlbW91 bnRpbmcgb2YgdGhl
IGZzKS4KCkxlbm5hcnQKCi0tIApMZW5uYXJ0IFBvZXR0ZXJpbmcgLSBSZWQg SGF0LCBJbmMuCl9f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f CnN5c3RlbWQtZGV2
ZWwgbWFpbGluZyBsaXN0CnN5c3RlbWQtZGV2ZWxAbGlzdHMuZnJlZWRlc2t0 b3Aub3JnCmh0dHA6
Ly9saXN0cy5mcmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5mby9zeXN0 ZW1kLWRldmVsCg==

Re: systemd kills mdmon if it was started manuallyby user

On Sat, 04.12.10 15:08, Andrey Borzenkov (arvidjaar@gmail.com) wrote:

> >> It is then killed by systemd during shutdown as part of user session.
> >> It results in dirty array on next boot.
> >>
> >> Is there any magic that allows daemon to be exempted from killing?
> >
> > While your raid should absolutely not be corrupted on next reboot
> > when mdmon receives a SIGTERM,
>
> This won't be corrupted but it will initiate rebuilt. I have reports
> that such rebuild may take hours, costing performance and loss of
> redundancy.

Well, eventually we need to be able to kill mdmon. Otherwise we might
not be able to remount the root dir r/o. How exactly is mdmon supposed
to behave on shutdown?

Lennart

--
Lennart Poettering - Red Hat, Inc.

Re: [systemd-devel] systemd kills mdmon if it was started manually by user

2011/1/7 Lennart Poettering :
>
> Well, I have been discussing this with Kay and we'll most likely add
> something like DontKillOnShutdown=yes or so, which if added to a unit

Make that KillOnShutdown=no, please.


--
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [systemd-devel] systemd kills mdmon if it was started manuallyby user

On Fri, 7 Jan 2011 01:38:27 +0100 Lennart Poettering
wrote:

> On Sat, 04.12.10 11:41, Andrey Borzenkov (arvidjaar@gmail.com) wrote:
>
> > If user starts array manually (mdadm -A -s as example) from within
> > user session and array needs mdmon, mdmon becomes part of user session
> > control group:
>
> Are you suggesting that mdadm forks off mdmon from within the user
> session? This is horribly ugly and broken and they shouldn't do that.

What alternative would you suggest?

A daemon needs to be running while certain md arrays are running and writable.

NeilBrown
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [systemd-devel] systemd kills mdmon if it was started manuallyby user

--Sig_/t8FbUxgLd9Qv0j7Eyn+0Vai
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

On Fri, 7 Jan 2011 02:09:32 +0100
Michael Biebl wrote:

> 2011/1/7 Lennart Poettering :
> >
> > Well, I have been discussing this with Kay and we'll most likely add
> > something like DontKillOnShutdown=3Dyes or so, which if added to a unit
>=20
> Make that KillOnShutdown=3Dno, please.

Agreed :) That reminds me of "hal-disable-polling --enable-polling"
( http://ur1.ca/2rmis )

--=20
With respect,
Roman

--Sig_/t8FbUxgLd9Qv0j7Eyn+0Vai
Content-Type: application/pgp-signature; name=signature.asc
Content-Disposition: attachment; filename=signature.asc

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAk0maboACgkQTLKSvz+PZwgp2wCdHHPVlgR6mLC5tPQUJ5qO GWbO
PbQAoIDHpklXCauKRSArPrw3av/PV2sV
=BVBc
-----END PGP SIGNATURE-----

--Sig_/t8FbUxgLd9Qv0j7Eyn+0Vai--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [systemd-devel] systemd kills mdmon if it was started manually by user

On Fri, 07.01.11 12:16, NeilBrown (neilb@suse.de) wrote:

>
> On Fri, 7 Jan 2011 01:38:27 +0100 Lennart Poettering
> wrote:
>
> > On Sat, 04.12.10 11:41, Andrey Borzenkov (arvidjaar@gmail.com) wrote:
> >
> > > If user starts array manually (mdadm -A -s as example) from within
> > > user session and array needs mdmon, mdmon becomes part of user session
> > > control group:
> >
> > Are you suggesting that mdadm forks off mdmon from within the user
> > session? This is horribly ugly and broken and they shouldn't do that.
>
> What alternative would you suggest?

Start it as a normal service like any other. But if you fork off the
daemon from the user session then the daemon will run in a very broken
context: the resource limits of the user apply, the audit trail will
point to the user (i.e. /proc/self/loginuid), the cgroup will be of the
user, the daemon cannot be supervised as every other daemon. Also, the
daemon will inherit all the other process properties from the user,
which is almost definitely wrong. i.e. the env block and so
on, the sig mask. gazillions of small little properties. Of course, a
big bunch of them you can reset in your code, but that's a race you
cannot win: the kernel adds new process properties all the time, and
you'd have to reset them manually.

It's is really essential that daemons are started from a clean process
environment, and are detached from the user session. SysV kinda provides
that, for everything started on boot and in a limited way for stuff
started via /sbin/service. systemd provides that too and much more
correct. But just forking off things just like that is not a good
solution.

A thinkable, relatively simple solution in a systemd world is to pull in
the mdmon service from the udev device. The udev device would do all the
necessary matching to figure out whether mdmon is needed or not. If you
care about non-systemd environments something like this of course
becomes a lot more complex.

> A daemon needs to be running while certain md arrays are running and writable.

Well, but auto-spawning it from the user session is not really a usable solution.

Lennart

--
Lennart Poettering - Red Hat, Inc.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: systemd kills mdmon if it was started manually

T24gVHVlLCBKYW4gMjUsIDIwMTEgYXQgNzoyOCBBTSwgTGVubmFydCBQb2V0 dGVyaW5nCjxsZW5u
YXJ0QHBvZXR0ZXJpbmcubmV0PiB3cm90ZToKPiBPbiBUdWUsIDI1LjAxLjEx IDA2OjU4LCBBbmRy
ZXkgQm9yemVua292IChhcnZpZGphYXJAbWFpbC5ydSkgd3JvdGU6Cj4KPj4g PiBzeXN0ZW1kIHN1
cHBvcnRzIGluc3RhbnRpYXRlZCBzZXJ2aWNlcywgZm9yIGV4YW1wbGUgdG8g ZGVhbCB3aXRoIHRo
ZQo+PiA+IGdldHR5cyAoZS5nLiAiZ2V0dHlAdHR5NS5zZXJ2aWNlIikuIEl0 IHNob3VsZCBiZSB0
cml2aWFsIHRvIHVzZSB0aGUgc2FtZQo+PiA+IGZvciBtZG1vbiAoZS5nLiAi bWRtb25AbWQzLnNl
cnZpY2UiKS4KPj4gPgo+PiBUaGF0J3MgcmlnaHQsIGJ1dCB0aGUgbmFtZXMg YXJlIG5vdCBrbm93
biBpbiBhZHZhbmNlIGFuZCBjYW4gY2hhbmdlCj4+IGJldHdlZW4gcmVib290 cy4gVGhpcyBtZWFu
cyBzdWNoIHVuaXRzIGhhdmUgdG8gYmUgZ2VuZXJhdGVkCj4+IGR5bmFtaWNh bGx5LCBleGlzdCB1
bnRpbCByZWJvb3QgKHJhbWZzPykgYW5kIGJlIHJlbW92ZWQgd2hlbiBhcnJh eSBpcwo+PiBkZXN0
cm95ZWQuIE5vdCBzdXJlIGl0IGlzIHJlYWxseSBtYW5hZ2VhYmxlLgo+Cj4g SG1tPyBJdCBzaG91
bGQgYmUgc3VmZmljaWVudCB0byBqdXN0IHdyaXRlIHRoZSBzZXJ2aWNlIHRl bXBsYXRlIHByb3Bl
cmx5Cj4gKCJtZG1vbkAuc2VydmljZSIpIGFuZCB0aGVuIGluc3RhbnRpYXRl IGl0IHdoZW4gbmVl
ZGVkIHdpdGggInN5c3RlbWN0bAo+IHN0YXJ0IG1kbW9uQHh5ei5zZXJ2aWNl IiBvciBzb21ldGhp
bmcgZXF1aXZhbGVudC4gaXRNcyBhIG1hdHRlciBvZgo+IGlzc3VpbmcgYSBz aW5nbGUgZGJ1cyBj
YWxsLgo+Cj4+IEFuZCB3aGljaCBpbnN0YW5jZSBzaG91bGQgZ2VuZXJhdGUg dGhlbT8gbWRhZG0/
Cj4KPiBpIHRoaW5rIGl0IGlzIG11Y2ggbmljZXIgdG8gc3Bhd24gdGhlIG5l Y2Vzc2FyeSBtZGFk
bSBzZXJ2aWNlIGluc3RhbmNlCj4gZnJvbSBhIHVkZXYgcnVsZSwKClllcywg dGhpcyBjYW4gYmUg
ZG9uZSByZWxhdGl2ZWx5IGVhc2lseTsgYXMgcHJvb2Ygb2YgY29uY2VwdDoK ClNVQlNZU1RFTSE9
ImJsb2NrIiwgR09UTz0ic3lzdGVtZF9tZF9lbmQiCkFDVElPTiE9ImNoYW5n ZSIsIEdPVE89InN5
c3RlbWRfbWRfZW5kIgpLRVJORUwhPSJtZCoiLCBHT1RPPSJzeXN0ZW1kX21k X2VuZCIKQVRUUntt
ZC9tZXRhZGF0YV92ZXJzaW9ufT09ImV4dGVybmFsOltBLVphLXpdKiIsIFJV Tis9Ii9iaW4vc3lz
dGVtY3RsCnN0YXJ0IG1kbW9uQCVrLnNlcnZpY2UiCkxBQkVMPSJzeXN0ZW1k X21kX2VuZCIKCndo
ZXJlIG1kb25ALnNlcnZpY2UgaXMKCgpbVW5pdF0KRGVzY3JpcHRpb249bWRt b24gc2VydmljZQpC
aW5kVG89ZGV2LSVpLmRldmljZQpBZnRlcj1kZXYtJWkuZGV2aWNlCgpbU2Vy dmljZV0KVHlwZT1m
b3JraW5nClBJREZpbGU9L2Rldi8ubWRhZG0vJWkucGlkCkV4ZWNTdGFydD0v c2Jpbi9tZG1vbiAt
LXRha2VvdmVyICVpCgoKV2l0aCB0aGUgcmVzdWx0Cgpbcm9vdEBsb2NhbGhv c3Qgfl0jIHN5c3Rl
bWN0bCBzdGF0dXMgbWRtb25AbWQxMjcuc2VydmljZQptZG1vbkBtZDEyNy5z ZXJ2aWNlIC0gbWRt
b24gc2VydmljZQogICAgICAgICAgTG9hZGVkOiBsb2FkZWQgKC9ldGMvc3lz dGVtZC9zeXN0ZW0v
bWRtb25ALnNlcnZpY2UpCiAgICAgICAgICBBY3RpdmU6IGFjdGl2ZSAocnVu bmluZykgc2luY2Ug
VHVlLCAwOCBGZWIgMjAxMSAwOTo0MzozMAotMDUwMDsgNW1pbiBhZ28KICAg ICAgICAgUHJvY2Vz
czogMTQ2NyBFeGVjU3RhcnQ9L3NiaW4vbWRtb24gLS10YWtlb3ZlciAlaQoo Y29kZT1leGl0ZWQs
IHN0YXR1cz0wL1NVQ0NFU1MpCiAgICAgICAgTWFpbiBQSUQ6IDE0NjggKG1k bW9uKQogICAgICAg
ICAgQ0dyb3VwOiBuYW1lPXN5c3RlbWQ6L3N5c3RlbS9tZG1vbkAuc2Vydmlj ZS9tZDEyNwogICAg
ICAgICAgICAgICAgICDilJQgMTQ2OCAvc2Jpbi9tZG1vbiAtLXRha2VvdmVy IG1kMTI3CgpTZXR0
aW5nIFNZU1RFTURfV0FOVFMgd291bGQgYmUgbW9yZSBlbGVnYW50IHNvbHV0 aW9uLCBidXQgaXQg
ZG9lcyBub3QKd29yayB3aXRoIGN1cnJlbnQgc3lzdGVtZCBpbXBsZW1lbnRh dGlvbi4gSXQgaXMg
Y2FwYWJsZSBvZiBzdGFydGluZwpyZXF1ZXN0ZWQgdW5pdHMgb25seSBvbiAi YWRkIiBldmVudCAo
ZWZmZWN0aXZlbHkgdGhlIHZlcnkgZmlyc3QgdGltZQpkZXZpY2UgYmVjb21l cyBwbHVnZ2VkKSwg
d2hpbGUgbWRtb24gbXVzdCBiZSBzdGFydGVkIG9uICJjaGFuZ2UiCmV2ZW50 LCBhcyBvbmx5IHRo
ZW4gd2Uga25vdyB3aGV0aGVyIG1kbW9uIGlzIHJlcXVpcmVkIGF0IGFsbC4K ClJ1bm5pbmcgbWRt
b24gdmlhIHN5c3RlbWQgaW4gdGhpcyB3YXkgb3BlbnMgdXAgaW50ZXJlc3Rp bmcKcG9zc2liaWxp
dHkuIEUuZy4gc2VydmljZSBjb3VsZCBiZSBkZWNsYXJlZCAiaW1tb3J0YWwi IGFuZCBiZSBleGVt
cHQKZnJvbSB1c3VhbCBzaHV0ZG93biBzZXF1ZW5jZSAuLi4gb3IgaXMgaXQg cG9zc2libGUgdG8g
ZG8gYWxyZWFkeT8KCkFjdHVhbGx5IGl0IGNhbiBiZSBpbXBsZW1lbnRlZCBl dmVuIHdpdGhvdXQg
bWRhZG0gcGF0Y2hlczsgYXBwYXJlbnRseQppdCBpcyBwb3NzaWJsZSB0byBz dXBwcmVzcyBub3Jt
YWwgc3RhcnRpbmcgb2YgbWRtb24gYnkgc2V0dGluZwpNREFETV9OT19NRE1P Tj0xCgo+ICAgICAg
ICAgICAgICAgICAgICAgICAgICAgYnV0IHlvdSBjb3VsZCBldmVuIHJ1biBp dCBmcm9tIG1kYWRt
IGJ5IGludm9raW5nIG9uZQo+IGRidXMgY2FsbCBmcm9tIGl0Lgo+CgpJdCBt YXkgdHVybiBvdXQg
dG8gYmUgbmVjZXNzYXJ5IHN0aWxsLiBJZiBjb250YWluZXIgbmVlZHMgbWRt b24sCmFycmF5cyBp
dCBjb250YWlucyB3b24ndCBiZWNvbWUgcmVhZC13cml0ZSB1bnRpbCBtZG1v biBpcyBzdGFydGVk
LiBJZgptZG1vbiBpcyBzdGFydGVkIGFzeW5jaHJvbm91c2x5IGJ5IHVkZXYs IHRoZXJlIGlzIHdp
bmRvdyB3aGVyZSBzb21lb25lCm1heSB0cnkgdG8gdXNlIGFycmF5IGJlZm9y ZSBpdCBpcyBydy4g
QXMgdHJpdmlhbCBleGFtcGxlLCBtb3VudCB1bml0CndoaWNoIGRlcGVuZHMg b24gbWQgZGV2aWNl
IHVuaXQuCgpJIGRvIG5vdCB0aGluayBtZGFkbSBtYWludGFpbmVyIHdpbGwg YmUgaGFwcHkgdG8g
YWRkIEQtQnVzIGRlcGVuZGVuY3kKdG8gc29tZXRoaW5nIHRoYXQgaXMgbGlr ZWx5IHRvIGJlIGlu
Y2x1ZGVkIGluIGluaXRyZCB0aG91Z2ggOikgQnV0IG1heQpiZSB3ZSBjb3Vs ZCBzaW1wbHkgdHJ5
IGV4ZWNsKCIvYmluL3N5c3RlbWN0bCIsICJzdGFydCIsIC4uLikgYmVmb3Jl CmN1cnJlbnQgZXhl
Y2woIi9zYmluL21kbW9uIiwuLi4gKT8KX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19f
X19fX19fX19fX19fX18Kc3lzdGVtZC1kZXZlbCBtYWlsaW5nIGxpc3QKc3lz dGVtZC1kZXZlbEBs
aXN0cy5mcmVlZGVza3RvcC5vcmcKaHR0cDovL2xpc3RzLmZyZWVkZXNrdG9w Lm9yZy9tYWlsbWFu
L2xpc3RpbmZvL3N5c3RlbWQtZGV2ZWwK

Re: [systemd-devel] systemd kills mdmon if it was started manuallyby user

On Fri, 04.02.11 22:55, Andrey Borzenkov (arvidjaar@mail.ru) wrote:

> >> That's right, but the names are not known in advance and can change
> >> between reboots. This means such units have to be generated
> >> dynamically, exist until reboot (ramfs?) and be removed when array is
> >> destroyed. Not sure it is really manageable.
> >
> > Hmm? It should be sufficient to just write the service template properly
> > ("mdmon@.service") and then instantiate it when needed with "systemctl
> > start mdmon@xyz.service" or something equivalent. itMs a matter of
> > issuing a single dbus call.
> >
> >> And which instance should generate them? mdadm?
> >
> > i think it is much nicer to spawn the necessary mdadm service instance
> > from a udev rule,
>
> Yes, this can be done relatively easily; as proof of concept:
>
> SUBSYSTEM!="block", GOTO="systemd_md_end"
> ACTION!="change", GOTO="systemd_md_end"
> KERNEL!="md*", GOTO="systemd_md_end"
> ATTR{md/metadata_version}=="external:[A-Za-z]*", RUN+="/bin/systemctl
> start mdmon@%k.service"
> LABEL="systemd_md_end"

Nah, it's much better to simply use the SYSTEMD_WANTS var on the device.

Something like this:

....., ENV{SYSTEMD_WANTS}="mdmon@%k.service"

That way the device unit will simply have a wants dep on the service
unit, and this is prefectly discoverable.

> Setting SYSTEMD_WANTS would be more elegant solution, but it does not
> work with current systemd implementation. It is capable of starting
> requested units only on "add" event (effectively the very first time
> device becomes plugged), while mdmon must be started on "change"
> event, as only then we know whether mdmon is required at all.

Oha, so you are actually aware of SYSTEMD_WANTS. Hmm. I need to think
about this. Why does md employ the change event? Is this really
necessary, smells a bit foul.

> Running mdmon via systemd in this way opens up interesting
> possibility. E.g. service could be declared "immortal" and be exempt
> from usual shutdown sequence ... or is it possible to do already?

A service needs to conflict with shutdown.target to be shut down when we
go down normally. If your service does not conflict with shutdown.target
then it will stay around and be killed only after systemd is gone and
PID1 is systemd-shutdown which then kills all processes remaining
(independent of any idea of "service") and the unmounts all file
systems. Normally all services conflict with shutdown.target implicitly,
which you can turn off by setting DefaultDependencies=.

> Actually it can be implemented even without mdadm patches; apparently
> it is possible to suppress normal starting of mdmon by setting
> MDADM_NO_MDMON=1

A this point mdmon is simply broken: if glibc or mdmon itself (or any
lib it is using) is upgraded, then mdmon will keep referencing the old
..so or binary as long as it is running. This means that the fs these
files are on cannot be remounted r/o. However mdmon insists on being
shutdown only after all fs got remounted ro. So you have a cyclic
ordering loop here: mdmon wants to be shut down after the remount, but
we need to shut it down before the remount.

This is unfixable unless a) mdmon learns reexecution of itself without
losing state (like most init systems so), or b) mdmon would stop
insisting on being shutdown only after the remount.

In my eyes b) is very much preferebale: It should be possible to shut
down mdmon like any other service. And if then some md related code
still needs to be run on late shutdown this should be done from a new
process. I would be willing to add some hooks for this, so that we can
execute arbitrary drop-in processes as part of the final shutdown loop.

Lennart

--
Lennart Poettering - Red Hat, Inc.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: systemd kills mdmon if it was started manually

On Tue, Feb 8, 2011 at 12:48 PM, Lennart Poettering
wrote:
> On Fri, 04.02.11 22:55, Andrey Borzenkov (arvidjaar@mail.ru) wrote:
>
>> >> That's right, but the names are not known in advance and can change
>> >> between reboots. This means such units have to be generated
>> >> dynamically, exist until reboot (ramfs?) and be removed when array is
>> >> destroyed. Not sure it is really manageable.
>> >
>> > Hmm? It should be sufficient to just write the service template properly
>> > ("mdmon@.service") and then instantiate it when needed with "systemctl
>> > start mdmon@xyz.service" or something equivalent. itMs a matter of
>> > issuing a single dbus call.
>> >
>> >> And which instance should generate them? mdadm?
>> >
>> > i think it is much nicer to spawn the necessary mdadm service instance
>> > from a udev rule,
>>
>> Yes, this can be done relatively easily; as proof of concept:
>>
>> SUBSYSTEM!="block", GOTO="systemd_md_end"
>> ACTION!="change", GOTO="systemd_md_end"
>> KERNEL!="md*", GOTO="systemd_md_end"
>> ATTR{md/metadata_version}=="external:[A-Za-z]*", RUN+="/bin/systemctl
>> start mdmon@%k.service"
>> LABEL="systemd_md_end"
>
> Nah, it's much better to simply use the SYSTEMD_WANTS var on the device.
>
> Something like this:
>
> ...., ENV{SYSTEMD_WANTS}="mdmon@%k.service"
>
> That way the device unit will simply have a wants dep on the service
> unit, and this is prefectly discoverable.
>
>> Setting SYSTEMD_WANTS would be more elegant solution, but it does not
>> work with current systemd implementation. It is capable of starting
>> requested units only on "add" event (effectively the very first time
>> device becomes plugged), while mdmon must be started on "change"
>> event, as only then we know whether mdmon is required at all.
>
> Oha, so you are actually aware of SYSTEMD_WANTS. Hmm. I need to think
> about this. Why does md employ the change event? Is this really
> necessary, smells a bit foul.
>

I am probably the wrong one to ask, but here is what happens when
array is started (from udev perspective)

UDEV [1297507039.109828] add /devices/virtual/block/md127 (block)
UDEV_LOG=3
ACTION=add
DEVPATH=/devices/virtual/block/md127
SUBSYSTEM=block
DEVNAME=/dev/md127
DEVTYPE=disk
SEQNUM=1742
UDISKS_PRESENTATION_NOPOLICY=1
MAJOR=9
MINOR=127
TAGS=:systemd:

After this event device goes "plugged" and SYSTEMD_WANTS (if any) are
triggered. But at this point we have zero information about array to
decide anything.

UDEV [1297507039.211940] change /devices/virtual/block/md127 (block)
UDEV_LOG=3
ACTION=change
DEVPATH=/devices/virtual/block/md127
SUBSYSTEM=block
DEVNAME=/dev/md127
DEVTYPE=disk
SEQNUM=1743
MD_LEVEL=container
MD_DEVICES=2
MD_METADATA=ddf
MD_UUID=f8362f39:0436b20f:cf338104:afec436e
MD_DEVNAME=ddf0
UDISKS_PRESENTATION_NOPOLICY=1
MAJOR=9
MINOR=127
DEVLINKS=/dev/disk/by-id/md-uuid-f8362f39:0436b20f:cf338104: afec436e
/dev/md/ddf0
TAGS=:systemd:

At this point we know it is container, know that it has external
metadata and know that we need external metadata handler (mdmon). But
it is too late for systemd.

>
>> Actually it can be implemented even without mdadm patches; apparently
>> it is possible to suppress normal starting of mdmon by setting
>> MDADM_NO_MDMON=1
>
> A this point mdmon is simply broken: if glibc or mdmon itself (or any
> lib it is using) is upgraded, then mdmon will keep referencing the old
> .so or binary as long as it is running. This means that the fs these
> files are on cannot be remounted r/o. However mdmon insists on being
> shutdown only after all fs got remounted ro. So you have a cyclic
> ordering loop here: mdmon wants to be shut down after the remount, but
> we need to shut it down before the remount.
>

Ehh ...

a) mdmon is perfectly capable of restarting, it is already used to
take over mdmon launched in initrd. The problem is to know when to
restart - i.e. when respective libraries are changed. This is a job
for package management in distribution. It is already employed for
glibc, systemd and some others and can just as well be employed for
mdmon. And this is totally unrelated to systemd :)

b) having binary launched off some fs should not prevent this fs to be
remountd ro - binaries are not opened rw

> This is unfixable unless a) mdmon learns reexecution of itself without
> losing state (like most init systems so), or b) mdmon would stop
> insisting on being shutdown only after the remount.
>

As far as I can tell, both is true today; but remounting is not
enough, unfortunately.

> In my eyes b) is very much preferebale: It should be possible to shut
> down mdmon like any other service. And if then some md related code
> still needs to be run on late shutdown this should be done from a new
> process. I would be willing to add some hooks for this, so that we can
> execute arbitrary drop-in processes as part of the final shutdown loop.
>

mdmon is needed to ensure metadata were correctly updated. So it needs
to exist as long as metadata *may* be updated. For practical purposes
it means - until file system is unmounted and flushed to disks. I am
not sure that remounting ro stops all activity (at least, mounting ro
definitely *writes* to device using some filesystems).

Re: systemd kills mdmon if it was started manuallyby user

On Tue, 08.02.11 13:52, Andrey Borzenkov (arvidjaar@mail.ru) wrote:

> I am probably the wrong one to ask, but here is what happens when
> array is started (from udev perspective)

[...]

> After this event device goes "plugged" and SYSTEMD_WANTS (if any) are
> triggered. But at this point we have zero information about array to
> decide anything.

[...]

> At this point we know it is container, know that it has external
> metadata and know that we need external metadata handler (mdmon). But
> it is too late for systemd.

Kay, do you know why this "change" event is used here? Any chance we can
get rid of it?

>
> >
> >> Actually it can be implemented even without mdadm patches; apparently
> >> it is possible to suppress normal starting of mdmon by setting
> >> MDADM_NO_MDMON=1
> >
> > A this point mdmon is simply broken: if glibc or mdmon itself (or any
> > lib it is using) is upgraded, then mdmon will keep referencing the old
> > .so or binary as long as it is running. This means that the fs these
> > files are on cannot be remounted r/o. However mdmon insists on being
> > shutdown only after all fs got remounted ro. So you have a cyclic
> > ordering loop here: mdmon wants to be shut down after the remount, but
> > we need to shut it down before the remount.
> >
>
> Ehh ...
>
> a) mdmon is perfectly capable of restarting, it is already used to
> take over mdmon launched in initrd. The problem is to know when to
> restart - i.e. when respective libraries are changed. This is a job
> for package management in distribution. It is already employed for
> glibc, systemd and some others and can just as well be employed for
> mdmon. And this is totally unrelated to systemd :)

Really, you are sying there is a synchronous way to make mdmon reexec
itself? How does that work?

> b) having binary launched off some fs should not prevent this fs to be
> remountd ro - binaries are not opened rw

If you run a binary and then the package manager replaces it then the
running instance will still refer to the old copy and this will have the
effect that the file isn't actually deleted until the proces
exits/execs. And because that is the way it is the kernel will refuse
unmounting of the fs until you terminated/reexeced your process.

> > This is unfixable unless a) mdmon learns reexecution of itself without
> > losing state (like most init systems so), or b) mdmon would stop
> > insisting on being shutdown only after the remount.
>
> As far as I can tell, both is true today; but remounting is not
> enough, unfortunately.

So, you are saying we can shut down mdmon without ill effects early?

> > In my eyes b) is very much preferebale: It should be possible to shut
> > down mdmon like any other service. And if then some md related code
> > still needs to be run on late shutdown this should be done from a new
> > process. I would be willing to add some hooks for this, so that we can
> > execute arbitrary drop-in processes as part of the final shutdown loop.
>
> mdmon is needed to ensure metadata were correctly updated. So it needs
> to exist as long as metadata *may* be updated. For practical purposes
> it means - until file system is unmounted and flushed to disks. I am
> not sure that remounting ro stops all activity (at least, mounting ro
> definitely *writes* to device using some filesystems).

Well, the root file systems cannot be unmounted, only remounted.

So, is there a way to invoke mdmon so that it flushes all metadata
changes to disk and immediately terminates then this should be all we
need for a clean solution. We'd then shutdown the normal instances of
mdmon down like any other daemon and simply invoke this metadata
flushing command as part of late shutdown.

Lennart

--
Lennart Poettering - Red Hat, Inc.

Re: systemd kills mdmon if it was started manually

On Tue, Feb 8, 2011 at 2:07 PM, Lennart Poettering
wrote:
> On Tue, 08.02.11 13:52, Andrey Borzenkov (arvidjaar@mail.ru) wrote:
>
>> I am probably the wrong one to ask, but here is what happens when
>> array is started (from udev perspective)
>
> [...]
>
>> After this event device goes "plugged" and SYSTEMD_WANTS (if any) are
>> triggered. But at this point we have zero information about array to
>> decide anything.
>
> [...]
>
>> At this point we know it is container, know that it has external
>> metadata and know that we need external metadata handler (mdmon). But
>> it is too late for systemd.
>
> Kay, do you know why this "change" event is used here? Any chance we can
> get rid of it?
>
>>
>> >
>> >> Actually it can be implemented even without mdadm patches; apparently
>> >> it is possible to suppress normal starting of mdmon by setting
>> >> MDADM_NO_MDMON=1
>> >
>> > A this point mdmon is simply broken: if glibc or mdmon itself (or any
>> > lib it is using) is upgraded, then mdmon will keep referencing the old
>> > .so or binary as long as it is running. This means that the fs these
>> > files are on cannot be remounted r/o. However mdmon insists on being
>> > shutdown only after all fs got remounted ro. So you have a cyclic
>> > ordering loop here: mdmon wants to be shut down after the remount, but
>> > we need to shut it down before the remount.
>> >
>>
>> Ehh ...
>>
>> a) mdmon is perfectly capable of restarting, it is already used to
>> take over mdmon launched in initrd. The problem is to know when to
>> restart - i.e. when respective libraries are changed. This is a job
>> for package management in distribution. It is already employed for
>> glibc, systemd and some others and can just as well be employed for
>> mdmon. And this is totally unrelated to systemd :)
>
> Really, you are sying there is a synchronous way to make mdmon reexec
> itself? How does that work?
>

I am not sure whether it qualifies as synchronous, but "mdmon
--takeover" will kill any existing mdmon for this and start monitoring
itself.

>> b) having binary launched off some fs should not prevent this fs to be
>> remountd ro - binaries are not opened rw
>
> If you run a binary and then the package manager replaces it then the
> running instance will still refer to the old copy and this will have the
> effect that the file isn't actually deleted until the proces
> exits/execs. And because that is the way it is the kernel will refuse
> unmounting of the fs until you terminated/reexeced your process.
>
>> > This is unfixable unless a) mdmon learns reexecution of itself without
>> > losing state (like most init systems so), or b) mdmon would stop
>> > insisting on being shutdown only after the remount.
>>
>> As far as I can tell, both is true today; but remounting is not
>> enough, unfortunately.
>
> So, you are saying we can shut down mdmon without ill effects early?
>

At least that's what I see. You can shutdown mdmon and continue to
work with file system, even if it is mounted rw. Under some conditions
mount will hang; i.e.

start array
kill mdmon
try to mount

mount will hang. If you start mdmon, it is mounted. But if you now

umount
kill mdmon
mount

it is mounted just fine.

>> > In my eyes b) is very much preferebale: It should be possible to shut
>> > down mdmon like any other service. And if then some md related code
>> > still needs to be run on late shutdown this should be done from a new
>> > process. I would be willing to add some hooks for this, so that we can
>> > execute arbitrary drop-in processes as part of the final shutdown loop.
>>
>> mdmon is needed to ensure metadata were correctly updated. So it needs
>> to exist as long as metadata *may* be updated. For practical purposes
>> it means - until file system is unmounted and flushed to disks. I am
>> not sure that remounting ro stops all activity (at least, mounting ro
>> definitely *writes* to device using some filesystems).
>
> Well, the root file systems cannot be unmounted, only remounted.
>
> So, is there a way to invoke mdmon so that it flushes all metadata
> changes to disk and immediately terminates then this should be all we
> need for a clean solution. We'd then shutdown the normal instances of
> mdmon down like any other daemon and simply invoke this metadata
> flushing command as part of late shutdown.


Hmm ... it looks like you just need to

start mdmon
do mdadm --wait-clean

After this you can kill mdmon again (assuming decide is no more in use).

Re: [systemd-devel] systemd kills mdmon if it was started manuallyby user

On Tue, 08.02.11 16:54, Andrey Borzenkov (arvidjaar@mail.ru) wrote:

> >> a) mdmon is perfectly capable of restarting, it is already used to
> >> take over mdmon launched in initrd. The problem is to know when to
> >> restart - i.e. when respective libraries are changed. This is a job
> >> for package management in distribution. It is already employed for
> >> glibc, systemd and some others and can just as well be employed for
> >> mdmon. And this is totally unrelated to systemd :)
> >
> > Really, you are sying there is a synchronous way to make mdmon reexec
> > itself? How does that work?
> >
>
> I am not sure whether it qualifies as synchronous, but "mdmon
> --takeover" will kill any existing mdmon for this and start monitoring
> itself.

I wonder if this is really fully synchronous, i.e. that a) there is no
point in time where mdmon is not running during this restart and b) the
mdmom --takeover command returns when the new daemon is fully up, and
not right-away.

> > Well, the root file systems cannot be unmounted, only remounted.
> >
> > So, is there a way to invoke mdmon so that it flushes all metadata
> > changes to disk and immediately terminates then this should be all we
> > need for a clean solution. We'd then shutdown the normal instances of
> > mdmon down like any other daemon and simply invoke this metadata
> > flushing command as part of late shutdown.
>
>
> Hmm ... it looks like you just need to
>
> start mdmon
> do mdadm --wait-clean
>
> After this you can kill mdmon again (assuming decide is no more in
> use).


Well, it would be nice if the md utils would offer something doing this
without spawning multiple processes and killing them again.

Lennart

--
Lennart Poettering - Red Hat, Inc.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [systemd-devel] systemd kills mdmon if it was started manuallyby user

On Tue, 08.02.11 12:07, Lennart Poettering (lennart@poettering.net) wrote:

> > At this point we know it is container, know that it has external
> > metadata and know that we need external metadata handler (mdmon). But
> > it is too late for systemd.
>
> Kay, do you know why this "change" event is used here? Any chance we can
> get rid of it?

So, it seems that the "change" event does make some sense here. I have
now added a new property to systemd: if you set SYSTEMD_READY=0 on a
udev device then systemd will consider it unplugged even if it shows up
in the udev tree. If this property is not set for a device, or is set to
1 we will conisder the device plugged.

To make this md stuff compatible with systemd we hence just need to set
SYSTEMD_READY=0 during the "new" event and drop it when the device is
fully set up.

Andrey, since you are playing around with this, do you happen to know
which attribute we should check to set SYSTEMD_READY=0 properly? It
would be cool if we could come up with a default rule for inclusion in
our systemd rules file that will ensure the device only shows up when it
is ready.

Lennart

--
Lennart Poettering - Red Hat, Inc.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html