Storage device enumeration script

This is a multi-part message in MIME format.
--------------050807030307090109060202
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Hi All,

Last November, I shared a shell script that helped me keep track of the specific hot-swap drives I had in the various slots of my servers. Although encouraged by Roman and John, I declined to make a project out of it.

I've since kicked it around some more, and thought a bit about supporting more than just the SCSI subsystem. The latest and greatest is still built around some standard executables: blkid, lspci, lsusb, sginfo, and smartctl. The original was similar to "lsscsi", but with controller details and device serial numbers.

New features:
Supports non-SCSI storage devices
Describes layered block devices
MD raid
LVM
generic device mapper
loop (partial)
Shows UUIDs
Shows mountpoints
Avoids repeating subtrees when enumerating raid devices

I struggled with the last item, until I gave up on bash. I needed to pass data to subroutines by reference, and bash is sorely lacking in that area. The new script is in python. I'm releasing this one under the GPL version 2.

Please give it a whirl.

Phil

--------------050807030307090109060202
Content-Type: text/plain;
name="lsdrv"
Content-Transfer-Encoding: 8bit
Content-Disposition: attachment;
filename="lsdrv"

#! /usr/bin/python
# -*- coding: utf-8 -*-
#
# lsdrv - Report on a system's disk interfaces and how they are used.
#
# Copyright (C) 2011 Philip J. Turmel
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 2.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.

import os, io, re
from subprocess import Popen, PIPE

#-------------------
# Handy base for objects as "bags of properties"
# inspired by Peter Norvig http://norvig.com/python-iaq.html
# Unlike the original, this one supplies 'None' instead of an attribute
# error when an explicitly named property has not yet been set.
class Struct(object):
def __init__(self, **entries):
self.__dict__.update(entries)
def __repr__(self, recurse=[]):
if self in recurse:
return type(self)
args = []
for (k,v) in vars(self).items():
if isinstance(v, Struct):
args.append('%s=%s' % (k, v.__repr__(recurse+[self])))
else:
args.append('%s=%s' % (k, repr(v)))
return '%s(%s)' % (type(self), ', '.join(args))
def clone(self):
return type(self)(**self.__dict__)
def __getattr__(self, attr):
return None

#-------------------
# Spawn an executable and collect its output. Equivalent to the
# check_output convenience function of the subprocess module introduced
# in python 2.7.
def runx(*args, **kwargs):
kwargs['stdout'] = PIPE
kwargs['stderr'] = PIPE
sub = Popen(*args, **kwargs)
out, err = sub.communicate()
return out

#-------------------
# Extract a matched expression from a string buffer
# If a match is found, return the given replace expression.
re1 = re.compile(r'([/:][a-zA-Z]*)0+([0-9])')
re2 = re.compile(r'Serial.+\'(.+)\'')
re3 = re.compile(r'Serial.+:(.+)')
def extractre(regexp, buffer, retexp=r'\1'):
mo = re.search(regexp, buffer)
if mo:
return mo.expand(retexp)
return None

#-------------------
# Extract shell variable assignments from a multiline string buffer
# This simple implementation returns everything after the equals sign
# as the value, including any quotes.
varsre = re.compile(r'^\s*([a-zA-Z][a-zA-Z0-9_]*)\s*=(.+)$', re.MULTILINE)
def extractvars(buffer):
vars=dict()
for mo in varsre.finditer(buffer):
vars[mo.group(1)] = mo.group(2)
return vars

#-------------------
# By Seo Sanghyeon. Some changes by Connelly Barnes and Phil Turmel.
def try_int(s):
try: return int(s)
except: return s
natsortre = re.compile(r'(\d+|\D+)')
def natsort_key(s):
if isinstance(s, str):
return map(try_int, natsortre.findall(s))
else:
try:
return tuple([natsort_keys(x) for x in s])
except:
return s
def natcmp(a, b):
return cmp(natsort_key(a), natsort_key(b))

#-------------------
# Convert device sizes expressed in kibibytes into human-readable
# sizes with a reasonable power-of-two suffix.
def k2size(k):
if k<1000:
return "%4.2fk" % k
m=k/1024.0
if m<1000:
return "%4.2fm" % m
g=m/1024.0
if g<1000:
return "%4.2fg" % g
t=g/1024.0
if t<1000:
return "%4.2ft" % t
p=t/1024.0
return "%4.2fp" % p

#-------------------
# Convert device sizes expressed as 512-byte sectors into human-readable
# sizes with a reasonable power-of-two suffix.
def sect2size(sectors):
return k2size(int(sectors)/2.0)

#-------------------
# Given a sysfs path to the parent of a physical block device, resolve the
# controller path, look it up in the list of known controllers, and return
# the corresponding struct object. If it's not present in the list, create
# the struct object w/ filled in details.
controllers=dict()
def probe_controller(cpathlink):
cpath = os.path.realpath(cpathlink)
if cpath in controllers:
return controllers[cpath]
while cpath and not os.path.exists(cpath+'/driver'):
cpath = os.path.dirname(cpath)
if cpath in controllers:
return controllers[cpath]
if not cpath:
return None
cntrlr = Struct(cpath=cpath, units=dict(),
abbrev=re1.sub(r'\1\2', cpath[12:]),
driver = os.path.realpath(cpath+'/driver').rsplit('/',1)[-1],
modpre = io.FileIO(cpath+'/modalias').read().split("\n",1)[0].split(' :',1)[0])
if cntrlr.modpre == 'pci':
cntrlr.description = runx(['lspci', '-s', cntrlr.abbrev.rsplit('/',1)[-1]]).split("\n",1)[0]
cntrlr.descriptors = ['PCI', '[%s]' % cntrlr.driver, cntrlr.description]
elif cntrlr.modpre == 'usb':
if os.path.exists(cpath+'/busnum'):
cntrlr.busnum = io.FileIO(cpath+'/busnum').read().split("\n",1)[0]
cntrlr.devnum = io.FileIO(cpath+'/devnum').read().split("\n",1)[0]
cntrlr.serial = io.FileIO(cpath+'/serial').read().split("\n",1)[0]
else:
parentpath = os.path.dirname(cpath)
cntrlr.busnum = io.FileIO(parentpath+'/busnum').read().split("\n",1)[0]
cntrlr.devnum = io.FileIO(parentpath+'/devnum').read().split("\n",1)[0]
cntrlr.serial = io.FileIO(parentpath+'/serial').read().split("\n",1)[0]
cntrlr.description = runx(['lsusb', '-s', cntrlr.busnum+':'+cntrlr.devnum]).split("\n",1)[0]
cntrlr.descriptors = ['USB', '[%s]' % cntrlr.driver, cntrlr.description, '{%s}' % cntrlr.serial]
else:
cntrlr.descriptors = ['Controller %s' % cntrlr.abbrev[1:], '[%s]' % cntrlr.driver, cntrlr.description]
controllers[cpath] = cntrlr
return cntrlr

#-------------------
# Given a link to a physical block device syspath, resolve the real device
# path, look it up in the list of known physical devices, and return
# the corresponding struct object. If it's not present in the list,
# create the struct object w/ filled in details, and probe its
# controller.
phydevs=dict()
def probe_device(devpathlink, nodestr):
devpath = os.path.realpath(devpathlink)
if devpath in phydevs:
return phydevs[devpath]
phy = Struct(dpath=devpath, node=nodestr,
vendor=io.FileIO(devpath+'/vendor').read().split("\n",1)[0]. strip(),
model=io.FileIO(devpath+'/model').read().split("\n",1)[0].st rip())
if os.path.exists(devpath+'/unique_id'):
phy.serial = io.FileIO(devpath+'/unique_id').read().split("\n",1)[0].stri p()
if not phy.serial:
phy.serial = extractre(re2, runx(['sginfo', '-s', '/dev/block/'+nodestr]))
if not phy.serial:
phy.serial = extractre(re3, runx(['smartctl', '-i', '/dev/block/'+nodestr]))
phy.name = "%s %s" % (os.path.realpath(devpath+'/subsystem').rsplit('/',1)[-1], devpath.rsplit('/',1)[-1])
phy.controller = probe_controller(os.path.dirname(devpath))
if phy.controller:
phy.controller.units[phy.name] = phy
phydevs[devpath] = phy
return phy

#-------------------
# Collect block device information and create dictionaries by kernel
# name and by device major:minor. Probe each block device and try to
# describe the filesystem or other usage.
blockbyname=dict()
blockbynode=dict()
sysclassblock="/sys/class/block/"
for x in os.listdir(sysclassblock):
nodestr=io.FileIO(sysclassblock+x+'/dev').read().split("\n") [0]
sizestr=sect2size(io.FileIO(sysclassblock+x+'/size').read(). split("\n")[0])
node = nodestr.split(':',1)
dev=Struct(name=x, node=nodestr, size=sizestr, major=int(node[0]), minor=int(node[1]), shown=False)
if os.path.exists(sysclassblock+x+'/device'):
dev.phy = probe_device(sysclassblock+x+'/device', nodestr)
if dev.phy:
dev.phy.block = dev
if os.path.exists(sysclassblock+x+'/holders'):
dev.holders = os.listdir(sysclassblock+x+'/holders')
else:
dev.holders = []
if os.path.exists(sysclassblock+x+'/slaves'):
dev.slaves = os.listdir(sysclassblock+x+'/slaves')
else:
dev.slaves = []
dev.partitions = [y for y in os.listdir(sysclassblock+x) if os.path.exists(sysclassblock+x+'/'+y+'/dev')]
dev.__dict__.update(extractvars(runx(['blkid', '-p', '-o', 'udev', '/dev/block/'+nodestr])))
if os.path.exists(sysclassblock+x+'/md'):
dev.isMD = True
dev.__dict__.update(extractvars(runx(['mdadm', '--export', '--detail', '/dev/block/'+nodestr])))
if dev.ID_FS_TYPE == 'linux_raid_member':
dev.hasMD = True
dev.__dict__.update(extractvars(runx(['mdadm', '--export', '--examine', '/dev/block/'+nodestr])))
if dev.holders:
mddir=sysclassblock+x+'/holders/'+dev.holders[0]+'/md/'
dev.MD_array_state = io.FileIO(mddir+'array_state').read().split("\n")[0]
dev.MD_array_size = io.FileIO(mddir+'array_size').read().split("\n")[0]
dev.MD_slot = io.FileIO(mddir+'dev-'+x+'/slot').read().split("\n")[0]
dev.MD_state = io.FileIO(mddir+'dev-'+x+'/state').read().split("\n")[0]
dev.FS = "MD %s (%s/%s) %s %s %s %s" % (dev.MD_LEVEL, dev.MD_slot, int(dev.MD_DEVICES), dev.size, dev.holders[0], dev.MD_array_state, dev.MD_state)
else:
dev.FS = "MD %s (%s) %s inactive" % (dev.MD_LEVEL, dev.MD_DEVICES, dev.size)
elif dev.ID_FS_TYPE and dev.ID_FS_TYPE[0:3] == 'LVM':
# Placeholder string for inactive physical volumes. It'll be
# overwritten when active PVs are scanned.
dev.FS = "PV %s (inactive)" % dev.ID_FS_TYPE
elif dev.ID_PART_TABLE_TYPE:
dev.FS = "Partitioned (%s) %s" % (dev.ID_PART_TABLE_TYPE, dev.size)
elif dev.ID_FS_TYPE:
dev.FS = "(%s) %s" % (dev.ID_FS_TYPE, dev.size)
else:
dev.FS = "Empty/Unknown %s" % dev.size
if dev.ID_FS_LABEL:
dev.FS += " '%s'" % dev.ID_FS_LABEL
if dev.ID_FS_UUID:
dev.FS += " {%s}" % dev.ID_FS_UUID
blockbyname[x] = dev
blockbynode[nodestr] = dev

#-------------------
# Collect information on mounted file systems and annotate the
# corresponding block device. Use the block device's major:minor node
# numbers, as the mount list often shows symlinks.
for x in io.FileIO('/proc/mounts').readlines():
if x[0:5] == '/dev/':
mdev, mnt = tuple(x.split(' ', 2)[0:2])
devstat = os.stat(mdev)
nodestr="%d:%d" % (os.major(devstat.st_rdev), os.minor(devstat.st_rdev))
if nodestr in blockbynode:
mntstat = os.statvfs(mnt)
dev = blockbynode[nodestr]
dev.mountdev = mdev
dev.mountpoint = mnt
dev.mountinfo = mntstat

#-------------------
# Collect information on LVM volumes and groups and annotate the
# corresponding block device. Use the block device's major:minor node
# numbers, as the mount list often shows symlinks.
vgroups = dict()
for x in runx(['pvs', '-o', 'pv_name,pv_used,pv_size,pv_uuid,vg_name,vg_size,vg_free,vg_ uuid', '--noheadings', '--separator', ' ']).split("\n"):
if x:
pv_name, pv_used, pv_size, pv_uuid, vg_name, vg_size, vg_free, vg_uuid = tuple(x.strip().split(' ',7))
devstat = os.stat(pv_name)
nodestr="%d:%d" % (os.major(devstat.st_rdev), os.minor(devstat.st_rdev))
if nodestr in blockbynode:
dev = blockbynode[nodestr]
dev.vg_name = vg_name
if not dev.hasLVM:
dev.hasLVM = True
dev.pv_used = pv_used
dev.pv_size = pv_size
dev.pv_uuid = pv_uuid
dev.FS = "PV %s %s/%s VG %s %s {%s}" % (dev.ID_FS_TYPE, pv_used, pv_size, vg_name, vg_size, pv_uuid)
if vg_name in vgroups:
vgroups[vg_name].PVs += [dev]
else:
vgroups[vg_name] = Struct(name=vg_name, size=vg_size, free=vg_free, uuid=vg_uuid, LVs=[], PVs=[dev])

for x in runx(['lvs', '-o', 'vg_name,lv_name,lv_path', '--noheadings', '--separator', ' ']).split("\n"):
if x:
vg_name, lv_name, lv_path = tuple(x.strip().split(' ',2))
devstat = os.stat(lv_path)
nodestr="%d:%d" % (os.major(devstat.st_rdev), os.minor(devstat.st_rdev))
if nodestr in blockbynode:
dev = blockbynode[nodestr]
dev.isLVM = True
dev.vg_name = vg_name
dev.__dict__.update(extractvars(runx(['lvs', '--rows', '-o', 'all', '--nameprefixes', '--noheadings', '--unquoted', lv_path])))
if vg_name in vgroups:
vgroups[vg_name].LVs += [dev]
else:
vgroups[vg_name] = Struct(name=vg_name, LVs=[dev], PVs=[])

def show_vgroup(indent, vg):
if vg.shown:
return
print "%s └─Volume Group %s (%s) %s free {%s}" % (indent, vg.name, ','.join([dev.name for dev in vg.PVs]), vg.free, vg.uuid)
show_blocks(indent+" ", vg.LVs)
vg.shown = True

#-------------------
# Given an indent level and list of block device names, recursively describe
# them.
continuation = ('│', '├')
corner = (' ', '└')
def show_blocks(indent, blocks):
blocks = [x for x in blocks if not x.shown]
for blk in blocks:
if blk == blocks[-1]:
branch=corner
else:
branch=continuation
print "%s %s─%s: %s" % (indent, branch[1], blk.name, blk.FS)
if blk.mountpoint:
print "%s %s └─Mounted as %s @ %s" % (indent, branch[0], blk.mountdev, blk.mountpoint)
elif blk.hasLVM:
show_vgroup(indent+" ", vgroups[blk.vg_name])
else:
subs = blk.partitions + blk.holders
subs.sort(natcmp)
if subs:
show_blocks("%s %s " % (indent, branch[0]), [blockbyname[x] for x in subs])
blk.shown = True

#-------------------
# Collect SCSI host / controller pairs from sysfs and create an ordered tree. Skip
# hosts that have targets, as they will already be in the list. Add empty physical
# device entries for hosts without targets.
scsidir = "/sys/bus/scsi/devices/"
scsilist = os.listdir(scsidir)
hosts = dict([(int(x[4:]), Struct(n=int(x[4:]), cpath=os.path.dirname(os.path.realpath(scsidir+x)), hpath='/'+x)) for x in scsilist if x[0:4]=='host'])

for n, host in hosts.items():
cntrlr = probe_controller(host.cpath)
if cntrlr :
targets = [x for x in os.listdir(host.cpath+host.hpath) if x[0:6]=='target']
if not targets:
phy = Struct(name='scsi %d:x:x:x [Empty]' % host.n)
cntrlr.units[phy.name] = phy

for cntrlr in controllers.values():
cntrlr.unitlist = cntrlr.units.keys()
if cntrlr.unitlist:
cntrlr.unitlist.sort(natcmp)
cntrlr.first = cntrlr.unitlist[0]
else:
cntrlr.first = ''

tree=[(cntrlr.first, cntrlr) for cntrlr in controllers.values()]
tree.sort(natcmp)

for f, cntrlr in tree:
print " ".join(cntrlr.descriptors)
if cntrlr.unitlist:
cntrlr.units[cntrlr.unitlist[-1]].last = True
branch = continuation
for key in cntrlr.unitlist:
phy = cntrlr.units[key]
if phy.last:
branch = corner
unitdetail = phy.name
if phy.vendor:
unitdetail += ' '+phy.vendor
if phy.model:
unitdetail += ' '+phy.model
if phy.serial:
unitdetail += " {%s}" % phy.serial.strip()
print ' %s─%s' % (branch[1], unitdetail)
if phy.block:
show_blocks(" %s " % branch[0], [phy.block])

unshown = [z.name for z in blockbynode.values() if z.size != '0.00k' and not z.shown]
unshown.sort(natcmp)
if unshown:
print "Other Block Devices"
show_blocks("", [blockbyname[x] for x in unshown])

--------------050807030307090109060202--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

On 26 May 2011 04:03, Phil Turmel wrote:
> Hi All,
>
> Last November, I shared a shell script that helped me keep track of t=
he specific hot-swap drives I had in the various slots of my servers. =C2=
=A0Although encouraged by Roman and John, I declined to make a project =
out of it.
>
> I've since kicked it around some more, and thought a bit about suppor=
ting more than just the SCSI subsystem.  The latest and greatest i=
s still built around some standard executables:  blkid, lspci, lsu=
sb, sginfo, and smartctl.  The original was similar to "lsscsi", b=
ut with controller details and device serial numbers.
>
> New features:
>        Supports non-SCSI storage devices
>        Describes layered block devices
>                MD raid
>                LVM
>                generic device=
mapper
>                loop (partial)
>        Shows UUIDs
>        Shows mountpoints
>        Avoids repeating subtrees when enumerating=
raid devices
>
> I struggled with the last item, until I gave up on bash.  I need=
ed to pass data to subroutines by reference, and bash is sorely lacking=
in that area.  The new script is in python.  I'm releasing t=
his one under the GPL version 2.
>
> Please give it a whirl.
>
> Phil
>

Awesome, however:

$ ./lsdrv.py
File "./lsdrv.py", line 301
print "%s └─Volume Group %s (%s) %s free {%s}" % (=
indent,
vg.name, ','.join([dev.name for dev in vg.PVs]), vg.free, vg.uuid)
^
SyntaxError: invalid syntax

Maybe it's a copy-paste thing from my mail that does it. Could you put
it somewhere, like a pastebin?

cheers,
/M
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

On 05/25/2011 11:10 PM, Mathias Burén wrote:
> On 26 May 2011 04:03, Phil Turmel wrote:
>> Hi All,
>>
>> Last November, I shared a shell script that helped me keep track of =
the specific hot-swap drives I had in the various slots of my servers. =
Although encouraged by Roman and John, I declined to make a project ou=
t of it.
>>
>> I've since kicked it around some more, and thought a bit about suppo=
rting more than just the SCSI subsystem. The latest and greatest is st=
ill built around some standard executables: blkid, lspci, lsusb, sginf=
o, and smartctl. The original was similar to "lsscsi", but with contro=
ller details and device serial numbers.
>>
>> New features:
>> Supports non-SCSI storage devices
>> Describes layered block devices
>> MD raid
>> LVM
>> generic device mapper
>> loop (partial)
>> Shows UUIDs
>> Shows mountpoints
>> Avoids repeating subtrees when enumerating raid devices
>>
>> I struggled with the last item, until I gave up on bash. I needed t=
o pass data to subroutines by reference, and bash is sorely lacking in =
that area. The new script is in python. I'm releasing this one under =
the GPL version 2.
>>
>> Please give it a whirl.
>>
>> Phil
>>
>=20
> Awesome, however:
>=20
> $ ./lsdrv.py
> File "./lsdrv.py", line 301
> print "%s └─Volume Group %s (%s) %s free {%s}" %=
(indent,
> vg.name, ','.join([dev.name for dev in vg.PVs]), vg.free, vg.uuid)
> ^
> SyntaxError: invalid syntax
>=20
> Maybe it's a copy-paste thing from my mail that does it. Could you pu=
t
> it somewhere, like a pastebin?

Yeah, should have thought of that first.

http://www.turmel.org/other/lsdrv

HTH,

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

On 26 May 2011 04:21, Phil Turmel wrote:
> On 05/25/2011 11:10 PM, Mathias Burén wrote:
>> On 26 May 2011 04:03, Phil Turmel wrote:
>>> Hi All,
>>>
>>> Last November, I shared a shell script that helped me keep track of=
the specific hot-swap drives I had in the various slots of my servers.=
 Although encouraged by Roman and John, I declined to make a proj=
ect out of it.
>>>
>>> I've since kicked it around some more, and thought a bit about supp=
orting more than just the SCSI subsystem.  The latest and greatest=
is still built around some standard executables:  blkid, lspci, l=
susb, sginfo, and smartctl.  The original was similar to "lsscsi",=
but with controller details and device serial numbers.
>>>
>>> New features:
>>>        Supports non-SCSI storage devices
>>>        Describes layered block devices
>>>                MD raid
>>>                LVM
>>>                generic devi=
ce mapper
>>>                loop (partia=
l)
>>>        Shows UUIDs
>>>        Shows mountpoints
>>>        Avoids repeating subtrees when enumerati=
ng raid devices
>>>
>>> I struggled with the last item, until I gave up on bash.  I ne=
eded to pass data to subroutines by reference, and bash is sorely lacki=
ng in that area.  The new script is in python.  I'm releasing=
this one under the GPL version 2.
>>>
>>> Please give it a whirl.
>>>
>>> Phil
>>>
>>
>> Awesome, however:
>>
>> $ ./lsdrv.py
>>   File "./lsdrv.py", line 301
>>     print "%s  └─Volume Group %s (%s)=
%s free {%s}" % (indent,
>> vg.name, ','.join([dev.name for dev in vg.PVs]), vg.free, vg.uuid)
>>                    =
                    =
              ^
>> SyntaxError: invalid syntax
>>
>> Maybe it's a copy-paste thing from my mail that does it. Could you p=
ut
>> it somewhere, like a pastebin?
>
> Yeah, should have thought of that first.
>
> http://www.turmel.org/other/lsdrv
>
> HTH,
>
> Phil
>

Aha, Archlinux defaults to python3. If I run the script with python2
it works. :)

~/bin $ sudo python2 lsdrv
PCI [ahci] 00:0b.0 SATA controller: nVidia Corporation MCP79 AHCI
Controller (rev b1)
├─scsi 0:0:0:0 ATA Corsair CSSD-F60 {10326505580009990=
027}
â”=82 └─sda: Partitioned (dos) 55.90g
â”=82 ├─sda1: (ext4) 100.00m 'ssd_boot' {ae879=
f86-73a4-451f-bb6b-e778ad1b57d6}
â”=82 â”=82 └─Mounted as /dev/sda1 @ /boo=
t
â”=82 ├─sda2: (swap) 2.00g 'ssd_swap' {a28e32f=
a-628c-419a-9693-ca88166d230f}
â”=82 └─sda3: (ext4) 53.80g 'ssd_root' {6e812e=
d7-01c4-4a76-ae31-7b3d36d847f5}
â”=82 └─Mounted as /dev/disk/by-label/ssd_r=
oot @ /
├─scsi 1:0:0:0 ATA WDC WD20EARS-00M {WD-WCAZA1022443}
â”=82 └─sdb: Partitioned (dos) 1.82t
â”=82 └─sdb1: MD raid6 (1/7) 1.82t md0 clean i=
n_sync 'ion:0'
{e6595c64-b3ae-90b3-f011-33ac3f402d20}
â”=82 └─md0: PV LVM2_member 9.08t/9.08t VG =
lvstorage 9.08t
{YLEUKB-klxF-X3gF-6dG3-DL4R-xebv-6gKQc2}
â”=82 └─Volume Group lvstorage (md0) 0 =
free {
Xd0HTM-azdN-v9kJ-C7vD-COcU-Cnn8-6AJ6hI}
â”=82 └─dm-0: (ext4) 9.08t 'storage'
{0ca82f13-680f-4b0d-a5d0-08c246a838e5}
â”=82 └─Mounted as /dev/mapper/lvs=
torage-storage @ /raid6volume
├─scsi 2:0:0:0 ATA WDC WD20EARS-00M {WD-WMAZ20152590}
â”=82 └─sdc: Partitioned (dos) 1.82t
â”=82 └─sdc1: MD raid6 (3/7) 1.82t md0 clean i=
n_sync 'ion:0'
{e6595c64-b3ae-90b3-f011-33ac3f402d20}
├─scsi 3:0:0:0 ATA WDC WD20EARS-00M {WD-WMAZ20188479}
â”=82 └─sdd: Partitioned (dos) 1.82t
â”=82 └─sdd1: MD raid6 (2/7) 1.82t md0 clean i=
n_sync 'ion:0'
{e6595c64-b3ae-90b3-f011-33ac3f402d20}
├─scsi 4:x:x:x [Empty]
└─scsi 5:x:x:x [Empty]
PCI [sata_mv] 05:00.0 SCSI storage controller: HighPoint Technologies,
Inc. RocketRAID 230x 4 Port SATA-II Controller (rev 02)
├─scsi 6:0:0:0 ATA WDC WD20EARS-00M {WD-WCAZA3609190}
â”=82 └─sde: Partitioned (dos) 1.82t
â”=82 └─sde1: MD raid6 (6/7) 1.82t md0 clean i=
n_sync 'ion:0'
{e6595c64-b3ae-90b3-f011-33ac3f402d20}
├─scsi 7:0:0:0 ATA SAMSUNG HD204UI {S2HGJ1RZ800964}
â”=82 └─sdf: Partitioned (dos) 1.82t
â”=82 └─sdf1: MD raid6 (4/7) 1.82t md0 clean i=
n_sync 'ion:0'
{e6595c64-b3ae-90b3-f011-33ac3f402d20}
├─scsi 8:0:0:0 ATA WDC WD20EARS-00M {WD-WCAZA1000331}
â”=82 └─sdg: Partitioned (dos) 1.82t
â”=82 └─sdg1: MD raid6 (0/7) 1.82t md0 clean i=
n_sync 'ion:0'
{e6595c64-b3ae-90b3-f011-33ac3f402d20}
└─scsi 9:0:0:0 ATA SAMSUNG HD204UI {S2HGJ1RZ800850}
└─sdh: Partitioned (dos) 1.82t
└─sdh1: MD raid6 (5/7) 1.82t md0 clean in_sync '=
ion:0'
{e6595c64-b3ae-90b3-f011-33ac3f402d20}


Sweet.

/M
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

--Sig_/GLf020wENWlQ8bPA_0j1Bky
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

On Wed, 25 May 2011 23:21:52 -0400
Phil Turmel wrote:

> Yeah, should have thought of that first.
>=20
> http://www.turmel.org/other/lsdrv

Traceback (most recent call last):
File "/r/scripts/lsdrv", line 198, in
dev.phy =3D probe_device(sysclassblock+x+'/device', nodestr)
File "/r/scripts/lsdrv", line 170, in probe_device
vendor=3Dio.FileIO(devpath+'/vendor').read().split("\n",1)[0 ].strip(),
File "/usr/lib/python2.6/io.py", line 631, in __init__
_fileio._FileIO.__init__(self, name, mode, closefd)
IOError: [Errno 2] No such file or directory:
'/sys/devices/platform/floppy.0/vendor'

--=20
With respect,
Roman

--Sig_/GLf020wENWlQ8bPA_0j1Bky
Content-Type: application/pgp-signature; name=signature.asc
Content-Disposition: attachment; filename=signature.asc

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAk3d5GMACgkQTLKSvz+PZwhGJgCeLOErEoSVz7Pj5nmoAbDS zecq
sXIAnju7CfbDxdF62Gnfza7E9VmCslhC
=WDJP
-----END PGP SIGNATURE-----

--Sig_/GLf020wENWlQ8bPA_0j1Bky--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

RE: Storage device enumeration script

I get an error right off the bat:

Traceback (most recent call last):
File "./lsdrv.txt", line 264, in
for x in runx(['pvs', '-o',
'pv_name,pv_used,pv_size,pv_uuid,vg_name,vg_size,vg_free,vg_ uuid',
'--noheadings', '--separator', ' ']).split("\n"):
File "./lsdrv.txt", line 50, in runx
sub = Popen(*args, **kwargs)
File "/usr/lib/python2.6/subprocess.py", line 623, in __init__
errread, errwrite)
File "/usr/lib/python2.6/subprocess.py", line 1141, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

I'm running Python 2.6.6 and mdadm 3.1.4 running on kernel 2.6.32-5-amd64


--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

On 26 May 2011 07:14, Leslie Rhorer wrote:
>
>        I get an error right off the bat:
>
> Traceback (most recent call last):
>  File "./lsdrv.txt", line 264, in
>    for x in runx(['pvs', '-o',
> 'pv_name,pv_used,pv_size,pv_uuid,vg_name,vg_size,vg_free,vg_ uuid',
> '--noheadings', '--separator', ' ']).split("\n"):
>  File "./lsdrv.txt", line 50, in runx
>    sub =3D Popen(*args, **kwargs)
>  File "/usr/lib/python2.6/subprocess.py", line 623, in __init__
>    errread, errwrite)
>  File "/usr/lib/python2.6/subprocess.py", line 1141, in _execute=
_child
>    raise child_exception
> OSError: [Errno 2] No such file or directory
>
> I'm running Python 2.6.6 and mdadm 3.1.4 running on kernel 2.6.32-5-a=
md64
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid"=
in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.ht=
ml
>

I get that in Ubuntu 11.04 on a laptop as well.

/M
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

May be setup some github repo for this?

On Thu, May 26, 2011 at 7:03 AM, Phil Turmel wrote:
> Hi All,
>
> Last November, I shared a shell script that helped me keep track of t=
he specific hot-swap drives I had in the various slots of my servers. =A0=
Although encouraged by Roman and John, I declined to make a project out=
of it.
>
> I've since kicked it around some more, and thought a bit about suppor=
ting more than just the SCSI subsystem. =A0The latest and greatest is s=
till built around some standard executables: =A0blkid, lspci, lsusb, sg=
info, and smartctl. =A0The original was similar to "lsscsi", but with c=
ontroller details and device serial numbers.
>
> New features:
> =A0 =A0 =A0 =A0Supports non-SCSI storage devices
> =A0 =A0 =A0 =A0Describes layered block devices
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0MD raid
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0LVM
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0generic device mapper
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0loop (partial)
> =A0 =A0 =A0 =A0Shows UUIDs
> =A0 =A0 =A0 =A0Shows mountpoints
> =A0 =A0 =A0 =A0Avoids repeating subtrees when enumerating raid device=
s
>
> I struggled with the last item, until I gave up on bash. =A0I needed =
to pass data to subroutines by reference, and bash is sorely lacking in=
that area. =A0The new script is in python. =A0I'm releasing this one u=
nder the GPL version 2.
>
> Please give it a whirl.
>
> Phil
>



--=20
Best regards,
[COOLCOLD-RIPN]
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

On Thu, May 26, 2011 at 7:21 AM, Phil Turmel wrote:
> On 05/25/2011 11:10 PM, Mathias Burén wrote:
>> On 26 May 2011 04:03, Phil Turmel wrote:
>>> Hi All,
>>>
>>> Last November, I shared a shell script that helped me keep track of=
the specific hot-swap drives I had in the various slots of my servers.=
 Although encouraged by Roman and John, I declined to make a proj=
ect out of it.
>>>
>>> I've since kicked it around some more, and thought a bit about supp=
orting more than just the SCSI subsystem.  The latest and greatest=
is still built around some standard executables:  blkid, lspci, l=
susb, sginfo, and smartctl.  The original was similar to "lsscsi",=
but with controller details and device serial numbers.
>>>
>>> New features:
>>>        Supports non-SCSI storage devices
>>>        Describes layered block devices
>>>                MD raid
>>>                LVM
>>>                generic devi=
ce mapper
>>>                loop (partia=
l)
>>>        Shows UUIDs
>>>        Shows mountpoints
>>>        Avoids repeating subtrees when enumerati=
ng raid devices
>>>
>>> I struggled with the last item, until I gave up on bash.  I ne=
eded to pass data to subroutines by reference, and bash is sorely lacki=
ng in that area.  The new script is in python.  I'm releasing=
this one under the GPL version 2.
>>>
>>> Please give it a whirl.
>>>
>>> Phil
>>>
>>
>> Awesome, however:
>>
>> $ ./lsdrv.py
>>   File "./lsdrv.py", line 301
>>     print "%s  └─Volume Group %s (%s)=
%s free {%s}" % (indent,
>> vg.name, ','.join([dev.name for dev in vg.PVs]), vg.free, vg.uuid)
>>                    =
                    =
              ^
>> SyntaxError: invalid syntax
>>
>> Maybe it's a copy-paste thing from my mail that does it. Could you p=
ut
>> it somewhere, like a pastebin?
>
> Yeah, should have thought of that first.
>
> http://www.turmel.org/other/lsdrv
>
> HTH,

Works in Debian Squeeze, but requires packages pciutils and sg3-utils
to be installed.

On Debian Lenny produces error:
root@gamma2:/tmp# python lsdrv
Traceback (most recent call last):
File "lsdrv", line 17, in
import os, io, re
ImportError: No module named io


>
> Phil
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid"=
in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.ht=
ml
>



--=20
Best regards,
[COOLCOLD-RIPN]
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

On 26/05/2011 04:03, Phil Turmel wrote:
> Hi All,
>
> Last November, I shared a shell script that helped me keep track of the specific hot-swap drives I had in the various slots of my servers. Although encouraged by Roman and John, I declined to make a project out of it.
>
> I've since kicked it around some more, and thought a bit about supporting more than just the SCSI subsystem. The latest and greatest is still built around some standard executables: blkid, lspci, lsusb, sginfo, and smartctl. The original was similar to "lsscsi", but with controller details and device serial numbers.
>
> New features:
> Supports non-SCSI storage devices
> Describes layered block devices
> MD raid
> LVM
> generic device mapper
> loop (partial)
> Shows UUIDs
> Shows mountpoints
> Avoids repeating subtrees when enumerating raid devices
>
> I struggled with the last item, until I gave up on bash. I needed to pass data to subroutines by reference, and bash is sorely lacking in that area. The new script is in python. I'm releasing this one under the GPL version 2.
>
> Please give it a whirl.

On CentOS 5:

[root@beast lsdrv]# ./lsdrv
Traceback (most recent call last):
File "./lsdrv", line 17, in ?
import os, io, re
ImportError: No module named io
[root@beast lsdrv]# python -V
Python 2.4.3

Maybe this is too old, so I got Python 2.6.5 from EPEL. Trying again:

[root@beast lsdrv]# python2.6 lsdrv
Traceback (most recent call last):
File "./lsdrv", line 192, in
for x in os.listdir(sysclassblock):
OSError: [Errno 2] No such file or directory: '/sys/class/block/'

So I changed sysclassblock to '/sys/block/' to suit EL5's 2.6.18 kernel
(so it might be nice ;-) if you could auto-detect which was present):

[root@beast lsdrv]# python2.6 lsdrv
Traceback (most recent call last):
File "lsdrv", line 198, in
dev.phy = probe_device(sysclassblock+x+'/device', nodestr)
File "lsdrv", line 170, in probe_device
vendor=io.FileIO(devpath+'/vendor').read().split("\n",1)[0]. strip(),
File "/usr/lib64/python2.6/io.py", line 631, in __init__
_fileio._FileIO.__init__(self, name, mode, closefd)
IOError: [Errno 2] No such file or directory:
'/sys/devices/platform/floppy.0/vendor'

This looks like the same error Roman had.

Cheers,

John.

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

--=-ujbDMj7IJh7KKPL7jo5U
Content-Type: multipart/mixed; boundary="=-ze2/GNr6sVeTvuH4ZFAv"


--=-ze2/GNr6sVeTvuH4ZFAv
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi,

Great tool, thanks for sharing.

I had to add some error handling to get it to work properly.
Currently it runs on Ubuntu 10.04, 10.10 and 11.04.

Added the check John Robinson asked for as well.

Attached patch for those interested.


--
Thanks
Torbjørn

On Wed, 2011-05-25 at 23:03 -0400, Phil Turmel wrote:
> Hi All,
>=20
> Last November, I shared a shell script that helped me keep track of the s=
pecific hot-swap drives I had in the various slots of my servers. Although=
encouraged by Roman and John, I declined to make a project out of it.
>=20
> I've since kicked it around some more, and thought a bit about supporting=
more than just the SCSI subsystem. The latest and greatest is still built=
around some standard executables: blkid, lspci, lsusb, sginfo, and smartc=
tl. The original was similar to "lsscsi", but with controller details and =
device serial numbers.
>=20
> New features:
> Supports non-SCSI storage devices
> Describes layered block devices
> MD raid
> LVM
> generic device mapper
> loop (partial)
> Shows UUIDs
> Shows mountpoints
> Avoids repeating subtrees when enumerating raid devices
>=20
> I struggled with the last item, until I gave up on bash. I needed to pas=
s data to subroutines by reference, and bash is sorely lacking in that area=
.. The new script is in python. I'm releasing this one under the GPL versi=
on 2.
>=20
> Please give it a whirl.
>=20
> Phil

--=20
Torbjørn Skagestad
Idé Til Produkt AS
torborn@itpas.no

--=-ze2/GNr6sVeTvuH4ZFAv
Content-Disposition: attachment; filename="lsdrv.patch"
Content-Type: text/x-patch; name="lsdrv.patch"; charset="UTF-8"
Content-Transfer-Encoding: base64

LS0tIGxzZHJ2CTIwMTEtMDUtMjYgMTE6MDY6NDQuMDAwMDAwMDAwICswMjAw DQorKysgbHNkcnYu
cHkJMjAxMS0wNS0yNiAxMTozNDo1MC4wMDAwMDAwMDAgKzAyMDANCkBAIC00 Nyw3ICs0NywxMiBA
QA0KIGRlZiBydW54KCphcmdzLCAqKmt3YXJncyk6DQogCWt3YXJnc1snc3Rk b3V0J10gPSBQSVBF
DQogCWt3YXJnc1snc3RkZXJyJ10gPSBQSVBFDQotCXN1YiA9IFBvcGVuKCph cmdzLCAqKmt3YXJn
cykNCisJdHJ5Og0KKwkJc3ViID0gUG9wZW4oKmFyZ3MsICoqa3dhcmdzKQ0K KwlleGNlcHQgT1NF
cnJvciBhcyBlOg0KKwkJcHJpbnQgIlVuYWJsZSB0byBleGVjdXRlICIgKyBz dHIoYXJnc1swXVsw
XSkNCisJCXByaW50IGUNCisJCWV4aXQoKQ0KIAlvdXQsIGVyciA9IHN1Yi5j b21tdW5pY2F0ZSgp
DQogCXJldHVybiBvdXQNCiANCkBAIC0xNjYsOSArMTcxLDEyIEBADQogCWRl dnBhdGggPSBvcy5w
YXRoLnJlYWxwYXRoKGRldnBhdGhsaW5rKQ0KIAlpZiBkZXZwYXRoIGluIHBo eWRldnM6DQogCQly
ZXR1cm4gcGh5ZGV2c1tkZXZwYXRoXQ0KLQlwaHkgPSBTdHJ1Y3QoZHBhdGg9 ZGV2cGF0aCwgbm9k
ZT1ub2Rlc3RyLA0KLQkJdmVuZG9yPWlvLkZpbGVJTyhkZXZwYXRoKycvdmVu ZG9yJykucmVhZCgp
LnNwbGl0KCJcbiIsMSlbMF0uc3RyaXAoKSwNCi0JCW1vZGVsPWlvLkZpbGVJ TyhkZXZwYXRoKycv
bW9kZWwnKS5yZWFkKCkuc3BsaXQoIlxuIiwxKVswXS5zdHJpcCgpKQ0KKwl0 cnk6DQorCQlwaHkg
PSBTdHJ1Y3QoZHBhdGg9ZGV2cGF0aCwgbm9kZT1ub2Rlc3RyLA0KKwkJCXZl bmRvcj1pby5GaWxl
SU8oZGV2cGF0aCsnL3ZlbmRvcicpLnJlYWQoKS5zcGxpdCgiXG4iLDEpWzBd LnN0cmlwKCksDQor
CQkJbW9kZWw9aW8uRmlsZUlPKGRldnBhdGgrJy9tb2RlbCcpLnJlYWQoKS5z cGxpdCgiXG4iLDEp
WzBdLnN0cmlwKCkpDQorCWV4Y2VwdCBJT0Vycm9yOg0KKwkJcmV0dXJuIE5v bmUNCiAJaWYgb3Mu
cGF0aC5leGlzdHMoZGV2cGF0aCsnL3VuaXF1ZV9pZCcpOg0KIAkJcGh5LnNl cmlhbCA9IGlvLkZp
bGVJTyhkZXZwYXRoKycvdW5pcXVlX2lkJykucmVhZCgpLnNwbGl0KCJcbiIs MSlbMF0uc3RyaXAo
KQ0KIAlpZiBub3QgcGh5LnNlcmlhbDoNCkBAIC0xODksNiArMTk3LDggQEAN CiBibG9ja2J5bmFt
ZT1kaWN0KCkNCiBibG9ja2J5bm9kZT1kaWN0KCkNCiBzeXNjbGFzc2Jsb2Nr PSIvc3lzL2NsYXNz
L2Jsb2NrLyINCitpZihub3Qgb3MucGF0aC5leGlzdHMoc3lzY2xhc3NibG9j aykpOg0KKwlzeXNj
bGFzc2Jsb2NrID0gIi9zeXMvYmxvY2svIg0KIGZvciB4IGluIG9zLmxpc3Rk aXIoc3lzY2xhc3Ni
bG9jayk6DQogCW5vZGVzdHI9aW8uRmlsZUlPKHN5c2NsYXNzYmxvY2sreCsn L2RldicpLnJlYWQo
KS5zcGxpdCgiXG4iKVswXQ0KIAlzaXplc3RyPXNlY3Qyc2l6ZShpby5GaWxl SU8oc3lzY2xhc3Ni
bG9jayt4Kycvc2l6ZScpLnJlYWQoKS5zcGxpdCgiXG4iKVswXSkNCkBAIC0y NTAsNyArMjYwLDEw
IEBADQogCQlkZXZzdGF0ID0gb3Muc3RhdChtZGV2KQ0KIAkJbm9kZXN0cj0i JWQ6JWQiICUgKG9z
Lm1ham9yKGRldnN0YXQuc3RfcmRldiksIG9zLm1pbm9yKGRldnN0YXQuc3Rf cmRldikpDQogCQlp
ZiBub2Rlc3RyIGluIGJsb2NrYnlub2RlOg0KLQkJCW1udHN0YXQgPSBvcy5z dGF0dmZzKG1udCkN
CisJCQl0cnk6DQorCQkJCW1udHN0YXQgPSBvcy5zdGF0dmZzKG1udCkNCisJ CQlleGNlcHQgT1NF
cnJvcjoNCisJCQkJbW50c3RhdCA9IE5vbmUNCiAJCQlkZXYgPSBibG9ja2J5 bm9kZVtub2Rlc3Ry
XQ0KIAkJCWRldi5tb3VudGRldiA9IG1kZXYNCiAJCQlkZXYubW91bnRwb2lu dCA9IG1udA0K


--=-ze2/GNr6sVeTvuH4ZFAv--

--=-ujbDMj7IJh7KKPL7jo5U
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEABECAAYFAk3eITQACgkQ4RjLWGFqSIaERQCfeUGDfezAnAWemhYSSFOg dU4D
5iIAnjoUHs21wRknXpG6ySN1Bfn2rvlT
=K19z
-----END PGP SIGNATURE-----

--=-ujbDMj7IJh7KKPL7jo5U--

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

On 26/05/2011 10:45, Torbjørn Skagestad wrote:
> Hi,
>
> Great tool, thanks for sharing.
>
> I had to add some error handling to get it to work properly.
> Currently it runs on Ubuntu 10.04, 10.10 and 11.04.
>
> Added the check John Robinson asked for as well.
>
> Attached patch for those interested.

Thanks, Torbjørn!

Getting there:

[root@beast lsdrv]# python2.6 lsdrv
PCI [pata_marvell] 03:00.0 IDE interface: Marvell Technology Group Ltd.=
=20
88SE6121 SATA II Controller (rev b2)
└─scsi 0:0:0:0 HL-DT-ST DVD-RAM GH22NP20
└─sr0: Empty/Unknown 1.00g
PCI [ahci] 00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10=20
=46amily) SATA AHCI Controller
├─scsi 2:0:0:0 ATA Hitachi HDS72101
â”=82 └─sda: Empty/Unknown 931.51g
Traceback (most recent call last):
File "lsdrv", line 387, in
show_blocks(" %s " % branch[0], [phy.block])
File "lsdrv", line 339, in show_blocks
show_blocks("%s %s " % (indent, branch[0]), [blockbyname[x] for x=20
in subs])
KeyError: 'sda1'

Now, something's not getting picked up about sda. Looking at Mathias'=20
"sweet" output, it's not coping with the (DOS) partition table. Another=
=20
variation on my kernel's /sys or still to old a Python or ...?

Cheers,

John.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

Thanks Torbjørn !

I tend to short the error-checking when I'm putting something together.=
. Clearly, this script was less baked than I thought.

On 05/26/2011 05:45 AM, Torbjørn Skagestad wrote:
> @@ -166,9 +171,12 @@
> devpath =3D os.path.realpath(devpathlink)
> if devpath in phydevs:
> return phydevs[devpath]
> - phy =3D Struct(dpath=3Ddevpath, node=3Dnodestr,
> - vendor=3Dio.FileIO(devpath+'/vendor').read().split("\n",1)[0 ].stri=
p(),
> - model=3Dio.FileIO(devpath+'/model').read().split("\n",1)[0]. strip(=
))
> + try:
> + phy =3D Struct(dpath=3Ddevpath, node=3Dnodestr,
> + vendor=3Dio.FileIO(devpath+'/vendor').read().split("\n",1)[0 ].str=
ip(),
> + model=3Dio.FileIO(devpath+'/model').read().split("\n",1)[0]. strip=
())
> + except IOError:
> + return None
> if os.path.exists(devpath+'/unique_id'):
> phy.serial =3D io.FileIO(devpath+'/unique_id').read().split("\n",1=
)[0].strip()
> if not phy.serial:

This hunk will have to be done differently. "phy" needs to be set with=
out vendor & model if they can't be read. I'll make a helper function =
to read the first line of a file, or return None.

On 05/26/2011 04:11 AM, CoolCold wrote:
> May be setup some github repo for this?
>=20

Yes, I'll do this later today. Obviously needed. :(

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

On 05/26/2011 02:16 AM, Mathias Burén wrote:
> On 26 May 2011 07:14, Leslie Rhorer wrote:
>>
>> I get an error right off the bat:
>>
>> Traceback (most recent call last):
>> File "./lsdrv.txt", line 264, in
>> for x in runx(['pvs', '-o',
>> 'pv_name,pv_used,pv_size,pv_uuid,vg_name,vg_size,vg_free,vg_ uuid',
>> '--noheadings', '--separator', ' ']).split("\n"):
>> File "./lsdrv.txt", line 50, in runx
>> sub =3D Popen(*args, **kwargs)
>> File "/usr/lib/python2.6/subprocess.py", line 623, in __init__
>> errread, errwrite)
>> File "/usr/lib/python2.6/subprocess.py", line 1141, in _execute_chi=
ld
>> raise child_exception
>> OSError: [Errno 2] No such file or directory
>>
>> I'm running Python 2.6.6 and mdadm 3.1.4 running on kernel 2.6.32-5-=
amd64
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid=
" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>=20
> I get that in Ubuntu 11.04 on a laptop as well.
>=20
> /M

Does a plain install of Ubuntu include the LVM utilities? Can you run =
"pvs" directly?

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

On 05/26/2011 05:59 AM, John Robinson wrote:
> On 26/05/2011 10:45, Torbjørn Skagestad wrote:
>> Hi,
>>
>> Great tool, thanks for sharing.
>>
>> I had to add some error handling to get it to work properly.
>> Currently it runs on Ubuntu 10.04, 10.10 and 11.04.
>>
>> Added the check John Robinson asked for as well.
>>
>> Attached patch for those interested.
>=20
> Thanks, Torbjørn!
>=20
> Getting there:
>=20
> [root@beast lsdrv]# python2.6 lsdrv
> PCI [pata_marvell] 03:00.0 IDE interface: Marvell Technology Group Lt=
d. 88SE6121 SATA II Controller (rev b2)
> └─scsi 0:0:0:0 HL-DT-ST DVD-RAM GH22NP20
> └─sr0: Empty/Unknown 1.00g
> PCI [ahci] 00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10 =
=46amily) SATA AHCI Controller
> ├─scsi 2:0:0:0 ATA Hitachi HDS72101
> â”=82 └─sda: Empty/Unknown 931.51g
> Traceback (most recent call last):
> File "lsdrv", line 387, in
> show_blocks(" %s " % branch[0], [phy.block])
> File "lsdrv", line 339, in show_blocks
> show_blocks("%s %s " % (indent, branch[0]), [blockbyname[x] for x=
in subs])
> KeyError: 'sda1'
>=20
> Now, something's not getting picked up about sda. Looking at Mathias'=
"sweet" output, it's not coping with the (DOS) partition table. Anothe=
r variation on my kernel's /sys or still to old a Python or ...?

Hmmm. I'll set up another CentOS VM. If I recall correctly, that kern=
el has the original block devices in folders named '.../block:sda1' ins=
tead of '.../block/sda1'. I'll have to identify this in the initial sw=
eep through sysfs.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

--=-gxjlM4t6zjJuKSaepG9n
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, 2011-05-26 at 07:39 -0400, Phil Turmel wrote:
> Thanks Torbjørn !
>=20
> I tend to short the error-checking when I'm putting something together...=
Clearly, this script was less baked than I thought.
>=20
> On 05/26/2011 05:45 AM, Torbjørn Skagestad wrote:
> > @@ -166,9 +171,12 @@
> > devpath =3D os.path.realpath(devpathlink)
> > if devpath in phydevs:
> > return phydevs[devpath]
> > - phy =3D Struct(dpath=3Ddevpath, node=3Dnodestr,
> > - vendor=3Dio.FileIO(devpath+'/vendor').read().split("\n",1)[0 ].strip(=
),
> > - model=3Dio.FileIO(devpath+'/model').read().split("\n",1)[0]. strip())
> > + try:
> > + phy =3D Struct(dpath=3Ddevpath, node=3Dnodestr,
> > + vendor=3Dio.FileIO(devpath+'/vendor').read().split("\n",1)[0 ].strip=
(),
> > + model=3Dio.FileIO(devpath+'/model').read().split("\n",1)[0]. strip()=
)
> > + except IOError:
> > + return None
> > if os.path.exists(devpath+'/unique_id'):
> > phy.serial =3D io.FileIO(devpath+'/unique_id').read().split("\n",1)[=
0].strip()
> > if not phy.serial:
>=20
> This hunk will have to be done differently. "phy" needs to be set withou=
t vendor & model if they can't be read. I'll make a helper function to rea=
d the first line of a file, or return None.

That makes sense.
Guess I was too trigger happy here and ignored some devices in the
process.
>=20
> On 05/26/2011 04:11 AM, CoolCold wrote:
> > May be setup some github repo for this?
> >=20
>=20
> Yes, I'll do this later today. Obviously needed. :(
>=20
> Phil
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html

--=20
Torbjørn Skagestad
Idé Til Produkt AS
torborn@itpas.no

--=-gxjlM4t6zjJuKSaepG9n
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEABECAAYFAk3ePvMACgkQ4RjLWGFqSIYrbgCdEBNhPP9YgsjpQYzG3vw8 y3cR
6X8An0C04vpr/LiaqlnBJ53yRN/Ngj79
=/lQZ
-----END PGP SIGNATURE-----

--=-gxjlM4t6zjJuKSaepG9n--

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

On 05/26/2011 04:24 AM, CoolCold wrote:
[...]
> On Debian Lenny produces error:
> root@gamma2:/tmp# python lsdrv
> Traceback (most recent call last):
> File "lsdrv", line 17, in
> import os, io, re
> ImportError: No module named io

Huh. The 'io' module is v2.6 and above. Another reason to make a helper function for reading these files.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

--=-afAwg1qHC1EGE1vh/MaI
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, 2011-05-26 at 07:49 -0400, Phil Turmel wrote:
> On 05/26/2011 05:59 AM, John Robinson wrote:
> > On 26/05/2011 10:45, Torbjørn Skagestad wrote:
> >> Hi,
> >>
> >> Great tool, thanks for sharing.
> >>
> >> I had to add some error handling to get it to work properly.
> >> Currently it runs on Ubuntu 10.04, 10.10 and 11.04.
> >>
> >> Added the check John Robinson asked for as well.
> >>
> >> Attached patch for those interested.
> >=20
> > Thanks, Torbjørn!
> >=20
> > Getting there:
> >=20
> > [root@beast lsdrv]# python2.6 lsdrv
> > PCI [pata_marvell] 03:00.0 IDE interface: Marvell Technology Group Ltd.=
88SE6121 SATA II Controller (rev b2)
> > └─scsi 0:0:0:0 HL-DT-ST DVD-RAM GH22NP20
> > └─sr0: Empty/Unknown 1.00g
> > PCI [ahci] 00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10 Fa=
mily) SATA AHCI Controller
> > ├─scsi 2:0:0:0 ATA Hitachi HDS72101
> > â”=82 └─sda: Empty/Unknown 931.51g
> > Traceback (most recent call last):
> > File "lsdrv", line 387, in
> > show_blocks(" %s " % branch[0], [phy.block])
> > File "lsdrv", line 339, in show_blocks
> > show_blocks("%s %s " % (indent, branch[0]), [blockbyname[x] for x i=
n subs])
> > KeyError: 'sda1'
> >=20
> > Now, something's not getting picked up about sda. Looking at Mathias' "=
sweet" output, it's not coping with the (DOS) partition table. Another vari=
ation on my kernel's /sys or still to old a Python or ...?
>=20
> Hmmm. I'll set up another CentOS VM. If I recall correctly, that kernel=
has the original block devices in folders named '.../block:sda1' instead o=
f '.../block/sda1'. I'll have to identify this in the initial sweep throug=
h sysfs.
>=20

For CentOS 5.4 (2.6.18-128.1.14) it is /sys/block/sda/sda[1-9]


> Phil
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html

--=20
Torbjørn Skagestad
Idé Til Produkt AS
torborn@itpas.no

--=-afAwg1qHC1EGE1vh/MaI
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEABECAAYFAk3eQfEACgkQ4RjLWGFqSIZ99wCdFBqdaJ56Or8uMlaY/The Co5t
ldkAnjqsWO2Z8EpEeB3a7OMOvqkOoDjH
=g7oO
-----END PGP SIGNATURE-----

--=-afAwg1qHC1EGE1vh/MaI--

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

On 05/26/2011 07:39 AM, Phil Turmel wrote:
> On 05/26/2011 04:11 AM, CoolCold wrote:
>> May be setup some github repo for this?
>>
>=20
> Yes, I'll do this later today. Obviously needed. :(

Done. With part of Torbjørn's patch and further fixes as proper c=
ommits.

https://github.com/pturmel/lsdrv

if you just want the latest script:

https://github.com/pturmel/lsdrv/raw/HEAD/lsdrv

Bug reports welcome. Patches, too.

I'm especially interested in users with hardware raid controllers... n=
ot all of them expose their devices through the SCSI subsystem. I've a=
n older HP SmartArray here to play with, so I know it basically works. =
I'm interested in enumerating the physical drives hooked to these cont=
roller subsystems.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

On 26 May 2011 18:46, Phil Turmel wrote:
> On 05/26/2011 07:39 AM, Phil Turmel wrote:
>> On 05/26/2011 04:11 AM, CoolCold wrote:
>>> May be setup some github repo for this?
>>>
>>
>> Yes, I'll do this later today.  Obviously needed. :(
>
> Done.  With part of Torbjørn's patch and further fixes as p=
roper commits.
>
> https://github.com/pturmel/lsdrv
>
> if you just want the latest script:
>
> https://github.com/pturmel/lsdrv/raw/HEAD/lsdrv
>
> Bug reports welcome. Patches, too.
>
> I'm especially interested in users with hardware raid controllers... =
 not all of them expose their devices through the SCSI subsystem. =
 I've an older HP SmartArray here to play with, so I know it basic=
ally works.  I'm interested in enumerating the physical drives hoo=
ked to these controller subsystems.
>
> Phil
>

I think all looks good now:

~/bin $ sudo python2 ./lsdrv
PCI [ahci] 00:0b.0 SATA controller: nVidia Corporation MCP79 AHCI
Controller (rev b1)
├─scsi 0:0:0:0 ATA Corsair CSSD-F60 {10326505580009990=
027}
â”=82 └─sda: Partitioned (dos) 55.90g
â”=82 ├─sda1: (ext4) 100.00m 'ssd_boot' {ae879=
f86-73a4-451f-bb6b-e778ad1b57d6}
â”=82 â”=82 └─Mounted as /dev/sda1 @ /boo=
t
â”=82 ├─sda2: (swap) 2.00g 'ssd_swap' {a28e32f=
a-628c-419a-9693-ca88166d230f}
â”=82 └─sda3: (ext4) 53.80g 'ssd_root' {6e812e=
d7-01c4-4a76-ae31-7b3d36d847f5}
â”=82 └─Mounted as /dev/disk/by-label/ssd_r=
oot @ /
├─scsi 1:0:0:0 ATA WDC WD20EARS-00M {WD-WCAZA1022443}
â”=82 └─sdb: Partitioned (dos) 1.82t
â”=82 └─sdb1: MD raid6 (1/7) 1.82t md0 clean i=
n_sync 'ion:0'
{e6595c64-b3ae-90b3-f011-33ac3f402d20}
â”=82 └─md0: PV LVM2_member 9.08t/9.08t VG =
lvstorage 9.08t
{YLEUKB-klxF-X3gF-6dG3-DL4R-xebv-6gKQc2}
â”=82 └─Volume Group lvstorage (md0) 0 =
free {
Xd0HTM-azdN-v9kJ-C7vD-COcU-Cnn8-6AJ6hI}
â”=82 └─dm-0: (ext4) 9.08t 'storage'
{0ca82f13-680f-4b0d-a5d0-08c246a838e5}
â”=82 └─Mounted as /dev/mapper/lvs=
torage-storage @ /raid6volume
├─scsi 2:0:0:0 ATA WDC WD20EARS-00M {WD-WMAZ20152590}
â”=82 └─sdc: Partitioned (dos) 1.82t
â”=82 └─sdc1: MD raid6 (3/7) 1.82t md0 clean i=
n_sync 'ion:0'
{e6595c64-b3ae-90b3-f011-33ac3f402d20}
├─scsi 3:0:0:0 ATA WDC WD20EARS-00M {WD-WMAZ20188479}
â”=82 └─sdd: Partitioned (dos) 1.82t
â”=82 └─sdd1: MD raid6 (2/7) 1.82t md0 clean i=
n_sync 'ion:0'
{e6595c64-b3ae-90b3-f011-33ac3f402d20}
├─scsi 4:x:x:x [Empty]
└─scsi 5:x:x:x [Empty]
PCI [sata_mv] 05:00.0 SCSI storage controller: HighPoint Technologies,
Inc. RocketRAID 230x 4 Port SATA-II Controller (rev 02)
├─scsi 6:0:0:0 ATA WDC WD20EARS-00M {WD-WCAZA3609190}
â”=82 └─sde: Partitioned (dos) 1.82t
â”=82 └─sde1: MD raid6 (6/7) 1.82t md0 clean i=
n_sync 'ion:0'
{e6595c64-b3ae-90b3-f011-33ac3f402d20}
├─scsi 7:0:0:0 ATA SAMSUNG HD204UI {S2HGJ1RZ800964}
â”=82 └─sdf: Partitioned (dos) 1.82t
â”=82 └─sdf1: MD raid6 (4/7) 1.82t md0 clean i=
n_sync 'ion:0'
{e6595c64-b3ae-90b3-f011-33ac3f402d20}
├─scsi 8:0:0:0 ATA WDC WD20EARS-00M {WD-WCAZA1000331}
â”=82 └─sdg: Partitioned (dos) 1.82t
â”=82 └─sdg1: MD raid6 (0/7) 1.82t md0 clean i=
n_sync 'ion:0'
{e6595c64-b3ae-90b3-f011-33ac3f402d20}
└─scsi 9:0:0:0 ATA SAMSUNG HD204UI {S2HGJ1RZ800850}
└─sdh: Partitioned (dos) 1.82t
└─sdh1: MD raid6 (5/7) 1.82t md0 clean in_sync '=
ion:0'
{e6595c64-b3ae-90b3-f011-33ac3f402d20}

Nice.

/M
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

--Sig_/U8bc2agIIE4YwITjg6MrNXc
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

On Thu, 26 May 2011 13:46:59 -0400
Phil Turmel wrote:

> if you just want the latest script:
>=20
> https://github.com/pturmel/lsdrv/raw/HEAD/lsdrv

I already reported this earlier, but still in this version:

Traceback (most recent call last):
File "./lsdrv", line 274, in
probe_block('/sys/block/'+x)
File "./lsdrv", line 226, in probe_block
dev.phy =3D probe_device(blkpath+'/device', nodestr)
File "./lsdrv", line 193, in probe_device
vendor=3Dfileline1(devpath+'/vendor'),
File "./lsdrv", line 49, in fileline1
fh =3D open(filename, 'r')
IOError: [Errno 2] No such file or directory:
'/sys/devices/platform/floppy.0/vendor'

--=20
With respect,
Roman

--Sig_/U8bc2agIIE4YwITjg6MrNXc
Content-Type: application/pgp-signature; name=signature.asc
Content-Disposition: attachment; filename=signature.asc

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAk3ek8cACgkQTLKSvz+PZwhcZwCeOsZhUSC8JO+XtmJmkZqx LxQs
kXwAnic8BppA+UY62A9qakq+cV991EHk
=/FJd
-----END PGP SIGNATURE-----

--Sig_/U8bc2agIIE4YwITjg6MrNXc--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

Hi Roman,

On 05/26/2011 01:54 PM, Roman Mamedov wrote:
> On Thu, 26 May 2011 13:46:59 -0400
> Phil Turmel wrote:
>
>> if you just want the latest script:
>>
>> https://github.com/pturmel/lsdrv/raw/HEAD/lsdrv
>
> I already reported this earlier, but still in this version:
>
> Traceback (most recent call last):
> File "./lsdrv", line 274, in
> probe_block('/sys/block/'+x)
> File "./lsdrv", line 226, in probe_block
> dev.phy = probe_device(blkpath+'/device', nodestr)
> File "./lsdrv", line 193, in probe_device
> vendor=fileline1(devpath+'/vendor'),
> File "./lsdrv", line 49, in fileline1
> fh = open(filename, 'r')
> IOError: [Errno 2] No such file or directory:
> '/sys/devices/platform/floppy.0/vendor'

I used the wrong exception type. Fix pushed. Try again?

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

--Sig_/grjAWo9KDyu5JOWPPkRrSSg
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

On Thu, 26 May 2011 14:02:41 -0400
Phil Turmel wrote:

> Hi Roman,
>=20
> On 05/26/2011 01:54 PM, Roman Mamedov wrote:
> > On Thu, 26 May 2011 13:46:59 -0400
> > Phil Turmel wrote:
> >=20
> >> if you just want the latest script:
> >>
> >> https://github.com/pturmel/lsdrv/raw/HEAD/lsdrv
> >=20
> > I already reported this earlier, but still in this version:
> >=20
> > Traceback (most recent call last):
> > File "./lsdrv", line 274, in
> > probe_block('/sys/block/'+x)
> > File "./lsdrv", line 226, in probe_block
> > dev.phy =3D probe_device(blkpath+'/device', nodestr)
> > File "./lsdrv", line 193, in probe_device
> > vendor=3Dfileline1(devpath+'/vendor'),
> > File "./lsdrv", line 49, in fileline1
> > fh =3D open(filename, 'r')
> > IOError: [Errno 2] No such file or directory:
> > '/sys/devices/platform/floppy.0/vendor'
>=20
> I used the wrong exception type. Fix pushed. Try again?

Now it locks up with 100% CPU load and no output, I waited for a couple of
minutes. On Ctrl-C:

^CTraceback (most recent call last):
File "./lsdrv", line 274, in
probe_block('/sys/block/'+x)

$ ls /sys/block/
etherd!e1.5 etherd!e2.1 md0 md2 sda sdc sde sdg
etherd!e1.6 fd0 md1 md4 sdb sdd sdf

The first two devices are actually down at this moment, maybe that's the
reason? Still I'd expect not 100% CPU load by lsdrv, but 0% CPU and 100%
iowait in this case.

Output of the old (bash) lsdrv:

Controller device @ pci0000:00/0000:00:06.0 [pata_amd]
IDE interface: nVidia Corporation CK804 IDE (rev f2)
host8: [Empty]
host9: [Empty]
Controller device @ pci0000:00/0000:00:07.0 [sata_nv]
IDE interface: nVidia Corporation CK804 Serial ATA Controller (rev f3)
host6: /dev/sdd ATA Hitachi HDS5C302 {SN: ..............}
host7: /dev/sde ATA WDC WD15EADS-00S {SN: ..............}
Controller device @ pci0000:00/0000:00:08.0 [sata_nv]
IDE interface: nVidia Corporation CK804 Serial ATA Controller (rev f3)
host10: /dev/sdf ATA WDC WD20EADS-00S {SN: ..............}
host11: /dev/sdg ATA WDC WD20EADS-00S {SN: ..............}
Controller device @ pci0000:00/0000:00:0d.0/0000:02:00.0 [ahci]
SATA controller: Marvell Technology Group Ltd. 88SE9123 PCIe SATA 6.0 Gb/s
controller (rev 10) host4: /dev/sdc ATA Hitachi HDS5C302 {SN: .............=
..}
host5: [Empty]
Controller device @ pci0000:00/0000:00:0e.0/0000:01:00.0 [sata_mv]
SCSI storage controller: Marvell Technology Group Ltd. 88SX7042 PCI-e 4-p=
ort
SATA-II (rev 02) host0: [Empty]
host1: [Empty]
host2: /dev/sda ATA ST31000528AS {SN: ..............}
host3: /dev/sdb ATA Hitachi HDS72202 {SN: ..............}


--=20
With respect,
Roman

--Sig_/grjAWo9KDyu5JOWPPkRrSSg
Content-Type: application/pgp-signature; name=signature.asc
Content-Disposition: attachment; filename=signature.asc

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAk3emBMACgkQTLKSvz+PZwhLQQCeJoqmJ1PCyfI4P2t5fjBT wnhV
t4QAn0TqRkcF7adkY9E71jXCbT8GvBAD
=FbN5
-----END PGP SIGNATURE-----

--Sig_/grjAWo9KDyu5JOWPPkRrSSg--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

On 05/26/2011 02:12 PM, Roman Mamedov wrote:
[...]
> Now it locks up with 100% CPU load and no output, I waited for a couple of
> minutes. On Ctrl-C:
>
> ^CTraceback (most recent call last):
> File "./lsdrv", line 274, in
> probe_block('/sys/block/'+x)
>
> $ ls /sys/block/
> etherd!e1.5 etherd!e2.1 md0 md2 sda sdc sde sdg
> etherd!e1.6 fd0 md1 md4 sdb sdd sdf
>
> The first two devices are actually down at this moment, maybe that's the
> reason? Still I'd expect not 100% CPU load by lsdrv, but 0% CPU and 100%
> iowait in this case.

Sounds like an infinite loop, or infinite recursion. Could you apply the temporary patch below so I can see how far the probing got?

> Output of the old (bash) lsdrv:
>
> Controller device @ pci0000:00/0000:00:06.0 [pata_amd]
> IDE interface: nVidia Corporation CK804 IDE (rev f2)
> host8: [Empty]
> host9: [Empty]
> Controller device @ pci0000:00/0000:00:07.0 [sata_nv]
> IDE interface: nVidia Corporation CK804 Serial ATA Controller (rev f3)
> host6: /dev/sdd ATA Hitachi HDS5C302 {SN: ..............}
> host7: /dev/sde ATA WDC WD15EADS-00S {SN: ..............}
> Controller device @ pci0000:00/0000:00:08.0 [sata_nv]
> IDE interface: nVidia Corporation CK804 Serial ATA Controller (rev f3)
> host10: /dev/sdf ATA WDC WD20EADS-00S {SN: ..............}
> host11: /dev/sdg ATA WDC WD20EADS-00S {SN: ..............}
> Controller device @ pci0000:00/0000:00:0d.0/0000:02:00.0 [ahci]
> SATA controller: Marvell Technology Group Ltd. 88SE9123 PCIe SATA 6.0 Gb/s
> controller (rev 10) host4: /dev/sdc ATA Hitachi HDS5C302 {SN: ..............}
> host5: [Empty]
> Controller device @ pci0000:00/0000:00:0e.0/0000:01:00.0 [sata_mv]
> SCSI storage controller: Marvell Technology Group Ltd. 88SX7042 PCI-e 4-port
> SATA-II (rev 02) host0: [Empty]
> host1: [Empty]
> host2: /dev/sda ATA ST31000528AS {SN: ..............}
> host3: /dev/sdb ATA Hitachi HDS72202 {SN: ..............}

The old code never attempted to recurse into the layers of block devices, so it can't have recursion problems.

Phil

8<-------------------------

diff --git a/lsdrv b/lsdrv
index d1caaf8..37728c1 100755
--- a/lsdrv
+++ b/lsdrv
@@ -145,6 +145,7 @@ def sect2size(sectors):
# the struct object w/ filled in details.
controllers=dict()
def probe_controller(cpathlink):
+ print "Probing controller %s" % cpathlink
cpath = os.path.realpath(cpathlink)
if cpath in controllers:
return controllers[cpath]
@@ -186,6 +187,7 @@ def probe_controller(cpathlink):
# controller.
phydevs=dict()
def probe_device(devpathlink, nodestr):
+ print "Probing device %s" % devpathlink
devpath = os.path.realpath(devpathlink)
if devpath in phydevs:
return phydevs[devpath]
@@ -214,6 +216,7 @@ def probe_device(devpathlink, nodestr):
blockbyname=dict()
blockbynode=dict()
def probe_block(blocklink):
+ print "Probing block %s" % blocklink
name=blocklink.rsplit('/', 1)[-1]
if name in blockbyname:
return
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

Hi,

I'm hitting this as well.It seems to be triggered by my non-existing
floppy device.

As there is no real floppy device connected, no driver is loaded, so we
never see a /driver directory. This loop then goes all the way up to /,
where it loops forever at 100% cpu.


from probe_controller()
151 while cpath and not os.path.exists(cpath+'/driver'):
152 cpath =3D os.path.dirname(cpath)
153 if cpath in controllers:
154 return controllers[cpath]


On Thu, 2011-05-26 at 14:22 -0400, Phil Turmel wrote:
> On 05/26/2011 02:12 PM, Roman Mamedov wrote:
> [...]
> > Now it locks up with 100% CPU load and no output, I waited for a co=
uple of
> > minutes. On Ctrl-C:
> >=20
> > ^CTraceback (most recent call last):
> > File "./lsdrv", line 274, in
> > probe_block('/sys/block/'+x)
> >=20
> > $ ls /sys/block/
> > etherd!e1.5 etherd!e2.1 md0 md2 sda sdc sde sdg
> > etherd!e1.6 fd0 md1 md4 sdb sdd sdf
> >=20
> > The first two devices are actually down at this moment, maybe that'=
s the
> > reason? Still I'd expect not 100% CPU load by lsdrv, but 0% CPU and=
100%
> > iowait in this case.
>=20
> Sounds like an infinite loop, or infinite recursion. Could you apply=
the temporary patch below so I can see how far the probing got?
>=20
> > Output of the old (bash) lsdrv:
> >=20
> > Controller device @ pci0000:00/0000:00:06.0 [pata_amd]
> > IDE interface: nVidia Corporation CK804 IDE (rev f2)
> > host8: [Empty]
> > host9: [Empty]
> > Controller device @ pci0000:00/0000:00:07.0 [sata_nv]
> > IDE interface: nVidia Corporation CK804 Serial ATA Controller (re=
v f3)
> > host6: /dev/sdd ATA Hitachi HDS5C302 {SN: ..............}
> > host7: /dev/sde ATA WDC WD15EADS-00S {SN: ..............}
> > Controller device @ pci0000:00/0000:00:08.0 [sata_nv]
> > IDE interface: nVidia Corporation CK804 Serial ATA Controller (re=
v f3)
> > host10: /dev/sdf ATA WDC WD20EADS-00S {SN: ..............}
> > host11: /dev/sdg ATA WDC WD20EADS-00S {SN: ..............}
> > Controller device @ pci0000:00/0000:00:0d.0/0000:02:00.0 [ahci]
> > SATA controller: Marvell Technology Group Ltd. 88SE9123 PCIe SATA=
6.0 Gb/s
> > controller (rev 10) host4: /dev/sdc ATA Hitachi HDS5C302 {SN: .....=
........}
> > host5: [Empty]
> > Controller device @ pci0000:00/0000:00:0e.0/0000:01:00.0 [sata_mv]
> > SCSI storage controller: Marvell Technology Group Ltd. 88SX7042 P=
CI-e 4-port
> > SATA-II (rev 02) host0: [Empty]
> > host1: [Empty]
> > host2: /dev/sda ATA ST31000528AS {SN: ..............}
> > host3: /dev/sdb ATA Hitachi HDS72202 {SN: ..............}
>=20
> The old code never attempted to recurse into the layers of block devi=
ces, so it can't have recursion problems.
>=20
> Phil
>=20
> 8<-------------------------
>=20
> diff --git a/lsdrv b/lsdrv
> index d1caaf8..37728c1 100755
> --- a/lsdrv
> +++ b/lsdrv
> @@ -145,6 +145,7 @@ def sect2size(sectors):
> # the struct object w/ filled in details.
> controllers=3Ddict()
> def probe_controller(cpathlink):
> + print "Probing controller %s" % cpathlink
> cpath =3D os.path.realpath(cpathlink)
> if cpath in controllers:
> return controllers[cpath]
> @@ -186,6 +187,7 @@ def probe_controller(cpathlink):
> # controller.
> phydevs=3Ddict()
> def probe_device(devpathlink, nodestr):
> + print "Probing device %s" % devpathlink
> devpath =3D os.path.realpath(devpathlink)
> if devpath in phydevs:
> return phydevs[devpath]
> @@ -214,6 +216,7 @@ def probe_device(devpathlink, nodestr):
> blockbyname=3Ddict()
> blockbynode=3Ddict()
> def probe_block(blocklink):
> + print "Probing block %s" % blocklink
> name=3Dblocklink.rsplit('/', 1)[-1]
> if name in blockbyname:
> return

--=20
Torbjørn Skagestad
Ide Til Produkt A/S

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

I guess we should move off this list, may be continue on github comment=
s.

On Thu, May 26, 2011 at 10:42 PM, Torbj=F8rn Skagestad no> wrote:
> Hi,
>
> I'm hitting this as well.It seems to be triggered by my non-existing
> floppy device.
>
> As there is no real floppy device connected, no driver is loaded, so =
we
> never see a /driver directory. This loop then goes all the way up to =
/,
> where it loops forever at 100% cpu.
>
>
> from probe_controller()
> 151 while cpath and not os.path.exists(cpath+'/driver'):
> 152 =A0 =A0 =A0 =A0 =A0 =A0 cpath =3D os.path.dirname(cpath)
> 153 =A0 =A0 =A0 =A0 =A0 =A0 if cpath in controllers:
> 154 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 return controllers[cpath]
>
>
> On Thu, 2011-05-26 at 14:22 -0400, Phil Turmel wrote:
>> On 05/26/2011 02:12 PM, Roman Mamedov wrote:
>> [...]
>> > Now it locks up with 100% CPU load and no output, I waited for a c=
ouple of
>> > minutes. On Ctrl-C:
>> >
>> > ^CTraceback (most recent call last):
>> > =A0 File "./lsdrv", line 274, in
>> > =A0 =A0 probe_block('/sys/block/'+x)
>> >
>> > $ ls /sys/block/
>> > etherd!e1.5 =A0etherd!e2.1 =A0md0 =A0md2 =A0sda =A0sdc =A0sde =A0s=
dg
>> > etherd!e1.6 =A0fd0 =A0 =A0 =A0 =A0 =A0md1 =A0md4 =A0sdb =A0sdd =A0=
sdf
>> >
>> > The first two devices are actually down at this moment, maybe that=
's the
>> > reason? Still I'd expect not 100% CPU load by lsdrv, but 0% CPU an=
d 100%
>> > iowait in this case.
>>
>> Sounds like an infinite loop, or infinite recursion. =A0Could you ap=
ply the temporary patch below so I can see how far the probing got?
>>
>> > Output of the old (bash) lsdrv:
>> >
>> > Controller device @ pci0000:00/0000:00:06.0 [pata_amd]
>> > =A0 IDE interface: nVidia Corporation CK804 IDE (rev f2)
>> > =A0 =A0 host8: [Empty]
>> > =A0 =A0 host9: [Empty]
>> > Controller device @ pci0000:00/0000:00:07.0 [sata_nv]
>> > =A0 IDE interface: nVidia Corporation CK804 Serial ATA Controller =
(rev f3)
>> > =A0 =A0 host6: /dev/sdd ATA Hitachi HDS5C302 {SN: ..............}
>> > =A0 =A0 host7: /dev/sde ATA WDC WD15EADS-00S {SN: ..............}
>> > Controller device @ pci0000:00/0000:00:08.0 [sata_nv]
>> > =A0 IDE interface: nVidia Corporation CK804 Serial ATA Controller =
(rev f3)
>> > =A0 =A0 host10: /dev/sdf ATA WDC WD20EADS-00S {SN: ..............}
>> > =A0 =A0 host11: /dev/sdg ATA WDC WD20EADS-00S {SN: ..............}
>> > Controller device @ pci0000:00/0000:00:0d.0/0000:02:00.0 [ahci]
>> > =A0 SATA controller: Marvell Technology Group Ltd. 88SE9123 PCIe S=
ATA 6.0 Gb/s
>> > controller (rev 10) host4: /dev/sdc ATA Hitachi HDS5C302 {SN: ....=
.........}
>> > =A0 =A0 host5: [Empty]
>> > Controller device @ pci0000:00/0000:00:0e.0/0000:01:00.0 [sata_mv]
>> > =A0 SCSI storage controller: Marvell Technology Group Ltd. 88SX704=
2 PCI-e 4-port
>> > SATA-II (rev 02) host0: [Empty]
>> > =A0 =A0 host1: [Empty]
>> > =A0 =A0 host2: /dev/sda ATA ST31000528AS {SN: ..............}
>> > =A0 =A0 host3: /dev/sdb ATA Hitachi HDS72202 {SN: ..............}
>>
>> The old code never attempted to recurse into the layers of block dev=
ices, so it can't have recursion problems.
>>
>> Phil
>>
>> 8<-------------------------
>>
>> diff --git a/lsdrv b/lsdrv
>> index d1caaf8..37728c1 100755
>> --- a/lsdrv
>> +++ b/lsdrv
>> @@ -145,6 +145,7 @@ def sect2size(sectors):
>> =A0# the struct object w/ filled in details.
>> =A0controllers=3Ddict()
>> =A0def probe_controller(cpathlink):
>> + =A0 =A0 print "Probing controller %s" % cpathlink
>> =A0 =A0 =A0 cpath =3D os.path.realpath(cpathlink)
>> =A0 =A0 =A0 if cpath in controllers:
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 return controllers[cpath]
>> @@ -186,6 +187,7 @@ def probe_controller(cpathlink):
>> =A0# controller.
>> =A0phydevs=3Ddict()
>> =A0def probe_device(devpathlink, nodestr):
>> + =A0 =A0 print "Probing device %s" % devpathlink
>> =A0 =A0 =A0 devpath =3D os.path.realpath(devpathlink)
>> =A0 =A0 =A0 if devpath in phydevs:
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 return phydevs[devpath]
>> @@ -214,6 +216,7 @@ def probe_device(devpathlink, nodestr):
>> =A0blockbyname=3Ddict()
>> =A0blockbynode=3Ddict()
>> =A0def probe_block(blocklink):
>> + =A0 =A0 print "Probing block %s" % blocklink
>> =A0 =A0 =A0 name=3Dblocklink.rsplit('/', 1)[-1]
>> =A0 =A0 =A0 if name in blockbyname:
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 return
>
> --
> Torbj=F8rn Skagestad
> Ide Til Produkt A/S
>
>



--=20
Best regards,
[COOLCOLD-RIPN]
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

On 05/26/2011 02:58 PM, CoolCold wrote:
> I guess we should move off this list, may be continue on github comments.

I enabled the wiki on github for now. At some point I'll create a dev list.

https://github.com/pturmel/lsdrv/wiki

And I've created a issue for the floppy drive problem(s).

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

RE: Storage device enumeration script

> -----Original Message-----
> From: linux-raid-owner@vger.kernel.org [mailto:linux-raid-
> owner@vger.kernel.org] On Behalf Of Phil Turmel
> Sent: Thursday, May 26, 2011 6:42 AM
> To: Mathias Bur=E9n
> Cc: lrhorer@satx.rr.com; linux-raid@vger.kernel.org
> Subject: Re: Storage device enumeration script
>=20
> On 05/26/2011 02:16 AM, Mathias Bur=E9n wrote:
> > On 26 May 2011 07:14, Leslie Rhorer wrote:
> >>
> >> I get an error right off the bat:
> >>
> >> Traceback (most recent call last):
> >> File "./lsdrv.txt", line 264, in
> >> for x in runx(['pvs', '-o',
> >> 'pv_name,pv_used,pv_size,pv_uuid,vg_name,vg_size,vg_free,vg_ uuid',
> >> '--noheadings', '--separator', ' ']).split("\n"):
> >> File "./lsdrv.txt", line 50, in runx
> >> sub =3D Popen(*args, **kwargs)
> >> File "/usr/lib/python2.6/subprocess.py", line 623, in __init__
> >> errread, errwrite)
> >> File "/usr/lib/python2.6/subprocess.py", line 1141, in _execute_c=
hild
> >> raise child_exception
> >> OSError: [Errno 2] No such file or directory
> >>
> >> I'm running Python 2.6.6 and mdadm 3.1.4 running on kernel 2.6.32-=
5-
> amd64
> >>
> >>
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe linux-ra=
id"
> in
> >> the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at http://vger.kernel.org/majordomo-info.html
> >>
> >
> > I get that in Ubuntu 11.04 on a laptop as well.
> >
> > /M
>=20
> Does a plain install of Ubuntu include the LVM utilities? Can you ru=
n
> "pvs" directly?

I can't speak to Ubuntu, but Debian evidently does not. I don't
know what "pvs" is, but neither bash nor Python recognize it as a file
anywhere in the path.

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

On 27/05/11 08:13, Leslie Rhorer wrote:
>>
> I can't speak to Ubuntu, but Debian evidently does not. I don't
> know what "pvs" is, but neither bash nor Python recognize it as a file
> anywhere in the path.
apt-get install lvm2.

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

--Sig_/bEreq8MRAGbCPtHpo+GiOOb
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

On Fri, 27 May 2011 08:16:07 +0800
Brad Campbell wrote:

> On 27/05/11 08:13, Leslie Rhorer wrote:
> >>
> > I can't speak to Ubuntu, but Debian evidently does not. I don't
> > know what "pvs" is, but neither bash nor Python recognize it as a file
> > anywhere in the path.
> apt-get install lvm2.

Sure, but lsdrv shouldn't assume lvm2 is installed or require it to be
installed. Not everyone uses LVM, and simply installing it automatically ad=
ds
things to initramfs (PV/VG/LV detection?), which can slow down boot-up proc=
ess.

--=20
With respect,
Roman

--Sig_/bEreq8MRAGbCPtHpo+GiOOb
Content-Type: application/pgp-signature; name=signature.asc
Content-Disposition: attachment; filename=signature.asc

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAk3fIXAACgkQTLKSvz+PZwi44ACffi2ciSy0oJNWlAhubjT7 TyXg
JNQAn0g42yhVEut36SrFOcleZbY0qgS7
=TPpE
-----END PGP SIGNATURE-----

--Sig_/bEreq8MRAGbCPtHpo+GiOOb--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

On 26/05/2011 10:59, John Robinson wrote:
[...]
> [root@beast lsdrv]# python2.6 lsdrv
> PCI [pata_marvell] 03:00.0 IDE interface: Marvell Technology Group Lt=
d.
> 88SE6121 SATA II Controller (rev b2)
> └─scsi 0:0:0:0 HL-DT-ST DVD-RAM GH22NP20
> └─sr0: Empty/Unknown 1.00g
> PCI [ahci] 00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10
> Family) SATA AHCI Controller
> ├─scsi 2:0:0:0 ATA Hitachi HDS72101
> â”=82 └─sda: Empty/Unknown 931.51g
> Traceback (most recent call last):
> File "lsdrv", line 387, in
> show_blocks(" %s " % branch[0], [phy.block])
> File "lsdrv", line 339, in show_blocks
> show_blocks("%s %s " % (indent, branch[0]), [blockbyname[x] for x in =
subs])
> KeyError: 'sda1'
>
> Now, something's not getting picked up about sda. Looking at Mathias'
> "sweet" output, it's not coping with the (DOS) partition table. Anoth=
er
> variation on my kernel's /sys or still to old a Python or ...?

Still seeing this. I added a print command so I can see that=20
dev.partitions is being populated successfully. I'm not entirely sure=20
where dev.ID_ etc are supposed to be coming from, but if it's that=20
`blkid -p -o udev /dev/block/8:0` then I'm afraid CentOS 5's blkid=20
doesn't understand the -p or -o udev options, it doesn't produce any=20
output for whole drives with partition tables, and there isn't a=20
/dev/block directory. It's blkid 1.0.0 from e2fsprogs 1.39-23.el5_5.1.

If that knocks CentOS 5 support on the head then so be it...

Cheers,

John.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

On 27/05/2011 10:15, John Robinson wrote:
[...]
> I'm not entirely sure where dev.ID_ etc are supposed to be coming
> from, but if it's that `blkid -p -o udev /dev/block/8:0` then I'm
> afraid CentOS 5's blkid doesn't understand the -p or -o udev options,
> it doesn't produce any output for whole drives with partition tables,
> and there isn't a /dev/block directory. It's blkid 1.0.0 from
> e2fsprogs 1.39-23.el5_5.1.
>
> If that knocks CentOS 5 support on the head then so be it...

Hmm, udevinfo might be of some use. Still doesn't say it's found a DOS
partition table, but it does get you e.g. ID_FS_TYPE=linux_raid_member
and perhaps `file -s` will tell you there's DOS partition table (sort of).

Cheers,

John.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

On Fri, 27 May 2011, Roman Mamedov wrote:

> On Fri, 27 May 2011 08:16:07 +0800
> Brad Campbell wrote:
>
>> On 27/05/11 08:13, Leslie Rhorer wrote:
>>>>
>>> I can't speak to Ubuntu, but Debian evidently does not. I don't
>>> know what "pvs" is, but neither bash nor Python recognize it as a file
>>> anywhere in the path.
>> apt-get install lvm2.
>
> Sure, but lsdrv shouldn't assume lvm2 is installed or require it to be
> installed. Not everyone uses LVM, and simply installing it automatically adds
> things to initramfs (PV/VG/LV detection?), which can slow down boot-up process.

As well as not using LVM, some of us don't even use udev... Or kernels
with modules...

Can someone post the output of this utility so I can see what it's
actually doing?

(I don't have python on all my servers either)

Cheers,

Gordon
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

On 05/27/2011 05:44 AM, John Robinson wrote:
> On 27/05/2011 10:15, John Robinson wrote:
> [...]
>> I'm not entirely sure where dev.ID_ etc are supposed to be coming
>> from, but if it's that `blkid -p -o udev /dev/block/8:0` then I'm
>> afraid CentOS 5's blkid doesn't understand the -p or -o udev options,
>> it doesn't produce any output for whole drives with partition tables,
>> and there isn't a /dev/block directory. It's blkid 1.0.0 from
>> e2fsprogs 1.39-23.el5_5.1.
>>
>> If that knocks CentOS 5 support on the head then so be it...
>
> Hmm, udevinfo might be of some use. Still doesn't say it's found a DOS
> partition table, but it does get you e.g. ID_FS_TYPE=linux_raid_member and perhaps `file -s` will tell you there's DOS partition table (sort of).

I'll look into this when I have a new CentOS 5 VM installed on my laptop. I do want lsdrv to work with all of the CentOS 5 releases.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

--=-MXhvSv7HsB0jwIbfTe/t
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi,

Some info can be found here https://github.com/pturmel/lsdrv

The output will typically be like this:

Controller platform [None]
└─platform floppy.0
└─fd0: Empty/Unknown 4.00k
PCI [pata_jmicron] 03:00.1 IDE interface: JMicron Technology Corp. JMB362/J=
MB363 Serial ATA Controller (rev 02)
├─scsi 0:x:x:x [Empty]
└─scsi 2:x:x:x [Empty]
PCI [ahci] 03:00.0 SATA controller: JMicron Technology Corp. JMB362/JMB363 =
Serial ATA Controller (rev 02)
├─scsi 12:0:0:0 ATA THROTTLE
â”=82 └─sdu: Empty/Unknown 7.51g
â”=82 ├─sdu1: Empty/Unknown 243.00m
â”=82 â”=82 └─Mounted as /dev/sdu1 @ /boot
â”=82 ├─sdu2: Empty/Unknown 1.00k
â”=82 └─sdu5: Empty/Unknown 7.27g
â”=82 ├─dm-0: Empty/Unknown 6.90g
â”=82 â”=82 └─Mounted as /dev/mapper/serve=
r-root @ /
â”=82 └─dm-1: Empty/Unknown 368.00m
└─scsi 13:0:0:0 ATA WDC WD10EACS-00Z
└─sdv: Empty/Unknown 931.51g
└─md1: Empty/Unknown 3.64t
└─dm-3: Empty/Unknown 3.64t
└─Mounted as /dev/mapper/big1 @ /mnt/big1
PCI [pcieport] 00:1c.0 PCI bridge: Intel Corporation 82801H (ICH8 Family) P=
CI Express Port 1 (rev 02)
├─scsi 15:0:0:0 ATA WDC WD20EARS-00J
â”=82 └─sdy: Empty/Unknown 1.82t
â”=82 └─sdy1: Empty/Unknown 1.82t
â”=82 └─md3: Empty/Unknown 7.28t
â”=82 └─dm-5: Empty/Unknown 7.28t
â”=82 └─Mounted as /dev/mapper/big3 @ /mn=
t/big3
├─scsi 15:0:1:0 ATA WDC WD20EARS-00J
â”=82 └─sdz: Empty/Unknown 1.82t
â”=82 └─sdz1: Empty/Unknown 1.82t
├─scsi 15:0:2:0 ATA WDC WD20EARS-00J
â”=82 └─sda: Empty/Unknown 1.82t
â”=82 └─sda1: Empty/Unknown 1.82t
├─scsi 15:0:3:0 ATA WDC WD20EARS-00J
â”=82 └─sdw: Empty/Unknown 1.82t
â”=82 └─sdw1: Empty/Unknown 1.82t
└─scsi 15:0:4:0 ATA WDC WD20EARS-00J
└─sdae: Empty/Unknown 1.82t
└─sdae1: Empty/Unknown 1.82t
---snip---


On Fri, 2011-05-27 at 11:45 +0100, Gordon Henderson wrote:
> On Fri, 27 May 2011, Roman Mamedov wrote:
>=20
> > On Fri, 27 May 2011 08:16:07 +0800
> > Brad Campbell wrote:
> >
> >> On 27/05/11 08:13, Leslie Rhorer wrote:
> >>>>
> >>> I can't speak to Ubuntu, but Debian evidently does not. I don't
> >>> know what "pvs" is, but neither bash nor Python recognize it as a fil=
e
> >>> anywhere in the path.
> >> apt-get install lvm2.
> >
> > Sure, but lsdrv shouldn't assume lvm2 is installed or require it to be
> > installed. Not everyone uses LVM, and simply installing it automaticall=
y adds
> > things to initramfs (PV/VG/LV detection?), which can slow down boot-up =
process.
>=20
> As well as not using LVM, some of us don't even use udev... Or kernels=
=20
> with modules...
>=20
> Can someone post the output of this utility so I can see what it's=20
> actually doing?
>=20
> (I don't have python on all my servers either)
>=20
> Cheers,
>=20
> Gordon
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html

--=20
Torbjørn Skagestad
Idé Til Produkt AS
torborn@itpas.no

--=-MXhvSv7HsB0jwIbfTe/t
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEABECAAYFAk3fingACgkQ4RjLWGFqSIZ3+QCfZLVS381tb1aIRwTCBU5K /i1L
q2cAn0iw9hmFkpETNWdNsiqA8Edge8bY
=MQX3
-----END PGP SIGNATURE-----

--=-MXhvSv7HsB0jwIbfTe/t--

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

This message is in MIME format. The first part should be readable text,
while the remaining parts are likely unreadable without MIME-aware tools.

--180569313-157037677-1306496574=:25096
Content-Type: TEXT/PLAIN; charset=iso-8859-1; format=flowed
Content-Transfer-Encoding: 8BIT

On Fri, 27 May 2011, Torbjørn Skagestad wrote:

> Hi,
>
> Some info can be found here https://github.com/pturmel/lsdrv
>
> The output will typically be like this:

Ok. Intersting, thanks.

I generllyy know what's inside my servers, but I suppose visualising it is
good, or having to work on someone elses server...

Gordon


>
> Controller platform [None]
> ??platform floppy.0
> ??fd0: Empty/Unknown 4.00k
> PCI [pata_jmicron] 03:00.1 IDE interface: JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (rev 02)
> ??scsi 0:x:x:x [Empty]
> ??scsi 2:x:x:x [Empty]
> PCI [ahci] 03:00.0 SATA controller: JMicron Technology Corp. JMB362/JMB363 Serial ATA Controller (rev 02)
> ??scsi 12:0:0:0 ATA THROTTLE
> ? ??sdu: Empty/Unknown 7.51g
> ? ??sdu1: Empty/Unknown 243.00m
> ? ? ??Mounted as /dev/sdu1 @ /boot
> ? ??sdu2: Empty/Unknown 1.00k
> ? ??sdu5: Empty/Unknown 7.27g
> ? ??dm-0: Empty/Unknown 6.90g
> ? ? ??Mounted as /dev/mapper/server-root @ /
> ? ??dm-1: Empty/Unknown 368.00m
> ??scsi 13:0:0:0 ATA WDC WD10EACS-00Z
> ??sdv: Empty/Unknown 931.51g
> ??md1: Empty/Unknown 3.64t
> ??dm-3: Empty/Unknown 3.64t
> ??Mounted as /dev/mapper/big1 @ /mnt/big1
> PCI [pcieport] 00:1c.0 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 1 (rev 02)
> ??scsi 15:0:0:0 ATA WDC WD20EARS-00J
> ? ??sdy: Empty/Unknown 1.82t
> ? ??sdy1: Empty/Unknown 1.82t
> ? ??md3: Empty/Unknown 7.28t
> ? ??dm-5: Empty/Unknown 7.28t
> ? ??Mounted as /dev/mapper/big3 @ /mnt/big3
> ??scsi 15:0:1:0 ATA WDC WD20EARS-00J
> ? ??sdz: Empty/Unknown 1.82t
> ? ??sdz1: Empty/Unknown 1.82t
> ??scsi 15:0:2:0 ATA WDC WD20EARS-00J
> ? ??sda: Empty/Unknown 1.82t
> ? ??sda1: Empty/Unknown 1.82t
> ??scsi 15:0:3:0 ATA WDC WD20EARS-00J
> ? ??sdw: Empty/Unknown 1.82t
> ? ??sdw1: Empty/Unknown 1.82t
> ??scsi 15:0:4:0 ATA WDC WD20EARS-00J
> ??sdae: Empty/Unknown 1.82t
> ??sdae1: Empty/Unknown 1.82t
> ---snip---
>
>
> On Fri, 2011-05-27 at 11:45 +0100, Gordon Henderson wrote:
>> On Fri, 27 May 2011, Roman Mamedov wrote:
>>
>>> On Fri, 27 May 2011 08:16:07 +0800
>>> Brad Campbell wrote:
>>>
>>>> On 27/05/11 08:13, Leslie Rhorer wrote:
>>>>>>
>>>>> I can't speak to Ubuntu, but Debian evidently does not. I don't
>>>>> know what "pvs" is, but neither bash nor Python recognize it as a file
>>>>> anywhere in the path.
>>>> apt-get install lvm2.
>>>
>>> Sure, but lsdrv shouldn't assume lvm2 is installed or require it to be
>>> installed. Not everyone uses LVM, and simply installing it automatically adds
>>> things to initramfs (PV/VG/LV detection?), which can slow down boot-up process.
>>
>> As well as not using LVM, some of us don't even use udev... Or kernels
>> with modules...
>>
>> Can someone post the output of this utility so I can see what it's
>> actually doing?
>>
>> (I don't have python on all my servers either)
>>
>> Cheers,
>>
>> Gordon
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
> --
> Torbjørn Skagestad
> Idé Til Produkt AS
> torborn@itpas.no
>
--180569313-157037677-1306496574=:25096--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

On 05/26/2011 11:58 PM, Roman Mamedov wrote:
> On Fri, 27 May 2011 08:16:07 +0800
> Brad Campbell wrote:
>=20
>> On 27/05/11 08:13, Leslie Rhorer wrote:
>>>>
>>> I can't speak to Ubuntu, but Debian evidently does not. I don't
>>> know what "pvs" is, but neither bash nor Python recognize it as a f=
ile
>>> anywhere in the path.
>> apt-get install lvm2.
>=20
> Sure, but lsdrv shouldn't assume lvm2 is installed or require it to b=
e
> installed. Not everyone uses LVM, and simply installing it automatica=
lly adds
> things to initramfs (PV/VG/LV detection?), which can slow down boot-u=
p process.

With the latest contribution from Torbj=F8rn, lsdrv will now report the=
missing utility, then continue. If one of the devices is in fact an L=
VM element, the block device recursion code should still show the basic=
relationships, without the volume group details. Testing that case wo=
uld be appreciated.

https://github.com/pturmel/lsdrv

If you find more bugs, or have other suggestions, using the issue track=
er and the wiki on github would minimize the chatter on linux-raid.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

Hi Phil,
Thanks for doing this work off your own back, I'm sure it'll help a lot
of people.

My 2p would be to ask if you could have a comment at the start of the
file with the prerequisite applications/packages named it may help the
people that it doesn't work for (initially). The other thing is that
the virtualbox install of Ubuntu 11.04 that I just completed did not
contain the LVM2 package.

regards
Simon

On 26/05/2011 13:00, Phil Turmel wrote:
> On 05/26/2011 04:24 AM, CoolCold wrote:
> [...]
>> On Debian Lenny produces error:
>> root@gamma2:/tmp# python lsdrv
>> Traceback (most recent call last):
>> File "lsdrv", line 17, in
>> import os, io, re
>> ImportError: No module named io
> Huh. The 'io' module is v2.6 and above. Another reason to make a helper function for reading these files.
>
> Phil
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message tomajordomo@vger.kernel.org
> More majordomo info athttp://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

On Tue, May 31, 2011 at 10:51 PM, Simon McNair =
wrote:
> Hi Phil,
> Thanks for doing this work off your own back, I'm sure it'll help a l=
ot of
> people.
>
> My 2p would be to ask if you could have a comment at the start of the=
file
> with the prerequisite applications/packages named it may help the peo=
ple
> that it doesn't work for (initially). =A0The other thing is that the
> virtualbox install of Ubuntu 11.04 that I just completed did not cont=
ain the
> LVM2 package.
There is debian packaging for this tool, checkout
https://github.com/pturmel/lsdrv
You should be able build your package with proper dependencies.


>
> regards
> Simon
>
> On 26/05/2011 13:00, Phil Turmel wrote:
>>
>> On 05/26/2011 04:24 AM, CoolCold wrote:
>> [...]
>>>
>>> On Debian Lenny produces error:
>>> root@gamma2:/tmp# python lsdrv
>>> Traceback (most recent call last):
>>> =A0 File "lsdrv", line 17, in
>>> =A0 =A0 import os, io, re
>>> ImportError: No module named io
>>
>> Huh. =A0The 'io' module is v2.6 and above. =A0Another reason to make=
a helper
>> function for reading these files.
>>
>> Phil
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid=
" in
>> the body of a message tomajordomo@vger.kernel.org
>> More majordomo info athttp://vger.kernel.org/majordomo-info.html
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid"=
in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
>



--=20
Best regards,
[COOLCOLD-RIPN]
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

Hi John,

On 05/27/2011 07:23 AM, Phil Turmel wrote:
> On 05/27/2011 05:44 AM, John Robinson wrote:
>> On 27/05/2011 10:15, John Robinson wrote:
>> [...]
>>> I'm not entirely sure where dev.ID_ etc are supposed to be coming
>>> from, but if it's that `blkid -p -o udev /dev/block/8:0` then I'm
>>> afraid CentOS 5's blkid doesn't understand the -p or -o udev options,
>>> it doesn't produce any output for whole drives with partition tables,
>>> and there isn't a /dev/block directory. It's blkid 1.0.0 from
>>> e2fsprogs 1.39-23.el5_5.1.
>>>
>>> If that knocks CentOS 5 support on the head then so be it...
>>
>> Hmm, udevinfo might be of some use. Still doesn't say it's found a DOS
>> partition table, but it does get you e.g. ID_FS_TYPE=linux_raid_member and perhaps `file -s` will tell you there's DOS partition table (sort of).
>
> I'll look into this when I have a new CentOS 5 VM installed on my laptop. I do want lsdrv to work with all of the CentOS 5 releases.

I've been playing with lsdrv in a CentOS 5 VM, and found a number of items to address. The result has been pushed to github, with a detailed description. Please give it a whirl.

https://github.com/pturmel/lsdrv

(Further bug reports should be posted there... trying to keep the noise down on linux-raid.)

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Storage device enumeration script

Hi Simon,

On 05/31/2011 05:21 PM, CoolCold wrote:
> On Tue, May 31, 2011 at 10:51 PM, Simon McNair wrote:
>> Hi Phil,
>> Thanks for doing this work off your own back, I'm sure it'll help a lot of
>> people.
>>
>> My 2p would be to ask if you could have a comment at the start of the file
>> with the prerequisite applications/packages named it may help the people
>> that it doesn't work for (initially). The other thing is that the
>> virtualbox install of Ubuntu 11.04 that I just completed did not contain the
>> LVM2 package.
> There is debian packaging for this tool, checkout
> https://github.com/pturmel/lsdrv
> You should be able build your package with proper dependencies.

The latest version (this evening) reports on the utilities that didn't work, so you can look for what you need. Of course, if you have LVM volumes, but not the utilities, your system won't actually be able to start up those devices.

If you don't have LVM volumes, you can safely ignore the warning. Hmmm. Maybe I should suppress that if no LVM PVs are found in the first pass....

G'night!

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html