A horrible little ElementTree gotcha

Oct 20, 2009

What does this print:

from lxml import etree
doc = etree.fromstring('<a><b><c/></b></a>')
newdoc = etree.ElementTree(doc.find('b'))
print newdoc.xpath('/b/c')[0].xpath('/a')

The answer is: [<Element a at 817548c>]. The first point to note is that xpath() against an element is only relative to that element: any absolute XPaths enumerate from the top of the containing tree. The second point is that the shallow copying of etree means that _Element::xpath, unlike _ElementTree::xpath, evaluates absolute paths from the top of the original underlying tree! So even though there’s no <a> in newdoc, an absolute XPath on a child element can still reach it.
Yuck.

YouTube annoyance

Oct 19, 2009

How much time would it really take to order multi-part videos, so the suggestion at the end of the video is the next part? Please!

xVM and COMSTAR iSCSI

Oct 15, 2009
I recently had cause to try out COMSTAR for the first time, and I thought I'd write up the steps needed. Unfortunately, it's considerably more complex than the fall-over-easy shareiscsi=on ZFS feature.

Configuring the COMSTAR server

First install the storage-server packages and enable the services:

# svcadm enable -r stmf
# svcadm enable -r iscsi/target

We want to create a target group for each of our xVM guests, each of which will have one LUN in it. After creating the LUN, we define a "view" that allows that LUN to be visible for that target group:

# stmfadm create-tg domu-226
# zfs create -V 15G export/domu-226
# stmfadm create-lu /dev/zvol/rdsk/export/domu-226
Logical unit created: 600144F0C73ABF0F00004AD75DF2001A
# stmfadm add-view -t domu-226 600144F0C73ABF0F00004AD75DF2001A

Now we need to create the iSCSI target for this target group, that has our single LUN in it.

# itadm create-target -l domu-226
Target iqn.1986-03.com.sun:02:b8596bb9-9bb9-40e9-8cda-add6073ece46 successfully created

Here (finally) is our iSCSI Alias we can use in the clients. But we're not done yet. By default, this target will be able to see all LUNs not in a target group. So we need to make it a member of our domu-226 target group:

# stmfadm add-tg-member -g domu-226 iqn.1986-03.com.sun:02:b8596bb9-9bb9-40e9-8cda-add6073ece46
# stmfadm list-tg -v
Target Group: domu-226
        Member: iqn.1986-03.com.sun:02:b8596bb9-9bb9-40e9-8cda-add6073ece46

Configuring the iSCSI initiator (client)

We do this in the usual manner:

# svcadm enable -r svc:/network/iscsi/initiator:default
# iscsiadm add discovery-address 10.6.70.43:3260
# iscsiadm modify discovery --sendtargets enable

Installing a guest onto the LUN

We went through the above gymnastics so we can have a human-readable Alias for each of the domu's root LUNs. So now we can do:

# virt-install --paravirt --name domu-226 --ram 1024 --os-type solaris --os-variant opensolaris \
  --location nfs:10.5.235.28:/export/nv/x/latest --network bridge,mac=00:14:4f:0f:b5:3e \
  --disk path=/alias/domu-226,driver=phy,subdriver=iscsi \
  --nographics

Tags:

An annoying Python gotcha

Oct 10, 2009

Imagine you have this in mod.py:

import foo

class bar(object):
   ...

   def __del__(self):
       foo.cleanup(self.myhandle)

Seems fine right? In fact, there’s a nasty bug here. If I try to use this module in client.py like so:

import mod
mybar = bar()

Then you’re likely to get an exception when the program exits. This is because Python, for some bizarre reason, Nones out the globals in mod.py when taking down the interpreter. The actual __del__ method can be called sometime after this, and it ends up trying None.cleanup(), with the resultant AttributeError. It seems extremely bizarre that it happens in this order, but it does (a real example).

Kernel solipsism

Jun 4, 2009

Thomas Gleixner:

Exactly that’s the point. Adding dom0 makes life easier for a group of users who decided to use Xen some time ago, but what Ingo wants is technical improvement of the kernel… The kernel policy always was and still is to accept only those features which have a technical benefit to the code base.

It boggles the mind that someone could get things so backwards. The kernel exists to provide services to the outside world, not the other way around. By all means criticise the details of the Xen dom0 code, but this argument makes zero sense. How precisely did x86_64 support provide a technical benefit to the code base?

OpenSolaris 2009.06 guest domain on a Linux dom0

Jun 2, 2009
Just a quick note: you can follow the instructions I provided for the 2008.11 release, with one change. On a 64-bit machine, replace any instances of /boot/x86.microroot with /boot/amd64/x86.microroot. As of 2009.06, the boot archive is split into 32-bit and 64-bit variants. If you get a message like this:

krtld: failed to open '/platform/i86xpv/kernel/amd64/unix'

Then you've probably given the wrong combination of unix and microroot.

By the way, in my previous entry, I mentioned we were working on upstreaming our virt-install changes. During the Xen 3.3 work (more on which soon), I updated to the latest versions and got the needed parts into the upstream version. We've still some ZFS changes to push, but if you're running a recent enough version of Xen on Linux, you may well be able to use virt-install and skip all this horrible hacking!

BNP

May 18, 2009

Charlie Brooker on the BNP party political broadcast:

Nick Griffin’s first line is “Don’t turn it off!", which in terms of opening gambits is about as enticing as hearing someone shout “Try not to be sick!” immediately prior to intercourse.

Outputting XML in standard Python

Mar 26, 2009

Is it really this ugly? I expected something like this:

doc = xmldoc()
doc.start('foo', { 'id': 'blah' })
doc.start('sub')
doc.text('subtext')
doc.close('sub')
doc.close('foo')
print doc

and I thought I had it in SimpleXMLWriter. However, I have to jump hoops to get it to output to a string, and it doesn’t have any pretty-print. I tried using ElementTree, but that also doesn’t pretty print! libxml2 is horribly low-level. lxml seems to do pretty printing, but it’s still just as ugly as the best option I’ve found so far, xml.dom.minidom:

from xml.dom.minidom import Document
foo = doc.createElement('foo')
foo.setAttribute('id', 'blah')
doc.appendChild(foo)
sub = doc.createElement('sub')
sub.appendChild(doc.createTextNode('subtext'))
foo.appendChild(sub)

Yuck! If I’m building up a document, I almost always want to append directly at the last point: why do I have to keep track of all these elements by hand? I presume I’m missing some small standard helper module, but #python didn’t know about it. Anyone?

Scoble sets a new record

Mar 22, 2009

I really hate the word “friend.” It has no meaning anymore. No one can define what a friend is. Believe me, I’ve asked dozens of people to define it for me. My wife is my most “true” friend, for instance but if you trust her with picking a great wine (she doesn’t drink much) or picking a great sushi restaurant (she hates the stuff) you’ll be very disappointed. You’d be better off asking @garyvee about the wine even though you’ve never met him and he probably wouldn’t be listed among your “true” friends.

- Scoble

Might I gently suggest friendship isn’t about wine recommendations?

Sheesh

Mar 16, 2009

Apparently applications should be prepared to lose 60 minutes of data to work around the file system now.

Of course the notion that application should have explicit load/save operations is a nonsense already. Now we should “fix” one of the few places that never had this (ever seen a browser where you have to save your bookmarks explicitly when you quit?) to expose this implementation detail in a data-losing way again.