UPDATE: the canonical location for this information is now here - please check there, as it will be updated as necessary, unlike this blog entry.

As a final part to my entries on OpenSolaris and Xen, let's go through the steps needed to turn OpenSolaris into a dom0. Thanks to Trevor O for documenting this for 2008.05. And as before, expect this process to get much, much, easier soon!

I'm going to do the work in a separate BE, so if we mess up, we shouldn't have broken anything. So, first we create our BE:

$ pfexec beadm create -a -d xvm xvm

First, let's install the packages. If you've updated to the development version, a simple pkg install xvm-gui will work, but let's assume you haven't:

$ pfexec beadm mount xvm /tmp/xvm-be
$ pfexec pkg -R /tmp/xvm-be install SUNWvirt-manager SUNWxvm SUNWvdisk SUNWvncviewer
$ pfexec beadm umount xvm

Now we need to actually reboot into Xen. Unfortunately beadm is not yet aware of how to do this, so we'll have to hack it up. We're going to run some awk over the menu.lst file which controls grub:

$ awk '
/^title/ { xvm=0; }
/^title.xvm$/ { xvm=1; }
/^(splashimage|foreground|background)/ {
    if (xvm == 1) next
}
/^kernel\$/ {
    if (xvm == 1) {
       print("kernel\$ /boot/\$ISADIR/xen.gz")
       sub("^kernel\\$", "module$")
       gsub("console=graphics", "console=text")
       gsub("i86pc", "i86xpv")
       $2=$2 " " $2
    }
}
{ print }' /rpool/boot/grub/menu.lst >/var/tmp/menu.lst.xvm

Let's check that the awk script (my apologies) worked properly:

$ tail /var/tmp/menu.lst.xvm 
...
#============ End of LIBBE entry =============
title xvm
findroot (pool_rpool,0,a)
bootfs rpool/ROOT/xvm
kernel$ /boot/$ISADIR/xen.gz
module$ /platform/i86xpv/kernel/$ISADIR/unix /platform/i86xpv/kernel/$ISADIR/unix -B $ZFS-BOOTFS,console=text
module$ /platform/i86pc/$ISADIR/boot_archive
#============ End of LIBBE entry =============

Looks good. We'll move it into place, and reboot:

$ pfexec cp /rpool/boot/grub/menu.lst /rpool/boot/grub/menu.lst.saved
$ pfexec mv /var/tmp/menu.lst.xvm /rpool/boot/grub/menu.lst
$ pfexec reboot

This should boot you into xVM. If everything worked OK, let's enable the services:

$ svcadm enable -r xvm/virtd ; svcadm enable -r xvm/domains

At this point, you should be able to merrily go ahead and install domains!

Update: Todd Clayton pointed out the issue I've filed here: SUNWxvm needs to depend on SUNWvdisk. I've updated the instructions above with the workaround.

Update update: Rich Burridge has fixed it. Nice!

Tags: OpenSolaris Xen xVM

I think atheists and believers alike should agree to abide by the Advertising Standard Agency’s decision, when it comes - agreed?

After sitting through Expelled, I felt the need to cleanse my intellectual palate. Thankfully I had Bill Maher’s Religulous to watch. Coming across like a mongrel of Borat and An Inconvenient Truth, it’s an alleged documentary, played for laughs. And chunks of it are indeed very, very funny - the Cannabis Ministry guy comes to mind.

Consisting mainly of Maher tracking down the more comedic elements of out-there religion, it’s a wonder he
got most of these people to sign the release forms. I’m especially thinking of the senator who admitted he
was religious because he was stupid. Maher uses his incredulity at these people’s beliefs for humour, and he
does it rather well. It’s not the kind of film that’s supposed to make a serious point, though of course it does.

Particular highlights for me were the Vatican priest who dismissed Hell as a “silly idea”, and the Professor
Frink style inventions of The Institute for Science and Halacha, devoted to technology to work around the absurd orthodox Shabbat rules (cue the pneumatic wheelchair).

Recommended.

Yes, it’s a pointless end of year list! In no particular order.
Much of this stuff is by no means new in 2008, but it’s all new to me in 2008.

Sailor Jerry’s

Old news to most by now but the popularity of combining this vanilla-flavoured rum with coke and ice has yet to wane.

FFV

Imported via a visit to Helsinki earlier in the year, Fisherman’s Friend Vodka (both its name, and its recipe) took the world* by storm, and continues to delight.

Helsinki Travel Theodolites

Jim Beam’s, Disaronno, Coke, Ice and a slice of lime.

David Thomas Broughton versus 7 Hertz

“Can’t afford a pasty from Gregg’s bakery”. Still loving this album. His singing voice is an acquired taste, but I acquired it. I also admire anyone who doesn’t let their equipment blowing up stop them putting the song onto the CD.

Nick Cave And The Bad Seeds - Dig Lazarus Dig

Reminded me that I do actually really like Nick Cave.

Cut Copy - In Colours

Uncomplicated but enjoyable electro. A New Order I like.

Primal Scream - Swastika Eyes (live)

Entire audience hated it, but it was the only good bit in the whole gig. Still think they’re crap.

Trentemoller @ The Big Chill

I knew he’d be astonishing, because all his productions, and all his remixes, are astonishing. He was.

Leonard Cohen @ The Big Chill

The world is thanking his bent accountant for this tour.

There Will Be Blood

It’s not exactly a stretch for me to like a Paul Anderson film, but still…

Atonement

To my shame, I saw the film before the book, but I thought both were brilliant. Yes, even Keira Knightley.

No Country For Old Men

I was expecting to be bored crapless by this but was mesmerised instead.

The Shield Season 7

This final season knocked this back up to seriously rival The Wire for best thing ever. The entire season exulted in reminding you that Vic Mackey was a truly horrible person.

Sons Of Anarchy

A drama about gun-running Californian bikers sounds dumb, but this was brilliantly done. For some reason I kept being reminded of Cher though - that is never good.

Abode

Very, very, good restaurant 2 minutes from my flat == yay.

Try Thai

Very good value, with excellent battered scallops, in Chinatown (if you can call a single square a town (which you can’t, Manchester - sorry)).

The Deaf Institute

Well done for opening, Trof 3!

Sam Smith’s Organic Ale

I drank way too much of this.

Allgates Shining Light

And nowhere near enough of this.

My previous blog post described how to install OpenSolaris 2008.11 on a Solaris dom0 under Xen. This also works on with a Linux dom0. However, since upstream is missing some of our dom0 fixes, it's unfortunately more complicated. In particular, we can't use virt-install, as it doesn't know about Solaris ISOs, and later on, we can't use pygrub to boot from ZFS, since it doesn't know how to read such a filesystem. Bear with me, this gets a little awkward.

This example is using a 32-bit Fedora 8 installation. Your milage is likely to vary if you're using a different version, or another Linux distribution. First some of the configuration parameters you might want to change:

export name="domu-224"
export iso="/isos/osol-2008.11.iso"
export dompath="/export/guests/2008.11"
export rootdisk="$dompath/root.img"
export unixfile="/platform/i86xpv/kernel/unix"

If you're on 64-bit Linux, set unixfile="/platform/i86xpv/kernel/amd64/unix" instead. We need to create ourselves a 10Gb root disk:

mkdir -p $dompath
dd if=/dev/zero count=1 bs=$((1024 * 1024)) seek=10230 of=$rootdisk

Now let's use the configuration we need to install OpenSolaris:

cat >/tmp/domain-$name.xml <<EOF
<domain type='xen'>
 <name>$name</name>
 <bootloader>/usr/bin/pygrub</bootloader>
 <bootloader_args>--kernel=/platform/i86xpv/kernel/unix --ramdisk=/boot/x86.microroot</bootloader_args>
 <memory>1048576</memory>
 <on_reboot>destroy</on_reboot>
 <devices>
  <interface type='bridge'>
   <source bridge='eth0' />
   <--
       If you have a static DHCP setup, add the domain's MAC address here
       <mac address='00:16:3e:1b:e8:18' />
   -->
  </interface>
  <disk type='file' device='cdrom'>
   <driver name='file' />
   <source file='$iso' />
   <target dev='xvdc:cdrom' />
  </disk>
  <disk type='file' device='disk'>
   <driver name='file' />
   <source file='$rootdisk' />
   <target dev='xvda' />
  </disk>
 </devices>
</domain>
EOF

And start up the domain:

virsh create /tmp/domain-$name.xml
virsh console $name

Now you're dropped into the domain's console, and you can use the VNC trick I described to do the install. Answer the questions, wait for the domain to DHCP, then:

domid=`virsh domid $name`
ip=`/usr/bin/xenstore-read /local/domain/$domid/ipaddr/0`
port=`/usr/bin/xenstore-read /local/domain/$domid/guest/vnc/port`
/usr/bin/xenstore-read /local/domain/$domid/guest/vnc/passwd
vncviewer $ip:$port

At this point, you can proceed with the installation as normal. Before you reboot though, we need to do some tricks, due to the lack of ZFS support mentioned above. Whilst still in the live CD environment, bring up a terminal. We need to copy the new kernel and ramdisk to the Linux dom0. We can automate this via a handy script:

#/bin/bash

dom0=$1
dompath=$2
unixfile=/platform/i86xpv/kernel/$3/unix

root=`pfexec beadm list -H |  grep ';N*R;' | cut -d \; -f 1`
mkdir /tmp/root
pfexec beadm mount $root /tmp/root 2>/dev/null
mount=`pfexec beadm list -H $root | cut -d \; -f 4`
pfexec bootadm update-archive -R $mount
scp $mount/$unixfile root@$dom0:$dompath/kernel.$root
scp $mount/platform/i86pc/$3/boot_archive root@$dom0:$dompath/ramdisk.$root
pfexec beadm umount $root 2>/dev/null
echo "Kernel and ramdisk for $root copied to $dom0:$dompath"
echo "Kernel cmdline should be:"
echo "$unixfile -B zfs-bootfs=rpool/ROOT/$root,bootpath=/xpvd/xdf@51712:a"

For example, we might do:

/tmp/update_dom0 linux-dom0 /export/guests/2008.11
or on 64-bit:
/tmp/update_dom0 linux-dom0 /export/guests/2008.11 amd64

Now, you can finish the installation by clicking the reboot button. This will shut down the domain, ready to run. But first we need the configuration file for running the domain:

cat >/$dompath/$name.xml <<EOF
<domain type='xen'>
 <name>$name</name>
 <os>
  <kernel>$dompath/kernel.opensolaris</kernel>
  <initrd>$dompath/ramdisk.opensolaris</initrd>
  <cmdline>$unixfile -B zfs-bootfs=rpool/ROOT/opensolaris,bootpath=/xpvd/xdf@51712:a</cmdline>
 </os>
 <memory>1048576</memory>
 <devices>
  <interface type='bridge'>
   <source bridge='eth0'/>
  </interface>
  <disk type='file' device='disk'>
   <driver name='file' />
   <source file='$rootdisk' />
   <target dev='xvda' />
  </disk>
 </devices>
</domain>

virsh define $dompath/$name.xml
virsh start $name
virsh console $name

It should be booting, and you're (finally) done!

Updating the guest

Unfortunately we're not quite out of the woods yet. What we have works fine, but if we update the guest via pkg image-update, we'll need to make changes in dom0 to boot the new boot environment. The update_dom0 script above will do a fine job of copying out the new kernel and ramdisk for the BE that's active on reboot, but you also need to edit the config file. For example, if I wanted to boot into the new BE called opensolaris-1, I'd replace these lines:

<kernel>$dompath/kernel.opensolaris</kernel>
<initrd>$dompath/ramdisk.opensolaris</initrd>
<cmdline>$unixfile -B zfs-bootfs=rpool/ROOT/opensolaris,bootpath=/xpvd/xdf@51712:a</cmdline>

with these:

<kernel>$dompath/kernel.opensolaris-1</kernel>
<initrd>$dompath/ramdisk.opensolaris-1</initrd>
<cmdline>$unixfile -B zfs-bootfs=rpool/ROOT/opensolaris-1,bootpath=/xpvd/xdf@51712:a</cmdline>

then re-configure the domain (whist it's shut down) via virsh undefine $name ; virsh define $dompath/$name.xml.

Yes, we're aware this is rather over-complicated. We're trying to find the time to send our changes to virt-install upstream, as well as ZFS support. Eventually this will make it much easier to use a Linux dom0.

Tags: OpenSolaris Xen Linux

UPDATE: the canonical location for this information is now here - please check there, as it will be updated as necessary, unlike this blog entry.

As well obviously working with VirtualBox, OpenSolaris can also run as a guest domain under Xen. The installation CD ships with the paravirtual extensions so you can run it as a fully para-virtualized guest. This provides a significant advantage over fully-virtualized guests, or even guests with para-virtual drivers like Solaris 10 Update 6. Of course, if you choose to, you can still run OpenSolaris fully-virtualized (a.k.a. HVM mode), but there's little advantage to doing so.

One slight wrinkle is that Solaris guests don't yet implement the virtual framebuffer that the Xen infrastructure supports. Since OpenSolaris doesn't yet have a text-mode install, this means that to install such a PV guest, we need a way to bring up a graphical console.

With 2008.11, this is considerably easier. Presuming we're running a Solaris dom0 (either Nevada or OpenSolaris, of course), let's start an install of 2008.11:

# zfs create rpool/zvol
# zfs create -V 10G rpool/zvol/domu-220-root
# virt-install --nographics --paravirt --ram 1024 --name domu-220 -f /dev/zvol/dsk/rpool/zvol/domu-220-root -l /isos/osol-2008.11.iso

This will drop you into the console for the guest to ask you the two initial questions. Since they're not really important in this circumstance, you can just choose the defaults. This example presumes that you have a DHCP server set up to give out dynamic addresses. If you only hand out addresses statically based on MAC address, you can also specify the --mac option. As OpenSolaris more-or-less assumes DHCP, it's recommended to set one up.

Now we need a graphical console in order to interact with the OpenSolaris installer. If the guest domain successfully finished booting the live CD, a VNC server should be running. It has recorded the details of this server in XenStore. This is essentially a name/value config database used for communicating between guest domains and the control domain (dom0). We can start a VNC session as follows:

# domid=`virsh domid domu-220`
# ip=`/usr/lib/xen/bin/xenstore-read /local/domain/$domid/ipaddr/0`
# port=`/usr/lib/xen/bin/xenstore-read /local/domain/$domid/guest/vnc/port`
# /usr/lib/xen/bin/xenstore-read /local/domain/$domid/guest/vnc/passwd
DJP9tYDZ
# vncviewer $ip:$port

At the VNC password prompt, enter the given password, and this should bring up a VNC session, and you can merrily install away.

Implementation

The live CD runs a transient SMF service system/xvm/vnc-config. If it finds itself running on a live CD, it will generate a random VNC password, configure application/x11/x11-server to start Xvnc, and write the values above to XenStore. When application/graphical-login/gdm starts, it will read these service properties and start up the VNC server. The service system/xvm/ipagent tracks the IPv4 address given to the first running interface and writes it to XenStore.

By default, the VNC server is configured not to run post-installation due to security concerns. This can be changed though, as follows:

# svccfg -s x11-server
setprop options/xvm_vnc = "true"

Please remember that VNC is not secure. Since you need elevated privileges to read the VNC password from XenStore, that's sufficiently protected, as long as you always run the VNC viewer locally on the dom0, or via SSH tunnelling or some other secure method.

Note that this works even with a Linux dom0, although you can't yet use virt-install, as the upstream version doesn't yet "know about" OpenSolaris (more on this later).

Tags: OpenSolaris Xen xVM

They’re using the phrase “a sense of” even more than they used to. They refer to “the sense of jubilation” in the Democrat camp, or “a sense of taking stock” amongst Republicans. Practically every other sentence uses this moronic phrase. There’s a real sense of lazy journalism at the BBC.

On the “moronic phrase” note: “Yes We Can” is just about the stupidest campaign slogan ever.

Also, the election coverage on the BBC was surprisingly biased. Way too many left-leaning talking heads, very
little positive coverage of the McCain campaign. It wasn’t Fox News, of course, but it was bad: I expected better from the BBC. It was almost worth it to see John Bolton nearly explode with anger though.

Dear America: well done.

Argggh!

“People don’t like being preached at.”

!!!

“But in a twist which will have Christians in gales of laughter, the advertising campaign…is to be stuck on bendy-buses.”

May I suggest first that said Christians perhaps need to get out more, and second that “gales of laughter” doesn’t seem like a very Christian response?

“fellow humanists, not known for their generosity, wouldn’t stump up the cash.”

Aren’t you guys looking a little silly now?

Press release.