Converting a QEMU qcow2 cloud server image to an native disk img and putting on physical disk

Posted on 11th February 2018 by azio

Got this question at work a lot. Thought I’d finally get around to putting it down since it’s came up for me. I’ve got a virtual machine using virtio passthrough for my pcie, and I found actually that disk access via the qcow2 is pretty naff.

sudo apt-get install qemu-kvm

qemu-img convert windows10cloudimage.qcow2 -O raw diskimage.img

dd if=/path/to/windos10cloudimage.qcow2 of=/dev/sdc2

Please note in my case the physical partition I’d made was sdc2. I’d actually resized another 5TB disk I have in my system using gparted. Just so I can attach a physical partition with libvirtd. Evidently though libvirtd-manager doesn’t allow this business so I have to edit the xlm file in /etc/qemu/windows10.xml .

root@adam:/etc/libvirt/qemu# virsh  define /etc/libvirt/qemu/win10-uefi.xml 
Domain win10-uefi defined from /etc/libvirt/qemu/win10-uefi.xml
root@adam:/etc/libvirt/qemu# virt-manager

yeah baby!

You could alternatively do it all in one like below, though you may desire a copy of the img file as well as putting it to the disk.

qemu-img convert windows10.qcow2 -O raw /dev/sdc

Determining the size of a VHD in Xenserver

Posted on 4th January 2017 by azio

It is possible to use vhd-util scan -a -p to determine the size of a VHD in Xen server. It can be done as follows

vhd-util scan -a -p /var/run/sr-mount/96ca6e70-976e-32bd-837d-f1e7bade1262/e165978f-2ff7-4958-8642-7087be83d680.vhd

Aaron Mehar’s CBS to VHD solution for Rackspace Cloud

Posted on 11th November 2016 by azio

Hey. So another one of my colleagues put together this really awesome article. Although I was aware this could be done, he’s done a really good job or putting together the procedures, of turning your CBS BFV (boot from (network) volume) disk into a VHD file.

Rackspace CBS disks works over iscsi and are presented via the network. The difference between instance store on the hypervisor, (utilized by cloud-server images), and the disk store on the CBS is that the CBS disk is not a VHD, but an disk presented over network via iscsi.

So, to take a VHD, or an equivalent cloud-server image snapshot, you need to image the disk manually, as well as convert it to VHD.

Taking an image of a volume is not possible, and would not be downloadable. However there are some workarounds that can be done.

*** Please NOTE ****
This is not supported, and we can not assist beyond these instructions. I could provide some clarity if required, however, my collegaues may not be able to help should I become unavailable.

If you just want the data, then you could just download the data to your local machine, however, if you a VHD to create a local VM, then the below instructions will achieve this.

Steps

Please take special care, making a mistake working with partitioner can wipe all your data

1. Shutdown the server
2. Clone the disk, by Starting a volume clone and start the server back up.
3. Attach the newly created clone to the server
4. create another new CBS volume of a slightly larger size (+5GB is OK)

Now that is done, we can image the disk. You will need to ensure you have the corrects disks. The second disk with data should be xvbd and the new CBS should be xvdc

Create partition and filesystem for xvdbc. Please see this guide: https://support.rackspace.com/how-to/prepare-your-cloud-block-storage-volume/

the image xvdb to xvdc

   dd if=/dev/xvdb of=/mnt/cbsvolume1/myimage.dd

The download the image to your workstation, and install VirtualBox, and run the below command

   VBoxManage convertfromraw myfile.dd myfile.vhd --format VHD

Please take special care, making a mistake working with partitioner can wipe all your data

Killing a stuck sr-scan when vdi gets suck in vdi_deactivate

Posted on 10th November 2016 by azio

Simple fix for this one, kill the task directly by it’s PID. Using ps , grep and sed, to select correct process.

When executing this in live production, take especial care.

# xe task-list 
uuid ( RO)                : 7cb8b30b-563c-db9e-7682-e25aac2b2f07
          name-label ( RO): SR.scan
    name-description ( RO): 
              status ( RO): pending
            progress ( RO): 0.000
[root@21-16-246-494694 ~]# date
Thu Nov 10 11:41:45 UTC 2016
[root@xenhost ~]# kill -TERM $(ps fauxww | grep "[t]ap-ctl close" | sed -e "s/.*-p //" | awk '{print $1}')

All About NOVA and Xen Tools in Rackspace Cloud – why can’t I connect to my Windows server?

Posted on 9th November 2016 by azio

Why can’t I connect to my Rackspace Windows cloud-server, you ask? 2 important questions.

1. Is it a new build?
2. Is it using a custom image (a non rackspace base image).

(because the rackspace base images all have correct nova-agent and xen tools, so get networking information OK. But customer images don’t!). In the case you have run the below tests to see if nova-agent is running (or installed), you will need to install them.

Checking for the nova-agent and xe-guest-utilities

ps auxfwww | grep nova-agent
yum -qa xe-guest-utilities nova-agent
dpkg -l xe-guest-utilities nova-agent

Explanation and solution

Thanks for reaching out to us with your inquiry today. I’m glad to convey to you that I understand what the problem is with your cloud-server not being contactable.

Main reasons for breakage

The main reason why this is not working is most likely caused by some important pieces of software being missing. There is a piece of software called nova-agent, which is responsible for setting your cloud-servers IPV4 address, network subnet/mask, and ip routes, when it is first built. This is important, since the server image you built the server from, has different network details.

The rackspace build process giving networking detail to the VM is completely dependent on xe-guest-utilities and nova-agent

What has happened in this case, because the nova-agent wasn’t running on the cloud-server, the hypervisor software Rackspace use to automate cloud-server builds wasn’t able to contact the nova-agent running on your cloud-server, and therefore nova-agent wasn’t able to update the networking information. And hence, your not able to connect to it on it’s IPv4 address you are given at build time.

The steps to resolution: installing nova-agent and xen guest utilities
As such, nova-agent needs to be installed on the cloud-server you take the image from, it can be installed as follows:

https://community.rackspace.com/products/f/25/t/5694

Also nova-agent uses another piece of important software called xe-guest-utilities, or (Xen Tools) for your windows servers, this is an important ‘PV’ paravirtualization tools, responsible for seamless management of cloud-servers. Sorry that in this case it’s not working out seamlessly, but this can happen with images taken of servers which have had nova-agent disabled, uninstalled, or similar.

Upgrading the tools that nova-agent depends upon, can be installed by following the instructions at the following location:

https://support.rackspace.com/how-to/upgrade-citrix-xen-server-tools-for-windows-cloud-servers/

# Options of how to do this / Summary of Solution Steps

Naturally, you might be wondering how to achieve these changes, if you cannot RDP to the server. This is quite understandable, there are two ways to get this working;

Option 1) Manually install nova-agent on the current server you cannot access, then manually install the Xen Tools in the same way. This shall fix the OS on the server itself, and not the original image you built the server from. So it is important to create a new cloud-server image after performing these steps and us verifying tools + nova-agent installed correctly.

2) Manually install nova-agent on the source server you initially taken the image from, and install Xen Tools, then re-image the server, and then re-deploy. This should seamlesssly work each time on build with that image, provided the tools are installed. You will not need to recreate the image, since your fixing the problem on the cloud-server source that the original image was taken from.

I appreciate that these things are not 100% simple to get your head around and can be confusing for customers, I hope my explanation and summary makes this a little more painless to fix. Of course if you have additional questions, comments or concerns or don’t understand something I’ve said, please don’t hesitate to reach out to us, we are here to help!

Reinstalling the Linux Nova Agent for Rackspace Cloud

Posted on 17th October 2016 by azio

So, you have imaged a cloud-server and then built it but learnt ‘you got network issues bra’?’

Re-Install nova-agent using this automation script, on the original server:

curl -s novaagent.myconfig.info | sudo bash

Once this is done, you should be safe to re-image your server. Provided it’s just nova-agent and you don’t have other breakage. It will make sure that nova-agent is setting your networking detail at boot time.

Hope this helps!

Cheers &
Best wishes,
Adam

Configure Nested KVM for Intel & AMD based Machines

Posted on 29th April 2016 by azio

So, we are configuring some openstack and kvm stuff at work for some projects. We’re ‘cloudy’ guys. What can I say? 😀 One Issue I had when installing xenserver, underneath KVM.

(why would we do this?) In our testing environment we’re using a single OnMetal v2 server, and, instead of running xenserver directly on the server, and requiring additional servers, we are using a single 128GB RAM hypervisor for the test environment. One issue though is that Windows is only supported with xenserver when directly run on the ‘host’. Because Xen is running virtualized under KVM we have a problem.

Enter, tested virtualization support. Hardware virtualization assist support will now work for xenserver thru KVM, which means I can boot windows servers. YAY! uh.. 😉 kinda.

Check if Nested hardware virtualization assist is enabled

$cat /sys/module/kvm_intel/parameters/nested
N

It wasn’t 🙁 Lets enable it

Enable nested hardware virtualization assist

sudo rmmod kvm-intel
sudo sh -c "echo 'options kvm-intel nested=y' >> /etc/modprobe.d/dist.conf"
sudo modprobe kvm-intel

Ensure nested hardware virtualization is enabled

cat /sys/module/kvm_intel/parameters/nested
Y

modinfo kvm_intel | grep nested
parm:           nested:bool

It worked!

This can also be done for AMD systems simply substituting kvm_amd.

http://docs.openstack.org/developer/devstack/guides/devstack-with-nested-kvm.html

Installing KVM, libvirtd virt-manager and Xenserver for Rackspace onmetal using ZFS & X11 Forwarding

Posted on 27th April 2016 by azio

So, you want to run your own hypervisor using xenserver, but you want to have some of the flexibility of KVM too. This instructional guide explains how to install and configure KVM with virt-manager and with X11 forwarding. We will go step by step. In this case I am using a mac.

Step 1 – Create Rackspace onmetalv2 server

In this case I’ll be using a 40 cpu 128GB machine as the host utilizing the new onmetalv2 server range offered by Rackspace public cloud.

Please note that this is a bare metal server, not a cloud server, however it is offered by the same cloud platform at mycloud.rackspace.co.uk

Step 2 – Install and configure KVM

sudo yum update -y
sudo yum -y install kvm virt-manager libvirt virt-install qemu-kvm xauth dejavu-lgc-sans-fonts

Step 3 – Start and configure libvirtd

chkconfig libvirtd
service libvirtd status
service libvirtd restart
service libvirtd status

Step 4 – MAC SYSTEMS – Install X Quartz

For mac users simply install X Quartz, which can be found at http://www.xquartz.org/

Step 4a – Windows Systems – Install Xming

Windows users can get in on the action too, using xming which can be found at https://sourceforge.net/projects/xming/

Step 5 – MAC SYSTEMS ONLY – Configure X11 Forwarding

Xming will work out of the box for windows, but for Mac users you need to make sure you have enabled X11 forwarding.

touch ~/.ssh/config
echo "ForwardX11 yes" >> ~/.ssh/config

This simply allows X11 forwarding for Mac users which needs to be done at the client side. Then you can virtualize any application you like on the client, but running the application such as firefox , or even a virtual machine on the remote server. SSHv2 is beautiful. That’s it you’ve completed the most important steps.

Running virt-manager for the first time

 
[root@on-metal-test-2 ~]# virt-manager

After running the above command you will see something like the image below. You’ll see an X window open on your local client machine, which is associated with an application running on the remote server your connected to via SSH. This is pretty damn cool.

Lets take this further and install firefox to demonstrate how awesome this is!

yum install firefox -y

Now we’re using firefox thru ssh, much better and more convenient to use X11 forwarding for this, than using a proxy for instance on the client configured with tunnel or vpn.

Nice!

Lets take it a bit further and start installing xen server with KVM. I am very tempted to use ZFS for this since onmetal v2 has 2 1600GB disks…

Create partitions for KVM store

fdisk -l 
fdisk /dev/sdc

# type m , then type n, then type p, enter, enter, enter, enter, then type w

fdisk /dev/sdd

# type m , then type n, then type p, enter, enter, enter, enter, then type w

Create filesystem for KVM store

[root@on-metal-test-2 ~]# mkfs.ext3 /dev/sdc1 && mkfs.ext3 /dev/sdd1
mke2fs 1.42.9 (28-Dec-2013)
Discarding device blocks: done
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
97656832 inodes, 390624640 blocks
19531232 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
11921 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
	4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
	102400000, 214990848

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

mke2fs 1.42.9 (28-Dec-2013)
Discarding device blocks: done
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
97656832 inodes, 390624640 blocks
19531232 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
11921 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
	4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
	102400000, 214990848

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

Now we have created the filesystem. What about creating the ZFS partition. To do this we need to go thru a fairly laborious process (at least if you don’t know what your doing). As I discovered my yum installation wasn’t automatically providing the correct devel source for the kernel to use the ZFS DKMS module. As ZFS is really a native BSD package.

One of the problems I had was this

Loading new spl-0.6.5.6 DKMS files...
Building for 3.10.0-327.10.1.el7.x86_64
Module build for kernel 3.10.0-327.10.1.el7.x86_64 was skipped since the
kernel source for this kernel does not seem to be installed.
  Installing : zfs-dkms-0.6.5.6-1.el7.centos.noarch                                                                                                                       4/6
Loading new zfs-0.6.5.6 DKMS files...
Building for 3.10.0-327.10.1.el7.x86_64
Module build for kernel 3.10.0-327.10.1.el7.x86_64 was skipped since the
kernel source for this kernel does not seem to be installed.

This can be checked out in more detail by running an;

yum search --show-duplicates kernel-devel
# and
rpm -qa | grep kernel

This gave me the right version of the devel kernel I needed to install ZFS to my current kernel with a module, as opposed to completely recompiling the whole thing. Nice!

Install ZFS and kernel devel

sudo yum localinstall --nogpgcheck http://archive.zfsonlinux.org/epel/zfs-release.el7.noarch.rpm
sudo yum install epel-release

sudo yum install zfs kernel-devel-3.10.0-327.10.1.el7.x86_64

Enable ZFS

[root@on-metal-test-2 adam]# /sbin/modprobe zfs

Create the 2 disk mirror using ZFS

[root@on-metal-test-2 adam]# zpool create -f kvmstore mirror sdc1 sdd1

Check KVM store disk

[root@on-metal-test-2 adam]# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/md126p1    220G  2.3G  209G   2% /
devtmpfs         63G     0   63G   0% /dev
tmpfs            63G     0   63G   0% /dev/shm
tmpfs            63G   26M   63G   1% /run
tmpfs            63G     0   63G   0% /sys/fs/cgroup
tmpfs            13G  4.0K   13G   1% /run/user/0
kvmstore        1.5T     0  1.5T   0% /kvmstore

Run Virt manager to create Xenserver VM

Now we’ve created our partition and filesystem and configured ZFS we can run the virtual machines off the new kvm partition store. simples

Click top left icon on corner to create new VM

Download the Xenserver ISO to /root of hypervisor

root@on-metal-test-2 ~]# wget http://downloadns.citrix.com.edgesuite.net/10175/XenServer-6.5.0-xenserver.org-install-cd.iso
--2016-04-27 10:29:22--  http://downloadns.citrix.com.edgesuite.net/10175/XenServer-6.5.0-xenserver.org-install-cd.iso
Resolving downloadns.citrix.com.edgesuite.net (downloadns.citrix.com.edgesuite.net)... 104.86.110.32, 104.86.110.49
Connecting to downloadns.citrix.com.edgesuite.net (downloadns.citrix.com.edgesuite.net)|104.86.110.32|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 603744256 (576M) [application/octet-stream]
Saving to: ‘XenServer-6.5.0-xenserver.org-install-cd.iso’

100%[====================================================================================================================================>] 603,744,256 17.6MB/s   in 38s

Select Local Media (we’re going to use a Xenserver ISO)

Click browse, then press the bottom left + icon to add some pools. We’re going to add /root which has our iso in it, and we’re also going to add kvmstore aswell.

Congratulations you have now added the stores. Now all we need to do is finish configuring the VM.

We want to select the root partition now we have set up the pool, and choose the xenserver iso we just recently downloaded.

We are almost there now! Lets set the number of cpu and ram! Also lets make sure we use the kvmstore we just setup instead of the ‘main disk’ of the server.

Select our KVM store ‘pool’ on the left hand side, and then press + to add the kvmstore.qcow2 volume, see the images for illustration.

click choose volume at the bottom left to confirm! And finally name the server

awwww crap , we got this error because the libvirtd kvm configuration isnt running as root

This can be quickly resolved by editing the /etc/libvirt/qemu.conf and making sure user = “root” and group = “root” are present.

Job done!

Updating Citrix Xenserver 6.5 to SP1

Posted on 27th April 2016 by azio

I recently installed Xenserver on Rackspace OnMetal v2 and I wanted to make sure I had the latest supporting service packages.

It’s simple enough, just wget the update from citrix on their page and then patch-upload and patch-apply using the uuid of the patch and the uuid of the host. Simples.

Download Support Pack from Citrix

# wget -c http://downloadns.citrix.com.edgesuite.net/10340/XS65ESP1.zip

Unzip Support Pack

[root@xenserver-skucvxib ~]# unzip XS65ESP1.zip
Archive:  XS65ESP1.zip
  inflating: XS65ESP1.xsupdate
  inflating: XS65ESP1-src-pkgs.tar.bz2

Delete unused files

[root@xenserver-skucvxib ~]# rm XS65ESP1.zip
rm: remove regular file `XS65ESP1.zip'? y

Upload patch to Xen (and obtain the patch uuid)

[root@xenserver-skucvxib ~]# xe patch-upload file-name=XS65ESP1.xsupdate
7f2e4a3a-4098-4a71-84ff-b0ba919723c7

Remove all unnecessary files

[root@xenserver-skucvxib ~]# rm XS65ESP1*
rm: remove regular file `XS65ESP1-src-pkgs.tar.bz2'? y
rm: remove regular file `XS65ESP1.xsupdate'? y

Obtain uuid of host

root@xenserver-skucvxib ~]# xe host-list
uuid ( RO)                : 533a880b-6dd9-4d65-b930-55a3e4b27668
          name-label ( RW): xenserver-skucvxib
    name-description ( RW): Default install of XenServer

Apply patch

[root@xenserver-skucvxib ~]# xe patch-apply uuid=7f2e4a3a-4098-4a71-84ff-b0ba919723c7 host-uuid=533a880b-6dd9-4d65-b930-55a3e4b27668
5ee785d4-fc7d-dfe4-b250-26346b88898b is the local tools SR: scanning
Done
Preparing...                ##################################################
xen-device-model            ##################################################
Preparing...                ##################################################
xen-hypervisor              ##################################################
Preparing...                ##################################################
xen-tools                   ##################################################
Preparing...                ##################################################
xen-libs                    ##################################################
Preparing...                ##################################################
xen-dom0-tools              ##################################################
Preparing...                ##################################################
xen-dom0-libs               ##################################################
Preparing...                ##################################################
xen-ocaml-libs              ##################################################
Preparing...                ##################################################
guest-templates             ##################################################
Preparing...                ##################################################
lvm2                        ##################################################
Preparing...                ##################################################
upgrade-plugin              ##################################################
Preparing...                ##################################################
xapi-core                   ##################################################
Preparing...                ##################################################
xapi-xenopsd                ##################################################
Preparing...                ##################################################
xapi-rrdd                   ##################################################
Preparing...                ##################################################
Stopping XCP RRDD plugin xcp-rrdd-iostat: [  OK  ]
Stopping XCP RRDD plugin xcp-rrdd-squeezed: [  OK  ]
Stopping XCP RRDD plugin xcp-rrdd-xenpm: [  OK  ]
rrdd-plugins                ##################################################
Starting XCP RRDD plugin xcp-rrdd-iostat: [  OK  ]
Starting XCP RRDD plugin xcp-rrdd-squeezed: [  OK  ]
Starting XCP RRDD plugin xcp-rrdd-xenpm: [  OK  ]
Preparing...                ##################################################
blktap                      ##################################################
Preparing...                ##################################################
sm                          ##################################################
Preparing...                ##################################################
tzdata                      ##################################################
Preparing...                ##################################################
kernel                      ##################################################
unable to stat /sys/class/block//var/swap/swap.001: No such file or directory
Preparing...                ##################################################
vgpu                        ##################################################
Preparing...                ##################################################
linux-guest-loader-data     ##################################################
Preparing...                ##################################################
hwdata                      ##################################################
Preparing...                ##################################################
xenserver-transfer-vm       ##################################################
Preparing...                ##################################################
openvswitch                 ##################################################
Preparing...                ##################################################
v6d                         ##################################################
Preparing...                ##################################################
glibc                       ##################################################
Preparing...                ##################################################
glibc-common                ##################################################
Waiting for xapi to signal init complete
Removing any existing built-in templates
Regenerating built-in templates

We have now successfully applied Xen server SP1 to Xen Server 6.5.

Virt-manager won’t release mouse on Mac OS X

Posted on 26th April 2016 by azio

This was quite annoying, but thanks to Major hayden (thanks pal) I was able to resolve this issue by making a file on my Mac to make sure the bindings are there.

Make sure you close X Quartz first.

touch ~/.Xmodmap 
echo "clear Mod1" >> ~/.Xmodmap 
echo "keycode 66 = Alt_L"  >> ~/.Xmodmap 
echo "keycode 69 = Alt_R" >> ~/.Xmodmap 
echo "add Mod1 = Alt_L" >> ~/.Xmodmap 
echo "add Mod1 = Alt_R" >> ~/.Xmodmap

Job done! Now it works nicely 😀 This might be relevant to those using xen, kvm and libvirtd in particular.

haxed.me.uk

a place of infrastructure delights

Category Archives: xen