Mind the gap!

Greetings,

As if it hasn’t been too long, because I know that it has. Dear Diary, Health had a scare last year so I have not been frequently posting! Thanks to all those kind and remarkable friends and family that did send heartfelt messages of hope during times of real despair.

I am still quite ill, but will be making every effort to return to my regular posting schedule of techtips and other exciting infrastructure delights – now that I am well enough. In fact. I’m really excited about it and to start off I’m going to put together a really fancy guide for PCIE passthrough in Ubuntu 16.04. Which, afaik, pretty much doesn’t exist in any complete way on the internet. anywhere.

We continue on with the worthwhile cause on an inspiring mission. Hurrah!

Cheers &
Best wishes,
Adam

Controlling MySQL Queries, and killing if necessary.

Today we had a customer call in who had a problem with their magento site. What had happened was they had reached the maximum number of connections for their MySQL.

 250481 | someuser | localhost:7777 | somewebsite | Query   | 2464 | Writing to net               | SELECT /*!40001 SQL_NO_CACHE */ * FROM `mg_sales_flat_quote_address`                                 

| 250486 | someuser | localhost       | somewebsite | Query   | 2459 | Waiting for table level lock | INSERT INTO `mg_sales_flat_quote_address` (`quote_id`, `created_at`, `updated_at`, `customer_id`, `a |         0 |             0 |         0 |

.. and a whole load more waiting for table level lock were underneath

Solution is simple;

kill 250481

250481 is the processid of the select query causing these issues. I suspect it will go away, eventually but in the meantime, want to make sure the max connections don’t get reached, otherwise website will be down again!

Resetting trusted CA certificates in Redhat

I found this today, very handy link that explains how to regenerate the CA’s.

https://access.redhat.com/solutions/1549003

This didn’t actually resolve my error because my customer had a CAFile that had cert, but the CAPath was empty.. I’ve seen weird NSS -8172 errors before like this.

I’ll update this post once I figure out what it actually is causing it.

test

### Section 1: Global Environment
#
# The directives in this section affect the overall operation of Apache,
# such as the number of concurrent requests it can handle or where it
# can find its configuration files.
#

#
# Don't give away too much information about all the subcomponents
# we are running. Comment out this line if you don't mind remote sites
# finding out what major optional modules you are running
ServerTokens OS

Resizing PVHVM flavours Down via API

So, in the case you have a cloud-server, such as a Rackspace standard flavour instance, you might want to resize down.

The only problem is that when doing this it can result in the MBR being lost. Check out this great article from Jake Coe, explaining how it can be achieved.

https://community.rackspace.com/products/f/25/t/4716?_ga=1.244296325.1822315901.1458550977

Because of the sizes of larger vhds it will not let you downsizes these devices within the control panel. There may be a way to downsize them IF you are not using more then the required space on the server however there are some risks involved and once the downsize is completed there will be a manual fixed needed for the servers to get them to boot.

I would highly suggest that the following be attempted on a non production server as dataloss could happen.

Create an image of existing server you want to downsize.

Once the image is completed we can modify the min disk and ram to that of the smaller size needed.

On the new server once built to the larger size you will 1st need to write all zeros to empty space.

cat /dev/zero > /zero; rm -f /zero

Once this is done you will then downsize the server to the desired size.

Once the downsize is completed if this is a pvhvm instance the mbr will be gone and grub will need to be reinstalled

How I fixed a really broken CentOS Machine after an dist upgrade

So… you do basic dist upgrade.. you don’t really make an effort to take a cloud-server image (because your me, and are an idiot! and dont even follow the advice you give your own customers, yourself!)…

Now I’ve got that bit over.. I basically found that udev wasn’t installed. This might not seem like completely the end of the world, however, the fact that /dev/random /dev/urandom were missing, and /dev/null was a regular file.. we had big issues man.. I didn’t think I’d be able to get SSH to behave again, but actually it seems this is isolated to udev in most latest centos revisions.

I’m not going to give my kernel version, but you get the idea ;-D

[root@RESCUE-pirax-test ~]# ls -al
total 40
dr-xr-x---.  5 root root 4096 Dec  7 09:34 .
dr-xr-xr-x. 18 root root 4096 Dec  7 09:34 ..
-rw-r--r--.  1 root root   18 Dec 29  2013 .bash_logout
-rw-r--r--.  1 root root  176 Dec 29  2013 .bash_profile
-rw-r--r--.  1 root root  176 Dec 29  2013 .bashrc
drwxr-xr-x.  3 root root 4096 Dec  7 09:34 .cache
drwxr-xr-x.  3 root root 4096 Dec  7 09:34 .config
-rw-r--r--.  1 root root  100 Dec 29  2013 .cshrc
drwx------.  2 root root 4096 Dec  7 09:34 .ssh
-rw-r--r--.  1 root root  129 Dec 29  2013 .tcshrc
[root@RESCUE-pirax-test ~]# cd /mnt
[root@RESCUE-pirax-test mnt]# ls -al
total 112
dr-xr-xr-x. 23 root root  4096 Dec  7 09:32 .
dr-xr-xr-x. 18 root root  4096 Dec  7 09:34 ..
lrwxrwxrwx.  1 root root     7 Feb 23  2016 bin -> usr/bin
dr-xr-xr-x.  4 root root  4096 Dec  6 11:53 boot
drwxr-xr-x.  2 root root  4096 Apr 21  2016 customer
drwxr-xr-x.  2 root root  4096 Dec  7 09:47 dev
drwxr-xr-x.  8 root root  4096 Jun 10 10:02 documents
drwxr-xr-x.  3 root root  4096 Mar  1  2016 dump
drwxr-xr-x. 91 root root 12288 Dec  7 09:45 etc
drwxr-xr-x.  5 root root  4096 Nov 11 11:58 home
drwxr-xr-x.  3 root root  4096 Nov  9  2015 include
lrwxrwxrwx.  1 root root     7 Feb 23  2016 lib -> usr/lib
lrwxrwxrwx.  1 root root     9 Feb 23  2016 lib64 -> usr/lib64
drwx------.  2 root root 16384 Sep  3  2015 lost+found
drwxr-xr-x.  2 root root  4096 Aug 12  2015 media
drwxr-xr-x.  3 root root  4096 Jun 23 14:23 mnt
drwxr-xr-x.  4 root root  4096 Aug 12  2015 opt
drwxr-xr-x.  2 root root  4096 Sep  3  2015 proc
drwxr-xr-x. 26 root root  4096 Dec  7 09:36 root
drwxr-xr-x.  3 root root  4096 Dec  7 09:43 run
lrwxrwxrwx.  1 root root     8 Feb 23  2016 sbin -> usr/sbin
drwxr-xr-x.  2 root root  4096 Aug 12  2015 srv
drwxr-xr-x.  2 root root  4096 Sep  3  2015 sys
drwxrwxrwt.  7 root root  4096 Dec  7 09:46 tmp
drwxr-xr-x. 13 root root  4096 Feb 23  2016 usr
drwxr-xr-x. 22 root root  4096 Dec  7 09:31 var
drwxr-xr-x.  9 root root  4096 Oct 12  2015 wpscan
[root@RESCUE-pirax-test mnt]# cd ..
[root@RESCUE-pirax-test /]# chroot /mnt
[root@RESCUE-pirax-test /]# ls -al
total 112
dr-xr-xr-x. 23 root root  4096 Dec  7 09:32 .
dr-xr-xr-x. 23 root root  4096 Dec  7 09:32 ..
lrwxrwxrwx.  1 root root     7 Feb 23  2016 bin -> usr/bin
dr-xr-xr-x.  4 root root  4096 Dec  6 11:53 boot
drwxr-xr-x.  2 root root  4096 Apr 21  2016 customer
drwxr-xr-x.  2 root root  4096 Dec  7 09:47 dev
drwxr-xr-x.  8 root root  4096 Jun 10 10:02 documents
drwxr-xr-x.  3 root root  4096 Mar  1  2016 dump
drwxr-xr-x. 91 root root 12288 Dec  7 09:45 etc
drwxr-xr-x.  5 root root  4096 Nov 11 11:58 home
drwxr-xr-x.  3 root root  4096 Nov  9  2015 include
lrwxrwxrwx.  1 root root     7 Feb 23  2016 lib -> usr/lib
lrwxrwxrwx.  1 root root     9 Feb 23  2016 lib64 -> usr/lib64
drwx------.  2 root root 16384 Sep  3  2015 lost+found
drwxr-xr-x.  2 root root  4096 Aug 12  2015 media
drwxr-xr-x.  3 root root  4096 Jun 23 14:23 mnt
drwxr-xr-x.  4 root root  4096 Aug 12  2015 opt
drwxr-xr-x.  2 root root  4096 Sep  3  2015 proc
drwxr-xr-x. 26 root root  4096 Dec  7 09:36 root
drwxr-xr-x.  3 root root  4096 Dec  7 09:43 run
lrwxrwxrwx.  1 root root     8 Feb 23  2016 sbin -> usr/sbin
drwxr-xr-x.  2 root root  4096 Aug 12  2015 srv
drwxr-xr-x.  2 root root  4096 Sep  3  2015 sys
drwxrwxrwt.  7 root root  4096 Dec  7 09:46 tmp
drwxr-xr-x. 13 root root  4096 Feb 23  2016 usr
drwxr-xr-x. 22 root root  4096 Dec  7 09:31 var
drwxr-xr-x.  9 root root  4096 Oct 12  2015 wpscan
[root@RESCUE-pirax-test /]# su adam
[adam@RESCUE-pirax-test /]$ ssh root@localhost
cannot read from /dev/urandom, No such file or directory
[adam@RESCUE-pirax-test /]$ yum update mkinitrd
Loaded plugins: fastestmirror, langpacks
You need to be root to perform this command.
[adam@RESCUE-pirax-test /]$ exit
exit
[root@RESCUE-pirax-test /]# yum update mkinitrd
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: mirrors.vooservers.com
 * epel: epel.check-update.co.uk
 * extras: mirrors.vooservers.com
 * updates: mirrors.vooservers.com
No Match for argument: mkinitrd
No package mkinitrd available.
No packages marked for update
[root@RESCUE-pirax-test /]# yum provides initrd
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: mirrors.vooservers.com
 * epel: epel.check-update.co.uk
 * extras: mirrors.vooservers.com
 * updates: mirrors.vooservers.com
No matches found
[root@RESCUE-pirax-test /]# cd /dev
[root@RESCUE-pirax-test dev]# /sbin/MAKEDEV urandom;
bash: /sbin/MAKEDEV: No such file or directory
[root@RESCUE-pirax-test dev]# yum install udev
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: mirrors.vooservers.com
 * epel: epel.check-update.co.uk
 * extras: mirrors.vooservers.com
 * updates: mirrors.vooservers.com
Resolving Dependencies
--> Running transaction check
---> Package systemd.x86_64 0:219-19.el7_2.4 will be updated
--> Processing Dependency: systemd = 219-19.el7_2.4 for package: systemd-python-219-19.el7_2.4.x86_64
--> Processing Dependency: systemd = 219-19.el7_2.4 for package: systemd-sysv-219-19.el7_2.4.x86_64
---> Package systemd.x86_64 0:219-19.el7_2.13 will be an update
--> Processing Dependency: systemd-libs = 219-19.el7_2.13 for package: systemd-219-19.el7_2.13.x86_64
--> Running transaction check
---> Package systemd-libs.x86_64 0:219-19.el7_2.4 will be updated
--> Processing Dependency: systemd-libs = 219-19.el7_2.4 for package: libgudev1-219-19.el7_2.4.x86_64
---> Package systemd-libs.x86_64 0:219-19.el7_2.13 will be an update
---> Package systemd-python.x86_64 0:219-19.el7_2.4 will be updated
---> Package systemd-python.x86_64 0:219-19.el7_2.13 will be an update
---> Package systemd-sysv.x86_64 0:219-19.el7_2.4 will be updated
---> Package systemd-sysv.x86_64 0:219-19.el7_2.13 will be an update
--> Running transaction check
---> Package libgudev1.x86_64 0:219-19.el7_2.4 will be updated
---> Package libgudev1.x86_64 0:219-19.el7_2.13 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

=========================================================================================================================================================================================================================================================================
 Package                                                             Arch                                                        Version                                                              Repository                                                    Size
=========================================================================================================================================================================================================================================================================
Updating:
 systemd                                                             x86_64                                                      219-19.el7_2.13                                                      updates                                                      5.1 M
Updating for dependencies:
 libgudev1                                                           x86_64                                                      219-19.el7_2.13                                                      updates                                                       67 k
 systemd-libs                                                        x86_64                                                      219-19.el7_2.13                                                      updates                                                      358 k
 systemd-python                                                      x86_64                                                      219-19.el7_2.13                                                      updates                                                      100 k
 systemd-sysv                                                        x86_64                                                      219-19.el7_2.13                                                      updates                                                       54 k

Transaction Summary
=========================================================================================================================================================================================================================================================================
Upgrade  1 Package (+4 Dependent packages)

Total size: 5.7 M
Is this ok [y/d/N]: y
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : systemd-libs-219-19.el7_2.13.x86_64                                                                                                                                                                                                                  1/10
  Updating   : systemd-219-19.el7_2.13.x86_64                                                                                                                                                                                                                       2/10
  Updating   : systemd-sysv-219-19.el7_2.13.x86_64                                                                                                                                                                                                                  3/10
  Updating   : systemd-python-219-19.el7_2.13.x86_64                                                                                                                                                                                                                4/10
  Updating   : libgudev1-219-19.el7_2.13.x86_64                                                                                                                                                                                                                     5/10
  Cleanup    : systemd-sysv-219-19.el7_2.4.x86_64                                                                                                                                                                                                                   6/10
  Cleanup    : systemd-python-219-19.el7_2.4.x86_64                                                                                                                                                                                                                 7/10
  Cleanup    : systemd-219-19.el7_2.4.x86_64                                                                                                                                                                                                                        8/10
  Cleanup    : libgudev1-219-19.el7_2.4.x86_64                                                                                                                                                                                                                      9/10
  Cleanup    : systemd-libs-219-19.el7_2.4.x86_64                                                                                                                                                                                                                  10/10
  Verifying  : systemd-libs-219-19.el7_2.13.x86_64                                                                                                                                                                                                                  1/10
  Verifying  : systemd-sysv-219-19.el7_2.13.x86_64                                                                                                                                                                                                                  2/10
  Verifying  : systemd-219-19.el7_2.13.x86_64                                                                                                                                                                                                                       3/10
  Verifying  : systemd-python-219-19.el7_2.13.x86_64                                                                                                                                                                                                                4/10
  Verifying  : libgudev1-219-19.el7_2.13.x86_64                                                                                                                                                                                                                     5/10
  Verifying  : systemd-libs-219-19.el7_2.4.x86_64                                                                                                                                                                                                                   6/10
  Verifying  : systemd-sysv-219-19.el7_2.4.x86_64                                                                                                                                                                                                                   7/10
  Verifying  : systemd-219-19.el7_2.4.x86_64                                                                                                                                                                                                                        8/10
  Verifying  : libgudev1-219-19.el7_2.4.x86_64                                                                                                                                                                                                                      9/10
  Verifying  : systemd-python-219-19.el7_2.4.x86_64                                                                                                                                                                                                                10/10

Updated:
  systemd.x86_64 0:219-19.el7_2.13

Dependency Updated:
  libgudev1.x86_64 0:219-19.el7_2.13                             systemd-libs.x86_64 0:219-19.el7_2.13                             systemd-python.x86_64 0:219-19.el7_2.13                             systemd-sysv.x86_64 0:219-19.el7_2.13

Complete!
[root@RESCUE-pirax-test dev]# su adam
[adam@RESCUE-pirax-test dev]$ ssh root@localhost
cannot read from /dev/urandom, No such file or directory
[adam@RESCUE-pirax-test dev]$ /sbin/
Display all 526 possibilities? (y or n)
[adam@RESCUE-pirax-test dev]$ /sbin/MAKEDEV std
bash: /sbin/MAKEDEV: No such file or directory
[adam@RESCUE-pirax-test dev]$ mknod /dev/random c 1 9
mknod: ‘/dev/random’: Permission denied
[adam@RESCUE-pirax-test dev]$ exit
exit
[root@RESCUE-pirax-test dev]# mknod /dev/random c 1 9
[root@RESCUE-pirax-test dev]# su adam
[adam@RESCUE-pirax-test dev]$ ssh root@localhost
cannot read from /dev/urandom, No such file or directory
[adam@RESCUE-pirax-test dev]$ exit
exit
[root@RESCUE-pirax-test dev]# mknod /dev/urandom c 1 9
[root@RESCUE-pirax-test dev]# su adam
[adam@RESCUE-pirax-test dev]$ ssh root@localhost
Host key verification failed.
[adam@RESCUE-pirax-test dev]$ exit


Dec  7 09:23:55 pirax-test login: FAILED LOGIN 1 FROM tty1 FOR root, Authentication failure
Dec  7 09:32:00 pirax-test polkitd[1031]: Loading rules from directory /etc/polkit-1/rules.d
Dec  7 09:32:00 pirax-test polkitd[1031]: Loading rules from directory /usr/share/polkit-1/rules.d
Dec  7 09:32:00 pirax-test polkitd[1031]: Finished loading, compiling and executing 2 rules
Dec  7 09:32:00 pirax-test polkitd[1031]: Acquired the name org.freedesktop.PolicyKit1 on the system bus
Dec  7 09:32:10 pirax-test sshd[1375]: Server listening on 0.0.0.0 port 666.
Dec  7 09:32:10 pirax-test sshd[1375]: Server listening on :: port 666.
Dec  7 09:32:24 pirax-test unix_chkpwd[2692]: password check failed for user (root)
Dec  7 09:32:24 pirax-test login: pam_unix(login:auth): authentication failure; logname=LOGIN uid=0 euid=0 tty=tty1 ruser= rhost=  user=root
Dec  7 09:32:24 pirax-test login: pam_succeed_if(login:auth): requirement "uid >= 1000" not met by user "root"
Dec  7 09:32:27 pirax-test login: FAILED LOGIN 1 FROM tty1 FOR root, Authentication failure
Dec  7 09:32:32 pirax-test unix_chkpwd[2694]: password check failed for user (root)
Dec  7 09:32:32 pirax-test login: pam_succeed_if(login:auth): requirement "uid >= 1000" not met by user "root"
Dec  7 09:32:34 pirax-test login: FAILED LOGIN 2 FROM tty1 FOR root, Authentication failure
Dec  7 09:32:39 pirax-test unix_chkpwd[2696]: password check failed for user (root)
Dec  7 09:32:39 pirax-test login: pam_succeed_if(login:auth): requirement "uid >= 1000" not met by user "root"
Dec  7 09:32:41 pirax-test login: FAILED LOGIN SESSION FROM tty1 FOR root, Authentication failure
Dec  7 09:32:41 pirax-test login: PAM 2 more authentication failures; logname=LOGIN uid=0 euid=0 tty=tty1 ruser= rhost=  user=root
Dec  7 10:05:17 pirax-test polkitd[1029]: Loading rules from directory /etc/polkit-1/rules.d
Dec  7 10:05:17 pirax-test polkitd[1029]: Loading rules from directory /usr/share/polkit-1/rules.d
Dec  7 10:05:17 pirax-test polkitd[1029]: Finished loading, compiling and executing 2 rules
Dec  7 10:05:17 pirax-test polkitd[1029]: Acquired the name org.freedesktop.PolicyKit1 on the system bus
Dec  7 10:05:27 pirax-test sshd[1375]: Server listening on 0.0.0.0 port 666.
Dec  7 10:05:27 pirax-test sshd[1375]: Server listening on :: port 666.
Dec  7 10:09:56 pirax-test unix_chkpwd[2791]: password check failed for user (root)
Dec  7 10:09:56 pirax-test login: pam_unix(login:auth): authentication failure; logname=LOGIN uid=0 euid=0 tty=tty1 ruser= rhost=  user=root
Dec  7 10:09:56 pirax-test login: pam_succeed_if(login:auth): requirement "uid >= 1000" not met by user "root"
Dec  7 10:09:58 pirax-test login: FAILED LOGIN 1 FROM tty1 FOR root, Authentication failure
Dec  7 10:10:07 pirax-test unix_chkpwd[2802]: password check failed for user (root)
Dec  7 10:10:07 pirax-test login: pam_succeed_if(login:auth): requirement "uid >= 1000" not met by user "root"
Dec  7 10:10:09 pirax-test login: FAILED LOGIN 2 FROM tty1 FOR root, Authentication failure
[root@RESCUE-pirax-test /]# adduser adam
adduser: user 'adam' already exists
[root@RESCUE-pirax-test /]# passwd adam
Changing password for user adam.
New password:
Retype new password:
passwd: all authentication tokens updated successfully.
[root@RESCUE-pirax-test /]# passwd root
Changing password for user root.
New password:
Retype new password:
passwd: all authentication tokens updated successfully.
[root@RESCUE-pirax-test /]# su adam
[adam@RESCUE-pirax-test /]$ ssh adam@localhost
Host key verification failed.
[adam@RESCUE-pirax-test /]$ exit
exit
[root@RESCUE-pirax-test /]# vi /etc/ss
ssh/ ssl/
[root@RESCUE-pirax-test /]# vi /etc/ss
ssh/ ssl/
[root@RESCUE-pirax-test /]# vi /etc/ssh/
moduli                    ssh_config                sshd_config               ssh_host_ecdsa_key        ssh_host_ecdsa_key.pub    ssh_host_ed25519_key      ssh_host_ed25519_key.pub  ssh_host_rsa_key          ssh_host_rsa_key.pub
[root@RESCUE-pirax-test /]# vi /etc/ssh/sshd_config
[root@RESCUE-pirax-test /]# su adam
[adam@RESCUE-pirax-test /]$ ssh adam@localhost
Host key verification failed.
[adam@RESCUE-pirax-test /]$ exit
exit
[root@RESCUE-pirax-test /]# mknod -m 666 /dev/tty c 5 0
[root@RESCUE-pirax-test /]# su adam
[adam@RESCUE-pirax-test /]$ ssh adam@localhost
The authenticity of host 'localhost (::1)' can't be established.
ECDSA key fingerprint is e1:0c:0f:95:94:55:84:00:0d:e9:36:42:1d:6d:9e:0c.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
adam@localhost's password:

It’s still kind of broken, but its getting less broken by the minute ;-D

Rackspace Cloud Server not coming up after building it

So! Perhaps you’ve read my article on nova-agent , the common cause of this issue? If you haven’t you should since it covers out well the importance of nova-agent.

However, nova-agent itself also comes unstuck if the machine nova-agent is installed on is missing the xe-linux-distribution service, this service is provided by the package xe-guest-utilities and can be installed by yourselves, in the case that installing/ ensuring nova-agent starts on boot does not fix your issues.

Specifically if your nova-agent log provides you this message, you know you need to install the xe-guest-utilities. Simplies!

Problem

# cat /var/log/nova-agent.log

2016-10-06 18:58:14,696 [ERROR] [EXC] Traceback (most recent call last):
2016-10-06 18:58:14,697 [ERROR] [EXC]   File "/usr/share/nova-agent/nova-agent.py", line 40, in 
2016-10-06 18:58:14,697 [ERROR] [EXC]     xs = plugins.XSComm()
2016-10-06 18:58:14,697 [ERROR] [EXC]   File "/usr/share/nova-agent/1.39.0/plugins/xscomm.py", line 43, in __init__
2016-10-06 18:58:14,697 [ERROR] [EXC]     self.xs_handle = pyxenstore.Handle()
2016-10-06 18:58:14,700 [ERROR] [EXC] PyXenStoreError: Couldn't open connection to the xenstore: No such file or directory
2016-10-06 18:58:14,701 [ERROR] failed to parse config file '/usr/share/nova-agent/nova-agent.py'

Solution

# Redhat and CentOS systems
yum install xe-guest-utilities

# Debian, Ubuntu and other apt based systems
apt-get install xe-guest-utilities

I hope that this is of some assistance, here is some more background information.

More details about nova-agent and xe-guest-utilities in Xen

Provided that you have definitely enabled nova-agent, and ensured that it is running (after restarting the original server), with ps auxfwww | grep nova-agent

then, you should be good to re-image the original server, and then rebuild out the second.

The reason why your server doesn’t appear to be coming up in the new build is for some reason, nova-agent service got disabled on boot-time, and as a result, the nova-agent service responsible for swapping out the network configuration of your cloud-server wasn’t started up when the server was built, and the automatic ip configuration change didn’t occur. This explains well the behaviour you’ve been seeing, and after looking in the backend the error code seems to confirm that the issue was that the nova-agent wasn’t running.

Provided that you’ve definitely installed nova-agent and confirmed it is running, as well as made sure it starts at boot time, as in the article I wrote, you should be good.

I hope that this explanation and clarification meets you well.

I can see that you’ve recently posted an additional issue that has been experienced with xe-linux-distribution (the cause of the PyXenStoreError). This secondary cause of the issue can be fixed by ensuring xe-linux-distribution is installed;

apt-get update;
apt-get install xe-guest-utilities

This should install the xen guest tools as required by nova-agent. This is required by the nova-agent in order for the networking data to be retrieved by nova-agent, whereas nova-agent itself applies the change, but these services both need to be running and installed for this to work properly!

I really hope that this is of some assistance ,of course if you have additional questions, comments or concerns please don’t hesitate to write back, and we can escalate this issue further for you. These instructions should fix your issues though! I hope this helps &

Cool Little script for downloading stuff


#!/bin/sh
# just use uuid's instead of sequential numbers hehe

for a in `seq 10000000 90000000`;
do
for b in `seq 1 10`;
do
‪#‎echo‬ http://cdn.anonymous.com/$a""_""$b"".user"
echo "wget http://cdn.anonymous.com/$a""_""$b"".user" | bash
#echo "curl http://cdn.anonymous.com/$a""_""$b".user -o "($a)"_"($b)".user"

filesize=`ls -al "$a"_"$b".user | awk '{print $5}'`
echo "FILESIZE= $filesize"

if [ "$filesize" -eq "49" ]
then
echo "404: Emtpy fakefile HTTP 200 detected! The end of this hidden usergroup was detected"
echo "Cleaning up.."
rm "$a"_"$b".user
break;

else
echo "200: Continuing "
fi

sleep 4

done

TCPDUMP command packet capture Usage

So, it’s been a little while since my last update. We’ve been quite busy recently, but for those interested in learning more about tcpdump and physically capturing packets.

List Interfaces that can be tcp dumped

tcpdump -D

Listen on Interface eth0

tcpdump -i eth0

Listen to Xenserver domain 16 on public net

tcpdump -i vif16.0 

Listen on any interface

tcpdump -i any

Super duper High verbosity tcpdump

tcpdump -vvvv -i eth0 

Be verbose and print data of each packet in both hex and ASCII

tcpdump -v -X -i eth0

Be less verbose

tcpdump -q 

Limit the capture of packets to 100

tcpdump -c 100 -i eth0 

Display IP addresses and port numbers instead of domain and service names when capturing packets (note: on some systems you need to specify -nn to display port numbers):

tcpdump -n

Capture any packets where the destination host is 192.168.1.1. Display IP addresses and port numbers:

tcpdump -n dst host 192.168.1.1

Capture any packets where the source host is 192.168.1.1. Display IP addresses and port numbers:

tcpdump -n src host 192.168.1.1

http://www.rationallyparanoid.com/articles/tcpdump.html