Using Freebsd unrar utility properly and extracting recursively

I noticed I had a lot of unrarred files and needed a way to completely unrar everything in a folder. I noticed a lot of the examples on the internets didnt work.

find Archive/ -name '*.part01.rar' -execdir unrar e {} \;

This basically hunts out all directories below Archive/ and extracts all archives starting with part01.rar, the problem with examples I found is they used regex for *.rar or unrar’s -r recursive feature which in my BSD system seemed not best way to do this.

I was pleased with the oneliner though. It might be useful for people with freenas systems.

Apache2 Module installed but not loaded

I came across a customer recently that had a module installed on their apache2 installation, but they couldn’t understand why it wasn’t loaded. In this case it was the filter module.

[root@box ~]# yum provides /usr/lib64/httpd/modules/
Loaded plugins: fastestmirror, replace
Loading mirror speeds from cached hostfile
drivesrvr                                                                                                                                                                                                            | 2.2 kB     00:00
httpd-2.2.15-59.el6.centos.x86_64 : Apache HTTP Server
Repo        : base
Matched from:
Filename    : /usr/lib64/httpd/modules/

httpd24u-2.4.27-1.ius.centos6.x86_64 : Apache HTTP Server
Repo        : ius
Matched from:
Filename    : /usr/lib64/httpd/modules/

httpd-2.2.15-60.el6.centos.4.x86_64 : Apache HTTP Server
Repo        : updates
Matched from:
Filename    : /usr/lib64/httpd/modules/

httpd-2.2.15-60.el6.centos.4.x86_64 : Apache HTTP Server
Repo        : installed
Matched from:
Other       : Provides-match: /usr/lib64/httpd/modules/

You can activated it by adding following line to httpd.conf;

It was simple to install just throw this in your httpd.conf

LoadModule filter_module modules/

Job done.

Retrieving Process List from Rackspace Cloud Server Monitoring

It is possible for you to use the Rackspace API to retrieve the Running Process List of a Cloud-server on your Rackspace account which has the rackspace cloud-server monitoring agent installed.


 curl -s -H "Content-Type: application/json" -H "X-Auth-Token: $token"$customeridhere/agents/$server_uuid_here/host_info/processes

Not that difficult to do, really !

Rapid Troubleshooting mail not arriving at its destination

Today I had a customer whose mail was not arriving at it’s destination. IN my customers case, they knew that it was arriving at the destination but going into the spam folder. This is likely due to blacklisting, however some ISP or clients dont know it’s reaching the spam folder at the other end, and since the most common cause of this happening is due to a missing SPF (Sending policy framework record), MX record or DKIM record it is possibly to rapidly check the DNS of each using dig, if the sender domain is known.

To check for the IP Blacklistings on a mailserver use it’s ip in one of the many spam a checker;

In other cases it might be being caused by email bounces, due to the PTR, MX or DKIM records, and not even getting into the inbox, you can see that on the sending mail server using a simple grep command;

cat /var/log/maillog | grep -i status=bounced

You probably want to save the file as well

cat /var/log/maillog | grep -i status=bounced > bouncedemail.txt 

If you wanted to know which domains bounced email, if you’ve ensured all sending domains are correctly configured via DNS..

[root@api ~]# cat /var/log/maillog | grep -i status=bounced | awk '{print $7}'

You could use sed to extract which domains failed..

Then you could use whois against the domains to reach out to the email contact with some automation that explains that ‘weve checked DKIM, MX and SPF and all are configured correctly and believe this is an error on your behalf, etc, blah blah’..

Such a thing would be a good idea to implement for some large email providers, and I’m sure you could automate the DNS checking as well. You could likely automate the whole thing, just by watching the logs and DNS records, and some intelligent grep and awking.

Not bad.

Calculating the Average Hits per minute en-mass for thousands of sites

So, I had a customer having some major MySQL woes, and I wanted to know whether the MySQL issues were query related, as in due to the frequency of queries alone, or the size of the database. VS it being caused by the number of visitors coming into apache, therefore causing more frequency of MySQL hits, and explaining the higher CPU usage.

The best way to achieve this is to inspect /var/log/httpd with ls -al,

First we take a sample of all of the requests coming into apache2, as in all of them.. provided the customer has used proper naming conventions this isn’t a nightmare. Apache is designed to make this easy for you by the way it is setup by default, hurrah!

[root@box-DB1 logparser]# time tail -f /var/log/httpd/*access_log > allhitsnow

real	0m44.560s
user	0m0.006s
sys	0m0.031s

Time command prefixed here, will tell you how long you ran it for.

[root@box-DB1 logparser]# cat allhitsnow | wc -l

The above command shows you the number of lines in allhitsnow file, which was written to with all the new requests coming into sites from all the site log files. Simples! 1590 queries a minute is quite a lot.

Lysncd fails with Error: Terminating since out of inotify watches.

This is a remarkably easy problem to solve.

From man:

The inotify API provides a mechanism for monitoring filesystem
       events.  Inotify can be used to monitor individual files, or to
       monitor directories.  When a directory is monitored, inotify will
       return events for the directory itself, and for files inside the

The problem is a kernel based value, that determines how much memory is used by processes such as lsyncd. Lets just double check what value it is set to presently:

Check Inotify max_user_watches value presently

[root@server php]# cat /proc/sys/fs/inotify/max_user_watches

Check the amount of available memory on the server to ensure it can deal with the increase

[root@server-new php]# free -m
             total       used       free     shared    buffers     cached
Mem:          7358       6577        781          1        376       2427
-/+ buffers/cache:       3773       3584
Swap:            0          0          0

The cache can be added to free, as a maximum saturation. Linux uses lots of memory it doesn’t really need for efficiency.

Each used inotify watch uses 1 kB RAM in 64 bit systems, for 32bit its half.

Increase Inotify watches to a higher value

sysctl fs.inotify.max_user_watches=200000

Enabling MySQL Slow Query Logs

In the case your seeing very long pageload times, and have checked your application. It always pays to check the way in which the database performs when interacting with the application, especially if they are on either same or seperate server, as these significantly affect the way that your application will run.

mysql> SET GLOBAL slow_query_log = 'ON' ;
Query OK, 0 rows affected (0.01 sec)

mysql> SET GLOBAL slow_query_log_file = '/slow_query_logs/slow_query_logs.txt';
Query OK, 0 rows affected (0.00 sec)

mysql> SET GLOBAL long_query_time = 5;
Query OK, 0 rows affected (0.00 sec)

Setting X-Frame-Options HTTP Header to allow SAME or NON SAME ORIGINS

It’s possible to increase the security of a webserver running a website, by ensuring that the X-FRAME-OPTIONS header pushes a header to the browser, which enforces the origin (server) serving the site. It prevents the website then providing objects which are not local to the site, in the stream. An admirable option for those which wish to increase their server security.

Naturally, there are some reasons why you might want to disable this, and in proper context, it can be secure. Always be sure to discuss with your pentester or PCI compliance officer, such considerations before proceeding, especially making sure that if you do not want to use SAME ORIGIN you always use the most secure option for the required task. Always check if there is a better way to achieve what your trying to do, when making such changes to your server configuration.

Insecure X-Frame-Option allows remote non matching origins

Header always append X-Frame-Options ALLOWALL

Secure X-Frame-Option imposes on the browser to not allow non origin(al) connections for the domain, which can prevent clickjack and other attacks.

Header always append X-Frame-Options SAMEORIGIN

Moving a WordPress site – much ado about nothing !

Have you noticed, there is all kinds of advise on the internet about the best way to move WordPress websites? There is literally a myriad of ways to achieve this. One of the methods I read on was:

Changing Your Domain Name and URLs

Moving a website and changing your domain name or URLs (i.e. from to, or to requires the following steps - in sequence.

    Download your existing site files.
    Export your database - go in to MySQL and export the database.
    Move the backed up files and database into a new folder - somewhere safe - this is your site backup.
    Log in to the site you want to move and go to Settings > General, then change the URLs. (ie from to ) - save the settings and expect to see a 404 page.
    Download your site files again.
    Export the database again.
    Edit wp-config.php with the new server's MySQL database name, user and password.
    Upload the files.
    Import the database on the new server.

I mean this is truly horrifying steps to take, and I don’t see the point at all. This is how I achieved it for one my customers.

1. Take customer Database Dump
2. Edit the database searching for 'siteurl' with vi
vi mysqldump.sql

And just swap out the values, confirming after editing the file;

[root@box]# cat somemysqldump.sql  | grep siteurl -A 2
(1, 'siteurl', '', 'yes'),
(2, 'home', '', 'yes'),
(3, 'blogname', 'My website name', 'yes'),

Job done, no stress

There might be additional bits but this is certainly enough for them to access the wp-admin panel. If you have problems add this line to the wp-config.php file;


Just before the line which says

/* That’s all, stop editing! Happy blogging. */

And then just do the import/restore as normal;

mysql -u newmysqluser -p newdatabase_to_import_to < old_database.sql

Simples! I really have no idea why it is made to be so complicated on other hosting sites or platforms.

Reverting a yum transaction & controlling auto-updates for yum-cron with excludes

Hey, so a customer was running yum-cron, the Redhat version of Canoninical’s unattended upgrades. An auto update for a package ran, which actually broke their backend.

# yum history info 172
Loaded plugins: replace, rhnplugin
This system is receiving updates from RHN Classic or Red Hat Satellite.
Transaction ID : 172
Begin time     : Sat May  6 05:45:29 2017
Begin rpmdb    : 742:11fcb243cc5701b9d2293d90cb4161e5edc34bb8
End time       :            05:45:31 2017 (2 seconds)
End rpmdb      : 742:0afbbf517315f18985b2d01d4c2e5250caf0afb5
User           : root <root>
Return-Code    : Success
Transaction performed with:
    Installed     rpm-4.11.3-21.el7.x86_64                @rhel-x86_64-server-7
    Installed     yum-3.4.3-150.el7.noarch                @rhel-x86_64-server-7
    Installed     yum-metadata-parser-1.1.4-10.el7.x86_64 @anaconda/7.1
    Installed     yum-rhn-plugin-2.0.1-6.1.el7_3.noarch   @rhel-x86_64-server-7
Packages Altered:
    Updated php56u-pecl-xdebug-2.5.1-1.ius.el7.x86_64 @rackspace-rhel-x86_64-server-7-ius
    Update                     2.5.3-1.ius.el7.x86_64 @rackspace-rhel-x86_64-server-7-ius
history info

The customer asked us to reverse the transaction, so I did, this was quite simple to do;

[root@web user]# yum history undo 172
Loaded plugins: replace, rhnplugin
This system is receiving updates from RHN Classic or Red Hat Satellite.
Undoing transaction 172, from Sat May  6 05:45:29 2017
    Updated php56u-pecl-xdebug-2.5.1-1.ius.el7.x86_64 @rackspace-rhel-x86_64-server-7-ius
    Update                     2.5.3-1.ius.el7.x86_64 @rackspace-rhel-x86_64-server-7-ius
Resolving Dependencies
--> Running transaction check
---> Package php56u-pecl-xdebug.x86_64 0:2.5.1-1.ius.el7 will be a downgrade
---> Package php56u-pecl-xdebug.x86_64 0:2.5.3-1.ius.el7 will be erased
--> Finished Dependency Resolution

Dependencies Resolved

 Package                                                  Arch                                         Version                                               Repository                                                                Size
 php56u-pecl-xdebug                                       x86_64                                       2.5.1-1.ius.el7                                       rackspace-rhel-x86_64-server-7-ius                                       205 k

Transaction Summary
Downgrade  1 Package

Total download size: 205 k
Is this ok [y/d/N]: y
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
php56u-pecl-xdebug-2.5.1-1.ius.el7.x86_64.rpm                                                                                                                                                                        | 205 kB  00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : php56u-pecl-xdebug-2.5.1-1.ius.el7.x86_64                                                                                                                                                                                1/2
  Cleanup    : php56u-pecl-xdebug-2.5.3-1.ius.el7.x86_64                                                                                                                                                                                2/2
  Verifying  : php56u-pecl-xdebug-2.5.1-1.ius.el7.x86_64                                                                                                                                                                                1/2
  Verifying  : php56u-pecl-xdebug-2.5.3-1.ius.el7.x86_64                                                                                                                                                                                2/2

  php56u-pecl-xdebug.x86_64 0:2.5.3-1.ius.el7

  php56u-pecl-xdebug.x86_64 0:2.5.1-1.ius.el7


In CentOS 7 and I believe RedHat RHEL 7 as well, if you don’t want to disable yum-cron altogether by running a yum remove yum-cron, you could exclude the specific package, or use wildcards to exclude all of them like php* , http*, etc.

vi /etc/yum/yum-cron.conf.

If you wish to exclude some packages from auto-update mechanism, you’ll have to add an exclude line, at the bottom of the file, in the base section.

exclude = kernel* php* httpd*

etc, I hope that this is of some assist,

cheers &
Best wishes,