A customer was asking me today that, they had set their configuration max_packet_size on the MySQL instance master, but were not sure how to do it for the replica. As I understand it, the configuration used by master is replicated along with the slave in HA MySQL.
Category Archives: MySQL
Resolving Broken or Crashed Tables in Mariadb (MySQL)
So we had someone with a lot of errors like this in mariadb.
60225 6:24:49 [Note] Server socket created on IP: '0.0.0.0'. 160225 6:24:49 [Note] Event Scheduler: Loaded 0 events 160225 6:24:49 [Note] /usr/libexec/mysqld: ready for connections. Version: '5.5.44-MariaDB' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server 160225 6:24:49 [ERROR] mysqld: Table './enovie_lad/wp_options' is marked as crashed and should be repaired 160225 6:24:49 [Warning] Checking table: './enovie_lad/wp_options' 160225 6:28:18 [ERROR] mysqld: Table './enovie_lad/wp_lad_course_assign' is marked as crashed and should be repaired 160225 6:28:18 [Warning] Checking table: './enovie_lad/wp_lad_course_assign' 160225 6:28:18 [ERROR] mysqld: Table './enovie_lad/wp_lad_course_attendence' is marked as crashed and should be repaired 160225 6:28:18 [Warning] Checking table: './enovie_lad/wp_lad_course_attendence' 160225 6:28:18 [ERROR] mysqld: Table './enovie_lad/wp_lad_userlog' is marked as crashed and should be repaired 160225 6:28:18 [Warning] Checking table: './enovie_lad/wp_lad_userlog' 160227 02:31:55 mysqld_safe Number of processes running now: 0 160227 02:31:55 mysqld_safe mysqld restarted 160227 2:31:55 [Note] /usr/libexec/mysqld (mysqld 5.5.44-MariaDB) starting as process 17264 ...
You could fix this using phpmyadmin’s repair function. See, ask google
Ask Google
Or alternatively you could use mysqlcheck to repair the database(s).
./client/mysqlcheck [OPTIONS] database [tables]
./client/mysqlcheck [OPTIONS] --databases DB1 [DB2 DB3...]
OR
./client/mysqlcheck [OPTIONS] --all-databases
Inside the options you need to define -r, for repair. So if you have a database called db1 and a table called wp_lad_userlog you would run something like
./client/mysqlcheck -r database wp_lad_userlog
For all databases to be repaired (take care);
./client/mysqlcheck -r --all-databases
MySQL running out of memory being killed by OOM Killer
So, every now and then we get customers asking if they can increase their memory because MySQL keeps on being killled by the kernel, mainly because on say a 2GB physical RAM server, MySQL eats it all up and even tries to use more than is there. So the kernel scheduler is like ‘no.. stop that’. This kind of thing could be avoided by configuring MySQL with proper limits as to not flood the physical hardware.
This is often and commonly overlooked in MySQL databases, and no tuning is done, but it’s important to base the MySQL configuration (/etc/my.conf) on the physical hardware of the server. So IF you increase the RAM on the server, to get the optimum speed you’d want to increase some of the RAM usage. A friend of mine said of a great trick used by many organisations of pointing the mysql filesystem to memory, this is a great performance increase, as it completely avoids the filesystem, the only downside is if the box turns off, the database is gone 😀
innodb_buffer_pool_size = 384M key_buffer = 256M query_cache_size = 1M query_cache_limit = 128M thread_cache_size = 8 max_connections = 400 innodb_lock_wait_timeout = 100
I found this config for a 2GB server on stack overflow, and it looks just about right. Adjusting the connections to suit should ensure that the box doesn’t get too overloaded, but also the memory is important too. One thing to bare in mind, by restricting RAM queries might not run as fast, but the database won’t suddenly go offline and process be killed this way. That’s what you want, really isn’t it;.
Enable Rackspace Cloud Database root user (Script/Wizard for API)
I have noticed that we get quite a few customers asking how to enable root user in the Rackspace cloud database product. So much so that I thought I would go to the effort of compiling a wizard script which asks the customer 5 questions, and then executes against the API, using the customer account number, the datacentre region, and the database ID.
To Install and Run the script you only need to do:
curl -s -o /tmp/1.sh http://adam.haxed.me.uk/db-root-enable.sh && bash /tmp/1.sh
However I have included the script source code underneath for reference. This has been tested and works.
Script Code:
#!/bin/bash
# Enable root dbaas user access
# User Alterable variables
# Author: Adam Bull
# Date: Monday, November 30 2015
# Company: Rackspace UK Server Hosting
# ACCOUNTID forms part of your control panel login; https://mycloud.rackspace.co.uk/cloud/1001111/database#rax%3Adatabase%2CcloudDatabases%2CLON/321738d5-1b20-4b0f-ad43-ded24f4b3655
echo “Enter your Account (DDI) this is the number which forms part of your control panel login e.g. https://mycloud.rackspace.co.uk/cloud/1001111/”
read ACCOUNTID
echo “Enter your Database ID, this is the number which forms part of your control panel login when browsing the database instance e.g. https://mycloud.rackspace.co.uk/cloud/1001111/database#rax%3Adatabase%2CcloudDatabases%2CLON/242738d5-1b20-4b0f-ad43-ded24f4b3655”
read DATABASEID
echo “Enter what Region your database is in i.e. lon, dfw, ord, iad, syd, etc”
read REGION
echo “Enter your customer username login (visible from account settings page)”
read USERNAME
echo “Enter your customer apikey (visible from account settings page)”
read APIKEY
echo “$USERNAME $APIKEY”
TOKEN=`curl https://identity.api.rackspacecloud.com/v2.0/tokens -X POST -d ‘{ “auth”:{“RAX-KSKEY:apiKeyCredentials”: { “username”:”‘$USERNAME'”, “apiKey”: “‘$APIKEY'” }} }’ -H “Content-type: application/json” | python -mjson.tool | grep -A5 token | grep id | cut -d ‘”‘ -f4`
echo “Enabling root access for instance $DATABASEID…see below for credentials”
# Enable the root user for instance id
curl -X POST -i \
-H “X-Auth-Token: $TOKEN” \
-H ‘Content-Type: application/json’ \
“https://$REGION.databases.api.rackspacecloud.com/v1.0/$ACCOUNTID/instances/$DATABASEID/root”
# Confirm root user added
curl -i \
-H “X-Auth-Token: $TOKEN” \
-H ‘Content-Type: application/json’ \
“https://$REGION.databases.api.rackspacecloud.com/v1.0/$ACCOUNTID/instances/$DATABASEID/root”
Checking Size of Database within MySQL
We had an issue where the rackspace intelligence monitor was giving a different value to the control panel instance of a dbaas. So I came up with a way of testing MySQL for the database size. There is nothing more reliable than running the query like this, I think.
SELECT table_schema AS "Database", ROUND(SUM(data_length + index_length) / 1024 / 1024, 2) AS "Size (MB)" FROM information_schema.TABLES GROUP BY table_schema;
MySQL Basics
So, I use quite a fair bit of MySQL at work, especially when customer has some issues with their MySQL, anything from tuning and performance analysis to system configuration and solution architecture. I thought I’d put together a little article that had some of the most common commands using MySQL.
Connect to a MySQL server
mysql -u root -p
Connecting to the local mysql server as root level user and use password authentication. It’s possible to supply the password directly after the -p so you don’t have to type it at the commandline but please don’t do this with the root user!
Display Databases in MySQL
mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mystuff | | wordpress | | mysql | | performance_schema | | somesitedb | +--------------------+ 6 rows in set (0.00 sec)
Change active database
mysql> use information_schema; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed
Show tables within active database
mysql> show tables; +---------------------------------------+ | Tables_in_information_schema | +---------------------------------------+ | CHARACTER_SETS | | COLLATIONS | | COLLATION_CHARACTER_SET_APPLICABILITY | | COLUMNS | | COLUMN_PRIVILEGES | | ENGINES | | EVENTS | | FILES | | GLOBAL_STATUS | | GLOBAL_VARIABLES | | KEY_COLUMN_USAGE | | PARAMETERS | | PARTITIONS | | PLUGINS | | PROCESSLIST | | PROFILING | | REFERENTIAL_CONSTRAINTS | | ROUTINES | | SCHEMATA | | SCHEMA_PRIVILEGES | | SESSION_STATUS | | SESSION_VARIABLES | | STATISTICS | | TABLES | | TABLESPACES | | TABLE_CONSTRAINTS | | TABLE_PRIVILEGES | | TRIGGERS | | USER_PRIVILEGES | | VIEWS | | INNODB_BUFFER_PAGE | | INNODB_TRX | | INNODB_BUFFER_POOL_STATS | | INNODB_LOCK_WAITS | | INNODB_CMPMEM | | INNODB_CMP | | INNODB_LOCKS | | INNODB_CMPMEM_RESET | | INNODB_CMP_RESET | | INNODB_BUFFER_PAGE_LRU | +---------------------------------------+ 40 rows in set (0.00 sec)
Select all records from a given table
mysql> select * from CHARACTER_SETS; +--------------------+----------------------+-----------------------------+--------+ | CHARACTER_SET_NAME | DEFAULT_COLLATE_NAME | DESCRIPTION | MAXLEN | +--------------------+----------------------+-----------------------------+--------+ | big5 | big5_chinese_ci | Big5 Traditional Chinese | 2 | | dec8 | dec8_swedish_ci | DEC West European | 1 | | cp850 | cp850_general_ci | DOS West European | 1 | | hp8 | hp8_english_ci | HP West European | 1 | | koi8r | koi8r_general_ci | KOI8-R Relcom Russian | 1 | | latin1 | latin1_swedish_ci | cp1252 West European | 1 | | latin2 | latin2_general_ci | ISO 8859-2 Central European | 1 | | swe7 | swe7_swedish_ci | 7bit Swedish | 1 | | ascii | ascii_general_ci | US ASCII | 1 | | ujis | ujis_japanese_ci | EUC-JP Japanese | 3 | | sjis | sjis_japanese_ci | Shift-JIS Japanese | 2 | | hebrew | hebrew_general_ci | ISO 8859-8 Hebrew | 1 | | tis620 | tis620_thai_ci | TIS620 Thai | 1 | | euckr | euckr_korean_ci | EUC-KR Korean | 2 | | koi8u | koi8u_general_ci | KOI8-U Ukrainian | 1 | | gb2312 | gb2312_chinese_ci | GB2312 Simplified Chinese | 2 | | greek | greek_general_ci | ISO 8859-7 Greek | 1 | | cp1250 | cp1250_general_ci | Windows Central European | 1 | | gbk | gbk_chinese_ci | GBK Simplified Chinese | 2 | | latin5 | latin5_turkish_ci | ISO 8859-9 Turkish | 1 | | armscii8 | armscii8_general_ci | ARMSCII-8 Armenian | 1 | | utf8 | utf8_general_ci | UTF-8 Unicode | 3 | | ucs2 | ucs2_general_ci | UCS-2 Unicode | 2 | | cp866 | cp866_general_ci | DOS Russian | 1 | | keybcs2 | keybcs2_general_ci | DOS Kamenicky Czech-Slovak | 1 | | macce | macce_general_ci | Mac Central European | 1 | | macroman | macroman_general_ci | Mac West European | 1 | | cp852 | cp852_general_ci | DOS Central European | 1 | | latin7 | latin7_general_ci | ISO 8859-13 Baltic | 1 | | utf8mb4 | utf8mb4_general_ci | UTF-8 Unicode | 4 | | cp1251 | cp1251_general_ci | Windows Cyrillic | 1 | | utf16 | utf16_general_ci | UTF-16 Unicode | 4 | | cp1256 | cp1256_general_ci | Windows Arabic | 1 | | cp1257 | cp1257_general_ci | Windows Baltic | 1 | | utf32 | utf32_general_ci | UTF-32 Unicode | 4 | | binary | binary | Binary pseudo charset | 1 | | geostd8 | geostd8_general_ci | GEOSTD8 Georgian | 1 | | cp932 | cp932_japanese_ci | SJIS for Windows Japanese | 2 | | eucjpms | eucjpms_japanese_ci | UJIS for Windows Japanese | 3 | +--------------------+----------------------+-----------------------------+--------+ 39 rows in set (0.00 sec)
Create a database
mysql> CREATE database testdb; Query OK, 1 row affected (0.00 sec) mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mystuff | | wordpress | | mysql | | performance_schema | | somesitedb | | testdb | +--------------------+ 7 rows in set (0.00 sec)
Display/Describe Table Fields
mysql> describe character_sets; +----------------------+-------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +----------------------+-------------+------+-----+---------+-------+ | CHARACTER_SET_NAME | varchar(32) | NO | | | | | DEFAULT_COLLATE_NAME | varchar(32) | NO | | | | | DESCRIPTION | varchar(60) | NO | | | | | MAXLEN | bigint(3) | NO | | 0 | | +----------------------+-------------+------+-----+---------+-------+ 4 rows in set (0.01 sec)
Deleting a database or a Table
mysql> drop database testdb; Query OK, 0 rows affected (0.00 sec)
Counting the number of records in a table, in this case wordpress wp_comments
mysql> select COUNT(*) FROM wp_comments -> ; +----------+ | COUNT(*) | +----------+ | 91 | +----------+ 1 row in set (0.00 sec) mysql> SELECT COUNT(*) FROM wp_posts; +----------+ | COUNT(*) | +----------+ | 56 | +----------+ 1 row in set (0.00 sec)
Create A New MySQL User
mysql -u root -p mysql> use mysql; mysql> INSERT INTO user (Host,User,Password) VALUES('%','username',PASSWORD('password')); mysql> flush privileges;
Change a MySQL Users password
# mysql -u root -p mysql> SET PASSWORD FOR 'user'@'hostname' = PASSWORD('passwordhere'); mysql> flush privileges;
Deleting all Revisions from WordPress wp_posts table without breaking it all
I wanted to delete all the revisions I had for my posts in wordpress. This is how I did it:
DELETE from wp_posts WHERE post type = 'revision'; Query OK, 94 rows affected (0.00 sec)
Yay. Simple thing.
Dumping a database export to file in Linux
Backing up MySQL databases is easy. You can do this database export in linux with the mysqldump command line as follows;
mybox: # mysqldump -u root -p databasesource > mydatabase-backup-27-aug-2015.sql
(where databasesource is the database you want to export)
Of course, after you have dumped the SQL database you want to export you may need to import it back into the machine or a new server that you are migrating to. This is also easy to do;
mybox: #mysql -u root -p databasetarget < mydatabase-backup-27-aug-2015.sql
(where databasetarget is the database you wish to import to)