06 Dec

Escaping “Lesson Learned” dealing with MySQL Databases & Case Sensitiviy

Do you ever need to transfer databases between servers? Different OSes?

Two common ways of data migration:
1. Create mysqldump and load it.
2. File transfer using SCP.

Case Sensitivity and Issue:
Yes, that can create big issues when you have to deal with systems having different case sensitivity.
E.g. on linux you can create directories with names “Kedar” or “kedar” which is not possible on windows!
It will be harmful when file-system itself restricting the names to lower case.
MySQL has a variable for that: lower_case_file_system.

Documentation says: This variable describes the case sensitivity of file names on the file system where the data directory is located. OFF means file names are case sensitive, ON means they are not case sensitive. This variable is read only because it reflects a file system attribute and setting it would have no effect on the file system.

Try loading data having different values for lower_case_file_system and you will get it!

How MySQL tries to solves this problem:
It asks user to handle the issue by changing system variable lower_case_table_names accordingly.
Check the options available:
0 : Table names are stored as specified and comparisons are case sensitive.
1 : Table names are stored in lowercase on disk and comparisons are not case sensitive.
2 : Table names are stored as given but compared in lowercase.

So where do I see the problem?
Here control is given to user with the variable lower_case_table_names and ignoring which can definitely lead to issues. These issues can be avoided by putting restrictions.
Also queries working on MAC / Windows may not work well on Linux / Other Flavours; cause problems on application side.

Documentation Says: You should not set this variable to 0 if you are running MySQL on a system that has case-insensitive file names (such as Windows or Mac OS X). If you set this variable to 0 on such a system and access MyISAM tablenames using different lettercases, index corruption may result.

Now, I recently read the change: PERFORMANCE_SCHEMA is renamed to lowercase in MySQL 5.5 to answer a bug [http://bugs.mysql.com/bug.php?id=57609] pointed by Marc Alff’s post [http://marcalff.blogspot.com/].

In one line I wish: lower_case_table_names, RIP!
I wish MySQL does the same for all tables in order to avoid case sensitivity problems :).

I feel MySQL should not allow upper cases. What do you think?

06 Dec

Life At Rest – A few lines for her

:Life at rest:

Oh my Dear,

insomniac for ur sleep,
u’r awesome when its deep,
pale skin, nighty hair,
mighty life and u’r so near…

warm breath, and ur charm,
makes me wait, till u wake,
while u sleep, lemme gaze,
beauty like u and i await….

Kedar's Angel

Kedar's Angel

sun wont rise, birdz wont sing,
oh dear, u oughta rest,
my life, im with u, wont leave,
i know u will rise and i wont laze..

its your time, waiting next door,
i have the keyz, letz go n open,
life.. i am waiting..
life u r resting, waiting..

i see u radiance, felicitous
you glow , weaknesses fading,
feeling u, i was waiting for u..
arise, arise and krex iz with u..

– Krex [with you forever]

PS: This 100th post is dedicated to my angel, my first poem for her!

05 Dec

Kedar and Madhumita Wedding Invitation 25 Dec 2010

Dear God,

You’re almighty and kind. You know everything about me and us. Still I’m writing this formally to you.
With blessings of our parents we’ve decided to “join” each other forever on 25th December 2010. (You know who)

Please execute below commands on our wedding day 🙂

GRANT ALL PRIVILEGES ON madhumita.* TO 'kedar'@'%' INDENTIFIED BY '******' WITH GRANT OPTION;
GRANT ALL PRIVILEGES ON kedar.* TO 'madhumita'@'%' INDENTIFIED BY '******' WITH GRANT OPTION;

Create table KedarMadhumita like God.BlessedLife;
insert into KedarMadhumita select life from kedar join madhumita on (christmasday) where love is true;
call bless(KedarMadhumita);

I’m always thankful and greatful to you God!

Okay,

Though the God will execute above commands on Christmas day, you my friend is invited to share the joy and happiness of our wedding.
So below is the invite architecture for you. 😉

How to get the Great News:

SELECT gr8news FROM madhumita;

How to get the Invitation:

SELECT invite AS Invitation FROM kedar JOIN madhumita ON (25122010);

How to get the Location:
SELECT venue AS Venue, wedDate AS Wedding_Date, wedtime AS Wedding_Time FROM kedar JOIN madhumita ON (25122010);

Wanna know a bit about our likings?

SELECT kedar.likes Kedar_Likes FROM kedar JOIN madhumita ON (25122010);
SELECT madhumita.likes Madhumita_Likes FROM kedar JOIN madhumita ON (25122010);


and ofcourse we like each other as well 🙂

Have doubts? What is there on 25th December? Contact me:
SELECT email as Email, phone AS Phone FROM kedar JOIN madhumita ON (25122010);

I’m sure many of you will try to execute:

SELECT * FROM OurStory;
0 Rows Affected. You do not have sufficient privileges. Please contact Kedar.

🙂

Okay enough !! For those who hate reading above stuff, below is clear-cut invitation for you 🙂

So here you go with Official Wedding Invite Card:

Kedar Madhumita Wedding Invitation

Kedar Madhumita Wedding Invitation

Thanks,
Kedar & Madhumita.

PS: Closed Event. Invitees Only. 🙂

30 Nov

Using LOAD DATA INFILE with Stored Procedure Workaround-MySQL

Okay! So here we will use Load Data syntax to load file into MySQL Server in a Stored procedure. Yep! It’s a workaround.

Download MySQL UDF:

[root@localhost kedar]# wget http://www.mysqludf.org/lib_mysqludf_sys/lib_mysqludf_sys_0.0.3.tar.gz

[refer: http://www.mysqludf.org/]

Extract and Install:

[root@localhost kedar]# tar -xzvf lib_mysqludf_sys_0.0.3.tar.gz
install.sh
lib_mysqludf_sys.c
lib_mysqludf_sys.html
lib_mysqludf_sys.so
lib_mysqludf_sys.sql
Makefile

[root@localhost kedar]# sh install.sh
Compiling the MySQL UDF
gcc -Wall -I/usr/include/mysql -I. -shared lib_mysqludf_sys.c -o /usr/lib/lib_mysqludf_sys.so
MySQL UDF compiled successfully

Please provide your MySQL root password
Enter password:
MySQL UDF installed successfully

Create function sys_exec as follows:

CREATE FUNCTION sys_exec RETURNS INT SONAME ‘lib_mysqludf_sys.so’;

sys_exec – executes an arbitrary command, and returns it’s exit code.

You also can similarly create functions:
sys_eval – executes an arbitrary command, and returns it’s output.
sys_get – gets the value of an environment variable.
sys_set – create an environment variable, or update the value of an existing environment variable.

Example – How to load txt file to MySQL using Stored Procedure & Load Data syntax:

Step-1. Creating table:

CREATE TABLE `t` ( `id` int(2) default NULL ) ENGINE=MyISAM

Step-2. Create a sample file to load:

vi loadtest.txt
1
2
3

Step-3. Create a shell script:

vi /tmp/load.sh
mysql -u mysql_user -p mysql_password -e “load data local infile \”$1\” into table $2;”

Step-4. Create a Stored Procedure:

DELIMITER $$
DROP PROCEDURE IF EXISTS `load_data_SP` $$
CREATE PROCEDURE `load_data_SP` (in_filepath varchar(100),in_db_table varchar(100))
BEGIN
declare exec_str varchar(500);
declare ret_val int;
set exec_str=concat(“sh /tmp/load.sh “,in_filepath,” “, in_db_table);
set ret_val=sys_exec(exec_str);
if ret_val=0 then
select “Success” as Result;
else
select “Please check file permissions and paths” as Result;
end if;
END $$
DELIMITER ;

Step 5. Execute:

CALL `load_data_SP`(‘/tmp/loadtest.txt’ , ‘test.t’);

…and that’s it Stored Procedure will return Success or Failure accordingly.

Make sure you’re having file permissions well set and MySQL can access the files.
Here I’ve kept the files under /tmp directory with chmod 777 & chown mysql:mysql to remove the permission-issue possibility.

Hope this helps.

18 Nov

10 Steps: MySQL Monitoring through Nagios: Install & Configure

Nagios is a powerful monitoring system and here we will learn how to monitor MySQL through Nagios. We will be installing Nagios, required plugins and configuring it to monitor MySQL Database Server.

Let’s unleash the power step by step:

Installing and configuring Nagios

Step-1 : Install required stuff:
yum install httpd
yum install gcc
yum install glibc*
yum install gd*

Step-2 :Create Nagios user account and group
useradd nagios
passwd nagios
groupadd nagcmd
usermod -G nagcmd nagios
usermod -G nagcmd apache

Step-3: Downloads:
Create directory:
mkdir NagiosSetup
cd NagiosSetup

Download nagios
wget -X Get "http://sourceforge.net/projects/nagios/files/nagios-3.x/nagios-3.2.1/nagios-3.2.1.tar.gz/download"

Download Nagios Plugins:
wget -X Get "http://sourceforge.net/projects/nagiosplug/files/nagiosplug/1.4.15/nagios-plugins-1.4.15.tar.gz/download"

Follow up articles:
1: Installing Percona Monitoring Tools for MySQL Nagios.
2: Adding remote MySQL Host to Nagios Setup.

Step-4: Install Nagios

tar -xzvf nagios-3.2.1.tar.gz
cd nagios-3.2.1
./configure --with-command-group=nagcmd
make all
make install
make install-config
make install-commandmode
make install-init
chkconfig --add nagios

[If you miss step “make install-init” you may get:: error reading information on service nagios: No such file or directory ]

Configure Nagios Web Interface:
make install-webconf
htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin

[specify password for nagios admin]

Step-5: Install plugins
tar xvf nagios-plugins-1.4.11.tar.gz
cd nagios-plugins-1.4.11
./configure --with-nagios-user=nagios --with-nagios-group=nagios
make
make install

Step-6: Verify Installation, Starting nagios for the first time

service nagios start
Browse: http://localhost/nagios

Here if you get Error:
“You don’t have permission to access /nagios/ on this server.”

Check /etc/httpd/conf/httpd.conf for DirectoryIndex.
If it’s not having index.php add it as follows:

vi /etc/httpd/conf/httpd.conf
DirectoryIndex index.php index.html index.html.var

Make sure you do restart apache(httpd) and nagios every time you change the config file. You must have php installed.

Monitoring MySQL:

Step-7: Download, Extract and install the MySQL Plugin:

wget http://labs.consol.de/wp-content/uploads/2010/10/check_mysql_health-2.1.3.tar.gz

tar -zxvf check_mysql_health-2.1.3.tar.gz
cd check_mysql_health-2.1.3
./configure --prefix=/usr/local/nagios --with-nagios-user=nagios --with-nagios-group=nagios --with-perl=/usr/bin/perl
make
make install

Step-8: Create database user:
grant usage, replication client on *.* to 'nagios'@'localhost' identified by 'nagios';

Step-9: Provide email address for nagiosadmin:
[Change contacts.cfg file accordingly.]
vi /usr/local/nagios/etc/objects/contacts.cfg

define contact{
contact_name nagiosadmin ; Short name of user
use generic-contact ; Inherit default values from generic-contact template (defined above)
alias Kedar ; Full name of user
email kedar@nitty-witty.com ; <<***** CHANGE THIS TO YOUR EMAIL ADDRESS ******
}

Step-10: Configuring Nagios to Monitor MySQL Server

vi /usr/local/nagios/etc/nagios.cfg
add following line:
cfg_file=/usr/local/nagios/etc/objects/mysqlmonitoring.cfg

Define check_mysql_health command as follows:

vi /usr/local/nagios/etc/objects/commands.cfg

define command{
command_name check_mysql_health
command_line $USER1$/check_mysql_health -H $ARG4$ --username $ARG1$ --password $ARG2$ --port $ARG5$ --mode $ARG3$
}

Enter services to be monitored in mysqlmonitoring.cfg:

vi /usr/local/nagios/etc/objects/mysqlmonitoring.cfg
Add:

define service{
use local-service
host_name localhost
service_description MySQL connection-time
check_command check_mysql_health!nagios!nagios!connection-time!127.0.0.1!3306!
}


define service{
use local-service
host_name localhost
service_description MySQL slave-io-running
check_command check_mysql_health!nagios!nagios!slave-io-running!127.0.0.1!3306!
}


define service{
use local-service
host_name localhost
service_description MySQL slave-sql-running
check_command check_mysql_health!nagios!nagios!slave-sql-running!127.0.0.1!3306!
}

Here we’ve monitored 3 services: Connection-time, io thread and sql thread (replication) status. You can monitor more parameters described here: http://labs.consol.de/nagios/check_mysql_health/

Note: Every time you change configuration file, verify before starting nagios using command:
/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

Finally start nagios service and you’re done with nagios installation and configuration for monitoring MySQL.

Download PDF

I hope you’ve found this useful.

-- Kedar Vaijanapurkar --