Игорь Олемской — практические заметки по системному администрированию Linux CentOS

Архив рубрики ‘Интересные RSS-выборки (новости)’

#2013 – Lost connection to MySQL server during query – OR: MySQL VOODOO! (перепечатка)

Комментариев нет

Today one of my scripts prompted the #2013 MySQL error while querying a huge innoDB table (31GB in ~154million rows). Some queries worked, some just failed. Looking at the logfiles, I saw the following message: InnoDB: Page checksum 1840120551 (32bit_calc: 1224736073), prior-to-4.0.14-form checksum 1811838366 InnoDB: stored checksum 3031359782, prior-to-4.0.14-form stored checksum 1811838366 InnoDB: Page lsn [...]

mysql-5.5.24 (перепечатка)

Комментариев нет

RPM of MySQL Community Server 5.5.24 GA are available in remi repository for fedora and Enterpise Linux (RHEL, CentOS, ...).

To read :

Introduction to MySQL 5.5
Changes in MySQL 5.5.24
MySQL 5.5 Reference Manual

This build use a spec file close to the one from Rawhide
WARNING: before upgrading, a full backup of your databases seems mandatory (a logical dump using mysqldump for example).
 
Simple to install, via yum:
yum --enablerepo=remi update mysql-server... Lire mysql-5.5.24

RPM PHP 5.3.13 for CentOS 6 (перепечатка)

Комментариев нет

Собрал последнюю версию Php 5.3.13 для CentOS 6.x. Скачать rpm для CentOS 6 можно здесь. Либо подключив репозиторий CentALT для CentOS 6.

10.05.2012

php-5.3.13 and php-5.4.3 (перепечатка)

Комментариев нет

RPM of PHP 5.3.13 are available for Enterprise Linux in remi repository.

RPM of PHP 5.4.3 are available for Fedora in remi repository and for Enterprise Linux in remi-test repository.

Read the announcement  : PHP 5.3.13 and PHP 5.4.3 Released!

Use  YUM to install :
yum --enablerepo=remi update php-\*
This versions will also be soon available in the official updates for:

Fedora 17 — PHP 5.4.3
Fedora 16 — PHP 5.3.13
Fedora 15 — PHP 5.3.13

For all questions or help, please use the Forums... Lire php-5.3.13 and php-5.4.3

Simple and efficient MongoDB Backup using script (перепечатка)

Комментариев нет

MongoDB Backup types and strategies are neatly explained in its documentation, which you can check here. In case you are not familiar with MongoDB backup types and strategies, please have a look at its documentation.

What I am describing here is a simple script which we are using since months to take MongoDB backup and transfer it over to our Backup server. Here are few things its doing:

  • As we have multiple MongoDB Replica Sets, the script identify current replica set and check whether current server is Master or Slave, exit if its Master. We take backup only from Slave host.
  • Take Backup using mongodump command.
  • Upon successful completion of dump, transfer that to our Backup server. Ensure that ssh key based authentication is setup between both servers to implement seamless and secure transfer. It creates new directory based on current timestamp under replicaset directory in specified path at Backup server and transfer dump there.
  • Log each steps described above, send alert mail if any step fail with description or send confirmation mail upon successful execution with essential details.
  • Sample confirmation mail is below:
Subject: Backup for  done on  at 09/May/12 08:00:01 AM
Body: Mongo Backup Status for  on 09/May/12 08:00:01 AM.
 
08:00:01 AM:  is slave and looks OK.
08:00:01 AM: Starting Dump, executing /usr/local/mongodb/mongo/bin/mongodump --out /databases/dump --host ...
08:34:06 AM: Mongodump command completed. Backup size is 25G.
08:34:10 AM: Directory created on backup server, copying data using scp...
08:57:00 AM: Copied dump to backup server in directory /home/backup/Mongodump/replicaset/datestamp/.
08:57:00 AM: Mongo Backup process completed successfully.

Here is the full bash script, please note that you need to update variables properly and have to check and update it to run in your environment which can be entirely different from mine. Its just an idea to automate MongoDB backup:

##
# Script to take mongo backup using mongodump and store it in Backup Server
##
 
#!/bin/bash
 
## Set variables
TodayDate=`date +"%d/%b/%g %r"`
DateStamp=`date +%d%m%y%H%M%S`
CurrentTime=`date +"%r"`
 
MongoBinPath="/usr/local/mongodb/mongo/bin"
ReplicaSet=`echo 'rs.status()' | $MongoBinPath/mongo | egrep "set" | awk -F \" '{print $4}'`
MongoHost=`hostname`
LocalBackupPath="/databases/dump"
 
LogFile="/var/log/mongo-backup.log"
IsOK=0
CmdStatus=""
 
BackupHost=xx.xx.xx.xx
BackupHostPath="/home/backup/mongobackup"
BackupHostPort=22
 
MailNotification="admin@domain.com anotheradmin@domain.com"
 
echo -e "Mongo Backup Status for $ReplicaSet on $TodayDate. \n" > $LogFile
 
## Check whether host is slave and in good state for backup
for i in `echo "rs.status()" | $MongoBinPath/mongo | egrep "name" | awk -F \" '{print $4}'| cut -f 1 -d :`; do
 IsMaster=`echo "db.isMaster()"| $MongoBinPath/mongo --host $i | grep ismaster|awk -F ":" '{print $2}' | cut -f 1 -d ,`;
 TheState=`echo "rs.status()"| $MongoBinPath/mongo --host $i | grep -i mystate | awk -F ":" '{print $2}' | cut -f 1 -d ,`;
 if [ $IsMaster == "false" -a $TheState -eq 2  ]; then
  MongoHost=$i
  IsOK=1
  echo "$CurrentTime: $MongoHost is slave and looks OK." >> $LogFile
  break
 fi
done
 
CurrentTime=`date +"%r"`
 
## Exit if not good
if [ $IsOK -eq 0 ]; then
   echo "$CurrentTime: Error: Either $MongoHost is not slave or not in good state. Aborting Backup, Please check!" >> $LogFile
   mail -s "Backup error for $ReplicaSet from $MongoHost on $TodayDate" $MailNotification < $LogFile    exit 1; fi ## Remove earlier backup CmdStatus=$(rm -rf $LocalBackupPath/*) ## Start backup process echo "$CurrentTime: Starting Dump, executing $MongoBinPath/mongodump --out $LocalBackupPath --host $MongoHost..." >> $LogFile
 
CmdStatus=`$MongoBinPath/mongodump --out $LocalBackupPath --host $MongoHost`
if [ $? -ne 0 ]; then
  echo "$CurrentTime: There is an issue while trying to take dump in $MongoHost. Aborting dump process, please check! " >> $LogFile
  cat $CmdStatus >> $LogFile
  mail -s "Backup error for $ReplicaSet from $MongoHost on $TodayDate" $MailNotification < $LogFile   exit fi CurrentTime=`date +"%r"` BackupSize=$(du -sh $LocalBackupPath/. | awk '{ print $1 }') echo "$CurrentTime: Mongodump command completed. Backup size is $BackupSize. " >> $LogFile
 
## dump is fine then scp it to backup server
## create directory first
CmdStatus=$(ssh -p $BackupHostPort $BackupHost "mkdir -p  $BackupHostPath/$ReplicaSet/$DateStamp")
 
if [ $? -ne 0 ]; then
  echo "$CurrentTime: Either failing to connect Backup server using ssh or destination directory already exist!" >> $LogFile
  cat $CmdStatus >> $LogFile
  mail -s "Backup error for $ReplicaSet from $MongoHost on $TodayDate" $MailNotification < $LogFile   exit fi CurrentTime=`date +"%r"` echo "$CurrentTime: Directory created on backup server, copying data using scp..." >> $LogFile
 
CmdStatus=`scp -P $BackupHostPort -r $LocalBackupPath/* $BackupHost:/$BackupHostPath/$ReplicaSet/$DateStamp/`
if [ $? -ne 0 ]; then
  echo "$CurrentTime: Unable to scp dump to $BackupHost:/$BackupHostPath/$ReplicaSet/$DateStamp using port $BackupHostPort. " >> $LogFile
  cat $CmdStatus >> $LogFile
  mail -s "Backup error for $ReplicaSet from $MongoHost on $TodayDate" $MailNotification < $LogFile   exit fi CurrentTime=`date +"%r"` echo "$CurrentTime: Copied dump to backup server in directory $BackupHost$BackupHostPath/$ReplicaSet/$DateStamp/." >> $LogFile
echo "$CurrentTime: Mongo Backup process completed successfully." >> $LogFile
mail -s "Backup for $ReplicaSet done on $MongoHost at $TodayDate" $MailRecipients $MailNotification < $LogFile

Its just a basic script and may needs further enhancements. In case you have suggestion/queries, please put it below in comments.

More Related and helpful articles:

 

Fedora 17 (перепечатка)

Комментариев нет

My main computer is now running Fedora 17 Beta (x86_64)
After Psyche, Shrike, Yarrow, Tettnang, Heidelberg, Stentz, Bordeaux, Zod, Moonshine, Werewolf, Sulphur, Cambridge, Leonidas, Constantine, Goddard, Laughlin, Lovelock and Verne, the hostname is now... DixSept (seventeen)... Lire Fedora 17

RPM PHP 5.3.12 for CentOS 6 (перепечатка)

Комментариев нет

Собрал последнюю версию Php 5.3.12 для CentOS 6.x. Скачать rpm для CentOS 6 можно здесь. Либо подключив репозиторий CentALT для CentOS 6.

04.05.2012

Kernel RHEL6 testing 042stab055.7 (перепечатка)

Комментариев нет

Another fix for race in ext4 (ext4_ioend_wait () hang). Added ext4 fsync optimization stats.

[ Change log/downloads... ]

--Kir 05:06, 4 May 2012 (EDT)

Обновил php до версии php 5.2.17-14 для CentOS 5.x (перепечатка)

Комментариев нет

Обновил php до версии 5.2.17-14 в репозитории CentALT для CentOS 5.x добавив патчи закрывающие Fix PHP-CGI query string parameter vulnerability, CVE-2012—1823. Fixed bug #61650 (ini parser crashes when using ${xxxx} ini variables (without apache2)). Fixed bug #61095 (Incorect lexing of 0×00*+). Fixed bug #61000 (Exceeding max nesting level doesn’t delete numerical vars). Fixed bug #60227 [...]

04.05.2012

php-5.3.12 and php-5.4.2 (перепечатка)

Комментариев нет

RPM of PHP 5.3.12 are available for Enterprise Linux in remi repository.

RPM of PHP 5.4.2 are available for Fedora in remi repository and for Enterprise Linux in remi-test repository.

Read the announcement  : PHP 5.3.12 and PHP 5.4.2 Released!

Use  YUM to install :
yum --enablerepo=remi update php-\*
For all questions or help, please use the Forums... Lire php-5.3.12 and php-5.4.2