Игорь Олемской — практические заметки по системному администрированию Linux CentOS

MongoDB Backup types and strategies are neatly explained in its documentation, which you can check here. In case you are not familiar with MongoDB backup types and strategies, please have a look at its documentation.

What I am describing here is a simple script which we are using since months to take MongoDB backup and transfer it over to our Backup server. Here are few things its doing:

• As we have multiple MongoDB Replica Sets, the script identify current replica set and check whether current server is Master or Slave, exit if its Master. We take backup only from Slave host.
• Take Backup using mongodump command.
• Upon successful completion of dump, transfer that to our Backup server. Ensure that ssh key based authentication is setup between both servers to implement seamless and secure transfer. It creates new directory based on current timestamp under replicaset directory in specified path at Backup server and transfer dump there.
• Log each steps described above, send alert mail if any step fail with description or send confirmation mail upon successful execution with essential details.
• Sample confirmation mail is below:

Subject: Backup for  done on  at 09/May/12 08:00:01 AM
Body: Mongo Backup Status for  on 09/May/12 08:00:01 AM.
 
08:00:01 AM:  is slave and looks OK.
08:00:01 AM: Starting Dump, executing /usr/local/mongodb/mongo/bin/mongodump --out /databases/dump --host ...
08:34:06 AM: Mongodump command completed. Backup size is 25G.
08:34:10 AM: Directory created on backup server, copying data using scp...
08:57:00 AM: Copied dump to backup server in directory /home/backup/Mongodump/replicaset/datestamp/.
08:57:00 AM: Mongo Backup process completed successfully.

Here is the full bash script, please note that you need to update variables properly and have to check and update it to run in your environment which can be entirely different from mine. Its just an idea to automate MongoDB backup:

##
# Script to take mongo backup using mongodump and store it in Backup Server
##
 
#!/bin/bash
 
## Set variables
TodayDate=`date +"%d/%b/%g %r"`
DateStamp=`date +%d%m%y%H%M%S`
CurrentTime=`date +"%r"`
 
MongoBinPath="/usr/local/mongodb/mongo/bin"
ReplicaSet=`echo 'rs.status()' | $MongoBinPath/mongo | egrep "set" | awk -F \" '{print $4}'`
MongoHost=`hostname`
LocalBackupPath="/databases/dump"
 
LogFile="/var/log/mongo-backup.log"
IsOK=0
CmdStatus=""
 
BackupHost=xx.xx.xx.xx
BackupHostPath="/home/backup/mongobackup"
BackupHostPort=22
 
MailNotification="admin@domain.com anotheradmin@domain.com"
 
echo -e "Mongo Backup Status for $ReplicaSet on $TodayDate. \n" > $LogFile
 
## Check whether host is slave and in good state for backup
for i in `echo "rs.status()" | $MongoBinPath/mongo | egrep "name" | awk -F \" '{print $4}'| cut -f 1 -d :`; do
 IsMaster=`echo "db.isMaster()"| $MongoBinPath/mongo --host $i | grep ismaster|awk -F ":" '{print $2}' | cut -f 1 -d ,`;
 TheState=`echo "rs.status()"| $MongoBinPath/mongo --host $i | grep -i mystate | awk -F ":" '{print $2}' | cut -f 1 -d ,`;
 if [ $IsMaster == "false" -a $TheState -eq 2  ]; then
  MongoHost=$i
  IsOK=1
  echo "$CurrentTime: $MongoHost is slave and looks OK." >> $LogFile
  break
 fi
done
 
CurrentTime=`date +"%r"`
 
## Exit if not good
if [ $IsOK -eq 0 ]; then
   echo "$CurrentTime: Error: Either $MongoHost is not slave or not in good state. Aborting Backup, Please check!" >> $LogFile
   mail -s "Backup error for $ReplicaSet from $MongoHost on $TodayDate" $MailNotification < $LogFile    exit 1; fi ## Remove earlier backup CmdStatus=$(rm -rf $LocalBackupPath/*) ## Start backup process echo "$CurrentTime: Starting Dump, executing $MongoBinPath/mongodump --out $LocalBackupPath --host $MongoHost..." >> $LogFile
 
CmdStatus=`$MongoBinPath/mongodump --out $LocalBackupPath --host $MongoHost`
if [ $? -ne 0 ]; then
  echo "$CurrentTime: There is an issue while trying to take dump in $MongoHost. Aborting dump process, please check! " >> $LogFile
  cat $CmdStatus >> $LogFile
  mail -s "Backup error for $ReplicaSet from $MongoHost on $TodayDate" $MailNotification < $LogFile   exit fi CurrentTime=`date +"%r"` BackupSize=$(du -sh $LocalBackupPath/. | awk '{ print $1 }') echo "$CurrentTime: Mongodump command completed. Backup size is $BackupSize. " >> $LogFile
 
## dump is fine then scp it to backup server
## create directory first
CmdStatus=$(ssh -p $BackupHostPort $BackupHost "mkdir -p  $BackupHostPath/$ReplicaSet/$DateStamp")
 
if [ $? -ne 0 ]; then
  echo "$CurrentTime: Either failing to connect Backup server using ssh or destination directory already exist!" >> $LogFile
  cat $CmdStatus >> $LogFile
  mail -s "Backup error for $ReplicaSet from $MongoHost on $TodayDate" $MailNotification < $LogFile   exit fi CurrentTime=`date +"%r"` echo "$CurrentTime: Directory created on backup server, copying data using scp..." >> $LogFile
 
CmdStatus=`scp -P $BackupHostPort -r $LocalBackupPath/* $BackupHost:/$BackupHostPath/$ReplicaSet/$DateStamp/`
if [ $? -ne 0 ]; then
  echo "$CurrentTime: Unable to scp dump to $BackupHost:/$BackupHostPath/$ReplicaSet/$DateStamp using port $BackupHostPort. " >> $LogFile
  cat $CmdStatus >> $LogFile
  mail -s "Backup error for $ReplicaSet from $MongoHost on $TodayDate" $MailNotification < $LogFile   exit fi CurrentTime=`date +"%r"` echo "$CurrentTime: Copied dump to backup server in directory $BackupHost$BackupHostPath/$ReplicaSet/$DateStamp/." >> $LogFile
echo "$CurrentTime: Mongo Backup process completed successfully." >> $LogFile
mail -s "Backup for $ReplicaSet done on $MongoHost at $TodayDate" $MailRecipients $MailNotification < $LogFile

Its just a basic script and may needs further enhancements. In case you have suggestion/queries, please put it below in comments.

More Related and helpful articles:

• Setup multiple MySQL database servers in a single Linux host
• Optimize MySQL on a large Database Server
• Recover or reset root password of MySQL and PostgreSQL Servers 
• Optimize and fix MySQL Server running slow without any load
• How to find out the clients connecting to your MySQL server
• Quickly repair huge corrupted or crashed table in MySQL
• Install and configure PhpMyAdmin to manage multiple MySQL Servers

*Kind Note*: This was written with a sense of humor to allow visitors quickly discover code anomaly and suggests fixes but if it is not up to your mark, please close your browser tab instead of making unnecessary noise. Thanks You!
I am putting a damn small thing here regarding bash script for fun.

Here it goes: Need to create a bash script which asks for a word from user, say either “one”, “two” or “three” and then check in single if statement (no else if section) that if its not “one” or “two” or “three” then print Not OK otherwise print OK.

Further, here is a snippet, check and figure out why its not working as expected:

#!/bin/bash
echo -n Enter a word:
read myword
 
if [[ $myword != "one" || $myword != "two" || $myword != "three" ]]; then
 echo Not OK
else
 echo OK
fi

Please put your suggestion/solution script in comments.

I found myself stuck in a particularly nasty situation a few weeks ago where I had two git branches with some commits that were mixed up. Some commits destined for a branch called development ended up in master. To make matters worse, development was rebased on top of master and the history was obviously mangled.

My goal was to find out which commits existed in development but didn't exist anywhere in master. From there, I needed to find out which commits existed in master that didn't exist in development. That would give me all of the commits that needed to be in the development branch.

I constructed this awful looking bash mess to figure out which commits were in development but not in master:

I had a list of commits that existed in development but not in master:

965cf71 Trollface
acda854 Some patch 2
bf1f3e2 Some patch 1
db1980c Packaging

From there, I could swap MASTER and DEV to figure out which commits existed in master but not in development. Only a couple of commits showed up and these were the ones which were committed and pushed to master inadvertently. After a couple of careful cherry picks and reversions, my branches were back to normal.

Compare commits between two git branches is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

How to run a script per second? Any easy way to check that no multiple instances of the script will run at a single time? I am cnvering 3 simple ways which helps you to fulfill such requirements: Using Cron If script needs to be executed every x minute or a frequency which is more [...]

Sometimes I want to FTP a file from one machine to another. Usually, I can do the transfer interactively, but every so often, I would like to have a shell script do the file transfer. This task has eluded me in the past, but I finally figured it out. I've not seen this particular trick documented in the past, so I submit it for your approval...

Original news source

Here is a simple one liner to check on download time of a webpage:

read more

Below is a simple bash script that uses the dictionary to go through 3 letter words and does a whois check to see if the domain is available. You probably won't find any of those as all would have been taken. May want to try with 5 or 6 letters instead, in which case change the value of «$count».

read more

С помощью данного скрипта можно синхронизировать папки сайтов, базы MySQL и конфиги виртуал-хостов апача и nginx с одного сервера на другой. Синхронизация баз происходит без остановку мускула.

There are of course various ways to check whether your mail server is now configured ok or not but what I found is that checking through telnet is quick and easy. let’s check our mail server now, it may be mail.youdomain.com or localhost depending on what you are using right now, here’s the full process: [...]

Below are one liners to clean out all comment and blank lines with grep and sed, usually in config files.

or

read more