MongoDB Backup types and strategies are neatly explained in its documentation, which you can check . In case you are not familiar with MongoDB backup types and strategies, please have a look at its documentation.
What I am describing here is a simple script which we are using since months to take MongoDB backup and transfer it over to our Backup server. Here are few things its doing:
- As we have multiple MongoDB Replica Sets, the script identify current replica set and check whether current server is Master or Slave, exit if its Master. We take backup only from Slave host.
- Take Backup using mongodump command.
- Upon successful completion of dump, transfer that to our Backup server. Ensure that ssh key based authentication is setup between both servers to implement seamless and secure transfer. It creates new directory based on current timestamp under replicaset directory in specified path at Backup server and transfer dump there.
- Log each steps described above, send alert mail if any step fail with description or send confirmation mail upon successful execution with essential details.
- Sample confirmation mail is below:
Subject: Backup for done on at 09/May/12 08:00:01 AM Body: Mongo Backup Status for on 09/May/12 08:00:01 AM. 08:00:01 AM: is slave and looks OK. 08:00:01 AM: Starting Dump, executing /usr/local/mongodb/mongo/bin/mongodump --out /databases/dump --host ... 08:34:06 AM: Mongodump command completed. Backup size is 25G. 08:34:10 AM: Directory created on backup server, copying data using scp... 08:57:00 AM: Copied dump to backup server in directory /home/backup/Mongodump/replicaset/datestamp/. 08:57:00 AM: Mongo Backup process completed successfully.
Here is the full bash script, please note that you need to update variables properly and have to check and update it to run in your environment which can be entirely different from mine. Its just an idea to automate MongoDB backup:
## # Script to take mongo backup using mongodump and store it in Backup Server ## #!/bin/bash ## Set variables TodayDate=`date +"%d/%b/%g %r"` DateStamp=`date +%d%m%y%H%M%S` CurrentTime=`date +"%r"` MongoBinPath="/usr/local/mongodb/mongo/bin" ReplicaSet=`echo 'rs.status()' | $MongoBinPath/mongo | egrep "set" | awk -F \" '{print $4}'` MongoHost=`hostname` LocalBackupPath="/databases/dump" LogFile="/var/log/mongo-backup.log" IsOK=0 CmdStatus="" BackupHost=xx.xx.xx.xx BackupHostPath="/home/backup/mongobackup" BackupHostPort=22 MailNotification="admin@domain.com anotheradmin@domain.com" echo -e "Mongo Backup Status for $ReplicaSet on $TodayDate. \n" > $LogFile ## Check whether host is slave and in good state for backup for i in `echo "rs.status()" | $MongoBinPath/mongo | egrep "name" | awk -F \" '{print $4}'| cut -f 1 -d :`; do IsMaster=`echo "db.isMaster()"| $MongoBinPath/mongo --host $i | grep ismaster|awk -F ":" '{print $2}' | cut -f 1 -d ,`; TheState=`echo "rs.status()"| $MongoBinPath/mongo --host $i | grep -i mystate | awk -F ":" '{print $2}' | cut -f 1 -d ,`; if [ $IsMaster == "false" -a $TheState -eq 2 ]; then MongoHost=$i IsOK=1 echo "$CurrentTime: $MongoHost is slave and looks OK." >> $LogFile break fi done CurrentTime=`date +"%r"` ## Exit if not good if [ $IsOK -eq 0 ]; then echo "$CurrentTime: Error: Either $MongoHost is not slave or not in good state. Aborting Backup, Please check!" >> $LogFile mail -s "Backup error for $ReplicaSet from $MongoHost on $TodayDate" $MailNotification < $LogFile exit 1; fi ## Remove earlier backup CmdStatus=$(rm -rf $LocalBackupPath/*) ## Start backup process echo "$CurrentTime: Starting Dump, executing $MongoBinPath/mongodump --out $LocalBackupPath --host $MongoHost..." >> $LogFile CmdStatus=`$MongoBinPath/mongodump --out $LocalBackupPath --host $MongoHost` if [ $? -ne 0 ]; then echo "$CurrentTime: There is an issue while trying to take dump in $MongoHost. Aborting dump process, please check! " >> $LogFile cat $CmdStatus >> $LogFile mail -s "Backup error for $ReplicaSet from $MongoHost on $TodayDate" $MailNotification < $LogFile exit fi CurrentTime=`date +"%r"` BackupSize=$(du -sh $LocalBackupPath/. | awk '{ print $1 }') echo "$CurrentTime: Mongodump command completed. Backup size is $BackupSize. " >> $LogFile ## dump is fine then scp it to backup server ## create directory first CmdStatus=$(ssh -p $BackupHostPort $BackupHost "mkdir -p $BackupHostPath/$ReplicaSet/$DateStamp") if [ $? -ne 0 ]; then echo "$CurrentTime: Either failing to connect Backup server using ssh or destination directory already exist!" >> $LogFile cat $CmdStatus >> $LogFile mail -s "Backup error for $ReplicaSet from $MongoHost on $TodayDate" $MailNotification < $LogFile exit fi CurrentTime=`date +"%r"` echo "$CurrentTime: Directory created on backup server, copying data using scp..." >> $LogFile CmdStatus=`scp -P $BackupHostPort -r $LocalBackupPath/* $BackupHost:/$BackupHostPath/$ReplicaSet/$DateStamp/` if [ $? -ne 0 ]; then echo "$CurrentTime: Unable to scp dump to $BackupHost:/$BackupHostPath/$ReplicaSet/$DateStamp using port $BackupHostPort. " >> $LogFile cat $CmdStatus >> $LogFile mail -s "Backup error for $ReplicaSet from $MongoHost on $TodayDate" $MailNotification < $LogFile exit fi CurrentTime=`date +"%r"` echo "$CurrentTime: Copied dump to backup server in directory $BackupHost$BackupHostPath/$ReplicaSet/$DateStamp/." >> $LogFile echo "$CurrentTime: Mongo Backup process completed successfully." >> $LogFile mail -s "Backup for $ReplicaSet done on $MongoHost at $TodayDate" $MailRecipients $MailNotification < $LogFile
Its just a basic script and may needs further enhancements. In case you have suggestion/queries, please put it below in comments.
More Related and helpful articles: