HowTo: Install MongoDB Replica Set with Docker Swarm in Multiple Regions

Visits: 4821

Running your MongoDB in replica set is a really great idea, using Docker is a quick way to deploy it.

The simplest way to create a multi region MongoDB Replica set using Docker Swarm is to use our scripts that are already written. They launch a CloudFormation and the scripts written below.

Please leave a comment below with any issues or thank yous.

https://aws.amazon.com/marketplace/pp/B01N9N0KFZ/

Then run the CloudFormation Script:

https://console.aws.amazon.com/cloudformation/

http://float.i.ng/replicated-mongodb-installation-instructions/

News for this project: Nov. 16 , 2017.

I have mostly completed the  bash script and redacted it so that it can use Multiple AWS regions. I have also created a CloudFormation file to run this easily.

IN order to get distributed Mongo to talk to each other from their docker containers in other regions, the /etc/hosts file of each container needs to the real Public IP address.

I have been running the commands that are below with docker swarm  to create a  MongoDB Replica set. It builds incredibly fast on AWS on separate instances. The AMI number is not accessible, so replace it with the AMI of your desire. The default of Ubuntu doesn’t work so great. I have also added an XFS volume.

This solution is now a simple  “one-click” solution in the AWS Marketplace, it might save you dozens of hours of work.

The script works on AWS  if you give  IAM priveges to the first instance. I used a custom Ubuntu, but will try with the plain AWS AMI.

Another issue that I encountered was that Docker Swarm could not get the replicas to communicate without opening all possible Docker and MongoDB Ports in the AWS EC2 Security Groups. Here is the settings that I used, You might be able to make it neater.

The issues still left open for this project include

• making a cloudformation script to work so that  clients easily launch MongoDB with replica set.
• making it work with lots more security since MongoDB defaults to work without a password.
• making it store the MongoDB Database files  in the pre-created XFS volume
• making it run in Multiple Regions and even multiple clouds since Docker Swarm can serve as the DNS.

Here is the script as of Oct 31 2017

#!/bin/bash
set -v
set -p 
#in Couldformation label the first one CharmingManager
#fixed SEARCH RESULTS FOR:  DISABLE TRANSPARENT HUGE PAGES (THP) — MONGODB MANUAL 3.4"
Disable Transparent Huge Pages
# delete keys that docker machine makes
# when running the script it needs the AMI number to run 
#Directions - In order to connect the Primary use or the URL to put in your MOngo Client  mongo  mongodb://PublicDNS:27017/test
#You can connect to any of the REplicas by accessing the Public IP of the Swarm Manager and changing the port XXXXXXX
#Directions - In order to connect to the Read Replicas go to the charming1 or 2 Instances and use their PublicDNS for the following command or point your mongo client to the following URL mongo  mongodb://PublicDNS:port/test for charming 1 use port 27018 for charming2 use port 27019
#Our installation has auto complete so to help with options for docker and docker-machine 
# create docker-machines
#make the volume of first machine 20 gb default
#DONT use cloudstor - use aws ec2 cli

# add apt update and upgrade to scripts
# think of adding docker-machine env to cloudformation userdata instead of whole complication with leaving the script on server.
#Variable that will be needed are instance type , key , volume size. voliume type, IOPS needed. PRETTY MUCH LIKE cloudformation for docker for Aws. use https://docs.docker.com/machine/drivers/aws/#default-amis for vars fields


#sleep 50
n=0
	while [[ $n -lt 3 ]]
	do 
aws ec2 create-volume --size 5 --region us-east-1 --availability-zone us-east-1b   --volume-type gp2  --tag-specifications 'ResourceType=volume,Tags=[{Key=charmingvol,Value='$n'}]'
n=$((n+1))
 done
 


n=0
	while [[ $n -lt 3 ]]
	do docker-machine create --driver amazonec2   --amazonec2-iam-instance-profile fullAccessEC2 --amazonec2-zone b  --amazonec2-ami  $1  charming$n 
	n=$((n+1))
 done


sleep 40
# regenerate-certs since Ubuntu does its thing and misses Docker commands
	n=0
	while [[ $n -lt 3 ]]
	do docker-machine regenerate-certs charming$n -f
	n=$((n+1))
done 

 
 n=0
	while [[ $n -lt 3 ]]
	do aws ec2 attach-volume --volume-id $(aws ec2 describe-volumes --filters Name=tag-key,Values="charmingvol" Name=tag-value,Values="$n"  --region us-east-1 | sed -n 's/.*"VolumeId": "\(.*\)",/\1/p') --instance-id $( aws ec2 describe-instances  --region us-east-1 --filters Name=instance-state-code,Values=16  Name=tag-key,Values="Name" Name=tag-value,Values="charming$n" | sed -n 's/.*"InstanceId": "\(.*\)",/\1/p')  --device /dev/sdc  --region us-east-1
	n=$((n+1))
 done
 
  echo "LABEL=cloudimg-rootfs   /        ext4   defaults,discard        0 0" > fstab
echo "/dev/xvdc /mnt/charming xfs rw,user,auto 0 0" >> fstab

 n=0
	while [[ $n -lt 3 ]]
	do 
    docker-machine ssh charming$n  sudo mkfs.xfs /dev/xvdc 
	docker-machine ssh charming$n sudo mkdir /mnt/charming	
	docker-machine scp fstab charming$n:
	docker-machine ssh charming$n sudo /bin/cp -f fstab /etc/fstab
    docker-machine ssh charming$n sudo mount -a 
	n=$((n+1))
 done
 
 #From create-mongo-replicas-7.sh
 
 # create swarm
docker swarm init --listen-addr $(/bin/hostname -i):2377 --advertise-addr $(hostname -i):2377
# get token for joining in script Also make 1 instead of 0
# join swarm - 
 n=0
	while [[ $n -lt 1 ]]
do export JOIN_TOKEN=`docker swarm join-token -q worker`
n=$((n+1))
 done
 n=0
	while [[ $n -lt 3 ]]
	do eval `docker-machine env charming$n`
	docker swarm join --token $JOIN_TOKEN $(/bin/hostname -i):2377
	n=$((n+1))
 done
# make network
 n=0
	while [[ $n -lt 1 ]]
       do eval `docker-machine env -u`
       docker network create --driver overlay --internal charmingnet
n=$((n+1))
 done
n=0
	while [[ $n -lt 3 ]]
	do docker node update --label-add mongo.rpl=$n $(docker node ls -q -f name=charming$n)
	n=$((n+1))
 done
#run docker service  - chANGE port number each time - change name too- 
#Following didnt work Oct - 19 -17 - could be issue with label
n=0
	while [[ $n -lt 3 ]]
	do docker service create --detach=false --network charmingnet --publish 270$((16+n+1)):27017  --mount type=bind,src=/mnt/charming,target=/data/db  --constraint 'node.labels.mongo.rpl=='$n'' --name mongo$n mongo:3.4  mongod   --replSet charming
	n=$((n+1))
 done
#Create Replicas in Mongo - will need number of replicas - skip this figure it out after running the rest of the set.
eval `docker-machine env charming0`
sleep 2
 docker exec  $(docker ps -q) mongo  --eval 'rs.initiate({ _id: "charming", members: [{ _id: 0, host: "mongo0:27017" }, { _id: 1, host: "mongo1:27017" }, { _id: 2, host: "mongo2:27017" }], settings: { getLastErrorDefaults: { w: "majority", wtimeout: 30000 }}})'

This is simpler older script, they all require an AMI that has docker-machine

#!/bin/bash

#put upgrade back on!#remove volumes as well containers and images#volume didnt take up!#Deal with Mongo errors – lower down#VERY clean images, containers, etc
# The folowing works on AWS  if you give  IAM preliges to the first instance. I used a custom Ubuntu, but will try with the plain AWS AMI.
# create docker-machines
n=0 while [[ $n -lt 3 ]] do docker-machine create –driver amazonec2 –amazonec2-zone b  –amazonec2-ami ami-7873be02 charming$n n=$((n+1)) done
sleep 40
# regenerate-certs since Ubuntu does its thing and misses Docker commands n=0 while [[ $n -lt 3 ]] do docker-machine regenerate-certs charming$n -f  n=$((n+1))done

 

# create swarmdocker swarm init –listen-addr $(hostname -i):2377 –advertise-addr $(hostname -i):2377
# get token for joining in script Also make 1 instead of 0

 

JOIN_TOKEN=`docker swarm join-token -q worker`

#join swarm –

n=0 while [[ $n -lt 3 ]] do eval `docker-machine env charming$n` docker swarm join –token $JOIN_TOKEN $(hostname -i):2377 n=$((n+1)) done
#make networkeval `docker-machine env -u`docker network create –driver overlay –internal charmingnet

n=0 while [[ $n -lt 3 ]] do docker node update –label-add mongo.rpl=$n $(docker node ls -q -f name=charming$n) n=$((n+1)) done
#run docker service  – chANGE port number each time – change name too-
#Following didnt work Oct – 19 -17 – could be issue with label
n=0 while [[ $n -lt 3 ]] do docker service create –network charmingnet –publish 270$((16+n+1)):27017  –mount type=bind,src=/mnt/charming,target=/data/db  –constraint ‘node.labels.mongo.rpl==’$n” –name mongo$n mongo:3.4  mongod   –replSet charming n=$((n+1)) done

 

#Create Replicas in Mongo – will need number of replicas – skip this figure it out after running the rest of the set.# eval `docker-machine env charming0`
# docker exec -it $(docker ps -qf label=com.docker.swarm.service.name=mongo0) $(docker ps –format “{{.Names}}”) mongo  –eval ‘rs.initiate({ _id: “charming”, members: [{ _id: 0, host: “mongo0:27017” }, { _id: 1, host: “mongo1:27017” }, { _id: 2, host: “mongo2:27017” }], settings: { getLastErrorDefaults: { w: “majority”, wtimeout: 30000 }}})’ exit

Also see:

•  http://charmingwebdesign.com/bootstrapping-a-docker-swarm-mode-cluster-semaphore/
• http://charmingwebdesign.com/bash-how-do-i-undo-the-command-eval-docker-machine-env-blog-stack-overflow/
• http://charmingwebdesign.com/docker-swarm-intro-tutorial-httpcharming-co-ilaws-marketplacelabsreadme-md-at-master-%c2%b7-dockerlabs/
• http://charmingwebdesign.com/docker-basics-amazon-ec2-container-service-fun/
• https://eladnava.com/deploy-a-highly-available-mongodb-replica-set-on-aws/

This is yet another option to manage networking so that Replica containers can talk to each other.

It seems like there are 4 options

hard code it with hosts file on main manager server

use kubernetes to manage the DNS

Use Docker Swarm – the link below discusses using Swarm

Amazon ECS  also has an option with an agent that configure route 53 DNS

Azure is starting to push their Container Service, but it is based on Open Source Kubernetes or Swarm

You can read about how this scripts is written.

Source: Running a MongoDB Replica Set on Docker 1.12 Swarm Mode: Step by Step

http://charmingwebdesign.com/running-a-mongodb-replica-set-on-docker-1-12-swarm-mode-step-by-step/

Relate Blog articles

http://charmingwebdesign.com/make-xfs-faield-mkfs-xfs-no-such-file-or-directory/

http://charmingwebdesign.com/kubernetes-running-mongodb-on-kubernetes-with-statefulsets/