Nomad setup with standalone Nomad Server with multiple worker nodes without use of Consul.

Let see at high level, what use cases does Nomad support.

1. Simple Container Orchestration (Easier to manage containers than k8s in my opinion)
2. Non-Container based Application Workload Orchestration (This would really excite many organizations that wanted to move quickly to cloud)

Resource Requirements for this setup

• 3 VMs or 3 LXC (Choice is yours) , you may setup this on your workstation/laptop or you can spin off compute in any cloud provider of your choice. If you want to know more about setting up LXCs ,here is my detailed blog https://livespotty.github.io/kubernetes-the-hard-way-on-lxd/docs/03-compute-resources.html

for this demonstration, I have used Oracle Cloud Infrastructure (OCI), since you get 4 CPU (ARM64) 24 GB always free :)

• Note: If you are using any of the cloud providers ensure you open these ports ( 4646,4647,4648,5648 )for inter VM communication. Also linux firewall is disabled.

sudo ufw status

sudo ufw disable

After creating 3 virtual servers with latest Ubuntu 20.4 OS, download the corresponding version of Nomad under the following link https://www.nomadproject.io/downloads

I have used Nomad v1.1.6. for setting up my development cluster environment.

List of servers and purpose

1. nomad-server :- This serves as standalone Nomad Control Server that would schedule jobs
2. nomad-node-1 :- Nomad node, where the jobs will be scheduled based on resource availability
3. nomad-node-2 :- Second Nomad server node, where the jobs will be scheduled based on resource availability

Step 1:

• Create data directory, say “/opt/nomad/data” on each server machines (all three)

Step 2:

• Copy the following configuration file to “nomad-nerver” (say, if your server private IP is 10.0.0.1) to “/opt/nomad/config” directory with the filename server.conf

data_dir  = "/opt/nomad/data"

bind_addr = "10.0.0.1" # the default

advertise {
  # Defaults to the first private IP address.
  http = "10.0.0.1"
  rpc  = "10.0.0.1"
  serf = "10.0.0.1:5648" # non-default ports may be specified
}

server {
  enabled          = true
  bootstrap_expect = 1
}
# if you enable client and set the value to true, you will be able to schedule jobs on server.
client {
  enabled       = false
}

plugin "raw_exec" {
  config {
    enabled = true
  }
}

ports {
  http = 4646
  rpc  = 4647
  serf = 4648
}

Step 3:

• Log on to “nomad-node-1” (Say your node1 IP is 10.0.0.2) and copy the following node.conf to the location “/opt/nomad/config”

data_dir  = "/opt/nomad/data"

bind_addr = "10.0.0.2" # the default

advertise {
  # Defaults to the first private IP address.
  http = "10.0.0.2"
  rpc  = "10.0.0.2"
  serf = "10.0.0.2:5648" # non-default ports may be specified
}

server {
  enabled          = false
  bootstrap_expect = 1
}

client {
  enabled       = true
  servers = ["10.0.0.1:4647"]
}

plugin "raw_exec" {
  config {
    enabled = true
  }
}

ports {
  http = 4646
  rpc  = 4647
  serf = 4648
}

Step 4:

• Log on to “nomad-node-2” (Say your node1 IP is 10.0.0.3) and copy the following node.conf to the location “/opt/nomad/config”

data_dir  = "/opt/nomad/data"

bind_addr = "10.0.0.3" # the default

advertise {
  # Defaults to the first private IP address.
  http = "10.0.0.3"
  rpc  = "10.0.0.3"
  serf = "10.0.0.3:5648" # non-default ports may be specified
}

server {
  enabled          = false
  bootstrap_expect = 1
}

client {
  enabled       = true
  servers = ["10.0.0.1:4647"]
}

plugin "raw_exec" {
  config {
    enabled = true
  }
}

ports {
  http = 4646
  rpc  = 4647
  serf = 4648
}

Now we have all the configurations ready, now its time to run the nomad process in each server

Step 5:

• Ensure you have the correct version of nomad installed, run the following command in “nomad-server”

sudo nomad agent -config=/opt/nomad/config/server.conf

Step 6:

• Run the following command in both the node servers “nomad-node-1” and “nomad-node-2”

sudo nomad agent -config=/opt/nomad/config/node.conf

How to test the installation

Run the following command and you will see the following output.

NOMAD_ADDR=http://10.0.0.1:4646 nomad server members

Name                 Address    Port  Status  Leader  Protocol  Build  Datacenter  Region
nomad-server.global  10.0.0.1   5648  alive   true    2         1.1.6  dc1         global

Testing the node servers

NOMAD_ADDR=http://10.0.0.1:4646 nomad node status

ID        DC   Name            Class   Drain  Eligibility  Status
8edd9637  dc1  nomad-node-2    <none>  false  eligible     ready
4181323f  dc1  nomad-node-1    <none>  false  eligible     ready

And thats it! you have nomad development cluster up and running. I will cover how to schedule simple jobs on Nomad in my next blog.

For more information, visit https://nomadproject.io