If your product involves microservices, API gateway & Service discovery will be an intrinsic part of your deployment architecture.

KrakenD is a stateless, distributed, high-performance API Gateway that helps you effortlessly adopt microservices. KrakenD is more than a typical proxy that forwards clients to backend services, but a powerful engine that can transform, aggregate or remove data from your own or third party services. KrakenD also implements the Backend for Frontend and Micro-frontends patterns to eliminate the necessity of dealing with multiple REST services, isolating clients from the micro-service implementation details.

Consul from HashiCorp is one of the early implementations of service mesh technology. It comes with a full-featured control plane with service discovery, configuration, and segmentation functionality. The best thing about Consul is the support for a variety of environments including traditional applications, VMs, containers, and orchestration engines such as Nomad and Kubernetes.

In this post will talk about how to configure Consul as a DNS SRV in KrakenD API gateway.

Just to give a sample deployment perspective, let’s name out few servers. We have all our servers running on Ubuntu 18.04.

Consul Server Node - 10.0.0.1 - ConsulServer
API Server         - 10.0.0.2 - APIServer
KrakenD Gateway    - 10.0.0.3 - GatewayServer

Step 1 — Download and Install

Download & install consul on all three servers. If you have chosen the pre-compiled version, then complete the following steps

sudo chown root:root consul
sudo mv consul /usr/local/bin/
consul --version

consul command features opt-in auto-completion for flags, sub commands, and arguments (where supported). Enable auto-completion.

consul -autocomplete-install
complete -C /usr/local/bin/consul consul

Create a unique, non-privileged system user to run Consul and create its data directory.

sudo useradd --system --home /etc/consul.d --shell /bin/false consul
sudo mkdir --parents /opt/consul
sudo chown --recursive consul:consul /opt/consul

Just incase you want to configure systemd

sudo touch /etc/systemd/system/consul.service

Add this configuration to the Consul service file:

[Unit]
Description="HashiCorp Consul - A service mesh solution"
Documentation=https://www.consul.io/
Requires=network-online.target
After=network-online.target
ConditionFileNotEmpty=/etc/consul.d/consul.hcl[Service]
Type=notify
User=consul
Group=consul
ExecStart=/usr/local/bin/consul agent -config-dir=/etc/consul.d/
ExecReload=/usr/local/bin/consul reload
KillMode=process
Restart=on-failure
LimitNOFILE=65536[Install]
WantedBy=multi-user.target

Step 2 — Start Consul Server

consul agent -dev -server -bind=10.0.0.1 -node ConsulServer

Add the ip address in the bind property

Step 2 — Start Consul Agent on API server

Let’s assume that an API server is running on port 8080.

Create a directory consul.d and a file ping.json with the following data

{"service": {"name": "ping", "tags": ["dev"], "port": 8080}}

Now start the consul agent with the following command

consul agent -dev -bind=10.0.0.2 -join=10.0.0.1 -node APIServer -config-dir=consul.d/

Each service assumes a FQDN in the format of service_name.service.consul. Consul will resolve the name to the exact service running in a specific data center and node.

Try running the command to check if things are fine.

$ consul members
Node          Address        Status  Type    Build  Protocol  DC   Segment
APIServer     10.0.0.2:8301  alive   server  1.7.1  2         dc1  <all>
ConsulServer  10.0.0.1:8301  alive   server  1.7.1  2         dc1  <all>$ dig @127.0.0.1 -p 8600 ping.service.consul SRV; <<>> DiG 9.11.3-1ubuntu1.11-Ubuntu <<>> @127.0.0.1 -p 8600 ping.service.consul SRV
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 16540
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 3
;; WARNING: recursion requested but not available;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;ping.service.consul.            IN      SRV;; ANSWER SECTION:
ping.service.consul.     0       IN      SRV     1 1 3001 APIServer.node.dc1.consul.;; ADDITIONAL SECTION:
APIServer.node.dc1.consul. 0    IN      A       10.0.0.4
APIServer.node.dc1.consul. 0    IN      TXT     "consul-network-segment=";; Query time: 0 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Fri Mar 06 09:15:53 UTC 2020
;; MSG SIZE  rcvd: 144

Step 3 — Configure DNS in Gateway server

This is one of the step that is pretty confusion and it took us a lot of time to figure out. Gateway server has to resolve the names that consul service exposes, say — ping.service.consul.

Add the following entry in /etc/resolv.confnameserver 127.0.0.1
#nameserver 127.0.0.53Add the following entry in /etc/systemd/resolved.conf[Resolve]
DNS=127.0.0.1
Domains=~consulConfigure iptablesiptables -t nat -A OUTPUT -d localhost -p udp -m udp --dport 53 -j REDIRECT --to-ports 8600
iptables -t nat -A OUTPUT -d localhost -p tcp -m tcp --dport 53 -j REDIRECT --to-ports 8600sudo systemctl restart systemd-resolved

Step 4 — Start Consul Agent on Gateway Server

./consul agent -dev -bind=10.0.0.3 -join=10.0.0.1 -node GatewayServer

Check if the gateway server is able to resolve the host

host -t SRV ping.service.consulshould give the following results:ping.service.consul has SRV record 1 1 3001 APIServer.node.dc1.consul.

Now you are good to start the gateway application.

Step 5 — Start KrakenD gateway application

Create a krakend configuration file

{
  "version": 2,
  "port": 8090,
  "extra_config": {
    "github_com/devopsfaith/krakend-cors": {
      "allow_methods": ["GET","POST","PUT","DELETE"],
      "allow_origins": ["*"],
      "expose_headers": ["Content-Length"],
      "max_age": "12h"
    },
    "sd_providers": {
      "providers": {"dns": true},
      "hosts": [
        {
          "sd": "dns",
          "host": "ping.service.consul"
        }
      ]
    },
    "github_com/devopsfaith/krakend-gologging": {
      "format": "default",
      "level": "DEBUG",
      "prefix": "[KRAKEND]",
      "stdout": true,
      "syslog": true
    },
    "github_com/devopsfaith/krakend-logstash": {
      "enabled": true
    }
  },
  "timeout": "3000ms",
  "cache_ttl": "300s",
  "output_encoding": "json",
  "name": "local-gateway",
  "endpoints": [
    {
      "endpoint": "/",
      "description": "This is a ping pong service",
      "group": "lnr",
      "method": "POST",
      "extra_config": {},
      "output_encoding": "no-op",
      "concurrent_calls": 1,
      "backend": [
        {
          "url_pattern": "/",
          "encoding": "no-op",
          "sd": "dns",
          "host": [
            "ping.service.consul"
          ],
          "disable_host_sanitize": true,
          "is_collection": true,
          "target": "",
          "method": "POST"
        }
      ]
    }
  ]
}

krakend run -c krakend.json -d -p 1234

Test the gateway now:

curl -v -d @request.json -H "Content-Type: application/json" http://localhost:1234/

Should give a ping API response.

Summary

KrakenD API with consul as service discovery gives a lot of flexibility for scaling up your microservices in a production environment.