Author

Browsing

Kafka only transfers data from producer to consumer in byte format and it doesn’t have a verification data mechanism. In fact, Kafka does not even know what kind of data it is sending or receiving; whether it is a string or integer. So Kafka needs a server to stand outside and handle data verification between producers and consumers.

RabbitMQ is an open-source message broker that acts as the intermediary or middleman for independent applications, giving them a common platform to communicate. Simply said; it is software where queues are defined, to which applications connect in order to transfer a message or messages.  The queue-manager software stores the messages until a receiving application connects and takes a message off the queue. 

For decades, organizations have been using a traditional relational database and trying to fit everything there, whether it is key/value-based user session data, unstructured log data, or analytics data for a data warehouse. However, the truth is, the relational database is meant for transaction data, and it doesn’t work very well for other data types.

This article is one of the tutorials about how to config API in Kong and Konga to manage microservices. Before read this article you need to know what is Kong and key concepts in Kong and install Kong on Docker.

In this tutorial we’ll use fake API from https://fakestoreapi.com/docs . We will call API to get list products, cart via Kong API gateway, then Kong will forward this request to https://fakestoreapi.com/products and https://fakestoreapi.com/carts

Adding Services

At first you need to access Services menu, click add new service button then fill form like this

NameAdd a unique name for your upstream service.
DescriptionService description.
TagsList of tags to identify a group of services together. Press ENTER for any kind of array values in Konga UI.
URLShorthand for setting HostPathProtocol with just one value. Note that this is only a feature in Konga, Kong doesn’t have it, when using it with the Admin API calls directly.

In this example we leave path field is empty (or /) because the API https://fakestoreapi.com/products has only 1 level in the path. If you want to call upstream http://host/base-path/path you’ll need to set up Path in the above dialog is base-path.

Adding routes

If you want to call API kong-gateway-adress/fake/products and Kong will forward this request to upstream https://fakestoreapi.com/products you will need to setup route like this. In Add Routes Dialog you only need to set Paths and Methods field. You should note that the value of Strip Path must be false.

FieldValue
NameAdd a unique name for your route.
HostsKong checks for the hostname present in the incoming request’s header. If you specify this value then the Hostname must be present for Kong to match the request to this route. This is suitable only if you want to block any request made outside this hostname. You can leave it null if not needed.
PathList of paths present in the incoming request. This is required to namespace the upstream endpoints. The client must send this prefix in the request, Kong will try to match any request’s path in this list of paths and based on the setting of strip_path the request will be proxied.
StripPathBoolean value, which configures Kong to strip the matching path from the incoming request to the upstream URL.

Now, you can open terminal or Postman and call

curl -i localhost:8000/fake/products

We realize that the response header will contain via:Kong. Kong will try to match the path /fake in this incoming request and look for the routes where the path is fake. Since it found the correct route, and we have set strip_path to True, Kong will just remove this particular path prefix while reverse proxying to the upstream URL. In this way, our upstream doesn’t need to be concerned about the path prefix as well.

Security API – Authenticate Kong API with plugin Key Auth on Service

In created service learncode24h, you click on Plugins and Add Plugin button

In the next screen, you need to specify key names (I use apikey in this sample), all other fields you can leave blank.

Adding Consumers

After setting up authentication for API, you need to generate apikey for clients before calling API. Go to Consumers link and click Create Consumer button.

Then you click to tab Credentials -> API KEYS and click CREATE API KEY button, in this dialog, you can enter your API key or leave it blank and Kong will generate an apikey for you.

Now you’ll need to add an apikey to the header when calling API localhost:8000/fake/products. Without you’ll get 401 – unauthorized error.

$ curl -i -H "apikey:{{your-api-key}}" http://localhost:8000/fake/products

 In real-world applications, the readers should only be able to read. Managing resources is something only administrators/editors should be allowed to do.

Our management route, — the one that accepts POST , PUT , PATCH and DELETE requests should only be accessible by certain consumer groups. In order to achieve that, we can use Access Control Lists plugin and specify Consumer Group for each route. For example:

  • Routes with GET method will be assigned for READER consumer group
  • Routes with POST/DELETE/PATH method will be assigned for EDITOR consumer group

Setting up Access Control Lists for routes

We’ll create one more consumer is learncode24h-editor with Write permission.

After that, you need to click on Groups on each consumer and set Group for per consumer like this. You can name the group anything.

Go to routes of learncode24h-service and click on Plugins then click ADD PLUGIN button

In allow field, we need to specify consumer group (don’t forget to press enter after typing)

Now, all clients with apikey in editor consumer group will be can call all APIS in this route.

About Kong API Gateway

Kong is an API gateway and platform. That means it is a form of middleware between computing clients and your API-based applications. Kong easily and consistently extends the features of your APIs. Some of the popular features deployed through Kong include authentication, security, traffic control, serverless, analytics & monitoring, request/response transformations and logging.

You can read more about what is Kong and concepts in Kong

How to install Kong on Docker

Step1: Create a Docker network

docker network create my-network

Step2: Install Postgress and prepare Postgres DB for Kong

docker run -d --name kong-database --network=my-network \
-e "POSTGRES_USER=kong" -e "POSTGRES_DB=kong" -e "POSTGRES_PASSWORD=kong" \
-p 5432:5432 postgres:9.6

docker run --rm --network=my-network \
-e "KONG_DATABASE=postgres" -e "KONG_PG_HOST=kong-database" \
-e "KONG_PG_PASSWORD=kong" kong:latest kong migrations bootstrap

Step3: Install and start Kong

docker run -d --name kong --network=my-network \
-e "KONG_DATABASE=postgres" -e "KONG_PG_HOST=kong-database" \
-e "KONG_PG_PASSWORD=kong" -e "KONG_PROXY_ACCESS_LOG=/dev/stdout" \
-e "KONG_ADMIN_ACCESS_LOG=/dev/stdout" -e "KONG_PROXY_ERROR_LOG=/dev/stderr" \
-e "KONG_ADMIN_ERROR_LOG=/dev/stderr" -e "KONG_ADMIN_LISTEN=0.0.0.0:8001, 0.0.0.0:8444 ssl" \
-p 8000:8000 -p 8443:8443 -p 8001:8001 -p 8444:8444 kong:latest

List docker containers

docker ps

Check Kong is started?

curl -X POST --url http://localhost:8001/services/ --data 'name=my-api' --data 'url=http://localhost:8080'

if Kong is started, you will receive a response like this

{
   "code":5,
   "name":"unique constraint violation",
   "message":"UNIQUE violation detected on '{name=\"my-api\"}'",
   "fields":{
      "name":"my-api"
   }
}

You can also check Kong is started by call curl -i http://localhost:8001/

What is Konga?

Konga is a fully-featured open-source, multi-user GUI, that makes the hard task of managing multiple Kong installations easy way.

It can be integrated with MySQL, postgresSQL, MongoDB databases out of the box, and provides the GUI for better understanding and maintain architecture.

Konga Features

  • Manage all Kong Admin API Objects.
  • Import Consumers from remote sources (Databases, files, APIs etc.).
  • Manage multiple Kong Nodes.
  • Backup, restore and migrate Kong Nodes using Snapshots.
  • Monitor Node and API states using health checks.
  • Email & Slack notifications.
  • Multiple users.
  • Easy database integration (MySQL, postgresSQL, MongoDB).

Install Konga

Prepare Konga database

docker run --rm --network=my-network pantsel/konga -c prepare -a postgres -u postgresql://kong:kong@kong-database:5432/konga_db

In the above command, you need to specify username, password and database Postgress in connection string postgresql://user:pass@kong-database-container-name:5432/konga-db-name

Install and start Konga

docker run -p 1337:1337 \
     --network=my-network \
     -e "DB_ADAPTER=postgres" \
     -e "DB_HOST=kong-database" \
     -e "DB_USER=kong" \
     -e "DB_PASSWORD=kong" \
     -e "DB_DATABASE=konga_db" \
     -e "KONGA_HOOK_TIMEOUT=120000" \
     -e "NODE_ENV=production" \
     --name konga \
     pantsel/konga

Access Konga at http://localhost:1337

Connect Konga to Kong

After installing Konga successfully, you need to connect Konga to Kong by accessing the connections link and creating a new connection. At this screen, you need to fill name and Kong Admin URL like this

Next, you need to activate the created connection by clicking on activate button.

If successful, you will receive a sidebar menu with full features like this:

What is Kong API Gateway?

Kong is Orchestration Microservice API Gateway. Kong provides a flexible abstraction layer that securely manages communication between clients and microservices via API. Also known as an API Gateway, API middleware or in some cases Service Mesh. It is available as open-source project in 2015, its core values are high performance and extensibility. The database could be either Cassandra or Postgres. From version 1.1 it has become possible to run kong without a database